Friday, 2018-09-14

*** longkb has joined #senlin00:19
openstackgerritDuc Truong proposed openstack/senlin master: [WIP] Reject actions if target resource is locked
*** fabian_ has joined #senlin00:58
openstackgerritMerged openstack/senlin-tempest-plugin master: pep8 test default use python3
*** longkb has quit IRC02:13
*** longkb has joined #senlin02:14
*** XueFeng has joined #senlin03:41
*** XueFeng has quit IRC04:27
*** XueFeng has joined #senlin04:28
dtruongok, let's start the weekly meeting05:30
dtruong#startmeeting senlin05:30
openstackMeeting started Fri Sep 14 05:30:39 2018 UTC and is due to finish in 60 minutes.  The chair is dtruong. Information about MeetBot at
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.05:30
*** openstack changes topic to " (Meeting topic: senlin)"05:30
openstackThe meeting name has been set to 'senlin'05:30
dtruonganybody here for the meeting?05:31
fabian_Hi,I'm on line.05:31
*** fabian_ is now known as chenyb405:31
dtruonghi chenyb405:32
chenyb4hi, dtruong05:32
dtruongok, let's get started05:33
dtruong#topic announcements05:33
*** openstack changes topic to "announcements (Meeting topic: senlin)"05:33
dtruongi have send out nominations for 2 new core members in the dev mailing list05:34
dtruongif you haven't voted yet, you have until monday morning to do so05:34
dtruonghi qiming05:34
dtruongi also send out a nomination for chenyb4 to join stable maintainers:05:35
dtruongagain voting is open until monday for current stable cores05:35
dtruongother news is that PTG is ongoing this week for other projects05:36
dtruongi saw a mention of senlin in the K8s SIG etherpad:05:37
dtruongfrom the notes it appears that they agreed on a common library for nova, heat and senlin05:38
dtruongfor autoscaling k8s05:38
dtruongi'll keep an eye out for any more news on that05:39
Qimingthat is weird05:39
Qimingwhy a common lib is needed?05:39
dtruongnot sure.  maybe they want to abstract the service which is performing the autoscaling away from k8s or magnum05:41
dtruongi'll try to join the k8s SIG IRC meetings and find out more05:41
Qimingthat would be great05:42
dtruongactually, i think they use slack not irc05:42
dtruongyea, i'll keep everyone updated if i find out more05:43
chenyb4dtruong, you need use Slack join sig05:43
dtruongthat's right.  i did join the slack channel before, but i rarely log in because there is a lot of discussions going on05:44
Qimingjust took a quick glance over the code here:
QimingIt is in pretty early stage05:45
Qimingand it is more about interfacing heat from k8s05:46
dtruongyea, this one directly uses heat05:47
dtruongwe'll have to see how the common library fits in with this05:47
dtruongok, let's move on to next topic05:48
dtruong#topic blueprint status05:49
*** openstack changes topic to "blueprint status (Meeting topic: senlin)"05:49
dtruongi have created a blueprint for fail fast on locked resources05:49
dtruongthe spec is here:
dtruongqiming, the spec is already merged but did you have any comments on it?05:50
Qimingwhat's the reason to skip the retry?05:50
Qimingah ... I see05:51
Qimingreading the problem description now05:51
dtruongtwo main reasons.  we have seen problems when a lot of API requests are being made05:52
Qimingmakes sense05:52
dtruongand the engine keeps retrying05:52
Qimingleave the choice to users again05:52
dtruongyes, because we have seen the engine spin at 100% for 1 hour trying to process all the requests on the same cluster05:52
dtruong*100% CPU usage05:53
Qiming409 error code is okay05:53
dtruongalso, i checked AWS autoscaling implementation and it does the same thing05:53
dtruongit will return error code if the ASG is already in use05:53
Qimingthe original assumption was that locks will be freed up pretty soon05:54
Qimingour design idea was mostly from JVM lock implementation05:54
dtruongwe will still keep the retry implemention because it is still possible that 2 actions can arrive at the same time05:55
dtruongbut this blueprint will prevent hundreds of request queuing up05:55
Qimingokay, just wanted to clarify where we are from. Little bit hints here:
Qimingin JVM, J9 specifically, there are three layers of locks for monitors. The inner most one is a spin loop05:57
Qimingwe borrowed that idea into senlin lock implementation05:57
Qimingthat doesn't mean the idea applies well to web services05:57
Qimingso .. no objections from me regarding the BP05:58
dtruongok, thanks.05:58
dtruongbtw, i do think the lock implemention is not bad and still useful because we can have multiple engines running05:58
dtruongthe locking will be handle to the concurrency between two engines operating on the same cluster05:59
dtruongok, the next blueprint is multiple detection modes06:00
dtruongplease look over it when you get a chance06:01
dtruongchenyb4: for the other items you added in
dtruongcan you create blueprints when you have time?06:02
dtruong#topic stein goal: python 306:03
*** openstack changes topic to "stein goal: python 3 (Meeting topic: senlin)"06:04
dtruongthis is on-going.06:04
dtruonga few patches for the tox environments were proposed by the python 3 champions and merged.06:04
Qimingokay ...06:05
dtruongany other updates on that topic, chenyb406:05
QimingI believe when we started senlin we aimed to support python3 and we aimed to support keystone v3 only06:05
dtruongi believe senlin runs fine with python 3.  all the unit tests and functional tests pass in python 3 environment06:06
dtruongso there shouldn't be much work for us to complete this goal06:07
Qiminggood to know06:07
chenyb4yes, most work in python3 was finish.06:08
dtruongok, thanks06:08
dtruongthe other stein community goal is upgrade checks06:09
dtruong#topic stein goal upgrade checks06:09
*** openstack changes topic to "stein goal upgrade checks (Meeting topic: senlin)"06:09
dtruongwe still need a volunteer for this. if anyone is interested, please let me know06:10
dtruongi probably also will send out an email to the dev mailing list to see if anyone is intestered on working on this06:10
chenyb4I am watching, but I have not started working yet.06:11
dtruongon the upgrade checkers?06:12
Qimingchenyb4, don't be shy when getting your hands dirty06:12
Qimingwe all do06:12
chenyb4I am looking at the implementation code of nova.06:13
Qimingjust checked, it looks clustering support is all in06:13
dtruongoh, we (blizzard) added that support in gophercloud06:14
dtruongwe also plan on adding terraform provider support for senlin06:14
dtruongfor the upgrade checks, the only potential problem that i can think of this upgrades to policies06:16
QimingI'm working on an internal workflow orchestrator recently06:16
dtruongfor example if we go from health policy v1.0 to v1.1 and the schema changes between the two versions06:17
Qimingthis orchestrator currently understands some VMware language06:17
Qimingas the next step, I'm gonna teach it how to speak terraform, how to speak k8s etc.06:18
dtruongcool. terraform is a common tool for cloud operators06:18
QimingI believe we have some version checks in place06:18
Qimingthe framework has room for improvement06:19
QimingI was thinking whether we can use oslo.versionedobject to manage the versioning of profiles and policies06:20
Qimingif we do that, we can get rid of the current schema parsing and version checking etc.06:21
Qimingit means we will be managing api requests/responses and resource representations all using the same technology06:21
Qimingfor versioning06:21
dtruongthat's not a bad idea06:22
eanderssonI like that06:22
dtruongit's a pretty big change but it would simplify the senlin code a lot06:22
dtruongok, let me put it down as an investigation item06:23
dtruong#action investigate using oslo.versionedobject for profiles and policies06:24
dtruongok, we have a few minutes left06:24
dtruong#topic reviews06:24
*** openstack changes topic to "reviews (Meeting topic: senlin)"06:24
eanderssonLets get the python 3.x test merged06:25
dtruonganybody have open patch sets that you would like to get looked at?06:25
eanderssonfor rocky06:25
dtruonglooks like we got +2 from qiming, so i will go ahead and add +1 workflow06:26
dtruongok, if there no other reviews, we can open the floor for discussion06:27
dtruong#topic open discussion06:27
*** openstack changes topic to "open discussion (Meeting topic: senlin)"06:27
dtruonganybody have anything they would like to discuss?06:28
dtruongok, i'll end the meeting then.06:29
dtruongthanks everybody for attending06:29
Qimingthanks guys06:29
*** openstack changes topic to "IRCLog: | Bugs: | Review:,n,z"06:29
openstackMeeting ended Fri Sep 14 06:29:47 2018 UTC.  Information about MeetBot at . (v 0.1.4)06:29
openstackMinutes (text):
*** AlexeyAbashkin has joined #senlin06:39
openstackgerritOpenStack Proposal Bot proposed openstack/senlin-dashboard master: Imported Translations from Zanata
*** AlexeyAbashkin has quit IRC08:51
*** AlexeyAbashkin has joined #senlin08:53
*** longkb has quit IRC10:01
*** chenyb4 has quit IRC10:06
*** chenyb4 has joined #senlin12:27
*** ChanServ sets mode: +rf #openstack-unregistered12:32
*** chenyb4 has quit IRC12:35
*** Qiming has quit IRC13:55
*** Qiming has joined #senlin13:59
*** AlexeyAbashkin has quit IRC16:17

Generated by 2.15.3 by Marius Gedminas - find it at!