Monday, 2016-10-31

*** thorst has quit IRC00:00
*** thorst has joined #openstack-powervm00:23
*** thorst has quit IRC00:26
*** thorst has joined #openstack-powervm00:27
*** thorst has quit IRC00:31
*** thorst has joined #openstack-powervm00:31
*** thorst has quit IRC00:37
*** tjakobs has joined #openstack-powervm00:41
*** efried has quit IRC00:42
*** YuYangWang has joined #openstack-powervm00:45
*** YuYangWang has quit IRC00:45
*** efried has joined #openstack-powervm00:53
*** tjakobs has quit IRC00:57
*** thorst has joined #openstack-powervm01:37
*** thorst has quit IRC01:41
*** thorst has joined #openstack-powervm01:41
*** thorst has quit IRC01:42
*** thorst has joined #openstack-powervm02:26
*** thorst has quit IRC02:27
*** thorst has joined #openstack-powervm02:31
*** thorst has quit IRC02:31
*** thorst has joined #openstack-powervm02:32
*** thorst has quit IRC02:36
*** tjakobs has joined #openstack-powervm02:39
*** tjakobs has quit IRC02:52
*** jwcroppe has quit IRC03:53
*** tjakobs has joined #openstack-powervm04:52
*** tjakobs has quit IRC05:22
*** thorst has joined #openstack-powervm05:35
*** thorst has quit IRC05:40
*** k0da has joined #openstack-powervm08:43
*** thorst has joined #openstack-powervm09:36
*** thorst has quit IRC09:41
*** seroyer has joined #openstack-powervm11:32
*** thorst has joined #openstack-powervm11:55
*** smatzek has joined #openstack-powervm12:23
*** kylek3h_away has quit IRC12:27
*** apearson has quit IRC12:32
*** mdrabe has joined #openstack-powervm12:35
thorstefried: you in yet?12:52
*** kylek3h_away has joined #openstack-powervm12:59
*** kylek3h_away has quit IRC12:59
*** kylek3h_away has joined #openstack-powervm13:00
*** apearson has joined #openstack-powervm13:08
efriedthorst, yes13:10
openstackLaunchpad bug 1587763 in oslo.messaging "py34 gate failure due to amqp can't find module 'vine'" [Undecided,Fix released] - Assigned to ChangBo Guo(gcb) (glongwave)13:11
thorstwe're impacted by that.13:11
thorstsorry...this one13:12
openstackLaunchpad bug 1637938 in oslo.messaging "Test fail with import error and timeout exception" [Undecided,New]13:12
*** edmondsw has joined #openstack-powervm13:12
thorstefried: I think I have a bad workaround...13:18
thorstset kombu to <4.013:18
efriedthorst, isn't this going to break the world?13:21
thorstone would think.  I ran rechecks on your nova stuff and it was OK13:21
thorstbut oslo-messaging is broke13:21
thorstso I think its fairly pervasive.13:21
thorstyeah, my workaround 'works'.  I would like to push it to unwedge us (as a partial fix)13:21
efriedfine by me.13:22
thorstI guess I'll hop in openstack-infra and give a heads up13:22
adreznecthorst: yeah, did you see my comment on your change about that?13:22
adreznecI started debugging on the plane but ran out of time to actually test before landing13:22
thorstadreznec: which change set?  I changed one to Parital-Fix...13:22
adreznecMust have missed it13:22
thorstahh, no13:23
thorstindependent debug13:23
adreznecMy question is why your job is ignoring upper-constraints13:23
adreznecSince that's set to 3.0.3713:23
thorsthow do upper constraints get defined.13:23
thorstsorry...bad word13:23
thorsthow do upper constraints get used.13:24
thorstright...but how do they get used13:24
thorstthis is just tox13:24
thorstdoes tox not respect upper-constraints?13:25
adreznecI wonder if there's not enforcement there for just tox runs13:25
adreznecI know devstack has the USE_CONSTRAINTS setting13:25
*** kylek3h_away is now known as kylek3h13:25
thorstadreznec: interesting13:26
adreznecWe're not explicitly listing kombu as a dependency in nova-powervm, are we?13:26
thorstI see why we're not enforcing it13:26
thorstI'll fix13:26
thorstno, but that's not why...13:26
thorstI got it13:26
thorstyou'll see in the review...its easier that way13:26
thorstjust a sec13:26
*** seroyer has quit IRC13:27
*** tblakes has joined #openstack-powervm13:31
*** jwcroppe has joined #openstack-powervm13:32
openstackgerritDrew Thorstensen (thorst) proposed openstack/nova-powervm: Use upper constraints
thorstadreznec efried: ^^13:34
adreznecthorst: makes sense13:35
adreznecWe probably need that for the others as well?13:35
adreznecStill though...13:36
openstackgerritDrew Thorstensen (thorst) proposed openstack/networking-powervm: Use upper constraints
thorstadreznec: yeah...I'm doing that now...gah  :-)13:36
efriedthorst, what's 'plain'?13:36
efriedWhere did you crib this from?13:37
adreznecplain = flat text13:37
efriedworks for me.13:37
*** smatzek has quit IRC13:37
openstackgerritDrew Thorstensen (thorst) proposed openstack/ceilometer-powervm: Use upper constraints
thorstI'll propose it to oslo-messaging as well.13:38
efriedIs it appropriate for oslo-messaging to be connected to the openstack requirements project that way?13:39
efriedI guess your reviewers will let you know...13:39
thorstwell, it is an openstack project?13:39
thorstefried: I think this needs to go back to mitaka...maybe liberty13:41
adreznecIIRC upper-constraints is broken in liberty13:41
adreznecUnless that got fixed13:42
thorstahh, OK - mitaka then13:44
adreznecI guess lets make sure it actually fixes things before worrying about backporting :P13:45
thorstOK - all branches are proposed up through mitaka13:46
thorstand yeah, it fixes13:46
thorstalready verified13:46
thorstcheck your review board.13:46
*** tjakobs has joined #openstack-powervm13:47
*** seroyer has joined #openstack-powervm13:48
efriedthorst, isn't 'SUSPENDED' a terminal state?13:49
thorstI wasn't sure if you would hair pin turn on suspended13:49
thorstlike you do when its powered off13:49
efriedLike, you can suspend the instance, then walk away indefinitely, then come back and resume it later on?13:49
thorstyeah, but you can also suspend and them immediately resume13:49
thorstlike the whole power off13:49
thorstwell, I'm assuming you can13:50
efriedWith power, there's an actual operation called reboot that does that.  Is there a corollary with suspend/resum?13:50
thorstefried: it doesn't even matter if there is a reboot13:50
thorstthe user could issue that command quickly13:50
thorstthe two in succession quickly13:50
efriedI contend that we don't need to use the event delay in that scenario.13:51
openstackgerritMerged openstack/nova-powervm: Use upper constraints
thorstI contend that we do for the same reason we need it for reboot...although I admit that reboot has a better timing window13:51
openstackgerritMerged openstack/ceilometer-powervm: Use upper constraints
thorstgah, looks like mitaka is borked for some other reason13:52
openstackgerritDrew Thorstensen (thorst) proposed openstack/nova-powervm: WIP: Clean up of OVS port on source host post migration
openstackgerritDrew Thorstensen (thorst) proposed openstack/nova-powervm: Refactor events into own class
openstackgerritDrew Thorstensen (thorst) proposed openstack/nova-powervm: WIP: Add delay queue for events
openstackgerritMerged openstack/networking-powervm: Use upper constraints
efriedwhat's the use case for quick suspend/resume?13:53
thorstefried: before we go down the state debate...which we should get to13:53
thorstcan I shift to getting builds going again?13:54
thorstwe're not quite out of woods yet.13:54
thorstmitaka...looks like new rules got pushed back13:54
thorstneed to find what change we fixed that under..13:56
thorstactually, looks like we merged that change back to stable/mitaka, but these are strings that no longer exist in newton13:58
thorstso we need a specific fix for these in stable/mitaka13:58
thorsthow do you want to handle...ignore the rule...or fix them?13:58
thorstmy vote is add an ignore rule.13:59
*** smatzek has joined #openstack-powervm14:00
thorstefried: I'll wait for your resp before I go down the rabbit hole14:03
openstackgerritMerged openstack/nova-powervm: Refactor events into own class
efriedthorst, fix 'em.14:05
efriedI can take it if you like.14:05
thorstefried: if you have time, I wouldn't mind pawning off to you  :-)14:06
efriedk, on it.14:06
thorstwell, if you're taking that...should we go back to SUSPENDED state discussion or want to revisit later?14:06
efriedthorst, do you have the bug handy?  The one we used for same issue in the other branches?14:07
thorstefried: no..couldn't find it easily14:07
efriedk, I'll track it down.14:07
efriedWe can talk about SUSPENDED.14:07
thorstseroyer: can you hairpin turn between SUSPENDING, SUSPENDED, RESUMING, RUNNING?14:10
thorstall within maybe 15 seconds...could you flow through those states (generally speaking)14:11
*** dwayne_ has joined #openstack-powervm14:12
efriedthorst, seroyer - while it might be interesting to know if you *could* do that, I think the real question is *would* you?14:12
thorstI again defer to seroyer there...14:12
efriedWhat's the symptom if you get it wrong?  That the VM shows up in the wrong state?14:13
thorstwe don't support pause (suspend) in openstack today...but not sure about other symptoms14:13
thorstyeah, wrong state shows up, allowing improper actions to be invoked against it14:13
efriedPermanently, or until some periodic task refreshes it?14:13
thorstuntil next sync_power_state14:13
efriedwhich happens how frequently?14:13
thorstconfigurable....but I think 5-10 minute intervals14:13
thorst'eventual consistency' I've heard it called before.14:14
efriedPresumably you can't resume a VM unless it's SUSPENDED, right?14:14
thorsttrue...and if the workflow takes more than 15 seconds...which I suspect it does14:15
efriedSo - in the absence of a 'reboot' equivalent for suspend/resume, given that you're already delaying 15s on SUSPENDING...14:15
thorstahh, so leave SUSPENDING, but not suspended.14:15
thorstyeah, I'm OK with that14:16
thorstI feel like splitting hairs, but I'm cool with it14:16
efried...there would be no way for a consumer to issue the resume quick enough to hurt things.14:16
efriedYeah, leave SUSPENDING for sure - that's clearly a transitional state.14:16
efriedAll assuming there's no 'reboot' equivalent for suspend/resume.14:16
thorstefried: you know the irony...this was supposed to be a very quick function for me to add14:16
thorstand it turned into a not so fun one.14:17
thorstway more complex than I thought originally, but happy with its current form14:17
efriedWere you imitating libvirt code?14:17
thorstimitating...making better... (in my opinion)14:17
efriedCause this is certainly a nontrivial thing to do.14:17
efriedSo unless you were straight copy/pasting, not sure where your quick-to-implement expectation came from ;-)14:18
thorstwhat I did is more verbose...but I know I am going to need this extended for the Overlay Eventing we'll need to add soon14:18
thorstefried: yeah, I wasn't a huge fan of what we had the objects more14:18
efriedEspecially since Power's state model is totally different from libvirt's and OpenStack's.14:18
thorstI think also...we need to start thinking about a new version tag.14:18
thorstfor stable/newton14:19
thorstsomething like 3.0.114:19
thorstwe have some significant fixes back there...14:19
efriedI can't find the bug for the logging interpolation thing.  Wonder if we subsumed it in some other change.  Okay if I create a new one?14:21
thorstefried: yep14:21
efriedaha, found it.  163244114:23
seroyerthorst, efried: No, I can't think of any reason to suspend/resume like a reboot.  Doesn't really make sense.14:35
efriedthanks seroyer14:35
*** tjakobs has quit IRC14:38
openstackgerritRavi Kumar Kota proposed openstack/nova-powervm: Clean up of OVS port on source host post migration
efriedthorst, oh, this is ugly.  I need both upper constraints and pep8 fixes.  Is there a way to do this other than combining both change sets?15:14
*** tjakobs has joined #openstack-powervm15:15
thorstefried: nope15:23
*** tblakes has quit IRC15:26
*** apearson has quit IRC15:30
*** apearson has joined #openstack-powervm15:31
*** tblakes has joined #openstack-powervm15:54
efriedthorst, adreznec:  (not sure why the bot isn't putting these in IRC)16:06
efriedThat squashes all the fixes for nova-powervm in mitaka - let's see if it builds.  Then gotta do similar (but, alas, not the same) for networking and ceilometer.16:07
adreznecMaybe it broke16:07
adreznecNot sure16:07
efriedthorst, I think this is a 911:
*** apearson has quit IRC16:09
*** apearson has joined #openstack-powervm16:09
efriedgr, adreznec, got some insight on this?
efriedDo we need to cap our nova req or something?16:12
*** AlexeyAbashkin has quit IRC16:13
*** AlexeyAbashkin has joined #openstack-powervm16:15
*** apearson has quit IRC16:16
*** apearson has joined #openstack-powervm16:26
*** apearson has quit IRC16:28
*** apearson has joined #openstack-powervm16:28
*** tblakes_ has joined #openstack-powervm16:32
*** tblakes has quit IRC16:32
*** tblakes_ is now known as tblakes16:33
*** seroyer has quit IRC16:51
*** seroyer has joined #openstack-powervm16:54
*** k0da has quit IRC16:56
thorstefried: Yeah, that seems hot.17:03
thorstbackporting to stable/newton?17:04
adreznecefried: sorry, was in another meeting17:09
adreznecDid you get that figured out?17:09
*** thorst has quit IRC17:10
*** thorst has joined #openstack-powervm17:11
*** thorst_ has joined #openstack-powervm17:12
*** thorst has quit IRC17:15
*** thorst_ has quit IRC17:16
*** esberglu has joined #openstack-powervm17:21
*** thorst has joined #openstack-powervm17:21
*** thorst has quit IRC17:21
*** thorst has joined #openstack-powervm17:22
*** apearson has quit IRC17:24
*** thorst has quit IRC17:26
*** thorst has joined #openstack-powervm17:26
*** jwcroppe has quit IRC17:27
*** jwcroppe has joined #openstack-powervm17:28
*** jwcroppe_ has joined #openstack-powervm17:31
*** jwcroppe has quit IRC17:32
*** apearson has joined #openstack-powervm17:36
*** jwcroppe_ has quit IRC17:42
thorstefried: I hard reset...anything I need to chase?17:53
*** apearson has quit IRC18:08
*** tblakes has quit IRC18:24
*** tblakes has joined #openstack-powervm18:26
*** apearson has joined #openstack-powervm19:00
*** k0da has joined #openstack-powervm19:10
*** apearson has quit IRC19:17
efriedadreznec, thorst, no, I haven't figured out yet19:34
efriedthorst, I think the source of the logging one wasn't backported, was it?19:36
efried ?19:36
openstackgerritMerged openstack/nova-powervm: Remove self.instance from ConfigDrivePowerVM log
*** tblakes has quit IRC19:47
*** smatzek has quit IRC19:58
*** tblakes has joined #openstack-powervm20:18
thorstefried: nope20:32
*** esberglu has quit IRC20:43
*** jwcroppe has joined #openstack-powervm20:44
*** jwcroppe has quit IRC20:55
*** seroyer has quit IRC21:14
*** openstackgerrit has quit IRC21:18
*** openstackgerrit has joined #openstack-powervm21:19
*** thorst has quit IRC21:19
*** thorst has joined #openstack-powervm21:19
*** apearson has joined #openstack-powervm21:20
*** thorst has quit IRC21:23
*** kylek3h has quit IRC21:37
*** thorst has joined #openstack-powervm21:45
*** thorst has quit IRC21:49
*** tjakobs has quit IRC22:23
*** tblakes has quit IRC22:34
*** thorst has joined #openstack-powervm22:37
*** edmondsw has quit IRC22:39
*** apearson has quit IRC22:45
*** thorst has quit IRC23:03
*** thorst has joined #openstack-powervm23:04
*** thorst has quit IRC23:17
*** kylek3h has joined #openstack-powervm23:22
*** apearson has joined #openstack-powervm23:29
*** seroyer has joined #openstack-powervm23:36
*** k0da has quit IRC23:47

Generated by 2.14.0 by Marius Gedminas - find it at!