Friday, 2018-05-11

*** chhagarw has joined #openstack-powervm04:13
*** chhavi__ has joined #openstack-powervm04:42
*** chhagarw has quit IRC04:46
openstackgerritChhavi Agarwal proposed openstack/nova-powervm master: iSCSI Live Migration Support
*** chhavi__ has quit IRC08:12
*** openstack has joined #openstack-powervm09:27
*** ChanServ sets mode: +o openstack09:27
*** openstackgerrit has joined #openstack-powervm10:34
openstackgerritStephen Finucane proposed openstack/nova-powervm stable/pike: Fix sphinx-docs job for stable branch
openstackgerritStephen Finucane proposed openstack/nova-powervm stable/ocata: Fix sphinx-docs job for stable branch
*** edmondsw has joined #openstack-powervm12:08
*** efried is now known as fried_rice12:35
*** openstackstatus has joined #openstack-powervm13:12
*** ChanServ sets mode: +v openstackstatus13:12
-openstackstatus- NOTICE: Due to a Zuul outage, patches uploaded to Gerrit between 09:00UTC and 12:50UTC, were not properly added to Zuul. Please recheck any patches during this window and apologies for the inconvenience.13:15
*** esberglu has joined #openstack-powervm13:44
*** tjakobs has joined #openstack-powervm13:47
openstackgerritEric Berglund proposed openstack/nova-powervm stable/pike: Stringify instance UUID
esbergluedmondsw: fried_rice: Wanna push through
esbergluIt's failing PowerVM CI because of circular dependency on
esbergluBut just a tox change so we're fine14:26
edmondswesberglu do we need to cherry-pick 567616 back to ocata as well?14:31
edmondswI'm guessing that's why CI failed14:31
edmondsw(without looking)14:31
fried_riceesberglu: was gonna ask same14:32
edmondswor did we just merge a sphinx change that we shouldn't have... ?14:33
esbergluedmondsw: fried_rice: Huh that actually passes the tempest run, one sec14:33
esbergluedmondsw: fried_rice: No the sphinx change for pike is good14:33
edmondswesberglu it looks like the tests on all passed... so why the -1 ?14:34
esbergluedmondsw: Yes exactly, I'm looking now14:35
esbergluedmondsw: Must be a problem with the on ocata14:38
esbergluedmondsw: Also looks like the very end of the jenkins console log is being cut off, so I can't see why14:38
esbergluI'll have to do a manual run, might not get to it today though14:39
esbergluWell I guess it makes sense that the jenkins console log is cut off, since process logs gets run before post_stack_vm_cleaner14:41
openstackgerritMerged openstack/nova-powervm stable/pike: Fix sphinx-docs job for stable branch
openstackgerritMerged openstack/nova-powervm stable/pike: Stringify instance UUID
openstackgerritEric Fried proposed openstack/nova-powervm stable/queens: Fix boot volume connectivity type discovery
*** openstackgerrit has quit IRC14:49
*** openstackgerrit has joined #openstack-powervm14:50
openstackgerritEric Berglund proposed openstack/nova-powervm stable/pike: DNM: CI Check - Pike
esbergluedmondsw: Oh missed your other question, we do not need this for ocata, the oslo.vo changes were only pike and later14:52
edmondswesberglu ok good14:52
edmondswesberglu CI passed on 567418 before the rebase, looks good to me... take a quick look and +2/+A ?14:55
edmondswesberglu efried once that's merged, I will propose a new stable/queens release to include all these recent fixes14:56
edmondswwe should do the same for pike once we get things working and merged there14:57
edmondswfried_rice I keep calling you efried...15:00
edmondswfried_rice is an issue with pike?15:01
openstackLaunchpad bug 1769648 in nova-powervm "Can't create IBMi VM with RBD backend storage" [Medium,Fix released] - Assigned to Eric Fried (efried)15:01
fried_riceedmondsw: I didn't think we supported RBD in pike15:02
fried_ricebut let me see if that bad conditional exists there...15:03
edmondswfried_rice yep, looks like rbd was added in queens15:04
fried_riceedmondsw: But yeah, theoretically the problem (uninitialized variable) still exists in pike:
fried_riceedmondsw: The fix certainly shouldn't hurt there.  But not sure if the risk is justified.15:05
edmondswfried_rice how well do we understand the case where this bug would appear?15:07
fried_ricewhoops, hold on, I went after the wrong branch.15:07
fried_riceyeah, same:
fried_riceedmondsw: Unfortunately, we friggin don't.15:08
fried_riceedmondsw: It depends on the bdms that come at us, and NOBODOY but NOBODY seems to have any freaking clue what those can actually look like irl.15:08
edmondswmdrabe any thoughts here?15:09
edmondswtalking about whether to backport fix for to pike15:09
openstackLaunchpad bug 1769648 in nova-powervm "Can't create IBMi VM with RBD backend storage" [Medium,Fix released] - Assigned to Eric Fried (efried)15:09
edmondswthe fix wasn't specific to rbd (which wasn't in pike) as far as we know15:09
fried_riceAs I say, the fix at *shouldn't* be able to hurt anything, as it's backward compatible afaict.15:09
mdrabeWhat the BDMs look like?15:10
fried_ricebut then again, maybe we wait to fix it until it breaks somebody.15:10
edmondswwe should try to be more proactive than that15:10
edmondswthat's actually one of the requirements for follows:stable-policy (which we don't technically do yet...)15:10
esberglufried_rice: edmondsw: 6552 when you get a second15:10
edmondswmdrabe I think the basic question is can we determine what caused us to go down the code path where that bug occurred, so we can understand when/whether it's gonna happen on pike15:11
fried_riceesberglu: done15:12
esberglufried_rice: tx15:12
edmondswalready merged when I got to it :)15:13
edmondswI'm glad I'm not facing you guys in a wild west showdown... ;)15:14
mdrabeedmondsw: Looks like that bug was something that could've been caught it UT / review maybe no?15:16
mdrabeIt's a bug in pike no less15:16
mdrabeCan a fix be backported?15:17
edmondswmdrabe yep... question is whether it's worth the risk to backport it to pike or not15:17
edmondswwe already backported to queens15:17
mdrabeHmm what's the risk we're concerned about?15:18
edmondswbreaking something?15:18
mdrabeSeems kinda broken already15:19
edmondswI mean breaking something else as we fix this15:19
edmondswI don't think that's overly likely... so if we think someone is gonna hit this on pike I would vote to backport it15:20
edmondswbut if we think nobody's really going to ever hit this on pike, there's little point15:20
edmondswhence the question of whether we think that code path will be exercised in pike, and when?15:20
openstackgerritMerged openstack/nova-powervm stable/queens: Fix boot volume connectivity type discovery
edmondswI'm going to be tagging new pike and queens releases soon to pick up some other fixes. If we are going to need this for pike, I'd want to do it soon so I don't have to do that again15:21
fried_riceedmondsw: Looking at the IBMi stuff earlier, I noticed that, at least for IBMi, we expect that sucker to return 'npiv', not 'fibre_channel'.  So at least the docstring is... questionable.15:21
fried_riceedmondsw: That's kind of unrelated though.15:22
edmondswfried_rice lovely15:23
edmondswfried_rice the docstring said 'npiv' before your change... why did you change it to 'fibre_channel' ?15:24
fried_riceshit, I think because of the unit test.15:25
mdrabeedmondsw: Tough to say if the path could happen in pike, but it is there. I'd say at least boot_conn_type could be predefined at defaulted.15:25
fried_ricemdrabe: edmondsw: It breaks if the bdm['connection_info'][['data'] contains neither 'connection-type' nor 'driver_volume_type'.  Which apparently turned out to be the case for RBD (why??) but again, no idea when/why/how it would be the case otherwise (in pike).15:25
fried_riceThe fix essentially just makes us default to 'vscsi' when ^ happens, because that seemed to be the intent of the calling method.  But even that doesn't really feel right.15:27
fried_riceI don't know where the BDM comes from in the first place.  If it's in any part populated by data we get from REST, then I would be quick to posit that we're not receiving/transferring something from REST in the RBD case, which is why those fields wind up absent.15:28
mdrabeefried: Looks like the BDM is created in the API, but I think the majority of the connection_info comes from cinder15:42
mdrabeAlso I think connection-type is a pvc thing, and driver_volume_type is community15:43
mdrabeedmondsw: Does pure nova-powervm work with vscsi?15:54
edmondswmdrabe if connection-type is a PowerVC thing, it shouldn't be in nova-powervm code...15:54
mdrabeedmondsw: Only reason I say that is because I don't see it anywhere else15:57
openstackgerritEric Fried proposed openstack/nova-powervm stable/ocata: Fix sphinx-docs job for stable branch
openstackgerritEric Fried proposed openstack/nova-powervm stable/pike: Use "$*" instead of "$@" in
*** fried_rice is now known as fried_rolls16:31
openstackgerritMatthew Edmonds proposed openstack/nova-powervm stable/queens: Sanitize the config drive UUID
openstackgerritMerged openstack/nova-powervm master: Refactor _vol_drv_iter to use bdm.is_volume
openstackgerritMerged openstack/nova-powervm stable/pike: Use "$*" instead of "$@" in
*** openstackstatus has quit IRC17:00
*** openstack has joined #openstack-powervm17:03
*** ChanServ sets mode: +o openstack17:03
edmondswesberglu ocata CI just failed here similar to before... tests pass but -1 vote:
esbergluedmondsw: Yep I know I will look into it when I have a chance to do a manual run. I'm pretty sure is failing, but the output gets cut off17:32
esbergluAnd I haven't redeployed CI to pick up the change to fix that yet17:33
edmondswdo you always have to redeploy to make changes? Seems like we should be able to ssh in and make a change manually, no?17:33
esbergluedmondsw: Yeah that's what I'm going to do for this one, I use the term redeploy loosely17:42
edmondswesberglu just noticed has been sitting a while... plz look when you get a minute18:07
esbergluedmondsw: ack18:08
esbergluedmondsw: Ah okay I see what's happening on ocata. The pypowervm version used in ocata is too low to be compatible with post_stack_vm_cleaner.py18:12
esbergluWow this has been broken for a while then, we must not have done anything in ocata for a long time18:13
edmondswwhy does need a newer version?18:13
esbergluIt uses the newer power off code18:15
esberglufrom pypowervm.tasks import power_opts18:15
esberglupower_opts doesn't exist in
esbergluWe updated the script a while back to use the newer stuff, I missed that I didn't work on ocata18:17
esbergluedmondsw: I'm just gonna pull the latest pypowervm if on ocata before running it18:24
esbergluGotta hop offline for a few, will put up a patch when I'm back18:25
*** esberglu has quit IRC18:25
*** esberglu has joined #openstack-powervm18:26
*** esberglu has quit IRC18:31
*** esberglu has joined #openstack-powervm18:44
*** fried_rolls is now known as fried_rice19:03
esbergluedmondsw: fried_rice: 6553 for ocata run fix19:10
fried_riceesberglu: reviewed, needs fix19:12
esberglufried_rice: You care whether develop or 1.1.15?19:12
esbergluOcata won't be around much longer, probably doesn't matter19:13
esberglufried_rice: Done19:13
fried_riceesberglu: Is a pull necessary after checking out develop?19:13
esberglufried_rice: I don't think so, it should be new enough from the image template build. If we want the latest develop yes, but if we're missing the last commit or two it won't matter19:16
fried_riceI just didn't know how the repo was cloned to begin with.  If you say it's good, it's good.19:16
esbergluOkay, no pull then19:17
openstackgerritMerged openstack/nova-powervm stable/ocata: Fix sphinx-docs job for stable branch
openstackgerritMerged openstack/nova-powervm stable/queens: Sanitize the config drive UUID
openstackgerritMerged openstack/networking-powervm master: Uncap eventlet and sync requirements
edmondswesberglu using develop will probably be fine as long as we don't break something19:38
esbergluedmondsw: One more way to catch it if we do :)19:39
esbergluedmondsw: They changed the EOL process right?19:40
edmondswesberglu yeah, we live in an "extended maintenance" world now19:40
esbergluedmondsw: So when do we stop testing in CI then?19:41
edmondswgood question :)19:42
edmondswfried_rice do we need a pip10 stable fix for queens?19:45
edmondswwe've merged patches only for pike and ocata19:45
edmondswthe sphinx change19:45
fried_riceIt's possible they successfully blacklisted pip10 in queens.  Stephen is who I would ask, if I didn't want to go trying to dig myself.  And he may be gone for the evening.19:47
edmondswoh, we don't have in queens19:47
fried_ricewas gonna say, the fix wouldn't hurt us either way, but if the file doesn't exist, then n/a.19:48
fried_riceedmondsw: btw, I punt on the vnc thing - any ideas?19:48
edmondswfried_rice none... I haven't ever looked at that stuff19:49
edmondswwas hoping you would know19:49
edmondswor that thorst would answer19:49
fried_riceYeah, I think thorst was the one to get that stuff working originally.19:49
*** tjakobs has quit IRC21:30
*** edmondsw has quit IRC21:58
*** edmondsw has joined #openstack-powervm22:02
*** edmondsw has quit IRC22:06
*** fried_rice is now known as efried22:16
*** edmondsw has joined #openstack-powervm22:29
*** esberglu has quit IRC22:30

Generated by 2.15.3 by Marius Gedminas - find it at!