Wednesday, 2017-06-21

openstackgerritlan proposed openstack/nova-powervm master: Fix html_last_updated_fmt for Python3
*** esberglu has joined #openstack-powervm02:02
*** esberglu has quit IRC02:07
*** thorst has joined #openstack-powervm03:15
*** esberglu has joined #openstack-powervm03:51
*** thorst has quit IRC03:54
*** esberglu has quit IRC03:55
*** jwcroppe has joined #openstack-powervm04:01
*** chhavi has joined #openstack-powervm04:03
*** adreznec has quit IRC05:08
*** esberglu has joined #openstack-powervm05:39
*** esberglu has quit IRC05:44
*** kylek3h has quit IRC05:58
*** adreznec has joined #openstack-powervm06:00
*** k0da has joined #openstack-powervm06:21
*** kylek3h has joined #openstack-powervm06:37
*** edmondsw has joined #openstack-powervm07:03
*** edmondsw has quit IRC07:08
*** esberglu has joined #openstack-powervm07:27
*** k0da has quit IRC07:28
*** esberglu has quit IRC07:31
*** esberglu has joined #openstack-powervm07:37
*** esberglu has quit IRC07:42
*** esberglu has joined #openstack-powervm08:23
*** chhavi has quit IRC08:28
*** esberglu has quit IRC08:28
*** k0da has joined #openstack-powervm08:40
*** edmondsw has joined #openstack-powervm08:51
*** edmondsw has quit IRC08:56
*** chhavi has joined #openstack-powervm09:21
*** esberglu has joined #openstack-powervm10:08
*** esberglu has quit IRC10:13
*** edmondsw has joined #openstack-powervm10:39
*** edmondsw has quit IRC10:45
*** chhavi has quit IRC10:47
*** smatzek has joined #openstack-powervm11:11
*** smatzek has quit IRC11:47
*** smatzek has joined #openstack-powervm11:52
*** kylek3h has quit IRC12:07
*** edmondsw has joined #openstack-powervm12:21
*** k0da has quit IRC12:32
*** efried has joined #openstack-powervm12:34
*** efried has quit IRC12:39
*** kylek3h has joined #openstack-powervm12:40
*** dwayne has quit IRC12:49
*** jwcroppe has quit IRC13:03
*** jwcroppe has joined #openstack-powervm13:03
*** jwcroppe has quit IRC13:03
*** jwcroppe has joined #openstack-powervm13:04
*** jpasqualetto has joined #openstack-powervm13:06
*** jwcroppe has quit IRC13:06
*** thorst has joined #openstack-powervm13:16
*** k0da has joined #openstack-powervm13:18
*** esberglu has joined #openstack-powervm13:19
*** thorst has quit IRC13:22
*** esberglu has quit IRC13:24
*** jwcroppe has joined #openstack-powervm13:30
*** thorst has joined #openstack-powervm13:32
*** esberglu has joined #openstack-powervm13:35
*** smatzek has quit IRC13:42
esbergluedmondsw: 549113:59
*** smatzek has joined #openstack-powervm13:59
esbergluthorst: ^13:59
esbergluThinking it might take a little while for
esbergluto merge. 5491 adds it to the IT patch list14:00
*** mdrabe has joined #openstack-powervm14:06
edmondswesberglu yeah... shame we haven't already gotten 5468 merged. Will this complicate that?14:11
openstackgerritMerged openstack/nova-powervm master: Updated from global requirements
esbergluedmondsw: Yeah a little bit. But not much. We might have a couple failed runs while everything is getting converted14:13
esbergluI would rather just get IT CI going by merging 5491 than waiting for 5468. I still need to update and test 546814:14
edmondswyep, I +2'd14:16
*** dwayne has joined #openstack-powervm14:55
*** thorst has quit IRC15:05
*** efried has joined #openstack-powervm15:06
*** jwcroppe has quit IRC15:07
*** thorst has joined #openstack-powervm15:12
*** thorst has quit IRC15:16
esbergluedmondsw: efried: Reworked and tested the openstack patching logic. It works and is ready for reviews. I also added the neo-os-ci side of the change15:16
esbergluWe were overly complicating the deployment of this change15:16
esbergluWe can just merge the powervm-ci change and live update the jenkins-jobs on the management system at the same time15:17
esbergluNo need to split up the powervm-ci change15:17
esbergluThere might be a few runs that don't behave exactly right while this transition happens, but I'm okay with that15:18
*** chhavi has joined #openstack-powervm15:18
*** thorst has joined #openstack-powervm15:36
edmondswesberglu I believe 5468 will break the CI until/unless 5492 is merged, because neo-os-ci will still be sending -n and that will be an "Invalid option"15:37
esbergluedmondsw: Yeah I'm saying I can just apply that change to the management node though. The merge of 5492 is independent of me applying it to the management node15:38
esbergluSince I will just be live updating the jenkins jobs15:38
*** thorst has quit IRC15:40
esbergluedmondsw: Going live with the external patching16:00
*** mdrabe has quit IRC16:01
edmondswesberglu +116:02
*** mdrabe has joined #openstack-powervm16:07
efriedesberglu Did pick up edmondsw's recheck this morning?16:10
efriedWould've expected results by now.16:11
esbergluefried: I was just looking at that. One of the zuul merger processes died16:11
esbergluI restarted it, but you need another rechec16:11
efriedesberglu Well, the existing job was successful16:11
efriedSo I'd be happy just to leave it alone.16:11
esbergluefried: Yeah sure16:12
efriedjust didn't want to start asking for reviews and then have another result come in that wasn't as favorable.16:12
esbergluefried: It's no longer in the queue you should be good to go16:13
*** thorst has joined #openstack-powervm16:14
edmondswefried you going to ask mriedem to merge that?16:15
edmondswwell, review anyway16:15
efriededmondsw Already on it.16:16
efriedAnd eating crow, as predicted.16:16
efriedmriedem +2, expect quick merge from sdague once he sees it.16:17
*** thorst has quit IRC16:19
*** thorst has joined #openstack-powervm16:27
*** thorst has quit IRC16:27
*** thorst has joined #openstack-powervm16:34
*** jwcroppe has joined #openstack-powervm16:35
edmondswefried tx16:37
*** thorst has quit IRC16:39
*** jwcroppe has quit IRC16:44
*** thorst has joined #openstack-powervm16:47
*** thorst has quit IRC16:51
*** efried has quit IRC16:53
*** chhavi has quit IRC17:12
*** thorst has joined #openstack-powervm17:19
*** jwcroppe has joined #openstack-powervm17:21
*** thorst has quit IRC17:24
*** thorst has joined #openstack-powervm17:28
*** thorst has quit IRC17:34
*** thorst has joined #openstack-powervm17:38
*** thorst has quit IRC17:40
openstackgerritMatthew Edmonds proposed openstack/nova-powervm master: Use for powervm nodename
edmondswefried I added the test_driver test to
esbergluedmondsw: efried: 5495. I didn't hit this in my manual testing because I was running with prep_devstack with -f doesn't checkout any patches17:42
edmondswthat's what I was saying seems pretty pointless, but I guess for consistency...17:42
esbergluShould have been more diligent looking checking the script17:43
*** jwcroppe has quit IRC17:45
*** thorst has joined #openstack-powervm17:53
*** thorst has quit IRC17:57
*** thorst has joined #openstack-powervm18:00
edmondswesberglu efried fyi, using github last night didn't work any better than, no matter how many times I tried both... just had to wait and it worked today18:02
edmondswso it wasn't a goo vs. github issue18:02
*** thorst has quit IRC18:05
esbergluDid you try using a different protocol? https vs git?18:05
esbergluWe've had poor performance using git in the past, but I thought it was just a temporary network issue18:05
*** jwcroppe has joined #openstack-powervm18:07
edmondswesberglu used git for both18:24
edmondswI think it was just a temporary network issue, since it worked today18:25
edmondswtoday I'm hitting a different issue...18:25
edmondswtee: /etc/neutron/neutron.conf: No such file or directory18:25
esbergluNot sure why devstack can't create those18:26
edmondswah, yeah, I remember that now that you mention it18:26
esbergluI think it sets up files in /etc/nova so it should be able to work in /etc/neutron as well18:26
esbergluMight be a devstack bug, might be a config thing18:26
edmondswyeah, it definitely sets up files in /etc/nova18:27
edmondswand /etc/cinder, etc.18:27
esbergluedmondsw: And prior to getting rid of neutron legacy it had access there18:28
edmondswyeah, can't be an env issue... has to be either devstack bug or a problem with our local.conf18:28
esbergluedmondsw: Yep. I just haven't looked into it yet18:28
esbergluI'm guessing local.conf18:28
edmondswwhere does log its output to?18:28
esbergluJust because I would think others would hit it if it is a devstack thing18:29
esberglu /opt/stack/logs/
esbergluWell I guess that's just what I LOGFILE set to something in the CI local.confs18:30
esbergluBut that's where it will be by default if you are stacking out of /opt/stack/devstack/ I believe18:31
edmondswcould it be because we're enabling plugins before we enable services?18:32
edmondswesberglu ^18:33
edmondswor the order in which we enable plugins18:33
edmondswwell, not the order of plugin enablement... it's the last one that's failing18:37
edmondswesberglu I think I did just find one issue..18:47
edmondswis_neutron_legacy_enabled checks for any services starting with q-18:47
esbergluAre you worried about pvm-q-sea-agt?18:48
edmondswI think it may find things that contain, not necessarily start with... not sure on the syntax18:50
esbergluis_neutron_legacy_enabled was returning false when I was testing18:50
edmondswbut one way or the other, 2017-06-21 18:24:02.264 | ++lib/neutron:is_neutron_legacy_enabled:102  return 118:50
*** jwcroppe has quit IRC18:50
esbergluedmondsw: Which means it isn't enabled18:51
esbergluedmondsw: Yeah that's what we would expect18:52
edmondswok, my bad18:52
esbergluHere it is18:56
esbergluIt's doing sudo install18:56
esbergluBut no sudo cp, sudo mkdir, etc18:56
esbergluIt should chown $NEUTRON_CONF_DIR to $STACK_USER after creating it18:58
esbergluedmondsw: If you want to give that a try quick18:58
edmondswesberglu $NEUTRON_CONF_DIR isn't being created, so chown won't help18:59
edmondswnot on its own, anyway18:59
esbergluOh I misunderstood what that install command was doing19:00
*** smatzek_ has joined #openstack-powervm19:03
*** smatzek has quit IRC19:03
esbergluedmondsw: Yeah that install command is creating the directory I think19:05
esbergluinstall -d [OPTION]... DIRECTORY... (3rd format)19:05
esberglu"In the third format, create all components of the given DIRECTORY(ies)."19:05
esbergluWell I guess ensuring it's created and transferring ownership to STACK_USER19:06
edmondswesberglu I just lost connectivity to my neo... :( but I'm sure the directory didn't exist19:07
edmondswyeah, I'd tried ll -d /etc/neutron earlier and got back ls: cannot access '/etc/neutron': No such file or directory19:07
esbergluedmondsw: You're hitting this during the ml2 plugin setup?19:08
edmondswesberglu during enable_plugin neutron19:09
esbergluedmondsw: Did you ever figure out what was causing your network issues previously? I can' access a bunch of systems right now either19:11
edmondswesberglu no, I'm trying to figure out what I can and can't reach right now19:12
edmondswseems like pok is having network issues19:12
*** thorst has joined #openstack-powervm19:19
*** thorst has quit IRC19:27
*** jpasqualetto has quit IRC19:36
*** jpasqualetto has joined #openstack-powervm19:36
esbergluedmondsw: Perhaps we just need to set STACK_USER=jenkins19:46
edmondswesberglu why jenkins, not neo?19:50
esbergluOh I guess neo for you19:51
esberglujenkins within the CI19:51
edmondswesberglu oh, right19:51
edmondswesberglu I thought of that, but I don't see why that should be necessary for neutron and not for nova, etc.19:51
edmondswI thought STACK_USER was just if you were running as root19:51
*** efried has joined #openstack-powervm19:52
esbergluedmondsw: Just spitballing ideas19:52
esbergluYou could be right19:53
efriedThought jenkins on powervm-ci was instant19:53
efriedSomething going on there?19:53
esbergluWhat do you mean by that first sentence19:54
esbergluefried: Oh I see what you mean. Pok network issues19:55
*** jwcroppe has joined #openstack-powervm20:00
*** openstackgerrit has quit IRC20:03
*** openstackgerrit has joined #openstack-powervm20:05
openstackgerritMerged openstack/nova-powervm master: Use for powervm nodename
*** jwcroppe_ has joined #openstack-powervm20:17
*** jwcroppe has quit IRC20:21
*** jpasqualetto has quit IRC20:34
*** smatzek_ has quit IRC20:39
*** mdrabe has quit IRC20:41
*** mdrabe has joined #openstack-powervm20:42
edmondswesberglu adding STACK_USER=neo in my local.conf didn't help20:53
esbergluedmondsw: Okay. I'm spinning up some instances to try some stuff. I'll let you know if I find a fix20:55
efriedesberglu Do you have an easy way to run IT/OOT CI with pypowervm 5453?21:05
esbergluefried: I can either put in a manual run or just live patch it long enough for us to push a CI Check patch through21:07
esbergluThe latter is less work21:08
efriedesberglu Cool, just want to sanity/regression test, make sure it doesn't break the world.21:08
openstackgerritEric Berglund proposed openstack/nova-powervm master: DNM: ci check
esbergluefried: ^ running on the above21:10
efriedesberglu Thanks!21:11
*** smatzek_ has joined #openstack-powervm21:12
esbergluefried: It failed to apply21:12
esbergluI have to go manual21:12
efriedDoes it need a rebase?21:12
esbergluefried: I'm think it just doesn't apply to the the CI version of pyowervm (1.1.5)21:14
esberglu*I think21:14
esbergluBut I can do it on a manual run21:14
*** mdrabe has quit IRC21:15
esbergluefried: I won't have time to push a full manual run through before I head out. I can run it in the background later tonight unless you don't mind waiting until tomorrow21:16
efriedIt can wait.21:17
esbergluefried: Okay. On the list for tomorrow21:17
edmondswesberglu I think I figured out why /etc/neutron didn't exist yet when devstack tried to access it... trying to figure out the best way to fix it now21:17
esbergluEnlighten me21:17
edmondswconfigure_neutron hadn't been run yet21:17
edmondswthat is what will create /etc/neutron21:17
edmondswI'm doing a test run where we enable_services before enable_plugins21:18
edmondswI *think* that's the right fix21:18
esbergluI thought you were hitting this in configure_neturon_new? Which is called by configure_neutron. Maybe I misunderstood earlier21:19
edmondswesberglu well, enable_service before enable_plugin didn't fix it, anyway21:20
esbergluSo it's running the neutron plugin config before the neutron config?21:20
edmondswesberglu no, hitting this in neutron_service_plugin_class_add_new21:20
edmondswcalled by eutron_service_plugin_class_add21:20
edmondswcalled by configure_qos_service_plugin21:21
esbergluWhat do you mean by enable_services before enable_plugins?21:21
esbergluIn the local.conf? I don't think order matters there21:21
edmondswI mean that I tried to move the enable_service lines before the enable_plugin lines in local.conf21:21
esbergluedmondsw: Yeah I don't think order matters in local.conf21:22
edmondswesberglu, yeah, probably right21:22
esbergluI think the problem is that devstack is trying to configure the ml2 plugin before it configures neutron21:22
esberglu(I'm still waiting on my nodes to come up so I'm still just taking shots based on the code)21:22
esbergluedmondsw: I think this might go all the way back to itself calling things in the wrong order21:24
openstackgerritEric Fried proposed openstack/nova-powervm master: WIP: Adopt new pypowervm power_off APIs
*** mdrabe has joined #openstack-powervm21:25
edmondswesberglu yes, the problem is definitely that devstack is trying to configure plugins (I believe it's qos, not ml2) before it configures neutron21:26
edmondswesberglu I'm tracking back through to figure out why21:26
edmondswesberglu configure_neutron is on line 1177 of, whereas run_phase21:29
edmondswrun_phase stack install is on line 941, and will trigger the plugin installs21:29
esbergluI wonder what the impacts of that reorder are...21:31
esberglu(says anyone trying to do anything with devstack)21:32
edmondswesberglu I think I'm going to open a bug and see if someone from neutron will bite21:33
edmondswat least can put in what I've figured out21:33
esbergluedmondsw: We have a super easy workaround for the time being. +121:33
*** smatzek_ has quit IRC21:38
efriedesberglu If there's a way to do this too, 5453 + OOT21:38
efriedPasses live sniff test, so I'm fairly confident.21:38
efriedBut what I don't want to do is merge 5453, cut a new release, funnel it through g-r, then find out it's busted.21:39
esbergluefried: Yep I can make that happen21:39
efriedesberglu Cool man, thanks.21:40
efriedAgain, tomorrow is soon enough.21:40
efriedWe're still waiting on chhavi to sign off on the iscsi stuff before we can cut pypowervm anyway.21:40
esbergluefried: Might get a chance kick it off in the background tonight. Planning on wrapping up some stuff tonight since I'm out friday21:40
*** esberglu has quit IRC21:41
openstackLaunchpad bug 1699609 in devstack "stack fails unable to find /etc/neutron/neutron.conf" [Undecided,New]21:45
edmondsweserglu I'll throw up a patch adding that bug to the TODO comment21:46
*** efried has quit IRC21:48
*** mdrabe has quit IRC21:48
edmondswesberglu efried 549921:54
*** esberglu has joined #openstack-powervm21:55
*** esberglu has quit IRC22:00
*** dwayne has quit IRC22:01
*** kylek3h has quit IRC22:08
*** jwcroppe has joined #openstack-powervm22:15
*** jwcroppe_ has quit IRC22:19
*** jwcroppe has quit IRC22:20
*** k0da has quit IRC22:23

Generated by 2.15.3 by Marius Gedminas - find it at!