Thursday, 2017-05-25

*** chas has joined #openstack-powervm00:24
*** chas has quit IRC00:29
*** jay1_ has joined #openstack-powervm00:35
*** smatzek has joined #openstack-powervm01:10
*** edmondsw has joined #openstack-powervm01:20
*** edmondsw has quit IRC01:25
*** thorst has joined #openstack-powervm01:46
*** thorst has quit IRC02:02
*** openstackgerrit has joined #openstack-powervm02:11
openstackgerritOpenStack Proposal Bot proposed openstack/networking-powervm master: Updated from global requirements
*** smatzek has quit IRC02:11
*** thorst has joined #openstack-powervm02:14
*** chas has joined #openstack-powervm02:24
*** chas has quit IRC02:30
*** thorst has quit IRC02:37
*** jay1_ has quit IRC02:42
*** jay1_ has joined #openstack-powervm02:55
*** edmondsw has joined #openstack-powervm03:08
*** edmondsw has quit IRC03:13
*** thorst has joined #openstack-powervm04:16
*** thorst has quit IRC04:18
*** chhavi has joined #openstack-powervm04:24
*** chas has joined #openstack-powervm04:25
*** thorst has joined #openstack-powervm04:26
*** chas has quit IRC04:30
*** thorst has quit IRC04:38
*** tjakobs has joined #openstack-powervm04:46
*** edmondsw has joined #openstack-powervm04:56
*** tjakobs has quit IRC04:56
*** edmondsw has quit IRC05:01
*** jay1_ has quit IRC05:21
*** jay1_ has joined #openstack-powervm05:33
*** thorst has joined #openstack-powervm05:38
*** thorst has quit IRC05:42
*** arunman has joined #openstack-powervm05:49
*** jay1_ has quit IRC05:56
*** thorst has joined #openstack-powervm06:09
*** jay1_ has joined #openstack-powervm06:19
*** chas has joined #openstack-powervm06:26
*** thorst has quit IRC06:27
*** chas has quit IRC06:30
*** edmondsw has joined #openstack-powervm06:45
*** edmondsw has quit IRC06:49
*** thorst has joined #openstack-powervm07:24
*** thorst has quit IRC07:29
*** thorst has joined #openstack-powervm08:25
*** chas has joined #openstack-powervm08:27
*** thorst has quit IRC08:29
*** chas has quit IRC08:31
*** k0da has joined #openstack-powervm08:35
*** thorst has joined #openstack-powervm09:26
*** thorst has quit IRC09:30
*** edmondsw has joined #openstack-powervm10:21
*** edmondsw has quit IRC10:25
*** thorst has joined #openstack-powervm10:27
*** chas has joined #openstack-powervm10:28
*** thorst has quit IRC10:31
*** chas has quit IRC10:33
*** smatzek has joined #openstack-powervm11:15
*** arunman has quit IRC11:43
*** thorst has joined #openstack-powervm11:58
*** edmondsw has joined #openstack-powervm11:58
*** kylek3h has quit IRC12:00
*** kylek3h has joined #openstack-powervm12:15
*** chas has joined #openstack-powervm12:29
*** chas has quit IRC12:33
*** kylek3h has quit IRC12:41
*** chas has joined #openstack-powervm12:45
*** efried has joined #openstack-powervm12:46
*** arunman has joined #openstack-powervm12:49
*** chas has quit IRC12:49
*** apearson has joined #openstack-powervm13:03
*** mdrabe has joined #openstack-powervm13:16
*** esberglu has joined #openstack-powervm13:32
*** jpasqualetto has joined #openstack-powervm13:34
*** jwcroppe has quit IRC13:38
esbergluefried: thorst: edmondsw: The console tests don't work within the CI environment since pypowervm is non-local13:41
edmondswesberglu ah, yes13:42
efriedesberglu thorst Is there gonna be any way around that?13:42
thorstwe could make a repeater of the repeater13:42
thorstbut...that'd have...issues13:42
thorstI think we issue a mkvterm command for instance.13:43
thorstI think maybe step 1 is to create a mkvterm facade...13:44
thorstand then from that figure out how to tie that facade into the backing vnc13:44
thorstmaybe we have a new 'pvm-ci' package that puts down that facade...13:45
efriedAnd to be clear, all of this would be work purely to make these couple of tempest tests run properly.13:45
thorstheh, I think our team even made those tempest tests13:46
esbergluI guess I don't have enough background on this, but it seems like more work than it's worth for the time being?13:50
thorstI'm pretty sure mriedem said we needed to do it?13:51
thorstI'm not so sold its super tough TBH13:51
thorstbut I'd need to brainstorm a bit more (otp atm unfortunately)13:51
*** jwcroppe has joined #openstack-powervm13:53
*** apearson has quit IRC14:03
*** tjakobs has joined #openstack-powervm14:41
*** chas has joined #openstack-powervm14:45
*** thorst_ has joined #openstack-powervm14:49
*** chas has quit IRC14:50
*** thorst has quit IRC14:51
*** mdrabe has quit IRC14:54
*** mdrabe has joined #openstack-powervm15:01
efriedthorst_ edmondsw esberglu (adreznec mdrabe) --
esbergluefried: Nice. I might piggyback and add CI section there as well15:08
edmondswefried excellent, thanks15:08
esbergluUnless you think a separate one would be better15:08
efriedesberglu No, here.15:08
esbergluK good. I like it being in one place15:08
efriedesberglu I already included the VNC tempest stuff (ll 6-11)15:08
edmondswefried do we want this all visible publicly?15:09
efriededmondsw I don't see why not.15:09
edmondswgood... I'll look over it15:09
efriededmondsw I'm in favor of working openly as much as possible.  If you see something specifically questionable, let me know.15:09
thorst_+2 to everything out there15:10
efriedthorst_ esberglu Okay if I respond to mriedem's email and let him know we've got some challenges to overcome wrt the VNC tests?15:11
mdrabeefried: Anything more of a priority than others? Or should I just pick something?15:11
efriedmdrabe Other than the stuff at the very top, nothing else really has implied priorities.15:12
efriedmdrabe Some items may be blocked by others.15:12
efriedE.g. we can't do SSP shared resource providers until the SSP change set is in and jaypipes' infra is all in place.15:13
mdrabegotcha, I'll digest it15:13
efriedmdrabe The lowest-hanging fruit is the "teeny changes" stuff starting on L2415:14
efriedOh, thorst_ can I give you L29?15:14
efriedCause I don't know how to respond to that, but you probably do.15:14
thorst_r.o.o down?15:17
esbergluFine for me15:18
efriedmdrabe esberglu Click the people icon in the top right of the etherpad and enter your nick.15:20
edmondswefried I think I'm inclined to prioritize documenting our driver in the support matrix for Pike... thoughts?15:21
efriededmondsw Agree.15:21
edmondswefried should we move that up to the top section?15:22
efriededmondsw Sure15:22
efriedThe stuff at the top doesn't have an order anymore (it did last night) so Ima take off the numbering.15:23
efriedesberglu Can I recheck yet?15:36
esbergluefried: I did just a little bit ago15:36
efriedoh, okay, thanks.15:36
esbergluefried: 40 minutes into the run right now15:37
*** k0da has quit IRC15:38
*** thorst_ has quit IRC15:39
*** thorst has joined #openstack-powervm16:00
*** thorst has quit IRC16:02
*** thorst has joined #openstack-powervm16:04
esbergluefried: thorst: edmondsw: What's the plan for OVS support for CI? Are we trying to get both ovs and sea runs working simultaneously?16:22
edmondswesbergu I thought that's what we'd said... work them both and see which one we get working first16:24
esbergluYeah in-tree16:25
esbergluBut what about OOT?16:25
efriedI thought the first step was to get OVS CI working OOT at all.16:25
efriedThen we can figure out if/how we're gonna do either/both.16:25
esbergluefried: Yeah I was just thinking about the necessary playbook changes long term. For the ovs ci poc it doesn't matter16:28
efriedesberglu It's possible to use multiple net drivers at the same time, right?16:30
efriedJust not sure how to tell tempest that.16:30
esbergluefried: We can loop back on this discussion later. Don't even a working ovs local.conf yet16:32
efriednod.  We'll need thorst involved anyway.16:33
esbergluefried: edmondsw: One of the vnc tests is failing the setup class with a port binding issue16:36
esbergluI put up a patch reverting the inclusion of those tests since they aren't working anyways16:37
efriedesberglu Consistently?16:37
efriedI thought we saw that issue intermittently anyawy16:37
esbergluefried: Yeah. There is still some wackiness going on with the in-tree CI with regards to networking16:38
esberglunetworks interfering with other tests and things like that16:38
efriedesberglu +2.  How were you testing those anyway?  Did you and thorst suss out the remotable business?16:38
esbergluI just added them to the whitelist manually and tested. Saw 0 failures and got ahead of myself and merged the patch before making sure they actually got ran16:40
esbergluHaven't done anything as far as getting them to work yet16:40
efriedthorst edmondsw esberglu rebased, ready for internal +1s before I pester sdague/mriedem.16:46
*** chas has joined #openstack-powervm16:46
efriedesberglu FYI, if it's just stupid Citrix XenServer CI that fails, you can say `xenserver: recheck` to save rechecking the world.16:50
efriedThe rest of the failures are non-voting (even the microsoft one at the bottom)16:50
esbergluefried: Yeah every CI has a specific recheck string (or at least are supposed to)16:51
efriedright; just saw you did a full recheck on
*** chas has quit IRC16:52
openstackgerritMerged openstack/nova-powervm master: Consume HttpNotFound
openstackgerritEric Larese proposed openstack/nova-powervm master: Clean up orphan tap devices on migrate
*** arunman has quit IRC17:30
efriededmondsw Were you going to take the support matrix?17:39
efriedGetting ready to pick off another work item, wanted to make sure we didn't collide.17:39
edmondswefried I will if it's still there when I free up, but that probably wouldn't be today17:40
efriededmondsw Okay.  I think mebbe I'll do get_inventory() instead.17:40
edmondswefried sounds good17:41
*** apearson has joined #openstack-powervm17:48
*** jay1_ has quit IRC18:22
efriedesberglu Things are getting "read-only file system"...18:35
efried...during devstack.18:35
efriedboth IT and OOT18:36
efriedAnd saw it somewhere else earlier too.18:36
esbergluefried: Yeah other runs are getting flooded with failures as well18:38
*** apearson has quit IRC18:38
efriedesberglu Never a dull moment in CIland.18:41
*** chas has joined #openstack-powervm18:47
*** chas has quit IRC18:52
esbergluefried: Looks like that happened to a set of runs all at the same time, hasn't happened since. Not really much info to go off of with only the console log18:59
efriedesberglu Were all those runs perchance using the same neo?18:59
efriedor the same vios?19:00
efriedor the same SAN disks?19:00
esbergluI don't think the neo gets logged in the console log either. Let me see if I can piece it together from the jenkins jobs19:01
*** k0da has joined #openstack-powervm19:03
efriedesberglu If it was transient, prolly not worth spending a lot of time nailing it down.19:04
esbergluefried: I think it's actually just bad patches19:04
efriedesberglu Bad patches on what?19:04
efriedNot the changes being tested19:05
*** apearson has joined #openstack-powervm19:05
esbergluThose two related changes both hit it multiple times19:05
esbergluAnd they appear to be blowing everything up19:06
*** chhavi has quit IRC19:06
efriedAnd that code is PRISTINE19:06
openstackgerritEric Larese proposed openstack/nova-powervm master: Clean up orphan tap devices on migrate
efriedthorst You have any particular objection to renaming Connect/Disconnect to Attach/Detach throughout?19:41
efriedSeems like more work than it's worth to me, but...19:41
efriedMaybe in a followon, assigned to edmondsw  ;-)19:42
edmondswefried lol19:42
edmondswtoo nit picky... not nit picky enough... ;)19:43
efriededmondsw can also tackle the "one space after a period" work item.19:43
esbergluefried: Looks like this is breaking us. Currently failed the jenkins check but it's W+1. Got time to help look?19:43
edmondswefried I knew you'd love that one...19:43
efriedI learned to touch-type in the 9th grade, and it was two spaces after a period.19:43
edmondswefried yeah, same here... it's an adjustment19:44
efriedI've been seeing this comment a lot lately, so I had to go look up the rationale.19:44
edmondswefried I always get dinged on that, so sharing the love19:44
efriedDistilled down, it's really "two spaces after a period if the font is fixed-width"19:44
efried...which it is, in code.19:44
efriedSo I feel justified NOT changing my 25-year-ingrained habit.19:45
esbergluefried: Tons of19:45
esberglu"NeutronAdminCredentialConfigurationInvalid: Networking client is experiencing an unauthorized exception."19:45
esbergluin n-cpu, looking in networking logs now19:45
efriedBut people seem to get pretty excited about it, and it's (barely) not worth the fight.19:45
efriedesberglu Ugh, I remember we saw that intermittently earlier.  Now we're seeing it all the time?19:46
esbergluOn this commit chain yes19:46
esbergluActually there are a ton of other errors in n-cpu too19:48
esberglu"ValidatorException: Insufficient available processing units on host for virtual machine 'pvm5-tempest-Ser-03f27746' (0.10 requested, 0.00 available)"19:48
esbergluI'm guessing that's whats actually hitting from this patch19:48
esbergluefried: If you look at _remove_deleted_instances_allocations in in that patch19:53
esbergluIt should be logging a warning when deleting stale allocations19:53
esbergluI'm not seeing that warning anywhere in the logs19:54
esbergluSo I think the allocations aren't being removed so it thinks there aren't any available hosts19:54
efriedesberglu I don't understand how this could have changed it, though.19:58
esbergluYeah I don't either. But we should be seeing that warning in the logs correct?19:59
efriedesberglu Guess I would have to see the logs for a run with the code as it stood prior to this change set...20:00
esbergluI just looked at one I don't see it there either...20:01
efriedesberglu Is the other one seeing the same error?20:01
esbergluefried: Nope20:02
thorstefried: no concern with the rename20:04
*** smatzek has quit IRC20:12
*** edmondsw has quit IRC20:37
*** edmondsw has joined #openstack-powervm20:44
*** chas has joined #openstack-powervm20:48
*** chas has quit IRC20:52
*** smatzek has joined #openstack-powervm21:00
*** jwcroppe has quit IRC21:07
*** jwcroppe has joined #openstack-powervm21:08
*** chas has joined #openstack-powervm21:09
*** smatzek has quit IRC21:11
*** jwcroppe has quit IRC21:12
*** chas has quit IRC21:14
*** jpasqualetto has quit IRC21:16
*** jwcroppe has joined #openstack-powervm21:20
*** thorst has quit IRC21:22
*** jwcroppe has quit IRC21:25
*** chas has joined #openstack-powervm21:30
*** apearson has quit IRC21:32
*** chas has quit IRC21:34
*** chas has joined #openstack-powervm21:40
*** jwcroppe has joined #openstack-powervm21:43
*** chas has quit IRC21:54
*** chas has joined #openstack-powervm21:54
*** chas has quit IRC21:59
*** jwcroppe has quit IRC21:59
*** jwcroppe has joined #openstack-powervm21:59
*** mdrabe has quit IRC22:03
efriedesberglu two things here:22:04
efriedOne, not sure our patching strategy is working for non-master branches.22:04
esbergluYeah definitely isn't. We don't do any checks for branches22:05
efriedTwo, we can remove the master patch from the patch list.22:05
efriedcause it merged.22:05
esbergluefried: Just be aware that if we remove that from the patch list right now22:06
esbergluAny patches that aren't rebased such that the --strict patch is included22:06
esbergluWill fail22:06
efriedI don't think so.22:06
efriedBecause it merged.22:07
efrieduhm, maybe you're right.22:07
esbergluYeah but CI checks out the change22:07
esbergluSo it depends where the parent is22:07
efriedYeah, and it doesn't rebase it on master22:07
esbergluefried: I actually removed this patch earlier today and put it back on for that reason22:08
esbergluWas gonna remove it later on once there are less patches that aren't up to date22:09
efriedokay, I dig it.22:09
esbergluWe have green runs going though again finally. Not a good success rate but it's something22:10
esbergluNext week is gonna be tempest debug week22:10
*** chas has joined #openstack-powervm22:15
*** chas has quit IRC22:20
*** thorst has joined #openstack-powervm22:20
*** esberglu has quit IRC22:22
*** thorst has quit IRC22:23
*** esberglu has joined #openstack-powervm22:23
*** tjakobs has quit IRC22:23
*** esberglu has quit IRC22:28
*** edmondsw has quit IRC22:34
*** edmondsw has joined #openstack-powervm22:34
*** edmondsw has quit IRC22:39
*** thorst has joined #openstack-powervm22:53
*** chas has joined #openstack-powervm22:56
*** chas has quit IRC23:01
*** thorst has quit IRC23:08
*** k0da has quit IRC23:17
*** apearson has joined #openstack-powervm23:31
*** edmondsw has joined #openstack-powervm23:35
*** chas has joined #openstack-powervm23:37
*** edmondsw has quit IRC23:40
*** chas has quit IRC23:42

Generated by 2.14.0 by Marius Gedminas - find it at!