Tuesday, 2017-08-15

*** zerick has joined #openstack-powervm00:00
*** mdrabe has joined #openstack-powervm00:00
*** dwayne has joined #openstack-powervm00:00
*** esberglu has joined #openstack-powervm00:00
*** kylek3h has joined #openstack-powervm00:00
*** efried_zzz has joined #openstack-powervm00:00
*** adi_____ has joined #openstack-powervm00:00
*** AndyWojo_ has joined #openstack-powervm00:00
*** tonyb has joined #openstack-powervm00:00
*** BorD_ has joined #openstack-powervm00:00
*** toan has joined #openstack-powervm00:00
*** ChanServ has joined #openstack-powervm00:00
*** tepper.freenode.net sets mode: +o ChanServ00:00
*** svenkat has joined #openstack-powervm00:14
*** thorst has joined #openstack-powervm00:16
*** svenkat has quit IRC01:35
*** thorst has quit IRC01:59
*** esberglu has quit IRC04:10
*** esberglu has joined #openstack-powervm04:54
*** esberglu has quit IRC04:59
*** chhavi has joined #openstack-powervm05:41
*** chhavi has quit IRC05:54
*** chhavi has joined #openstack-powervm05:55
*** k0da has joined #openstack-powervm06:46
*** thorst has joined #openstack-powervm07:03
*** thorst has quit IRC07:07
*** chhavi has quit IRC07:17
*** adreznec has quit IRC07:52
*** adreznec has joined #openstack-powervm07:58
*** esberglu has joined #openstack-powervm08:33
*** esberglu has quit IRC08:37
*** chhavi has joined #openstack-powervm08:51
*** k0da has quit IRC08:53
*** thorst has joined #openstack-powervm09:04
*** thorst has quit IRC09:08
*** thorst has joined #openstack-powervm10:05
*** edmondsw has joined #openstack-powervm10:06
*** thorst has quit IRC10:09
*** edmondsw has quit IRC10:10
*** esberglu has joined #openstack-powervm10:21
*** chas has joined #openstack-powervm10:22
*** k0da has joined #openstack-powervm10:23
*** esberglu has quit IRC10:26
*** k0da has quit IRC10:57
*** k0da has joined #openstack-powervm11:03
*** k0da has quit IRC11:24
*** thorst has joined #openstack-powervm11:25
*** thorst has quit IRC11:29
*** svenkat has joined #openstack-powervm12:00
*** thorst has joined #openstack-powervm12:03
*** svenkat_ has joined #openstack-powervm12:03
*** svenkat has quit IRC12:04
*** svenkat_ is now known as svenkat12:04
*** esberglu has joined #openstack-powervm12:09
*** esberglu has quit IRC12:14
*** edmondsw has joined #openstack-powervm12:46
*** apearson has joined #openstack-powervm12:49
*** esberglu has joined #openstack-powervm12:54
esberglu#startmeeting powervm_driver_meeting13:00
openstackMeeting started Tue Aug 15 13:00:37 2017 UTC and is due to finish in 60 minutes.  The chair is esberglu. Information about MeetBot at http://wiki.debian.org/MeetBot.13:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.13:00
openstackThe meeting name has been set to 'powervm_driver_meeting'13:00
*** chas has quit IRC13:01
edmondswefried_zzz is not going to make it today13:02
esberglu#topic In Tree Driver13:02
esbergluNothing going on IT still I believe13:03
edmondswI wanted to bring up our governance here13:03
edmondswand use of releases for the *-powervm projects13:03
edmondswthorst, did you see my note there?13:03
thorstedmondsw: I did, and I started to respond but got distracted13:03
edmondswdo you want to discuss here or?13:03
thorstlet me pull it back up13:04
thorstI think a lot of the assumptions we had when we started aren't reality now that we've grown and are going in tree13:04
edmondswyup, that's what I'm hoping13:04
thorstwe have a "don't rock the boat" approach, but if dims is recommending that, it's obviously worth investigating13:04
edmondswesberglu mdrabe forwarded the note to you...13:04
edmondswsorry for leaving off13:05
thorstI think a pvm-stackers style group makes sense13:05
thorstassuming the community would be on board13:05
edmondswcool, because I was thinking the same13:05
edmondswand maybe even move ceilometer-pvm there instead of under telemetry13:05
edmondswso they're all together?13:05
thorstwe knew that if we came into the community and asked for something like that early on...it'd have been seen as some agenda we were trying to drive13:06
thorstwhere as our end goal is to just get powervm enablement in the respective projects13:06
thorstI think we've demonstrated that to the community now13:06
edmondsw#action edmondsw to investigate creation of pvm-stackers project under governance13:07
edmondswthat's all I had for IT13:07
esberglu#topic Out of Tree Driver13:08
edmondswiSCSI is a mess...13:09
edmondswI've dug into that code some more and have a much better understanding of what we've got ahead of us13:09
thorstedmondsw: I think seroyer has someone that was going to be able to dedicate some time as well to that effort13:10
edmondswwe've got a meeting later today out of which I hope we'll have direction on priority and can put together a timeline13:10
edmondswthorst awesome13:10
edmondswso probably a bigger update on this next week13:10
esbergluCool. Any other OOT discussion items?13:11
mdrabeNVRAM event performance stuff testing yielded some issues. With evacuate itself not the change. I'm wading through them.13:12
edmondswnothing else from my end13:12
mdrabeit from me13:12
esberglu#topic PCI Passthru13:13
edmondswefried_zzz has started wading through the code there and noting issues13:13
esbergluYep saw that note. Nothing to discuss for the time being?13:14
edmondswyeah, not without him13:14
esberglu#topic PowerVM CI13:14
esbergluGonna try to link up with hsien today to look into some rest logs to find out why the compute service is occasionally failing to come up13:15
edmondswwe've gotta figure that out13:16
esbergluOther than that I've currently trying to take everything defined in the jenkins jobs and move them to the external scripts13:16
esbergluWhich should do 2 things13:16
esberglu1) Allow changes to those tasks to take effect immediately13:16
esberglu2) Ease the transition to zuul v3 (no more jenkins!)13:16
edmondswboth good things13:17
esbergluI want to start that transition to zuulv3 asap after it goes live13:17
edmondswwhat is the outlook for that?13:17
esberglumdrabe: It's not worth pursuing that jenkins upgrade anymore. We can just stay at the current version until we get rid of it13:18
esbergluedmondsw: They are releasing it in conjunction with the PTG13:18
mdrabewhew good to hear13:18
esberglumdrabe: If you have any extra time I can always use extra eyes on tempest failures. Not sure how busy you are elsewhere13:19
esbergluThat's all for CI13:20
esberglu#topic Driver Testing13:20
esbergluDoesn't look like jay is on13:20
edmondswesberglu I think we learned since our last meeting that we were wrong about the progress we thought we were making on the timeouts issues, right?13:20
esbergluedmondsw: Well we made some progress on the REST timeouts, but not the server build timeouts13:21
esbergluI'm planning on being 100% tempest failure debug for the rest of the week13:21
edmondswsounds good13:21
esbergluunless something else breaks which seems to happen every time I start really digging in13:22
edmondswI think we can move on to the next topic13:22
esberglu#topic Open Discussion13:22
esbergluNothing further from me13:23
edmondswI should have raised this under IT, but we should start looking at adding network support there now that master=queens13:23
edmondswand config driver, and vSCSI... once we can free someone up to look at those things13:24
esbergluDo we have an order that we would like to do that in?13:24
edmondswso mdrabe, esberglu, if you have any cycles let me know13:24
edmondswesberglu good question... not at the moment13:24
*** k0da has joined #openstack-powervm13:25
esbergluedmondsw: I would like to start getting my foot in the door on that. I can work some time in for it13:25
edmondswI need to look at the changesets we already have up and see what they still need13:25
edmondswyep, that's what I wanted to hear13:25
edmondswI think network is probably the priority, but if config_drive is easier we might tackle it first13:25
edmondswthat's it13:26
esbergluedmondsw: We can touch base later to figure out a gameplan13:26
esbergluNice and quick today. Thanks for joining13:26
openstackMeeting ended Tue Aug 15 13:26:58 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)13:27
openstackMinutes:        http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-08-15-13.00.html13:27
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-08-15-13.00.txt13:27
openstackLog:            http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-08-15-13.00.log.html13:27
edmondswesberglu we should do better about using the action tag during meetings... help me with that next time?13:28
esbergluedmondsw: Will do13:28
esbergluThe question there is whether people will ever go back to the minutes to look at the action items13:29
esbergluAction items to add stuff to the todo lists perhaps13:29
*** k0da has quit IRC13:31
*** k0da has joined #openstack-powervm14:07
esbergluedmondsw: I'm looking at the server timeouts right now. In the test in question there is an initial spawn which works fine. Then a rebuild starts. Before the rebuild spawn completes a delete comes in.14:23
esbergluSo the vm never reaches active status here because something is deleting the vm before the test completes14:24
esbergluBut I can't see where this delete is coming from14:24
edmondswesberglu looking14:35
*** efried_zzz is now known as efried14:37
edmondswesberglu we probably need to look at how rebuild works14:39
edmondswefried welcome back. you probably want to read the mtg minutes above and then maybe help us on this timeout issue if you have cycles14:39
edmondswesberglu I've got another mtg in 20 min but will look til then14:39
esbergluedmondsw: I had to manually rebase 570314:41
efriededmondsw Yuh, catching up now.14:41
esbergluCan I get another +2 and then we can go live with those changes14:41
edmondswesberglu +214:43
efriedesberglu Consider the possibility that the delete is coming in as part of cleanup because the test timed out.14:43
edmondswesberglu I think the delete is part of the rebuild14:43
efriedRight, that delete should be happening before the spawn starts.14:46
efried...the second spawn, that is.14:46
esbergluefried: That destroy flow completes before the second spawn flow starts14:47
efriedesberglu Right.  And the delete you're concerned about is coming in when?  During the second spawn?14:47
esberglu2nd spawn:
esbergluDestroy flow starts:
esbergluInstance deleted while rebuilding:
efriedesberglu And the ultimate error is a test timeout?14:50
efriedSo yeah, that second delete would be the test cleanup.  It's a symptom, not a cause.14:51
*** tjakobs has joined #openstack-powervm14:51
edmondswefried I don't think so14:51
edmondswit seems to start too early14:52
edmondswbut maybe14:52
esbergluefried: Nor do I. The build timeout is 20 min. This whole event from start to finish is about 12 min. Unless there is some other timeout in play that I'm not aware of14:53
*** k0da has quit IRC14:56
efriedYeah, there's a per-build timeout, nah?14:56
efried196s or something?14:56
esbergluYeah the per build timeout is 120014:58
esbergluHowever there are some additional tempest timeouts that we don't set. A timeout to wait for an image to become available and a timeout for network operations to complete. Both default to 30015:03
esbergluI take that back15:04
esberglu"Other services that do not define build_timeout will inherit this value"  with this value being the compute build timeout15:04
*** k0da has joined #openstack-powervm15:47
*** k0da has quit IRC15:53
esbergluedmondsw: efried: I missed the updated log path variable name in process logs in my manual rebase. 571216:09
edmondswesberglu are you intentionally removing the -e from bash?16:14
esbergluedmondsw: Yeah. Lets say we fail early in the stack. Some of the files won't exist and the script will fail out. Meaning nothing will get posted to the logserver. This way if something fails we at least have a chance that something will be copied16:15
*** k0da has joined #openstack-powervm16:21
*** k0da has quit IRC16:27
esbergluefried: edmondsw: One final process_logs fix. 5713. I really should have tested this on staging but didn't feel like spending 4 hours redeploying16:35
edmondswesberglu +2 but now you'll need to also update the name of the var for that in process_logs accordingly16:37
esbergluedmondsw: Yep will do16:38
edmondswzuul_log_path instead of zuul_base_log_path16:38
*** k0da has joined #openstack-powervm16:40
*** esberglu is now known as esberglu_lunch16:50
*** edmondsw has quit IRC17:35
*** thorst has quit IRC17:35
*** thorst has joined #openstack-powervm17:37
*** thorst has quit IRC17:41
*** thorst has joined #openstack-powervm17:54
*** thorst has quit IRC17:58
*** thorst has joined #openstack-powervm17:59
*** svenkat has quit IRC18:02
*** edmondsw has joined #openstack-powervm18:05
*** edmondsw has quit IRC18:05
*** edmondsw has joined #openstack-powervm18:06
*** esberglu_lunch is now known as esberglu18:11
*** chhavi has quit IRC18:14
efriedesberglu I think config drive is easier - or at least the most reasonable next in sequence.18:17
efriedIn general, we're porting code from nova-powervm with an eye to making nova change sets as small as possible, factoring out anything that's unnecessary for the code as it appears in nova (even if we're going to have to add it back in later), breaking things into the smallest possible functional pieces.18:18
esbergluefried: I thought there was a WIP patch for that but I'm not seeing it18:19
esbergluNvm it was abandoned found it18:19
efriedConfig drive already had one started (https://review.openstack.org/409404) but it's waaaay out of date.18:19
efriedyeah, sdague's bot abandons stuff that's old & untouched.18:19
efriedesberglu Hm, maybe it's not so out of date.  It could probably be resurrected very easily.18:20
*** jwcroppe has joined #openstack-powervm18:25
esbergluefried: You seeing log coloring here?18:30
*** dwayne has quit IRC18:31
efriedHow'd you do it?18:31
edmondswefried ++ on everything you were saying about config_drive18:31
edmondswI have a todo to go find the abandoned commits and resurrect them if nobody beats me to it18:32
esbergluI did absolutely nothing18:32
esbergluThought you might have18:32
esbergluThe filters are working too18:32
efriededmondsw I have the list of abandoned commits we care about.18:33
efriededmondsw But no point resurrecting 'em until we're gonna do something with 'em.18:33
edmondswefried ++18:34
efriedActually edmondsw I can drop 'em in the etherpad18:34
edmondswefried that would be excellent18:34
efriedMost of 'em are there, actually, under "Rebase and/or update and/or repropose (because currently authored by wangqwsh, who's no longer with us) existing in-tree change sets."18:35
*** apearson has quit IRC19:00
*** jwcroppe has quit IRC19:00
*** k0da has quit IRC19:03
*** tjakobs has quit IRC19:03
*** k0da has joined #openstack-powervm19:03
*** tjakobs has joined #openstack-powervm19:03
*** jwcroppe has joined #openstack-powervm19:29
*** apearson has joined #openstack-powervm19:30
*** jwcroppe has quit IRC20:06
*** jwcroppe has joined #openstack-powervm20:13
*** jwcroppe has quit IRC20:20
*** jwcroppe has joined #openstack-powervm20:24
*** apearson has quit IRC21:09
*** esberglu has quit IRC21:31
*** esberglu has joined #openstack-powervm21:32
*** esberglu has quit IRC21:36
*** esberglu has joined #openstack-powervm21:45
*** k0da has quit IRC21:45
*** esberglu has quit IRC21:45
*** esberglu has joined #openstack-powervm21:45
*** thorst has quit IRC22:12
*** edmondsw has quit IRC22:25
*** svenkat has joined #openstack-powervm22:32
*** thorst has joined #openstack-powervm23:13
*** thorst has quit IRC23:17
*** thorst has joined #openstack-powervm23:43
*** tjakobs has quit IRC23:47
*** thorst has quit IRC23:48
*** jwcroppe has quit IRC23:49
*** edmondsw has joined #openstack-powervm23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!