Tuesday, 2018-10-30

*** gyee has quit IRC00:04
*** artom has joined #openstack-meeting00:08
*** M0nk3Ee has joined #openstack-meeting00:08
*** mriedem_away has quit IRC00:15
*** Liang__ has joined #openstack-meeting00:19
*** Liang__ is now known as LiangFang00:20
*** longkb has joined #openstack-meeting00:46
*** hongbin has joined #openstack-meeting01:23
*** bcafarel has quit IRC01:40
*** cloudrancher has quit IRC01:41
*** cloudrancher has joined #openstack-meeting01:42
*** diablo_rojo has quit IRC01:56
*** bcafarel has joined #openstack-meeting02:00
*** mhen has quit IRC02:14
*** fanzhang has left #openstack-meeting02:18
*** mhen has joined #openstack-meeting02:19
*** armax has quit IRC02:34
*** yamahata has quit IRC02:48
*** iyamahat has quit IRC02:49
*** psachin has joined #openstack-meeting03:10
*** diablo_rojo has joined #openstack-meeting03:32
*** fanzhang has joined #openstack-meeting03:47
*** tpatil has joined #openstack-meeting03:50
*** sagara has joined #openstack-meeting03:59
samPHi all for masakari04:00
samP#startmeeting masakari04:00
openstackMeeting started Tue Oct 30 04:00:10 2018 UTC and is due to finish in 60 minutes.  The chair is samP. Information about MeetBot at http://wiki.debian.org/MeetBot.04:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.04:00
*** openstack changes topic to " (Meeting topic: masakari)"04:00
openstackThe meeting name has been set to 'masakari'04:00
samPHi all for masakari04:00
sagarasamP: Hi04:00
samPsagara: Hi04:01
tpatilHi04:01
samPtpatil: Hi04:01
samPLet's start04:01
samP#topic Critical Bugs/patches04:02
*** openstack changes topic to "Critical Bugs/patches (Meeting topic: masakari)"04:02
samPAny bug or patches to discuss?04:02
samPNeed to fix the status for fixed bugs and clean up all the bug reports.04:04
samPI will do it this week.04:04
sagarasamP: thanks04:04
samPIf any critical bugs or patches please bring them up any time04:04
*** ShilpaSD has joined #openstack-meeting04:04
samPLet's go to next topic04:04
samP#topic Stein Work Items04:05
*** openstack changes topic to "Stein Work Items (Meeting topic: masakari)"04:05
samPBTW I update the meeting wiki page04:05
samP#link https://wiki.openstack.org/wiki/Meetings/Masakari04:05
tpatilNotification implementation: We will push patches soon04:07
samPI list up current work items under "Discussion points"04:07
samPPlease report if you have any update on items04:07
samPtpatil: Thanks.04:07
tpatilSpecs : https://review.openstack.org/#/c/473057/04:08
tpatilPlease review above specs proposed by Honjo san long time ago04:08
*** tetsuro has joined #openstack-meeting04:10
tpatilSecondly, we are also working on add event info for each kind of masakari notification (HOST/INSTANCE/PROCESS)04:10
tpatilThis point I will add in the wiki page04:10
samPtpatil: OK, I will. This spec propose versioned notification format. Have you take a look about those formats?04:10
samPtpatil: goit, thanks04:11
*** bswartz has quit IRC04:11
tpatilYes, this format is common IMO for all services. Important point is payload in the notification format.04:12
samPtpatil: OK then, I will read this again and post my comments.04:14
*** janki has joined #openstack-meeting04:14
tpatilsamP: Thanks04:15
tpatilContinue ansible support for Masakari04:15
tpatilNot resumed yet in this cycle04:15
samPtpatil: I will share details about test environments, so you may starting the tests for this.04:17
samPtpatil: sorry for the delay04:17
tpatilsamP: Thank you04:17
samPAny other updates?04:19
*** bswartz has joined #openstack-meeting04:19
tpatilDeprecate python-masakariclient04:19
tpatilI will add deprecation warning04:20
tpatiland upload patch in this week04:20
samPtpatil: thanks. I will send the mail to ML about this.04:20
tpatilAdd functional tests to openstacksdk04:21
tpatil Some of the newly added tests are failing randomly so checking this issue04:21
*** diablo_rojo has quit IRC04:21
*** diablo_rojo_ has joined #openstack-meeting04:21
*** vishalmanchanda has joined #openstack-meeting04:22
*** hongbin has quit IRC04:23
samPtpatil: are these patches in gerrit?04:23
tpatilNot yet04:24
samPtpatil: OK got it.04:24
tpatilwill propose these patches in this week04:24
samPtpatil: NP, thanks04:24
samPDo not have updates for Ironic HA and Force Stonith04:25
samP#topic AOB04:26
*** openstack changes topic to "AOB (Meeting topic: masakari)"04:26
samPWe got a slot for project updates in Berlin Summit04:26
samP#link https://www.openstack.org/summit/berlin-2018/summit-schedule/events/22953/masakari-project-update04:26
samPPlease let me know if you wish to speak in this session.04:27
samPI am the current speaker, however I can ask them to add you.04:27
samPOr If you would like me to talk about any specific topic in this session, please let me know.04:28
samPI will try to share my slides before summit.04:28
samP(2) Openstack Technical Vision statement04:29
tpatilI think we should talk about the event details of workflow that we are planning to add in stein04:29
samPtpatil: sure, good point.04:30
samPPlease read the mail sent my Zane, about Openstack Technical Vision04:31
samP#link http://lists.openstack.org/pipermail/openstack-dev/2018-October/136015.html04:32
samPPlease review the vision statement04:32
samP#link https://review.openstack.org/#/c/592205/404:33
samPThat's all from my side.04:33
samPAny thing else?04:33
samPOtherwise, we could finish this meeting early.04:34
tpatilsamP: Will read vision statement and update you if I find any important point to be included in the vision statement for masakari04:34
tpatilNo other topics from my end04:35
samPtpatil: thanks04:35
sagarasamP: I also don't have other topics, thanks.04:35
samPOK then, if you have any other issues or need extended discussion about above please use openstack-dev ML with [masakari] or IRC  #openstack-masakari @freenode04:36
samPThank you all! and have nice week ahead!04:36
samP#endmeeting04:36
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"04:36
openstackMeeting ended Tue Oct 30 04:36:55 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)04:36
openstackMinutes:        http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-10-30-04.00.html04:36
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-10-30-04.00.txt04:36
openstackLog:            http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-10-30-04.00.log.html04:37
*** sagara has quit IRC04:37
*** diablo_rojo_ has quit IRC04:42
*** diablo_rojo has joined #openstack-meeting04:50
*** lpetrut has joined #openstack-meeting04:52
*** tpatil has quit IRC04:59
*** diablo_rojo has quit IRC05:00
*** tetsuro has quit IRC05:03
*** tetsuro has joined #openstack-meeting05:03
*** tetsuro_ has joined #openstack-meeting05:04
*** tetsuro has quit IRC05:08
*** iyamahat has joined #openstack-meeting05:10
*** yamahata has joined #openstack-meeting05:28
*** lpetrut has quit IRC05:28
*** tetsuro_ has quit IRC05:29
*** tetsuro has joined #openstack-meeting05:34
*** pcaruana has joined #openstack-meeting05:36
*** tetsuro has quit IRC05:38
*** pcaruana has quit IRC05:47
*** fanzhang has left #openstack-meeting05:53
*** ykatabam has quit IRC05:58
*** ykatabam has joined #openstack-meeting06:01
*** tetsuro has joined #openstack-meeting06:05
*** tetsuro has quit IRC06:09
*** tetsuro has joined #openstack-meeting06:10
*** jbadiapa has quit IRC06:10
*** vishalmanchanda has quit IRC06:16
*** mnaser has quit IRC06:16
*** vishalmanchanda has joined #openstack-meeting06:16
*** mnaser has joined #openstack-meeting06:17
*** masayukig[m]1 has quit IRC06:18
*** vishalmanchanda has quit IRC06:26
*** vishalmanchanda has joined #openstack-meeting06:26
*** dkrol has quit IRC06:28
*** beisner has quit IRC06:30
*** beisner has joined #openstack-meeting06:31
*** slaweq has quit IRC06:40
*** ShilpaSD has quit IRC06:41
*** tonyb has quit IRC06:50
*** Luzi has joined #openstack-meeting06:52
*** aojea has joined #openstack-meeting07:18
*** jbadiapa has joined #openstack-meeting07:36
*** arne_wiebalck has joined #openstack-meeting07:45
*** LiangFang has quit IRC07:45
*** Liang__ has joined #openstack-meeting07:47
*** dkushwaha has joined #openstack-meeting07:47
*** slaweq has joined #openstack-meeting07:49
*** tetsuro has quit IRC07:50
*** Liang__ is now known as LiangFang07:51
*** bhagyashris_ has joined #openstack-meeting07:52
*** arne_wiebalck has quit IRC07:53
*** arne_wiebalck has joined #openstack-meeting07:54
*** pcaruana has joined #openstack-meeting07:57
*** tpatil has joined #openstack-meeting07:57
*** joxyuki has joined #openstack-meeting07:59
*** pcaruana is now known as pcaruana|elisa|07:59
*** phuoc has joined #openstack-meeting08:00
*** zaitcev has quit IRC08:01
dkushwaha#startmeeting tacker08:02
openstackMeeting started Tue Oct 30 08:02:02 2018 UTC and is due to finish in 60 minutes.  The chair is dkushwaha. Information about MeetBot at http://wiki.debian.org/MeetBot.08:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.08:02
*** openstack changes topic to " (Meeting topic: tacker)"08:02
openstackThe meeting name has been set to 'tacker'08:02
dkushwaha#topic Roll Call08:02
*** openstack changes topic to "Roll Call (Meeting topic: tacker)"08:02
dkushwahawho is here for Tacker weekly meeting ?08:02
phuochi08:02
joxyukihi tewam08:02
longkbo/08:02
joxyukiteam08:02
dkushwahahi team08:04
dkushwahaok lets start08:06
dkushwaha#chair phuoc08:06
openstackCurrent chairs: dkushwaha phuoc08:06
dkushwaha#topic bp/mistral-monitor-policy08:07
*** openstack changes topic to "bp/mistral-monitor-policy (Meeting topic: tacker)"08:07
dkushwaha#link https://review.openstack.org/#/c/48692408:07
dkushwahaphuoc, had you chance to test this patch?08:07
*** kopecmartin|off is now known as kopecmartin08:08
phuocnot yet, I have limit resource for testing this feature08:08
dkushwahai am just working on hardcoaded project_name08:08
dkushwahahi team, coud you please review and test the patch.08:08
*** zaitcev has joined #openstack-meeting08:09
dkushwahaI am thinking to land this asap08:09
phuocI will find some time to test that patch08:09
dkushwahai feel it is framework change which needs proper testing(even after code merge), so we will have sufficient time before Stein release08:10
dkushwaha..for bugs etc08:10
*** ralonsoh has joined #openstack-meeting08:11
phuocI think it is fine08:11
dkushwaha#topic bp/vdu-auto-healing08:13
*** openstack changes topic to "bp/vdu-auto-healing (Meeting topic: tacker)"08:13
dkushwahabhagyashris_, do you have something on that?08:13
dkushwahajoxyuki, ^08:14
bhagyashris_I have fixed all the review comments and uploaded implementation patch https://review.openstack.org/#/c/612595/ for review08:14
bhagyashris_Request to review the patch:  https://review.openstack.org/#/c/612595/  Thank you :)08:15
dkushwahabhagyashris_, i have just looked on overview code, overall it looks good to me, I will review it by today.08:17
bhagyashris_Ok thank you :)08:17
phuocbhagyashris_, you did a good job :)08:17
bhagyashris_And Thank you for review the spec :)08:18
joxyukithanks for review team!08:18
dkushwahaphuoc, do you have any query on spec?08:18
bhagyashris_phuoc: Thank you :)08:19
dkushwahaor it can be merged?08:19
phuocI look at that patch08:19
phuocLGTM too08:19
phuocI think we can test and it can be merged08:19
dkushwahaphuoc, +108:20
joxyukiphouc, I cannot understand your comment on the spec https://review.openstack.org/#/c/603558/3/specs/stein/vdu-auto-healing.rst08:21
dkushwahaok, i will just test code patch, in case of no issue i will first merge spec, and then look on code08:21
joxyukiwhat do you mean "to move the monitoring policy to Policy Type"?08:21
phuocI mean we can move the monitoring policy in each VDU to Policy Type08:22
phuochttps://github.com/openstack/tacker/blob/master/samples/tosca-templates/vnfd/tosca-vnfd-alarm-respawn.yaml#L4008:23
phuocit will be placed in policies08:23
joxyukigot it08:23
phuocbut it is not now, we can improve it later08:24
joxyukisounds good :)08:24
dkushwahaphuoc, +108:24
dkushwahajoxyuki, do you have something more to discuss?08:25
joxyukino08:26
dkushwahaphuoc ^08:26
phuocI only have some minor patches need to review08:27
dkushwahaphuoc, ok08:27
phuochttps://review.openstack.org/#/c/611785, https://review.openstack.org/#/c/61147808:27
phuochttps://review.openstack.org/#/c/612273, and https://review.openstack.org/#/c/59045508:28
dkushwahajoxyuki, bhagyashris_ longkb , I request you all to please help on review part08:29
dkushwaha#topic Open Discussion08:30
*** openstack changes topic to "Open Discussion (Meeting topic: tacker)"08:30
dkushwahalongkb, Thanks for code cleanup patches.08:30
longkbthank you very much! dkushwaha08:31
longkbIf you need my help, please let me know about it08:31
dkushwahalongkb, sure.08:32
dkushwaha:)08:32
longkb+108:32
dkushwahaTeam, we have some very good initiatives specs by Aliaksandr08:33
dkushwaha#link  https://review.openstack.org/#/q/owner:grechny.alexander%2540gmail.com+status:open08:33
dkushwahabut seems slow progress on that.08:33
phuocyes, they are good features for Tacker08:34
dkushwahaanyone had chance to reach Aliaksandr or other contributors?08:35
phuocno, I don't have their contact08:37
dkushwahaok08:37
dkushwaha#topic BerlinSummit08:37
*** openstack changes topic to "BerlinSummit (Meeting topic: tacker)"08:37
dkushwahaphuoc, joxyuki bhagyashris_ longkb are you going to join summit?08:38
bhagyashris_yes08:38
longkbNo, I will not :(08:38
phuocI can not join this summit, I hope to see you guys next year08:39
joxyukiyes08:39
joxyukioh,,, that is sad news....08:39
dkushwahaphuoc, longkb,  will miss you guys08:39
*** e0ne has joined #openstack-meeting08:41
longkbMe too. Hope that we can stay together somewhere in the world :)08:41
dkushwahalongkb, I remembered our barcelona meetup :)08:42
dkushwahaok, Thanks Folks08:42
longkbNo, I only attend Sydney summit :) We have been on the same hotel08:42
dkushwahaClosing this meeting for now08:42
phuocThank you guys :)08:42
joxyukibye08:42
dkushwaha#endmeeting08:43
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"08:43
openstackMeeting ended Tue Oct 30 08:43:01 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)08:43
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tacker/2018/tacker.2018-10-30-08.02.html08:43
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tacker/2018/tacker.2018-10-30-08.02.txt08:43
longkbbye gúy08:43
openstackLog:            http://eavesdrop.openstack.org/meetings/tacker/2018/tacker.2018-10-30-08.02.log.html08:43
dkushwahalongkb, yea, it was sydny08:43
phuocdkushwaha, that's right :)08:44
dkushwahaphuoc, :)08:46
*** dkushwaha has left #openstack-meeting08:46
*** lpetrut has joined #openstack-meeting08:48
*** tpatil has quit IRC08:48
*** rossella_s has joined #openstack-meeting08:48
*** priteau has joined #openstack-meeting08:52
*** slaweq has quit IRC08:53
*** tetsuro has joined #openstack-meeting09:07
*** tetsuro has quit IRC09:39
*** joxyuki has quit IRC09:41
*** a-pugachev has joined #openstack-meeting09:44
*** tetsuro has joined #openstack-meeting09:48
*** tetsuro has quit IRC09:51
*** bhagyashris_ has quit IRC09:51
*** wznoinsk has quit IRC09:51
*** slaweq has joined #openstack-meeting10:08
*** priteau has quit IRC10:11
*** ttsiouts has joined #openstack-meeting10:22
*** e0ne has quit IRC10:32
*** shrasool has joined #openstack-meeting10:33
*** rohinijoshi has joined #openstack-meeting10:38
*** rohinijoshi has quit IRC10:38
*** LiangFang has quit IRC10:42
*** rohinijoshi has joined #openstack-meeting10:44
*** rohinijoshi has left #openstack-meeting10:45
*** shrasool has quit IRC10:47
*** e0ne has joined #openstack-meeting10:54
*** jbadiapa has quit IRC11:04
*** erlon has joined #openstack-meeting11:04
*** electrofelix has joined #openstack-meeting11:04
*** priteau has joined #openstack-meeting11:04
*** phuoc has left #openstack-meeting11:16
*** janki has quit IRC11:25
*** yamamoto has quit IRC11:51
*** yamamoto has joined #openstack-meeting11:53
*** larainema has joined #openstack-meeting11:54
*** jesusaur has quit IRC11:56
*** yamamoto has quit IRC11:58
*** dtrainor has joined #openstack-meeting12:01
*** jesusaur has joined #openstack-meeting12:01
*** jesusaur has quit IRC12:13
*** longkb has quit IRC12:14
*** ttsiouts has quit IRC12:16
*** raildo has joined #openstack-meeting12:16
*** tssurya has joined #openstack-meeting12:20
*** e0ne has quit IRC12:21
*** wznoinsk has joined #openstack-meeting12:28
*** wznoinsk has quit IRC12:29
*** janki has joined #openstack-meeting12:31
*** wznoinsk has joined #openstack-meeting12:31
*** ttsiouts has joined #openstack-meeting12:32
*** yamamoto has joined #openstack-meeting12:33
*** jesusaur has joined #openstack-meeting12:41
*** wznoinsk has quit IRC12:41
*** wznoinsk has joined #openstack-meeting12:45
*** wznoinsk has quit IRC12:47
*** wznoinsk has joined #openstack-meeting12:49
*** wznoinsk has quit IRC12:51
*** wznoinsk has joined #openstack-meeting12:57
*** wznoinsk has quit IRC12:57
*** awaugama has joined #openstack-meeting12:59
*** jbadiapa has joined #openstack-meeting13:02
*** lpetrut has quit IRC13:05
*** dustins has joined #openstack-meeting13:07
*** e0ne has joined #openstack-meeting13:08
*** psachin has quit IRC13:09
*** lpetrut has joined #openstack-meeting13:11
*** yamamoto has quit IRC13:12
*** mriedem has joined #openstack-meeting13:13
*** tpsilva has joined #openstack-meeting13:15
*** psachin has joined #openstack-meeting13:15
*** Liang__ has joined #openstack-meeting13:20
*** bobh has joined #openstack-meeting13:20
*** Liang__ is now known as LiangFang13:21
*** dklyle has quit IRC13:37
*** dklyle has joined #openstack-meeting13:37
*** ttsiouts has quit IRC13:40
*** ykatabam has quit IRC13:40
*** ttsiouts has joined #openstack-meeting13:42
*** wznoinsk has joined #openstack-meeting13:50
*** hongbin has joined #openstack-meeting13:52
*** dklyle has quit IRC14:02
*** david-lyle has joined #openstack-meeting14:02
*** yamamoto has joined #openstack-meeting14:21
*** kiennt26 has joined #openstack-meeting14:21
*** yamamoto has quit IRC14:26
*** eharney has joined #openstack-meeting14:28
*** pcaruana|elisa| has quit IRC14:33
*** e0ne has quit IRC14:33
*** armax has joined #openstack-meeting14:34
*** janki has quit IRC14:35
*** e0ne has joined #openstack-meeting14:37
*** psachin has quit IRC14:38
*** munimeha1 has joined #openstack-meeting14:38
*** raildo has quit IRC14:43
*** pcaruana|elisa| has joined #openstack-meeting14:45
*** a-pugachev_ has joined #openstack-meeting14:50
*** raildo has joined #openstack-meeting14:50
*** kiennt26 has quit IRC14:51
*** shintaro has joined #openstack-meeting14:51
*** Luzi has quit IRC14:52
*** a-pugachev has quit IRC14:53
*** a-pugachev_ is now known as a-pugachev14:53
*** yamamoto has joined #openstack-meeting14:59
*** jamesmcarthur has joined #openstack-meeting15:05
*** caboucha has joined #openstack-meeting15:06
*** ianychoi_ is now known as ianychoi15:07
*** yamamoto has quit IRC15:20
*** yamamoto has joined #openstack-meeting15:23
*** e0ne has quit IRC15:25
*** Leo_m has joined #openstack-meeting15:27
*** yamamoto has quit IRC15:28
*** gyee has joined #openstack-meeting15:29
*** kiennt26 has joined #openstack-meeting15:31
*** pcaruana|elisa| has quit IRC15:35
*** e0ne has joined #openstack-meeting15:35
*** Leo_m has quit IRC15:49
*** kiennt26 has quit IRC15:49
*** pcaruana|elisa| has joined #openstack-meeting15:50
*** Leo_m has joined #openstack-meeting15:51
*** yamamoto has joined #openstack-meeting15:57
*** pcaruana|elisa| has quit IRC16:00
slaweq#startmeeting neutron_ci16:00
openstackMeeting started Tue Oct 30 16:00:11 2018 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: neutron_ci)"16:00
openstackThe meeting name has been set to 'neutron_ci'16:00
slaweqhello everyone16:00
*** mlavalle has joined #openstack-meeting16:00
slaweqhi mlavalle :)16:00
mlavalleo/16:00
mlavalleam I late?16:00
slaweqno, You are first actually16:01
mlavalleif yes, blame frickler and bcafarel16:01
slaweq(except me who started this meeting)16:01
slaweqI know, it's always bcafarel's fault :P16:01
slaweqhaleyb, njohnston: hongbin: manjeets: CI meeting - are You around?16:01
bcafarelnot this time, I even held on a question I have for mlavalle :p16:01
njohnstono/16:02
bcafareland o/ btw16:02
manjeetso/16:02
* njohnston was lurking16:02
slaweqok, lets start then16:02
slaweq#topic Actions from previous meetings16:02
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"16:02
*** cloudrancher has quit IRC16:02
slaweqslaweq to continue checking how jobs will run on Bionic nodes16:02
slaweqI was checking it a bit16:03
slaweqI created etherpad https://etherpad.openstack.org/p/neutron_ci_on_bionic which I want to use to track the progress16:03
* manjeets is in other meeting 16:04
*** cloudrancher has joined #openstack-meeting16:04
njohnston+1 for the etherpad16:04
slaweqand also I want to have separate topic about this on today's meeting, so let's talk about it later, ok for You?16:04
slaweqI take it as "yes" :)16:05
slaweqnext action then:16:05
slaweqmlavalle to continue debugging issue with not reachable FIP in scenario jobs16:05
mlavalleyou guys are not going to believe this.....16:05
*** _alastor_ has joined #openstack-meeting16:05
mlavalleno hist over the past seven days since yesterday16:05
manjeetsslaweq, so all the jobs mentioned in etherpad should use ubuntu bionic ideally ?16:05
njohnstonmlavalle: really?  wow.16:06
mlavalleThis is the query I'm using http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22line%20143,%20in%20test_trunk_subport_lifecycle%5C%22&from=7d16:06
slaweqmlavalle: are You sure?16:06
slaweqI found something like http://logs.openstack.org/59/596959/6/check/neutron-tempest-plugin-dvr-multinode-scenario/fbc011b/testr_results.html.gz from yesterday for example16:06
slaweqisn't it issue like You were looking for?16:07
mlavalleyeah, it's the same issue16:09
mlavallethe query is not catching it16:09
mlavalleah, you know why?16:09
mlavalleit's the line number16:09
mlavalleI should remove the line number from the query16:10
mlavalleI took the query that slaweq left originally in the bug (that includes the line number) and just added the 7days16:10
slaweq:)16:10
slaweqsorry for that then16:11
*** dustins has quit IRC16:11
mlavallein that case....16:11
mlavalleI'll go back to it16:11
mlavalleLOL16:11
slaweqok, I will assign it to You again :)16:11
slaweqmlavalle to continue debugging issue with not reachable FIP in scenario jobs16:11
slaweq#action mlavalle to continue debugging issue with not reachable FIP in scenario jobs16:11
slaweqthx mlavalle for taking care of it16:12
slaweqnext one was:16:12
slaweqslaweq to check if failing test_ha_router_namespace_has_ipv6_forwarding_disabled is related to bug https://bugs.launchpad.net/neutron/+bug/179847516:12
openstackLaunchpad bug 1798475 in neutron "Fullstack test test_ha_router_restart_agents_no_packet_lost failing" [High,Confirmed]16:12
slaweqand I totaly forgot about this one as I didn't create card on my trello for it :/16:12
slaweqsorry for that16:12
slaweqI will assign it to myself for next week then16:12
slaweq#action slaweq to check if failing test_ha_router_namespace_has_ipv6_forwarding_disabled is related to bug https://bugs.launchpad.net/neutron/+bug/179847516:12
slaweqok, next one16:13
slaweqslaweq to increase neutron-tempest jobs timeouts16:13
slaweqpatch is merged already https://review.openstack.org/#/c/612809/16:13
slaweqdid You saw any new failures because of timeouts in those jobs in last few days?16:14
njohnstonI haven't seen anything going into TIMED_OUT state at least16:14
njohnstonin general things look healthier16:15
slaweqnjohnston: it wasn't TIMED_OUT, it was usually FAILED16:15
slaweqbut in job-output.txt.gz there was info about timeout16:15
*** lpetrut has quit IRC16:16
njohnstonOK, I haven't seen any of those lately but I haven't tried looking systematically16:16
slaweqI don't see anything like that in logstash in last few days so should be better IMO16:19
slaweqand lets just check as njohnston said :)16:19
slaweqok, so that's all from last week16:19
slaweq#topic Python 316:19
*** openstack changes topic to "Python 3 (Meeting topic: neutron_ci)"16:19
slaweqlets talk about switch CI jobs to python316:20
slaweqnjohnston: I think You are most up to date with it16:20
njohnstonI've been working on the change to the neutron-fu;;stack job we discussed previously: https://review.openstack.org/60474916:20
*** LiangFang has quit IRC16:20
njohnstonmy goal for this week is to start going through the jobs and adding python3, either by changing their ancestor zuul template or just by adding USE_PYTHON3 depending16:21
* bcafarel actually reading through it, as zuul seems happy now16:21
njohnstonI noticed that there are differences in the zuul templates - for example if you base things off of "tempest-full" you16:22
slaweqok, but didn't we agree in Denver that we should just switch all jobs to py3 and left only UT and functional with py27 too?16:22
njohnstonsorry by "adding python3" I meant adding it to existing jobs to convert them to py3 jobs16:23
slaweqok :)16:23
slaweqbasically what I think is that we should do:16:23
slaweq1. etherpad to track progress16:23
njohnstonSo 'tempest-full' will get more tests run than if you base off of 'tempest-full-py3' because the latter has things like object store turned off, which disables object tests - https://git.openstack.org/cgit/openstack/tempest/tree/.zuul.yaml#n11416:23
slaweq2. rename existing neutron-functional job to neutron-functional-python27 and switch neutron-functional to be py316:24
njohnstonslaweq: Will do, I'll get an etherpad together today16:24
slaweq3. Start switching other jobs to py3 as we decided16:24
slaweqwhat do You think about it mlavalle and njohnston ?16:24
mlavallesounds like a good plan16:24
slaweqand IMO we shouldn't do python35/36/37 jobs as it may depends on OS on which job is running16:25
*** cloudrancher has quit IRC16:25
slaweqso we should just have our jobs like e.g. neutron-fullstack to run using python316:25
njohnstonyes16:26
*** cloudrancher has joined #openstack-meeting16:26
slaweqmlavalle: ok for You?16:26
njohnstonwe may have unit tests for the subversions like in Zane's proposal16:26
njohnstonbut not for all the flavors of testing16:26
mlavalleyes16:26
slaweqyes, UT are fine and we already have different versions but we shouldn't do it for all other jobs16:26
mlavallenjohnston: when you say Zane's proposal, you refer to the messsage in the ML?16:27
njohnstonit's now a governance change: https://review.openstack.org/#/c/613145/16:27
njohnstonI encourage everyone to read it and give feedback16:28
njohnston"Resolution on keeping up with Python 3 releases"16:28
mlavalleyeah, about the same thing we stated in the message16:29
mlavalleso, yes, overall, I'm in agreement16:29
slaweqgreat :) njohnston will You do etherpad to track it?16:30
*** e0ne has quit IRC16:30
njohnstonalready started16:31
slaweqthx16:31
slaweqand do You want to propose this change for functional tests? or do You want me to do it?16:32
njohnstonI'll do it16:32
slaweqthx16:32
njohnston#action njohnston rename existing neutron-functional job to neutron-functional-python27 and switch neutron-functional to be py316:33
slaweq#action njohnston to create new neutron-functional-python27 job and switch existing one to python316:33
njohnston#undo16:33
njohnstonI think that undid mine... we'll have to see :-)16:33
slaweqahh, sorry :)16:33
njohnston#action njohnston make py3 etherpad16:33
slaweqok, I will remember only one of them ;)16:34
slaweqso next week we will be able to check how it's going and continue this work16:34
slaweqdo You have anything else related to python3 to talk about?16:35
njohnstonnope, thanks16:35
slaweqok, so lets move on16:35
slaweqnext topic16:35
slaweq#topic Ubuntu Bionic in CI jobs16:35
*** openstack changes topic to "Ubuntu Bionic in CI jobs (Meeting topic: neutron_ci)"16:35
slaweqas I said, etherpad for it is created https://etherpad.openstack.org/p/neutron_ci_on_bionic16:36
slaweqtoday I sent patch for neutron-tempest-plugin: https://review.openstack.org/#/c/614216/ to switch our jobs to Bionic16:36
slaweqit's for now DNM patch as I want to check how it will be16:36
slaweqahh, and one most important thing16:37
slaweqthere is devstack patch https://review.openstack.org/#/c/610977/ which adds Bionic nodesets which we can use16:37
slaweqin etherpad I listed what nodeset should be replaced with what new one (but it may not be perfect yet)16:38
slaweqI also did some DNM patch https://review.openstack.org/#/c/610997/16:38
slaweqit looks that for many jobs we are fine and they works good on Bionic16:39
slaweqthere are some small issues with fullstack jobs for example but it's not something really big16:39
slaweqand now the question is: how You want to perform that switch? should we first switch all jobs to python3 and then to Bionic? or do it "in parallel"?16:40
slaweqor Bionic first and then python3?16:40
slaweqany thoughts?16:40
mlavalleI like keeping things simple16:40
mlavalleone change at a time16:40
njohnstonagreed16:41
mlavalleif possible, let's convert a job to python316:41
njohnstonby the way the problematic part for fullstack - where ovs is compiled - can be discarded when we move to bionic16:41
mlavalleand then we move that to bionic16:41
slaweqthat was also my idea :) lets move to python3 first and then start switching to Bionic16:42
njohnstondo we know if grenade is ready for py3 yet?16:42
mlavalleno clue16:42
slaweqI don't know16:42
njohnston#action njohnston check if grenade is ready for py316:43
slaweqthx njohnston :)16:43
njohnstonI mean we'll discover it if our changes fail for grenade jobs, but it'd be nice to find out the plan16:43
slaweqnjohnston: I agree16:44
*** Leo_m has quit IRC16:44
slaweqok, so I guess we can move on to next topic now16:44
slaweq#topic Grafana16:44
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"16:44
slaweq#link http://grafana.openstack.org/dashboard/db/neutron-failure-rate16:44
*** Leo_m has joined #openstack-meeting16:45
*** iyamahat has quit IRC16:45
slaweqfullstack is about 30% of failures now16:46
slaweqbut it is related to this issue with stopping processes IMHO16:46
slaweqhttps://bugs.launchpad.net/neutron/+bug/179847216:46
openstackLaunchpad bug 1798472 in neutron "Fullstack tests fails because process is not killed properly" [High,Confirmed]16:46
slaweqand to https://bugs.launchpad.net/neutron/+bug/179847516:46
openstackLaunchpad bug 1798475 in neutron "Fullstack test test_ha_router_restart_agents_no_packet_lost failing" [High,Confirmed]16:46
slaweqand both aren't assignment yet16:47
*** yamahata has quit IRC16:47
slaweqI will try to check at least one of them if I will have some time this week16:47
slaweqin functional tests db_migration test is happening less often I think16:48
mlavalleI can try to help with the other one16:48
slaweqthx mlavalle16:48
mlavallecontingent on time availaility16:48
mlavallewhich one do you want slaweq?16:48
slaweqso I will take https://bugs.launchpad.net/neutron/+bug/179847216:49
openstackLaunchpad bug 1798472 in neutron "Fullstack tests fails because process is not killed properly" [High,Confirmed]16:49
slaweq#action slaweq to check Fullstack tests fails because process is not killed properly (bug 1798472)16:49
mlavalleok16:49
mlavalle#action mlavalle to check bug 179847516:50
openstackbug 1798475 in neutron "Fullstack test test_ha_router_restart_agents_no_packet_lost failing" [High,Confirmed] https://launchpad.net/bugs/179847516:50
*** armax has quit IRC16:50
slaweqgetting back to functional tests, db migration tests are less often after I increased timeout for them to 600 seconds, but unfortunatelly I saw it at lest once that it happend event with such timeout:     * http://logs.openstack.org/31/613231/1/gate/neutron-functional/441128f/job-output.txt.gz#_2018-10-25_18_54_53_99269016:51
slaweqso I think that it mayb be something different than only slow node16:51
slaweqlet's observe it for few more days and if You will spot it again, please add it to bug report: https://bugs.launchpad.net/neutron/+bug/168702716:52
openstackLaunchpad bug 1687027 in neutron "test_walk_versions tests fail with "IndexError: tuple index out of range" after timeout" [High,Confirmed] - Assigned to Slawek Kaplonski (slaweq)16:52
mlavalleack16:54
slaweqand that's all from fullstack/functional tests16:54
slaweqanything else You want to add?16:54
mlavalleno, thanks16:54
slaweqok, so let's now talk about16:55
slaweq#topic Periodic16:55
*** openstack changes topic to "Periodic (Meeting topic: neutron_ci)"16:55
slaweqfrom grafana I see that openstack-tox-py35-with-oslo-master is failing constantly since few days16:55
*** Leo_m has quit IRC16:55
slaweqexample from today: http://logs.openstack.org/periodic/git.openstack.org/openstack/neutron/master/openstack-tox-py35-with-oslo-master/7ae4398/testr_results.html.gz16:55
*** Leo_m has joined #openstack-meeting16:56
slaweqanyone wants to fix that?16:57
slaweqor I should take it?16:57
mlavalleI don't have time this week16:57
slaweqok, I will check it16:57
slaweq#action slaweq to check issue with openstack-tox-py35-with-oslo-master periodic job16:57
slaweqok, so that's all from my side for today16:58
slaweqanything else You want to add/ask maybe?16:58
mlavallenot from me16:58
slaweqok16:59
slaweqthx for attending16:59
mlavalleo/16:59
slaweq#endmeeting16:59
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"16:59
njohnstonI have a change to migrate the iptables_hybrid job to the new zuulv3 format https://review.openstack.org/#/c/604894/16:59
openstackMeeting ended Tue Oct 30 16:59:09 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:59
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-10-30-16.00.html16:59
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-10-30-16.00.txt16:59
njohnstonoops too late16:59
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2018/neutron_ci.2018-10-30-16.00.log.html16:59
slaweqsorry njohnston16:59
slaweq:)16:59
njohnstonnp :-)16:59
*** mlavalle has left #openstack-meeting16:59
slaweqlets talk about it in neutron channel then16:59
njohnstonok16:59
*** rossella_s has quit IRC16:59
*** shintaro has quit IRC17:00
*** rossella_s has joined #openstack-meeting17:01
*** rossella_s has quit IRC17:01
*** iyamahat has joined #openstack-meeting17:01
*** rossella_s has joined #openstack-meeting17:03
*** a-pugachev has quit IRC17:07
*** rossella_s has quit IRC17:11
*** tssurya has quit IRC17:19
*** yamahata has joined #openstack-meeting17:19
*** lpetrut has joined #openstack-meeting17:23
*** yamamoto has quit IRC17:24
*** ttsiouts has quit IRC17:28
*** ttsiouts has joined #openstack-meeting17:29
*** kopecmartin is now known as kopecmartin|off17:30
*** ttsiouts has quit IRC17:33
*** jamesmcarthur has quit IRC17:48
*** electrofelix has quit IRC17:50
*** bobh has quit IRC17:53
*** armax has joined #openstack-meeting17:54
*** bobh has joined #openstack-meeting17:55
*** Shrews has joined #openstack-meeting17:55
*** bobh has quit IRC17:59
*** cloudrancher has quit IRC18:01
*** cloudrancher has joined #openstack-meeting18:02
*** yamamoto has joined #openstack-meeting18:03
*** e0ne has joined #openstack-meeting18:10
*** yamamoto has quit IRC18:12
*** dustins has joined #openstack-meeting18:13
*** jamesmcarthur has joined #openstack-meeting18:19
*** jamesmcarthur has quit IRC18:22
*** diablo_rojo has joined #openstack-meeting18:24
*** e0ne has quit IRC18:25
*** bobh has joined #openstack-meeting18:29
*** lpetrut has quit IRC18:29
*** bobh has quit IRC18:33
*** bobh has joined #openstack-meeting18:41
*** njohnston has quit IRC18:44
clarkbAnyone here for the infra meeting?19:00
fungidepends on your definition of #here19:00
clarkbsome attention directed to this channel :)19:00
mordredand your definition of #anyone19:00
fungidoes seem like it's been a quiet day in the infra channel19:00
clarkbmordred: you are definitely an anyone19:00
mordred\o/19:01
clarkbya I've mostly been catching up on work already in flight19:01
clarkb#startmeeting infra19:01
openstackMeeting started Tue Oct 30 19:01:27 2018 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.19:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:01
*** openstack changes topic to " (Meeting topic: infra)"19:01
openstackThe meeting name has been set to 'infra'19:01
clarkb#topic Announcements19:01
*** openstack changes topic to "Announcements (Meeting topic: infra)"19:01
clarkbHopefully this doesn't surprise anyone but the Berlin summit is less than two weeks away19:01
*** tidwellr has left #openstack-meeting19:01
clarkbThis is your warning to do any necesary prep work :) I need to put together a project update and onboarding session19:02
clarkb#link https://civs.cs.cornell.edu/cgi-bin/vote.pl?id=E_aac97f1cbb6c61df&akey=b9e448b340787f0e Vote for OpenStack T release name19:02
clarkbtonyb put together a poll for us to pikc a name. You should really vote for train19:02
funginote that if you share an ip address you'll need to find an alternative from which to place your vote19:03
fungisay, an unpopular starbucks19:03
clarkbor via your phone?19:03
fungior maybe the public library (they still have those, right?)19:03
ianwo/19:03
clarkbAnything else worth announcing?19:04
funginot to my knowledge19:04
clarkb#topic Actions from last meeting19:04
*** openstack changes topic to "Actions from last meeting (Meeting topic: infra)"19:04
clarkb#link http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-10-23-19.01.txt minutes from last meeting19:04
*** weshay has joined #openstack-meeting19:05
clarkbNot in that ^ meeting but in the opendev messaging meeting I took an action to start drafting an email to talk about opendev more broadly19:05
clarkbI've got time set aside for that on this meeting agenda but if you want a sneak preview https://etherpad.openstack.org/p/WUNOTv8MuP19:05
clarkb#topic Priority Efforts19:05
* fungi sneaks a peek19:05
*** openstack changes topic to "Priority Efforts (Meeting topic: infra)"19:05
clarkb#topic Update Config Management19:06
*** openstack changes topic to "Update Config Management (Meeting topic: infra)"19:06
clarkbMay as well jump right in. I don't think much happened on this last week as we were distracted by zookeeper, openstacksdk, and nodepool related things (at least I was)19:06
clarkbgerrit query for reviews: topic:puppet-4 and topic:update-cfg-mgmt19:07
* mordred aplogizes for his hand in the openstacksdk things19:07
fungii, for one, am glad you choose to do openstacksdk things19:08
fungidon't apologize19:08
clarkbhave I missed related work on ^ worth talking about? I guess we may still need a decision on where to store the ara database? I think the conversation we had a week ago mostly ended up with maybe we should rebuild bridge server to be appropriately sized for ansible and mysql19:08
clarkbthen we reduce data leakage concerns and have server that performs as expected across the board19:08
fungiwell, there was already a trove instance created, so i thought we said in the interim we can use that in parallel with any work to replace the bridge server19:09
clarkbah19:09
mordredI support all of the choices19:09
fungibut that in sizing the replacement bridge.o.o we should take running a mysql db for ara into account19:09
clarkbI was distracted by zookeeper move at the time so fungi's memory likely better than mine19:09
clarkb++19:10
fungiyeah, the information leak is theoretical, and so short-term concerns while we're getting this up and running aren't as significant as the longer-term risks of continuing to use a separate db system19:10
clarkbsounds good19:10
clarkbmordred: hae a storyboard containerization update?19:11
mordredoh my19:11
* fungi hopes it's not an old margarine tub19:11
mordredso - I have decided to step back from that just a little bit- a bit more to wrap my head around19:11
mordredand gonna just do a normal update - so I can properly think about what the containerized version looks like19:12
mordredI'll hopefully get some patches up for review later today or early tomorrow19:13
funginow thinking that "margarine-tub" would have been a great alternative name for pbrx19:13
mordredfungi: still might be19:13
clarkbseems reasonable. I do think we shouldn't be afraid of doing the quicker get off trusty moves without blocking too hard on containers if it isn't straightforward upfront19:13
mordredyup19:13
mordredthat's kinda where I ended up19:13
*** e0ne has joined #openstack-meeting19:13
mordredalso - to be fair - I'm *also* overthinking it19:13
clarkbanything else related to updating config management before we go to storyboard?19:14
clarkb(this is the segway)19:14
fungiso basically upgrading storyboard-dev.o.o and then storyboard.o.o from trusty to xenial for now?19:14
mordredfungi: yah19:14
clarkber segue? english weird19:14
fungiwfm19:14
fungisanguine19:14
mordredI'd like to eventually end up with storyboard running in python3 in containers on bionic19:15
clarkbmordred: shiny19:15
mordredbut let's just make progress19:15
*** weshay is now known as weshay_doctor19:15
*** jamesmcarthur has joined #openstack-meeting19:15
clarkb#topic Storyboard19:15
*** openstack changes topic to "Storyboard (Meeting topic: infra)"19:15
clarkbI did review the storyboard spec19:15
fungime too! (finally)19:15
fungii also need to apologize to diablo_rojo for the pile of comments i left on it19:16
clarkband frickler too. Thank you all for doing that19:16
clarkbLooks like we are waiting on an update on that now?19:16
fungiif any of my comments are unclear or need added detail, please let me know and i'm happy to expand on them19:16
clarkbare tehre other storyboard updates? I had to miss the meeting last week due to dentist visit19:17
fungisome talk of forum planning19:18
fungithere's a remaining blockers to migration session on the forum schedule19:19
clarkbfungi: project moves have slowed down?19:19
fungi#link https://www.openstack.org/summit/berlin-2018/summit-schedule/events/22839/storyboard-migration-the-remaining-blockers Summit Session19:19
clarkbI'm guessing that is due to the remaining blockers?19:19
fungifor the moment, yes19:19
fungiwell, and also some roadblocks in testing them19:19
fungidiablo_rojo has discovered that test imports of neutron abruptly stop partway into what we would expect to have loaded19:20
fungisuspecting at this point that we're missing support for lp api result pagination or something along those lines19:20
clarkbI guess it is good that we test them then :)19:20
fungibut need to add some more debugging output to figure it out19:20
fungiindeed!19:21
mordredoh joy19:21
fungineutron is by far the largest single lp project we've attempted to import, so no surprises it's where we're finding new problems19:21
clarkbmordred: ^ probably want to coordinate the redeployment of -dev with neutron debugging19:22
clarkb(just to avoid deleting old server in the middle of a test migration)19:22
mordredclarkb: yes - I agree.19:22
clarkb#topic General Topics19:22
*** openstack changes topic to "General Topics (Meeting topic: infra)"19:22
clarkbOpendev. As mentioned earlier I've started drafting an email to communicate the things19:23
clarkbIt depends a bit on the work lsell will be doing foundation side, but I'm trying to set a tone of we aren't abandoning you and this is something people have wanted for a while19:23
clarkbI think it is also helpful that we now have data saying 98.5% of all the test resources we use are openstack related19:23
fungipoint of order. i move that any specs related to the opendev transition be deemed priority efforts19:23
mordred++19:23
* mordred seconds fungi's point of order19:24
clarkbThat said I am not sure I'm entirely happy with that draft so please give me feedback or suggest wys to make it better19:24
clarkbmordred: fungi I'll third that19:24
* mordred also knows that's not how robert's rules work19:24
fungiyep19:24
fungii was abusing terms for fun19:24
mordredme too19:24
clarkb#link https://etherpad.openstack.org/p/WUNOTv8MuP draft email announcement19:24
mordredjust wanted to be clear in case any robert's rules superfans were lurking19:25
clarkbnow the link shows up in the correct spot in the meeting logs19:25
clarkbI'll try to sync up with lsell real soon now to see how the other bits are going19:25
clarkbDoesn't look like the dns configurations are fully in place for opendev.org yet, but progress was amde on that lst week too19:26
clarkbI expect corvus will be motivated to finish that when back at work19:26
clarkbTrusty server upgrades. As hinted to earlier with storyboard this continue to happen. The zookeeper cluster work last week was basically a trusty to xenial upgrade for our zookeeper server. The trusty zookeeper is running on nodepool.openstack.org. It is on my list of todos today to delete that server19:27
clarkbthis server has been around for a long time and was a fairly important piece of the puzzle for us so if you know of anything that needs saving or cleanup before I delete it please let me know19:28
clarkb(possbily useful openstack api related scripts)19:28
fungialso, no obvious problems after switching from single-node zk to a cluster19:28
fungiso that was nice19:28
clarkbcorrect. I chceked the zk stat output on all three servers for days after and it has continued to look happy19:29
ianw++, now we just need to get it ssl :)19:29
clarkbcacti also makes it looks happy19:29
clarkbOnce nodepool.o.o is gone I'm going to grab another server off the list and go look at what that server needs. Maybe pbx?19:29
clarkbI feel like I've got some momentum with this and am hoping to keep doing a server ish a week if I can19:30
fungii guess that did eventually get rebuilt on ubuntu, huh?19:30
clarkbfungi: yup19:30
ianwi'm hoping to get to some upgrades too, if nothing else pops up19:30
fungiare the git servers and load balancer our only centos servers now?19:30
clarkbianw: great. Please do let me know if I can help (that goes for anyone doing this work I am highly motivated to get this type of thing done and happy to help where I can)19:30
clarkbfungi: yes19:31
clarkb#link https://etherpad.openstack.org/p/201808-infra-server-upgrades-and-cleanup list of servers that need upgrading19:31
clarkbZuul queue backlogs. Since the PTG we've been experiencing long zuul queue backlogs. There have been a varity or reasons for this over the last month and a half many of which we've fixed. But the general situation is that we are still suffering from this19:32
clarkb#link http://lists.openstack.org/pipermail/openstack-dev/2018-October/136141.html Most recent update from clarkb on the Zuul backlog situation19:32
clarkbThe good news is we are getting better data from zuul around where the cost is19:33
clarkbUsing that we can measure if our changes improve things19:33
clarkband also prioritize which work would be most effective19:33
mordred++ this is great19:33
clarkbAnother item worth pointing out is that I think we are in part running into lack of logging in jobs that help us understand where the time spend is within a job. I've been asking various projects to improve the logging so that the statement isn't just "infra is slow or unreliable" and instead is "on infra $task takes this long and is $resource bound"19:34
clarkbFor example tripleo points to docker image download costs. But as far as we know we don't have a measurement of that so step 0 is lets measure that then we'll know if changes improve it19:35
clarkb(or if it is even a major cost at all)19:35
clarkbOSH has performance issues but we don't knwo if they are related to cpu/disk/network/etc so to improve that we need that information19:35
mordredyup. logging here is also helpful, because while we do have docker mirrors, it may not always be clear if we're actually using them or not19:35
clarkbThere is zuul work to build in some performance data into base jobs too which should help19:36
clarkbI think we've taken for granted that devstack does all of this for you19:36
mordredyup19:36
clarkband now that we are doing more and more non devstack testing the lack of that data is apparent19:36
clarkbIf random projects drop by asking for help implementing that stuff any help you can give will likely go a long way19:37
clarkbAnd that was it for the agenda I had written down19:38
clarkb#topic Open Discussion19:38
*** openstack changes topic to "Open Discussion (Meeting topic: infra)"19:38
clarkbAnything else? we've got time so feel free19:38
fungii've got nothin'19:39
clarkbmy kids have a halloween parade thing tomorrow at school. I am not yet sure if I am expected to go view that19:40
clarkbbut I may be afk for an hour or two tomorrow for that19:40
clarkbfungi: ^ did the storm take precedence over halloween planning?19:40
clarkbI'll give this until 19:45 UTC and if we don't have anything to talk about by then I'll end the meeting19:41
fungiyeah, we sort of didn't decorate much19:42
fungigonna chill with some friends and frighten/rot the teeth of children in their neighborhood this year19:42
clarkboh! we get off DST this weekend in this country at least19:44
mordredoh - is that this weekend?19:45
clarkbfriendly reminder that meeting times are UTC so if your local government observes DST you may have the meeting shift by an hour19:45
fungiyeah, i need to skim my recurring schedule for any meetings which shift relative to dst19:45
fungior rather which follow dst and so shift in utc19:45
clarkbthe meeting will be an hour earlier in my local timezone next week19:46
clarkband with that I think I'll call the meeting. Thank you everyone19:46
clarkb#endmeeting19:46
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"19:46
openstackMeeting ended Tue Oct 30 19:46:28 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)19:46
openstackMinutes:        http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-10-30-19.01.html19:46
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-10-30-19.01.txt19:46
openstackLog:            http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-10-30-19.01.log.html19:46
fungithanks for chairing, clarkb!19:46
*** armax_ has joined #openstack-meeting19:49
*** armax has quit IRC19:50
*** armax_ is now known as armax19:50
*** gary_perkins has joined #openstack-meeting19:51
*** Shrews has left #openstack-meeting19:52
gary_perkinshmm...20:05
ssbarneahi!20:06
gary_perkinsWhat time does the meeting start? The UK changed back from daylight savings over the weekend20:06
clarkbgary_perkins: 1900utc or about an hour ago20:07
gary_perkinsahh, it's over already!! /facepalm20:08
gary_perkinsok, thanks :)20:08
fungiinfra meeting ran from 19:00 utc to roughly 19:4520:08
ssbarneayeah, so i missed it. but considering that nobody mentioned my name, probably i can just read the logs.20:08
fungiended early due to getting through the agenda in record time20:08
gary_perkinsYep, I'll read the logs, taa20:08
ssbarneaif anyone has some time, please read https://review.openstack.org/#/c/613753/ -- an idea I had on Sunday, not sure if really possible.20:10
*** pcaruana|elisa| has joined #openstack-meeting20:12
*** ralonsoh has quit IRC20:27
*** pcaruana|elisa| has quit IRC20:32
*** imacdonn has quit IRC20:33
*** priteau has quit IRC20:33
*** imacdonn has joined #openstack-meeting20:34
*** caboucha has quit IRC20:43
*** weshay_doctor is now known as weshay20:51
*** Leo_m has quit IRC20:51
*** rbudden has joined #openstack-meeting20:56
*** tssurya has joined #openstack-meeting20:58
*** cloudrancher has quit IRC20:59
*** trandles has joined #openstack-meeting20:59
*** oneswig has joined #openstack-meeting21:00
*** cloudrancher has joined #openstack-meeting21:00
*** bobh has quit IRC21:00
*** janders has joined #openstack-meeting21:01
jandersg'day everyone21:01
oneswigHi janders21:01
trandles\o21:01
*** martial__ has joined #openstack-meeting21:01
*** erlon has quit IRC21:02
oneswigI was just hanging back as I'd asked Martial to lead the ceremonies today - I'll be in and out21:02
martial__hello21:02
oneswigthe very chap :-)21:02
martial__(this is always weird to me: you are now identified for martial)21:02
martial__did you start the meeting Stig?21:02
oneswigno not yet, but I did run a bath :-)21:03
martial__#start-meeting  Scientific-SIG21:03
trandlesCheers for the bath :P21:03
oneswigalways thought you were more a hot tub kind of guy, down there in NM21:03
trandlesdon't think that worked martial__21:03
oneswigmartial__: startmeeting21:03
martial__nope seems it did not21:03
trandleshot springs21:04
martial__#startmeeting Scientific-SIG21:04
openstackMeeting started Tue Oct 30 21:04:19 2018 UTC and is due to finish in 60 minutes.  The chair is martial__. Information about MeetBot at http://wiki.debian.org/MeetBot.21:04
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:04
*** openstack changes topic to " (Meeting topic: Scientific-SIG)"21:04
openstackThe meeting name has been set to 'scientific_sig'21:04
oneswigaway we go :-)21:04
martial__#chair oneswig21:04
openstackCurrent chairs: martial__ oneswig21:04
janderstrandles: Blair would have a lot of those now :)21:04
martial__Agenda at21:05
martial__#link https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_October_30th_201821:05
martial__#topic  Berlin SIG BoF - Call for lightning talks21:05
*** openstack changes topic to "Berlin SIG BoF - Call for lightning talks (Meeting topic: Scientific-SIG)"21:05
martial__Come-thee, come-thee, the Scientific SIG-ness has an opening for its participants to discuss the latest and greatest and the barely working during its Lighting Talk session21:06
martial__we welcome the courageous, the brave and the crazy on our podium21:06
jandersbarely as in bare metal? :)21:06
oneswigoh, too good.21:07
oneswigI shall think up a topic now.21:07
oneswigAha, money for old rope - I'm presenting at Ceph day on the Monday before.  I can squeeze that into 3 minutes, sure21:07
martial__be thee brave friend? be thee stupendous with knowledge to share? Well thee are perfect for us ... come and bedazzle21:07
oneswigmartial__: been at the communion wine again? :-)21:08
martial__so ... add thy words to our pad of the ether and amaze the crowd21:08
martial__(selling it :) )21:09
oneswigSold!21:09
trandlesmartial__: when are you auditioning for a job at Medieval Times?  Just back from a stay at Excalibur in Vegas?21:09
martial__:)21:09
martial__so as the words above totally say in more words, come, be happy, present and ... profit21:10
trandlesSomeone get in touch with Arkady.  Best talk prize this go should be a sword-in-the-stone model. ;)21:10
martial__who in this crowd is going beyond Sir OneSwig21:10
oneswigI am lost for words21:10
martial__well as long as you host the Joust :)21:11
jandersI'm sure that we can quickly write a sword-extracting ansible module21:12
martial__:)21:12
oneswigmartial__: I hope you'll keep this up for your video message to address the SIG meeting21:12
martial__4am my time? I will be just in the right state of sleep :)21:12
*** raildo has quit IRC21:13
janderstime is relative and only a function of space21:13
oneswigAre preparations going well for SC?21:13
martial__but yes I am proposing to host a GTM for people who are interested to view the meetings remotely21:13
martial__(and maybe record it?)21:13
martial__so if you are interested in participating, add your name to21:14
martial__#link https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_October_30th_201821:14
martial__oops wait no21:14
martial__#link https://etherpad.openstack.org/p/Berlin-Scientific-SIG-BoF21:14
martial__^ that one21:14
martial__in truth that is the agenda I have for Today21:15
jandersin which case let me propose this agenda item:21:15
jandersIBM acquisition of Red Hat21:15
martial__ouch21:15
oneswigjanders: anything to avoid repeating your excellent presentation!?21:16
janderswhat are your thoughts on what this means to OpenStack and Ansible?21:16
martial__tim, are you staying until the end? (have to leave in 15 minutes)21:16
oneswigI see no harm for Ansible.21:16
jandersoneswig: I don't mind repeating if there is interest, but curious what people think21:16
*** dustins has quit IRC21:16
oneswigIBM's interest in OpenStack is not what it once was (true for many larger and more corpulent corporations)21:17
oneswigBut perhaps that is why they covet Red Hat, who have clearly made a stellar success from their contribution21:17
janderswhat's your feel regarding RHAT being left alone as promised as opposed to IBM making attempts of tighter integration of the two organisations?21:18
*** e0ne has quit IRC21:18
oneswigI think it looks bleak for Red Hat's business-centric offerings.21:18
oneswigAnd I'm not sure I'd back GlusterFS against SpectrumScale21:19
jandersI clearly wouldn't :)21:19
jandersif Ceph+CephFS didn't kill Gluster, Spectrum might21:19
martial__did not think of that I must say21:19
oneswigI assume there will be ever closer union.  Perhaps a bit like Hong Kong and mainland China21:19
martial__that said I truly saw this more as "we need cloud and open source" ...21:20
martial__wait and see I guess21:20
oneswigWho owns Pivotal (and RabbitMQ)?  Was that Dell?21:20
janderswhat are your bets on Microsoft buying Canonical now? :)21:21
*** eharney has quit IRC21:21
oneswigI'll bet you 500 beers that doesn't happen21:21
oneswig(very small ones)21:21
janders:D21:21
martial__ah yes since we are already in that ...21:21
martial__#topic AOB21:21
*** openstack changes topic to "AOB (Meeting topic: Scientific-SIG)"21:21
oneswigmartial__: will you have a beer night at SC for the Scientific SIG?21:22
jandersoneswig: why do you think Micosoft's acquisition of Canonical is unlikely?21:22
trandlesmartial__: I'm here21:22
martial__I can try to see who is around indeed (will send an email to the ML)21:22
martial__the SC18 counter dinner21:23
oneswigjanders: I am not sure what to expect nowadays, but still21:23
martial__janders: what next Microsoft buying Github?21:23
trandlesah, well, I'm only at SC18 from Monday night until early Wednesday morning (read: I'm only available on Tuesday for anything at SC18)21:23
janders..and Apple buying Microsoft? :)21:24
janders..and then Amazon will take over the main Linux distro21:24
*** jmlowe has joined #openstack-meeting21:24
martial__oh boy, let's conclude being on the record :)21:24
martial__any AOB to discuss?21:24
jmloweHi everybody21:24
trandlesDenver 19 Summit21:25
oneswigHi jmlowe21:25
trandlesI see on the foundation page that it's April 29-May 221:25
martial__Hey Mike, welcome21:25
oneswigThanks trandles, noted21:25
trandlesFWIW - The Rockies are playing a baseball game on May 3.  I know it's the night after the summit ends, but I might try organizing a group to go.21:25
jmloweI'd be interested21:25
martial__Stig, we are clearly on AOB, can you close? :)21:26
oneswigmartial__: I think so21:26
oneswigbon voyage martial__21:26
martial__thanks, gotta go pick up the younglings21:26
trandleslater all21:26
martial__team, see some of you at SC1821:26
janderssee you guys21:27
trandlesjmlowe: interest recorded ;)21:27
oneswigtrandles: are you still here?21:27
trandlesI am21:27
oneswigsmashing21:28
trandlespumpkins (is this word association? ;) )21:28
oneswigtrandles: jmlowe: did you see janders presentation from last week on mixing virt upon bare metal?21:28
oneswigHighly recommended.21:28
trandlesoneswig: might want to endmeeting21:28
jmloweI didn't got a  link?21:28
trandlesI saw janders link in the minutes but didn't click yet21:29
oneswigpumpkins - very topical. I ate an entire pumpkin pie when it became clear my family are not fans.21:29
jandershttps://docs.google.com/presentation/d/1fh6ZOq3DO-4V880Bn7Vxfm5I1kH5b0xKXHBLW2-sJXs/edit#slide=id.p21:29
trandles+1 pumpkin pies - what's not to like??21:29
jmloweDaylight savings ends this week for the US, makes participating a bit easier btw21:30
*** mriedem has quit IRC21:30
trandlesjanders: thx for the link21:30
trandlesreading now21:30
jmlowefor those of us in the eastern time zone21:30
oneswigjmlowe: good to hear it, although in 2 weeks no doubt you'll have your hands full21:30
jandersno worries :)21:30
oneswigjanders: how close are you to production workloads at scale on this system?21:31
janderswe're ready for power-POC with users on but we're delaying this to focus on procurement21:31
jandersfor full prod I'd like to do more work on resiliency and HA21:32
janderswe focused on capability first and we've got that and we're confident with that21:32
*** awaugama has quit IRC21:32
trandlesjanders: we have a network stress test application (I think it's MPI-based) that we use for some early acceptance testing...if I can get my hands on it, would you be willing to give it a go on (Super)Cloud?21:33
jandersnow we need to work on some implementation details that most people are not interested in21:33
janderstrandles: sure! jacob.anders@csiro.au21:33
oneswigtrandles: sounds like a quid pro quo arrangement there?21:33
trandlesjanders: thanks, I'll get back to you21:34
oneswigjanders: what generation of IB was it?  100G?21:35
jandersFDR/5621:35
jandersbut the final system will certainly have a newer interconnect :)21:36
jandersI do think we'll maintain significant investment in FDR - it's good enough for many things and there's potential of getting a fair bit of free FDR kit from other projects21:36
janderswe likely have some FDR14/40 in the mix too but it's transparent implementation wise so not sure what's the proportion21:37
oneswigThat 200G IB is taking a while to materialise21:37
jandersI meant FDR10/40 sorry21:38
jandersthat's right!21:38
jandersI bet it's got something to do with PCIe4 taking sweet forever too21:38
jandershave you guys had a chance to do any testing on HDR/200?21:39
oneswignot here21:39
janderssame here for now21:39
trandleswe're fully in OPA space on our latest machines, so no HDR here21:40
jandersfrom what I hear it should be very similar to drive to EDR as far as OpenStack is concerned, but I bet there'll be some catch somewhere :)21:41
janderstrandles: do you have any OpenStack on OPA?21:41
oneswigI've been working on OPA and Ceph again - a last push to get it working well21:41
jandersOPA doesn't have virtual function support or fabric segmentation support, right?21:42
trandlesI do not have OpenStack on OPA :(21:42
oneswigjanders: you want to speak to rbudden on OPA+OpenStack21:42
trandlesAFAIK OPA doesn't support SR-IOV, no21:42
oneswigI believe you're right21:42
trandlesbut +1 what oneswig says21:42
trandlesPSC's Bridges cluster is OPA, albeit a very early OPA21:43
jandersI don't believe I've met rbudden yet21:44
jmlowejanders: headed to sc'18?21:45
jandersno, OS Berlin21:45
trandlesjanders: https://github.com/jlothian/sysconfidence21:45
trandlesthat's the benchmark21:45
oneswigthanks trandles, might bookmark that myself21:46
jandersthanks! :)21:46
*** jamesmcarthur has quit IRC21:46
oneswigOK do we have any more AOB?21:47
trandlesI believe it was written by Oak Ridge originally.  Meant to stress the fabric on a traditional cluster.  It would be good to see what folks are getting on our scientific clouds, especially if you can test baremetal and from within VMs.21:47
oneswigtrandles: what does a good result look like?21:48
trandlesoneswig: I'll poke at our local expert and get back to you.  He fired off the link quickly and said "talk later."21:48
jandersI'll probably need to audit the HCAs switches and cables to make sure there's no FDR40 to get a good number21:49
oneswigOK thanks trandles.21:49
jandersfor multinode workloads on IB any inconsistencies are asking for trouble21:49
trandlesit's focused on latency, so in theory we should know lowest from the vendor21:50
oneswigShall we close the meeting?  Any more to add?21:51
trandlesnone from me, thanks everyone21:51
jandersI'm good - thanks guys!21:51
oneswigOK, it's a wrap...21:51
oneswig#endmeeting21:51
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"21:52
openstackMeeting ended Tue Oct 30 21:51:59 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)21:52
openstackMinutes:        http://eavesdrop.openstack.org/meetings/scientific_sig/2018/scientific_sig.2018-10-30-21.04.html21:52
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/scientific_sig/2018/scientific_sig.2018-10-30-21.04.txt21:52
openstackLog:            http://eavesdrop.openstack.org/meetings/scientific_sig/2018/scientific_sig.2018-10-30-21.04.log.html21:52
oneswigthanks all, speak in this timezone in ~4 weeks21:52
*** trandles has quit IRC21:53
*** rbudden has quit IRC21:53
*** munimeha1 has quit IRC21:55
*** ttsiouts has joined #openstack-meeting21:55
*** ttsiouts has quit IRC22:04
*** ttsiouts has joined #openstack-meeting22:05
*** ykatabam has joined #openstack-meeting22:07
*** david-lyle has quit IRC22:08
*** ttsiouts has quit IRC22:10
*** yamamoto has joined #openstack-meeting22:10
*** yamamoto has quit IRC22:15
*** oneswig has quit IRC22:23
*** dklyle has joined #openstack-meeting22:28
*** erlon has joined #openstack-meeting22:35
*** e0ne has joined #openstack-meeting22:49
*** tssurya has quit IRC22:51
*** janders has quit IRC22:52
*** fanzhang has joined #openstack-meeting22:54
*** tonyb has joined #openstack-meeting23:03
*** e0ne has quit IRC23:13
*** tpsilva has quit IRC23:18
*** yamamoto has joined #openstack-meeting23:23
*** Liang__ has joined #openstack-meeting23:28
*** armax has quit IRC23:29
*** Liang__ is now known as LiangFang23:30
*** ykatabam has quit IRC23:34
*** erlon has quit IRC23:35
*** ykatabam has joined #openstack-meeting23:38
*** erlon has joined #openstack-meeting23:48
*** hongbin has quit IRC23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!