Tuesday, 2020-01-07

*** jamesmcarthur has joined #openstack-meeting00:02
*** slaweq has joined #openstack-meeting00:11
*** slaweq has quit IRC00:15
*** ricolin_ has joined #openstack-meeting00:44
*** ricolin_ has quit IRC00:45
*** jamesmcarthur has quit IRC00:58
*** jamesmcarthur has joined #openstack-meeting01:15
*** ykatabam has quit IRC01:46
*** igordc has joined #openstack-meeting01:49
*** jamesmcarthur has quit IRC02:02
*** zbr_ has joined #openstack-meeting02:04
*** zbr has quit IRC02:07
*** zbr has joined #openstack-meeting02:10
*** slaweq has joined #openstack-meeting02:11
*** zbr_ has quit IRC02:11
*** slaweq has quit IRC02:16
*** mhen has quit IRC02:25
*** mrda has joined #openstack-meeting02:33
*** igordc has quit IRC02:43
*** slaweq has joined #openstack-meeting02:55
*** slaweq has quit IRC02:59
*** ykatabam has joined #openstack-meeting03:07
*** slaweq has joined #openstack-meeting03:11
*** rcernin_ has joined #openstack-meeting03:13
*** rcernin has quit IRC03:13
*** slaweq has quit IRC03:16
*** hongbin has joined #openstack-meeting03:34
*** rcernin_ has quit IRC03:56
*** rcernin has joined #openstack-meeting04:00
*** kiyofujin has joined #openstack-meeting04:03
*** ykatabam has quit IRC04:24
*** hongbin has quit IRC04:24
*** ykatabam has joined #openstack-meeting04:43
*** kiyofujin has quit IRC04:56
*** gyee has quit IRC05:00
*** slaweq has joined #openstack-meeting05:11
*** slaweq has quit IRC05:16
*** ociuhandu has joined #openstack-meeting05:30
*** ociuhandu has quit IRC05:35
*** links has joined #openstack-meeting05:46
*** pcaruana has joined #openstack-meeting06:00
*** pcaruana has quit IRC06:14
*** ykatabam has quit IRC06:53
*** rcernin has quit IRC06:54
*** e0ne has joined #openstack-meeting07:09
*** slaweq has joined #openstack-meeting07:11
*** slaweq has quit IRC07:17
*** jawad_axd has joined #openstack-meeting07:18
*** keiko-k has joined #openstack-meeting07:30
*** njohnston_ has joined #openstack-meeting07:40
*** njohnston has quit IRC07:41
ricolinmulti-arch initial meetup in 20 mins, feel free to join:)07:43
*** e0ne has quit IRC07:46
*** joxyuki37 has joined #openstack-meeting07:49
*** ralonsoh has joined #openstack-meeting07:49
ricolinforgot to mention the meeting will host in openstack-meeting-alt:)07:57
joxyuki37#startmeeting tacker08:02
openstackMeeting started Tue Jan  7 08:02:14 2020 UTC and is due to finish in 60 minutes.  The chair is joxyuki37. Information about MeetBot at http://wiki.debian.org/MeetBot.08:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.08:02
*** openstack changes topic to " (Meeting topic: tacker)"08:02
openstackThe meeting name has been set to 'tacker'08:02
*** slaweq has joined #openstack-meeting08:02
joxyuki37#topic roll call08:03
*** openstack changes topic to "roll call (Meeting topic: tacker)"08:03
joxyuki37PTL seems unvailable today so I will take after him.08:03
keiko-kHello08:04
joxyuki37hi keiko-k, waiting for other attendees...08:04
*** takahashi-tsc has joined #openstack-meeting08:07
*** brinzhang has joined #openstack-meeting08:09
joxyuki37seems no more attendees.08:16
joxyuki37keiko-k, takahashi-tsc, do you have anything to talk today?08:16
joxyuki37#topic OpenDiscussion08:17
*** openstack changes topic to "OpenDiscussion (Meeting topic: tacker)"08:17
keiko-kjoxyuki37 Nothing I have to talk.08:17
*** tesseract has joined #openstack-meeting08:17
takahashi-tscWe write Event alarm support spec.08:19
takahashi-tschttps://review.opendev.org/#/c/700167/08:19
takahashi-tscI'm wondering wheather the specification is correct or not... so I'd be glad if you get a review.08:21
joxyuki37takahashi-tsc: noted, I will review it in days.08:22
takahashi-tscI think implementation is possible. I'd like you to check specification, especially vnfd change.08:22
takahashi-tscThat's all from me.08:24
*** rsimai has joined #openstack-meeting08:25
joxyuki37thanks. the porpose of the spec looks reasnable. I will review it in detail.08:26
takahashi-tscThank you.08:27
joxyuki37seems no other topic today. let's end the meeting. thanks guys.08:27
joxyuki37#endmeeting08:28
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"08:28
openstackMeeting ended Tue Jan  7 08:28:47 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)08:28
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tacker/2020/tacker.2020-01-07-08.02.html08:28
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tacker/2020/tacker.2020-01-07-08.02.txt08:28
openstackLog:            http://eavesdrop.openstack.org/meetings/tacker/2020/tacker.2020-01-07-08.02.log.html08:28
*** pcaruana has joined #openstack-meeting08:29
*** takahashi-tsc has quit IRC08:31
*** links has quit IRC08:34
*** rpittau|afk is now known as rpittau08:36
*** links has joined #openstack-meeting08:37
*** ykatabam has joined #openstack-meeting08:45
*** joxyuki37 has quit IRC08:46
*** priteau has joined #openstack-meeting08:47
*** tridde has quit IRC08:53
*** tetsuro has quit IRC08:53
*** tetsuro has joined #openstack-meeting08:54
*** trident has joined #openstack-meeting08:55
*** maohongbo has joined #openstack-meeting08:56
*** jawad_axd has quit IRC08:57
*** maohongbo1 has joined #openstack-meeting09:00
*** brinzhang_ has joined #openstack-meeting09:01
*** maohongbo has quit IRC09:01
*** maohongbo1 is now known as maohongbo09:01
*** brinzhang_ has quit IRC09:03
*** brinzhang_ has joined #openstack-meeting09:03
*** jawad_axd has joined #openstack-meeting09:03
*** jawad_axd has quit IRC09:03
*** brinzhang has quit IRC09:04
*** slaweq_ has joined #openstack-meeting09:05
*** keiko-k has quit IRC09:06
*** dmellado has quit IRC09:07
*** slaweq has quit IRC09:07
*** dmellado has joined #openstack-meeting09:08
*** brinzhang has joined #openstack-meeting09:08
*** brinzhang_ has quit IRC09:11
*** maohongbo1 has joined #openstack-meeting09:14
*** maohongbo has quit IRC09:16
*** maohongbo1 is now known as maohongbo09:16
*** brinzhang_ has joined #openstack-meeting09:22
*** brinzhang has quit IRC09:25
*** hyunsikyang has joined #openstack-meeting09:32
*** hyunsikyang__ has quit IRC09:36
*** maohongbo1 has joined #openstack-meeting09:42
*** maohongbo has quit IRC09:43
*** maohongbo1 is now known as maohongbo09:43
*** ociuhandu has joined #openstack-meeting10:00
*** brinzhang has joined #openstack-meeting10:02
*** brinzhang_ has quit IRC10:05
*** ociuhandu has quit IRC10:08
*** slaweq_ is now known as slaweq10:16
*** ykatabam has quit IRC10:37
*** priteau has quit IRC10:37
*** ociuhandu has joined #openstack-meeting10:39
*** links has quit IRC10:45
*** links has joined #openstack-meeting10:49
*** dviroel has joined #openstack-meeting10:50
*** cschwede has quit IRC10:54
*** ociuhandu has quit IRC10:54
*** ykatabam has joined #openstack-meeting10:54
*** ociuhandu has joined #openstack-meeting10:55
*** ociuhandu has quit IRC10:56
*** ociuhandu has joined #openstack-meeting10:56
*** brinzhang_ has joined #openstack-meeting10:58
*** tetsuro has quit IRC11:01
*** brinzhang has quit IRC11:01
*** ociuhandu has quit IRC11:02
*** ociuhandu has joined #openstack-meeting11:05
*** ociuhandu has quit IRC11:06
*** ociuhandu has joined #openstack-meeting11:06
*** ociuhandu has quit IRC11:11
*** jraju__ has joined #openstack-meeting11:13
*** links has quit IRC11:13
*** brinzhang has joined #openstack-meeting11:14
*** rpittau is now known as rpittau|bbl11:16
*** brinzhang_ has quit IRC11:17
*** brinzhang_ has joined #openstack-meeting11:22
*** ykatabam has quit IRC11:24
*** brinzhang has quit IRC11:26
*** Lucas_Gray has joined #openstack-meeting11:40
*** enriquetaso has joined #openstack-meeting11:59
*** bbowen has quit IRC12:03
*** zbr has quit IRC12:11
*** zbr_ has joined #openstack-meeting12:11
*** lpetrut has joined #openstack-meeting12:33
*** rfolco has joined #openstack-meeting12:45
*** jawad_axd has joined #openstack-meeting12:58
*** bbowen has joined #openstack-meeting13:03
*** brinzhang has joined #openstack-meeting13:09
*** brinzhang_ has quit IRC13:13
*** e0ne has joined #openstack-meeting13:20
*** e0ne has quit IRC13:25
*** e0ne_ has joined #openstack-meeting13:25
*** rpittau|bbl is now known as rpittau13:26
*** Lucas_Gray has quit IRC13:30
*** Lucas_Gray has joined #openstack-meeting13:42
ricolinauto-scaling SIG and self-healing SIG meeting will start in 15 mins in  #openstack-meeting feel free to join us:)13:44
*** Lucas_Gray has quit IRC13:46
*** priteau has joined #openstack-meeting13:46
*** liuyulong has joined #openstack-meeting13:47
*** diablo_rojo has joined #openstack-meeting13:47
*** Lucas_Gray has joined #openstack-meeting13:49
*** ociuhandu has joined #openstack-meeting13:54
*** Lucas_Gray has quit IRC13:55
*** e0ne_ has quit IRC13:55
*** Lucas_Gray has joined #openstack-meeting13:59
ricolin#startmeeting self_healing14:00
openstackMeeting started Tue Jan  7 14:00:12 2020 UTC and is due to finish in 60 minutes.  The chair is ricolin. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
*** openstack changes topic to " (Meeting topic: self_healing)"14:00
openstackThe meeting name has been set to 'self_healing'14:00
ricolin#topic roll call14:00
*** openstack changes topic to "roll call (Meeting topic: self_healing)"14:00
ricolinThis meeting is for auto-scaling and self-healing both:)14:00
*** ociuhandu has quit IRC14:01
ricolinlooks like we won't get enough people to start meeting today14:03
*** brinzhang_ has joined #openstack-meeting14:03
ricolinfor meeting logs, we now start the effort to combine two SIGs into one, and merge the meeting, events will be the first step. We need some discussion on what else we should merge too14:04
ricolinAnd if you missed the meeting, it will be biweekly-even meeting schedule as shows in http://eavesdrop.openstack.org/#Self-healing_SIG_and_Auto-scaling_SIG_Meeting14:05
ricolin#link http://eavesdrop.openstack.org/#Self-healing_SIG_and_Auto-scaling_SIG_Meeting14:05
*** brinzhang has quit IRC14:06
ricolinCI jobs for autoscaling with heat and monasca is currently WIP, will build zuul job in repo once we have the test ready, and than we start to ask projects to cross-project gating with it if possible14:08
*** witek has joined #openstack-meeting14:09
ricolinand besides CI the guildeline should start next phase tasks too which IMO should include some detail guideline14:09
ricolinas shows in story14:09
ricolin#link https://storyboard.openstack.org/#!/story/200704614:09
ricolinhere's story for cross-project CI if you're interested in14:09
ricolin#link https://storyboard.openstack.org/#!/story/200575214:10
witekhi ricolin, sorry for being late14:11
ricolinNP14:11
ricolinwe have this scheduled in rush14:11
ricolinthe meeting patch actually landed few hours ago:)14:11
*** ociuhandu has joined #openstack-meeting14:11
witekthanks for working on auto-scaling testing jobs14:12
ricolinwitek, did you see my comments yet?14:12
witekyes, I pointed to your changes in Monasca meeting one hour ago14:13
ricolinthe test in https://review.opendev.org/#/c/700970 still fail because I miss-written the template, which I will correct it later today14:14
ricolinwitek, how's that goes14:14
witekwhat do you mean?14:15
ricolinI mean, you pointed my changes in Monasca meeting, is there's any feedback for those changes14:16
witekSumit has promised to review the changes14:16
ricolincool14:17
witekyou also got a review comment from Doug14:17
ricolinokay, will take a look14:17
ricolinthanks14:17
ricolin#topic open discussion14:17
*** openstack changes topic to "open discussion (Meeting topic: self_healing)"14:18
ricolinbesides the CI job14:18
ricolinIt will be great if we can start putting some detail guideline down too14:18
witekwhere are the necessary libraries getting installed in these CI jobs for heat_tempest_plugin?14:19
witekin particular python-monascaclient?14:19
*** pcaruana has quit IRC14:20
*** eharney has joined #openstack-meeting14:21
ricolinit's collected when install any project require python-monascaclient14:21
witekthat would be heat itself14:21
witekand because listed in Zuul's 'required-projects' it should use HEAD, right?14:22
*** ociuhandu has quit IRC14:22
witekand not tagged version14:23
ricolinyes14:23
ricolinif we check under /opt/stack/python-monascaclient it will be master14:23
ricolinand pip install will directly go to master branch14:24
*** njohnston_ is now known as njohnston14:24
*** brinzhang has joined #openstack-meeting14:24
ricolincorrection,  /opt/stack/python-monascaclient it will HEAD we point to14:26
witekso https://review.opendev.org/700886 should be testing your fix from python-monascaclient now, but it still fails14:27
ricolinindeed14:27
*** brinzhang_ has quit IRC14:28
ricolinmaybe that patch is not working too, I thought pip might be get the master branch from opendev site, but it directly use HEAD too https://982697e52cb2ad6aee54-6c5d61f7d9b09f80ec4de0e8f6953df6.ssl.cf5.rackcdn.com/700970/7/check/heat-functional/0d22f09/controller/logs/pip3-freeze.txt14:28
witekyes, that's why I was wondering if there is some other magic pulling in the released client version14:30
*** jraju__ has quit IRC14:30
ricolineverything we test was installed under /opt/stack/tempest/.tox/tempest14:30
ricolinwe can check if something override in that tox env14:31
*** pcaruana has joined #openstack-meeting14:32
*** ociuhandu has joined #openstack-meeting14:33
ricolinwill update the test template later, see if got time for monascaclient14:35
ricolinfeel free to udpate that patch as you wish too14:35
witekwill have a look at this as well, thanks a lot for your effort14:36
ricolinit's only 1.5 hours for me today:)14:36
*** Lucas_Gray has quit IRC14:36
ricolinNP14:36
*** brinzhang_ has joined #openstack-meeting14:36
*** brinzhang_ has quit IRC14:38
ricolinI also think we should take actions to invite members from related project teams to join this meeting14:38
*** brinzhang_ has joined #openstack-meeting14:38
*** witek has left #openstack-meeting14:39
*** witek has joined #openstack-meeting14:39
*** brinzhang has quit IRC14:40
*** brinzhang_ has quit IRC14:40
witekricolin: good point14:40
*** brinzhang_ has joined #openstack-meeting14:41
*** jeremyfreudberg has joined #openstack-meeting14:42
*** brinzhang_ has quit IRC14:42
ricolin#action ricolin will invite members from related project teams to join next meeting14:42
*** ociuhandu has quit IRC14:42
*** brinzhang_ has joined #openstack-meeting14:43
ricolin#action witek and ricolin works on the Heat+Monasca CI job14:43
ricolin#action we need to find people to volunteer to work on detail guideline14:43
ricolinanything else?:)14:43
witekI guess it's a good start14:44
ricolinwitek, think so too:)14:45
ricolinOkay, let's end this meeting time14:46
ricolinthanks witek14:46
witekthanks Rico, see you around14:46
ricolin#endmeeting14:46
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"14:47
openstackMeeting ended Tue Jan  7 14:46:59 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:47
openstackMinutes:        http://eavesdrop.openstack.org/meetings/self_healing/2020/self_healing.2020-01-07-14.00.html14:47
ricolinsee you tomorrow:)14:47
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/self_healing/2020/self_healing.2020-01-07-14.00.txt14:47
openstackLog:            http://eavesdrop.openstack.org/meetings/self_healing/2020/self_healing.2020-01-07-14.00.log.html14:47
*** jawad_axd has quit IRC14:49
*** jawad_axd has joined #openstack-meeting14:50
*** hrw has joined #openstack-meeting14:50
*** zbr_ is now known as zbr|rover14:50
*** witek has quit IRC14:51
ricolinMulti-Arch SIG  meeting in 8 mins:) feel free to join us:)14:53
*** jawad_ax_ has joined #openstack-meeting14:53
*** Lucas_Gray has joined #openstack-meeting14:54
ricolinalso logs is generated for our meeting earlier today http://eavesdrop.openstack.org/meetings/multi_arch/14:55
*** jawad_axd has quit IRC14:55
ricolintake a look if you like:)14:55
ricolinfor other information see https://etherpad.openstack.org/p/Multi-arch14:55
*** jawad_ax_ has quit IRC14:58
ricolin#startmeeting multi_arch15:00
openstackMeeting started Tue Jan  7 15:00:23 2020 UTC and is due to finish in 60 minutes.  The chair is ricolin. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: multi_arch)"15:00
openstackThe meeting name has been set to 'multi_arch'15:00
ricolin#topic roll call15:00
jeremyfreudbergo/15:00
*** openstack changes topic to "roll call (Meeting topic: multi_arch)"15:00
ricolinjeremyfreudberg, o/15:00
ricolinhrw, :)15:01
ricolinjeremyfreudberg, for some quick sync with you, we discuss about creating Storyboard and repo for documentation for multi-arch15:02
ricolinand besides current CI resources, we will also have around 400 vcpus environments to join CI too15:03
jeremyfreudbergnice15:04
ricolinalso we got mrda to help on ppc64le arch too15:05
hrwricolin: s/400 vcpus/400 arm64 vcpus/15:05
jeremyfreudberghrw, right, i figured15:05
hrwit is multiarch sig. not aarch64 sg15:05
ricolinso it might not just ARM64 but also ppc64le too15:05
hrwit is multiarch sig. not aarch64 sig15:05
ricolinoops:)15:05
jeremyfreudbergricolin: yeah mrda is on my team here at RH with tonyb15:06
jeremyfreudbergi'm glad he is also participating15:06
ricolinme too15:06
hrwjeremyfreudberg: OSP team?15:06
jeremyfreudberghrw: yes, the three of us do multiarch stuff for red hat's openstack product15:07
hrwjeremyfreudberg: I am in RH ARM team. platform enablement. assigned to Linaro where I work on arm64 in openstack15:07
jeremyfreudberghrw: right, i think i've seen your (real) name in some internal mailing lists, good to be talking to you out here in the community15:08
*** ociuhandu has joined #openstack-meeting15:08
ricolinalso there's documents idea for `oh, on arm64 you need to do XYZ in other way - here is how and why` and `use cases, who, and issues they had`15:09
ricolinthat's almost all we covered in last meeting15:09
ricolin#topic greeting:)15:09
*** openstack changes topic to "greeting:) (Meeting topic: multi_arch)"15:09
hrwI will be mostly away - trying to sort out some issue15:10
ricolinhrw, got it15:10
jeremyfreudbergi don't have too much to say today15:10
jeremyfreudbergi read the log from the earlier meeting15:10
jeremyfreudbergit seems we are on the right track15:10
ricolinjeremyfreudberg, NP, we mostly just like to check if there's any plan or idea for init this SIG15:11
ricolinand also make sure the meeting schedule is correct comfortable to people15:12
ricolindidn't got chance to check with tonyb15:12
ricolinbut will sync with him once he back to works15:12
ricolinalso we need to find candidate for chairs15:13
jeremyfreudbergright15:13
jeremyfreudbergi definitely endorse tonyb15:13
ricolinneed to check with ianw_pto and tonyb to see if any of them is interested15:13
ricolinI think tonyb will be a good candidate to lead this effort too15:13
jeremyfreudberghe has been in the community a long time and is the ringleader of my little group at red hat15:14
jeremyfreudberghe is also very busy all the time, so we will wait to see what he says15:14
ricolinwe can have about three chairs, and I'm looking for at least two chairs besides me15:15
ricolinI'm find to step down the chair role as soon as all these on track:)15:15
ricolinso if tonyb is okay with the role we should find one more IMO15:17
ricolinjeremyfreudberg, got any thoughts for at current stage of this SIG?:)15:20
jeremyfreudbergricolin: i was just thinking it would be good to have as much diversity among chairs as possible. diversity of company and diversity of architecture knowledge15:21
ricolin+115:22
*** ociuhandu has quit IRC15:22
jeremyfreudbergbesides that, i think we are good, i am happy to help in any way15:22
jeremyfreudbergi look forward to seeing the creation of repo for docs15:23
jeremyfreudbergi am excited to contribute there15:23
ricolinjeremyfreudberg, are you able to help on working on the docs together?:)15:23
ricolinThere's two kind of docs idea pop up from last meeting15:24
ricolindocs like `oh, on arm64 you need to do XYZ in other way - here is how and why` and `use cases, who, and issues they had`15:24
ricolinwhich I think will be good to have as inital docuemnts and build more content from there15:25
jeremyfreudbergricolin: i can help with all of that...15:26
ricolinI already take actions to build repo and storyboard, once they're ready, we can add tasks in storyboard and docs in repo15:27
jeremyfreudbergcool15:27
ricolinjeremyfreudberg, that will be super!15:27
ricolin#action jeremyfreudberg will help with docs once docs repo is ready:)15:28
jeremyfreudbergyup15:28
ricolinthanks jeremyfreudberg :)15:29
ricolinalso Multi-Arch SIG is officially founded15:30
ricolinyou can find it in https://governance.openstack.org/sigs/ now15:30
ricolinplease take a look if the SIG scope looks correct to you15:31
*** lpetrut has quit IRC15:31
ricolin#action ricolin sync today's meeting info with people who signed up in etherpad https://etherpad.openstack.org/p/Multi-arch15:32
ricolinjeremyfreudberg, anything else you think we should cover today?15:33
jeremyfreudbergricolin: i think the description is good. but i wonder, doe the average contributor know what "multiple architecture" really means? maybe we can make clear in the description that we care about cpu arch15:34
jeremyfreudbergit's a very minor point, but i just like things to be as clear as possible15:34
ricolinjeremyfreudberg, I think it's a good idea15:35
jeremyfreudbergricolin: i can propose a change for it this week15:35
ricolincool15:36
ricolinjust change it in https://opendev.org/openstack/governance-sigs/src/branch/master/sigs.yaml15:36
jeremyfreudbergyup15:36
jeremyfreudbergok, i don't have anything else to say today15:37
ricolin#action jeremyfreudberg will help to update governance-sigs to make more clear description for Multi-Arch SIG15:37
ricolinthat's already way to productive as the very first two meetings:D15:37
jeremyfreudbergyes, glad to see so much progress and participation already15:38
jeremyfreudberg:D15:38
ricolinme too:)15:38
hrw;)15:38
ricolinsince we running biweekly meeting, it's cool to see we have something to achieve15:39
ricolinbtw the meeting ics can be found in http://eavesdrop.openstack.org/#Multi-Arch_SIG_Meeting :)15:39
ricolinOkay let's end the meeting and call it a day:)15:41
jeremyfreudbergyup, bye all!15:41
ricolinthanks jeremyfreudberg hrw :)15:41
ricolin#endmeeting15:41
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"15:41
openstackMeeting ended Tue Jan  7 15:41:19 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:41
openstackMinutes:        http://eavesdrop.openstack.org/meetings/multi_arch/2020/multi_arch.2020-01-07-15.00.html15:41
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/multi_arch/2020/multi_arch.2020-01-07-15.00.txt15:41
openstackLog:            http://eavesdrop.openstack.org/meetings/multi_arch/2020/multi_arch.2020-01-07-15.00.log.html15:41
hrwthanks folks15:41
*** hrw has left #openstack-meeting15:41
*** jeremyfreudberg has left #openstack-meeting15:41
*** witek has joined #openstack-meeting15:42
*** Lucas_Gray has quit IRC15:54
slaweq#startmeeting neutron_ci16:00
openstackMeeting started Tue Jan  7 16:00:17 2020 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: neutron_ci)"16:00
slaweqhi16:00
openstackThe meeting name has been set to 'neutron_ci'16:00
njohnstono/16:00
ralonsohhi16:01
*** witek has left #openstack-meeting16:01
bcafarelo/16:01
slaweqok, lets start as we have our regular attendees already16:02
slaweqGrafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate16:02
slaweq#topic Actions from previous meetings16:02
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"16:02
*** eharney has quit IRC16:02
slaweqactually this time those are "actions from previous year" even :)16:03
slaweqand the first one is16:03
slaweqnjohnston to check failing NetworkMigrationFromHA in multinode dvr job16:03
*** jawad_axd has joined #openstack-meeting16:04
slaweqI think we lost njohnston now16:04
*** ociuhandu has joined #openstack-meeting16:05
njohnstonsorry, network hiccup.  I'll work on this later today16:05
slaweqok :)16:06
slaweqso I will set it as reminder for next week too16:06
slaweq#action njohnston to check failing NetworkMigrationFromHA in multinode dvr job16:06
bcafarelas long as it is not for next year :)16:06
slaweqbcafarel: LOL, this time no16:07
slaweqonly for next week16:07
slaweqnext one16:07
slaweqslaweq to talk with gmann about finish of tempest plugins migration16:07
slaweqI talked with gmann about it today16:07
slaweqso he is aware of that already16:08
slaweqand that's all actions from last meeting on my list16:08
slaweqlets move on16:08
slaweq#topic Stadium projects16:08
*** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)"16:08
slaweqnjohnston: any updates about dropping py2 support?16:08
njohnstonI took a look at the outstanding work16:09
njohnstonWe have work proceeding on networking-odl, networking-bagpipe, networking-midonet, and neutron-fwaas16:09
njohnstonalthough I am not sure the author of the fwaas change is engaged anymore16:09
njohnstonmidonet in particular is close to a merge16:09
njohnstonso the stadium is in a good position16:10
slaweqthat's good16:10
njohnstonthe only remaining ones are neutron-lib and neutron proper16:10
njohnstonI will file changes for those after the meeting16:10
slaweqI can take care of those16:10
slaweqahh, ok16:10
slaweqthx16:10
bcafarelso good progress overall then?16:10
njohnstonthere is one existing for neutron-lib but it includes things like removing six and other 2.7-hostile changes, which I want to separate out16:10
slaweqyes, I remember that one16:11
njohnstonso I think I will just edit that change16:11
slaweq+1 for that16:11
njohnstonbcafarel: Yes, definitely good progress16:11
slaweqthx njohnston for the update16:12
slaweqthat's really great work16:12
njohnstonas a team we are getting good at pulling together for these community-wide efforts :-)16:13
*** ociuhandu has quit IRC16:13
slaweqanything else related to the stadium projects You want to discuss today?16:13
njohnstonFor the zuulv3 migration, the standouts are networking-bgpvpn, midonet, neutron-dynamic-routing, and neutron-vpnaas16:14
njohnstonthose are the ones that still need work, otherwise everything is covered16:14
*** ociuhandu has joined #openstack-meeting16:15
slaweqok, I will try to find finally some time to work on at least some of them16:15
*** gyee has joined #openstack-meeting16:16
*** rfolco is now known as rfolcOUT16:16
slaweqok, next topic16:16
slaweq#topic Grafana16:16
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"16:16
slaweq#link http://grafana.openstack.org/dashboard/db/neutron-failure-rate16:16
slaweqwe don't have too much data from last days, especially from gate queue16:17
ralonsohyeah, just waiting for https://review.opendev.org/#/c/701018/16:17
slaweqin check queue, the biggest problem IMO are thise neutron-tempest-plugin failures16:18
slaweqralonsoh: exactly16:18
njohnstonyes, definitely, those are killer16:18
ralonsohI would delay any CI errors analysis until we have this patch16:19
ralonsoh(but I'm very lazy too)16:19
bcafarellaziness can be good :)16:19
slaweqralonsoh: this bug affects "only" neutron-tempest-plugin jobs16:19
slaweqand in fact other jobs are in pretty good shape (at lest not worts than usually)16:20
njohnstonralonsoh: Her, laziness is #1 on Larry Wall's three virtues of a great programmer16:20
njohnston*hey16:20
ralonsohhahah16:20
slaweq:)16:20
slaweqanything else related to the grafana?16:21
slaweqor can we move on to the next topic?16:21
ralonsohok for me16:22
njohnston+1 to move on16:22
slaweqok16:22
slaweq#topic fullstack/functional16:22
*** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)"16:22
slaweqI was looking today on recent failures16:22
slaweqand I found 3 various failed tests which we may check now16:23
slaweqfirst one:16:23
slaweqneutron.tests.functional.agent.test_firewall.FirewallTestCase.test_ingress_udp_rule16:23
*** electrofelix has joined #openstack-meeting16:23
slaweqhttps://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_f37/697317/8/check/neutron-functional/f37a608/testr_results.html.gz16:23
slaweqdid You ever saw such failure before?16:25
ralonsohnot very usual, do you want to implement some kind of retrial context?16:25
njohnstonthat is new and different to me16:26
ralonsohbut, IMO, the connection handler under the hood should be enough16:26
*** ociuhandu has quit IRC16:28
slaweqok, lets just be aware of it for now16:28
slaweqmaybe it will not happen often (or even will not happen at all anymore :))16:28
slaweqok, next one16:30
slaweqthose are 2 failures of different tests, but error looks kind of similar IMO16:30
slaweqhttps://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ea8/677092/11/check/neutron-functional/ea8fe46/testr_results.html.gz16:30
slaweqand https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_19d/701003/1/check/neutron-functional/19d8a0a/testr_results.html.gz16:31
*** eharney has joined #openstack-meeting16:31
slaweqone is timeout during namespace creation (again)16:31
slaweqand second is timeout during bridge creation16:31
slaweqbut it's linuxbridge16:31
slaweqso IMHO netlink is common part of both of those issues16:32
slaweqdid You saw similar issues before?16:33
ralonsohyes, many times16:33
ralonsohat least with the ns creation16:33
ralonsohnot so many, but some of them, related to the br creation16:34
slaweqok, so I think we should open bug for that, right?16:35
ralonsohyes, I'll do it16:35
ralonsohat least the one related to the bridge16:35
ralonsohthe problem is I have spent days with those privsep/netlink errors16:35
ralonsohand I don't see the cause16:36
slaweqralonsoh: I know You were debugging and trying to fix it16:36
slaweqit's not so often now but still happens from time to time16:36
*** ociuhandu has joined #openstack-meeting16:37
slaweqralonsoh: thx for reporting this on LP16:37
slaweq#action ralonsoh to report bug for timeout related to bridge creation16:38
slaweqok, that's all from my side about functional/fullstack jobs16:38
slaweqanythig else You want to add here?16:38
ralonsohno16:38
bcafarelme neither16:39
slaweqok, lets move on16:40
slaweq#topic Tempest/Scenario16:40
*** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)"16:40
slaweqwe already talked about issue with neutron-tempest-plugin jobs16:40
slaweqI also reported today new bug for dvr jobs: https://bugs.launchpad.net/neutron/+bug/185864216:40
openstackLaunchpad bug 1858642 in neutron "paramiko.ssh_exception.NoValidConnectionsError error cause dvr scenario jobs failing" [High,Confirmed]16:40
slaweqit happens quite often that tests fails due to such error16:40
slaweqand I saw it only in dvr jobs (which are non-voting fortunatelly)16:41
slaweqbut if someone would have some time, it would be good to take a look16:41
*** rsimai has quit IRC16:41
slaweqnext problen is that neutron-tempest-plugin-fwaas job is unstable16:42
slaweqI reported it here: https://bugs.launchpad.net/neutron/+bug/185864516:42
openstackLaunchpad bug 1858645 in neutron "Neutron-fwaas tempest job is not stable" [Critical,Confirmed]16:42
slaweqand I proposed patch to make those jobs non-voting and remove from gate queue for now16:43
slaweqhttps://review.opendev.org/#/c/701371/16:43
slaweqif someone interested in neutron-fwaas could take a look, that would be great16:43
njohnstonwill do16:44
slaweqthx njohnston16:44
njohnstondespite my lack of interest in neutron-fwaas :-)16:44
slaweq:)16:44
slaweqand that's all what I have for scenario jobs for today16:44
*** ociuhandu has quit IRC16:46
slaweqanything else related to scenario jobs You have or can we move on?16:46
njohnston+1 to move on16:47
slaweqok, last topic for today from me16:47
slaweq#topic Periodic16:47
*** openstack changes topic to "Periodic (Meeting topic: neutron_ci)"16:47
slaweqstill our mariadb job is failing 100% due to issue in Mariadb 10.116:48
slaweqis there anyone who has some cycles to check how to upgrade mariadb in this job?16:48
ralonsohhttps://review.opendev.org/#/c/698980/1/tools/fixup_stuff.sh16:48
ralonsohI think this patch won't fix it16:49
ralonsohbecause fixup_ubuntu is not being called16:49
ralonsohI'll try again this week16:49
slaweqyes, I think it's called only in functional/fullstack jobs16:49
slaweqok, thx ralonsoh16:50
slaweqif You would need any help with this, please ping me16:50
slaweq#action ralonsoh to take a look how to use newer Maridb in periodic job16:50
ralonsohthanks16:50
slaweqok, that's all from my side for today16:51
slaweqanything else You want to discuss today?16:51
ralonsohno thanks16:51
njohnstonnothing more from me16:51
bcafarelnothing here (except happy new ci year)16:51
slaweqhappy new CI year :)16:51
slaweqahh, one thing16:51
*** diablo_rojo has quit IRC16:52
slaweqI just pushed my blog post to be online: http://kaplonski.pl/blog/failed_builds_per_patch/16:52
slaweqit's about number of "rechecks" in our CI16:52
njohnston+116:52
* bcafarel adds to the "to read" tabs16:52
slaweqif You have few minutes, please take a look and tell me if that makes any sense for You and if such metric may be useful for us somehow16:53
ralonsoh+116:53
slaweqand thx njohnston for help with english in this post :)16:53
*** mattw4 has joined #openstack-meeting16:53
njohnstonmy pleasure, your English is already very good :-)16:54
slaweqthx njohnston :)16:54
slaweqok, I think we can finish our meeting now16:54
slaweqthx for attending16:54
slaweqand see You all online16:54
slaweqo/16:54
slaweq#endmeeting16:54
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"16:54
openstackMeeting ended Tue Jan  7 16:54:50 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:54
njohnstono/16:54
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-01-07-16.00.html16:54
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-01-07-16.00.txt16:54
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-01-07-16.00.log.html16:54
ralonsohbye16:55
bcafarelo/16:55
*** pcaruana has quit IRC17:05
*** rpittau is now known as rpittau|afk17:09
*** ociuhandu has joined #openstack-meeting17:15
*** ociuhandu has quit IRC17:25
*** pcaruana has joined #openstack-meeting17:28
*** ociuhandu has joined #openstack-meeting17:31
*** ociuhandu has quit IRC17:37
*** ociuhandu has joined #openstack-meeting17:39
*** priteau has quit IRC17:45
*** ociuhandu has quit IRC17:46
*** jawad_axd has quit IRC17:53
*** simon-AS5591 has joined #openstack-meeting17:58
*** brinzhang has joined #openstack-meeting18:03
*** jawad_axd has joined #openstack-meeting18:04
*** brinzhang_ has quit IRC18:06
*** jawad_axd has quit IRC18:44
*** trident has quit IRC18:48
*** trident has joined #openstack-meeting18:49
*** AJaeger has joined #openstack-meeting18:52
clarkbanyone else here for the infra meeting?19:00
corvuso/19:00
clarkb#startmeeting infra19:01
openstackMeeting started Tue Jan  7 19:01:07 2020 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.19:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:01
*** openstack changes topic to " (Meeting topic: infra)"19:01
openstackThe meeting name has been set to 'infra'19:01
clarkb#link http://lists.openstack.org/pipermail/openstack-infra/2020-January/006568.html Our Agenda19:01
clarkbNot a whole lot on the agenda19:01
clarkbI think most of us are catching up on things after holidays19:01
clarkb#topic Announcements19:01
*** openstack changes topic to "Announcements (Meeting topic: infra)"19:01
zbr|rovero/o/19:01
clarkbHappy new year and welcome back! that is all I had for announcements19:01
AJaegero/19:02
clarkb#topic Actions from last meeting19:02
*** openstack changes topic to "Actions from last meeting (Meeting topic: infra)"19:02
clarkb#link http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-12-17-19.01.txt minutes from last meeting19:02
clarkbNo actions from last meeting. It was pretty low key as I expect today to be.19:02
clarkb#topic Priority Efforts19:02
*** openstack changes topic to "Priority Efforts (Meeting topic: infra)"19:02
clarkb#topic OpenDev19:02
*** openstack changes topic to "OpenDev (Meeting topic: infra)"19:02
clarkbFirst up on this mordred has a change to upgrade gitea to 1.10.2 ready to go19:03
*** Shrews has joined #openstack-meeting19:03
clarkbreading the changelog I don't expect this will fix the weird git bug we've notice19:03
clarkbbut seems to include a fair number of other bug fixes19:03
clarkbThe other opendev subject is the governance draft email thread19:04
clarkb#link http://lists.openstack.org/pipermail/openstack-infra/2019-December/006537.html Governance email19:04
mordredyah. I don't expect it to be a noticable upgrade19:04
clarkbI'd like to bring that up with the openstack TC soon and start moving this process officially19:05
mordredbut figure it's best to keep on the latest to keep deltas down19:05
mordredI did check the templates and there are no relevant template changes19:05
clarkbmaybe give it until next week for further feedback and for people to return from holidays on both our side and the TC side then bring it up in an office hour there?19:05
clarkbfungi: ^ you probably have a good read on the best way to reach out to them. Would starting with a governance change be better?19:05
fungiwhat would the governance change entail?19:07
clarkbmordred: ++ on keeping up with them19:07
fungiremoving projects from the projects.yaml?19:07
clarkbfungi: at the very least we'd edit the governance reference project.yaml file to remove the opdnev bits of infra19:07
fungiyeah, okay, that makes sense19:07
fungii don't see any reason to delay on proposing that change19:08
fungithe official way to "reach out" to the openstack tc is to send e-mail to the openstack-discuss mailing list with [tc] in the subject19:08
fungibut also following up with them in one of the office hours (thursdays are the best-attended) can't hurt19:08
clarkbk, I guess I can propose the change then send email pointing to the change and our thread on the subject and take it from there19:09
clarkbI'll put that on my list of things for next week to give anyone last chance at further input on our thread19:09
fungiand there's a scheduled monthly irc meeting for next week you could get some announcement on the agenda for if you wanted, there's time19:09
clarkb#action clarkb push openstack governance change to pull out opendev and start discussion with tc via openstack-discuss19:09
clarkbAnything else on opendev before we continue?19:11
clarkb#topic Update Config Management19:12
*** openstack changes topic to "Update Config Management (Meeting topic: infra)"19:12
clarkbmordred: I think there has been some movement on review-dev's containerization. The change to switch it over merged but wasn't applied when I last looked (pre holiday) beacuse the hsot was in the emergency file19:12
clarkbmordred: is that still the case or are we automating the containerized deployment of gerrit on -dev now?19:13
mordredno - still the case - I hadn't dug in fully on that yet because holidays19:13
mordredlet's get that finished this week19:13
mordredI landed a few more of the patches19:13
clarkb#action mordred finish up switch to automated container deployment of Gerrit on review-dev. Changes are landing but need to remove -dev from the emergency file.19:14
clarkbmordred: ^ does that capture it?19:14
mordredhttps://review.opendev.org/#/c/691800/ https://review.opendev.org/#/c/692003 and https://review.opendev.org/#/c/691777/ could each use one more reviewer19:14
mordredclarkb: yup19:14
clarkbheh they have three different reviewers too. I've pulled them up and will try to get to the two I haven't read yet19:15
mordredalso unrelated but related, https://review.opendev.org/#/c/690505 needs a 2nd19:15
clarkbAre there any other configuration mgmt migration related changes or topics to discuss?19:16
clarkb#topic General Topics19:18
*** openstack changes topic to "General Topics (Meeting topic: infra)"19:18
*** ralonsoh has quit IRC19:18
clarkbfungi: any movement on the wiki? I expect not considering your boat touring19:18
funginope19:19
fungii need to refresh my memory on where i left off19:19
clarkbk19:19
clarkbFor the static.openstack.org migration ianw has some chagnes that need reviewing19:19
clarkb#link https://review.opendev.org/#/q/status:open+topic:static.opendev.org Next up on static.o.o migration19:19
fungii remember i got openid auth working, but i think i needed to do some more with extensions19:19
clarkbif others can help review those that will be great. They are on my list along with mordred's now19:20
clarkbI think ianw returns next week and would be good to have those changes sorted and ready before that19:20
mordredyay ianw returning!19:21
clarkbThat takes us to project renames. There is a single project requesting a rename.19:22
fungiany idea on the degree of urgency around it?19:22
clarkboh you know what19:23
clarkbI think this ended up being a retire in place then fork if development wanted to continue situation19:23
clarkbrather than a true rename19:23
AJaegerclarkb: that's not a rename, drop it.19:23
clarkbAJaeger: thank you for confirming19:23
fungiyeah, if it was a repo move out of the openstack namespace, then it's not a rename19:23
AJaegerwe retired completely - and reimported19:23
clarkbok we don't need to schedule a downtime19:23
AJaegerso, nothing more for us to do - let me remove from wiki19:23
clarkbI'll update our agenda to reflect this19:23
clarkboh go for it AJaeger thank you19:23
fungiopenstack tc policy is that once a repository is an official deliverable, it can't ever move out of the openstack namespace because consumers of the source code may not discover that it's ceased being part of openstack otherwise19:24
fungiso they require those projects to fork and the originals retired19:25
clarkbI was going to suggest the week ending january 24 as a good gerrit downtime option but that doesn't seem necessary now. That said depending on how mordred's gerrit container work goes that may still be a good opportunity for a downtime to siwtch things over then19:26
fungiseems reasonable to me19:26
mordredclarkb: I will be on PTO the week ending jan 24 - but depending on how the conatiner work is before then maybe?19:27
clarkbmordred: or the week after looks pretty clear too19:27
clarkbno rush!19:27
mordred++19:27
fungii'll be transiting to/from fosdem at the end of the month, so may not be around depending on the day19:28
fungispecifically, i'll be on airplanes the 30th/31st19:28
fungiif that's the proposal19:28
fungibut if others are around to help, that shouldn't be a blocker19:29
clarkbya we'll sort it out once we've got -dev cut over19:29
clarkbThis takes us to our regularly scheduled agenda19:30
clarkb#topic Open Discussion19:30
*** openstack changes topic to "Open Discussion (Meeting topic: infra)"19:30
AJaegerDo we still need these repos: x/pbrx, x/dox, openstack/js-openstack-lib - or shall I retire one of these?19:30
corvusmordred: ^ i'm pretty sure the 1st 2 are retirable?19:30
mordredx/pbrx and x/dox should both be retirable19:31
AJaegerok, will take care of these two19:31
mordredI don't know anything about openstackj/js-openstack-lib - If the horizon team isn't using it19:31
fungimordred: i noticed pbrx is in the openstackclient requirements.txt19:31
mordredI believe that was written originally as an attempt to be like openstacksdk but in javascript19:31
clarkbthe js repo is very quiet, but ya double checking with horizon would probably be a good thing19:31
mordredfungi: aroo?19:32
fungiyou might want to hunt around for anything using pbrx before retiring19:32
mordredoh - I think it's using ... yeah19:32
mordredwait - don't retire it19:32
mordredthere's two things in pbrx - the container stuff which we stoppped using19:32
mordredand the siblings logic - so that people can do a siblings install locally19:32
fungiso maybe pbrx just becomes the latter bits?19:32
mordredyeah - maybe that's the best idea19:33
AJaegeraccording to codesearch nobody uses js-openstack-lib19:33
*** electrofelix has quit IRC19:33
AJaegerBut I can ask amotoki19:33
AJaegerso, no retiring of pbrx - ok19:33
mordredAJaeger: yeah. my hunch is that it's fine19:33
mordred(to retire js-openstack-lib)19:34
mordredbut asking amotoki sounds great19:34
clarkbunrelated, the airship team is trying to figure out how to run their testing on our CI system. The avenue they seem to be pushing for is to add an airship specific cloud to nodepool that can run jobs on very large (60GB ram?) test nodes. They need a lot of memory to run large nested VMs which I've tried to point out is often unreliable and with a single cloud backend would result in lots of19:34
clarkbflakyness. Better instead to run multinode jobs to spread the load on smaller test nodes and then they can run in many clouds. But to get a better understanding of their actual requirements set up a call at 1600UTC thursday with them. I suggested we use room 6001 on pbx19:34
AJaegerSo, my proposal: I'll check with amotoki about js-openstack-lib and then send email to retire x/dox and js-openstack-lib.19:34
clarkbothers are more than welcome to join the call. If they can't get pbx working my plan was to try meet.jit.si as an alternative19:34
Shrewsclarkb: which cloud?19:35
clarkbShrews: I believe it would be a private cloud run by one of their interested parties19:35
clarkb(not a public cloud)19:35
Shrewsah19:35
Shrewsi was about to have objections  :)19:35
fungiyou can still have objections19:36
Shrewsfungi: i object to that19:36
mordredclarkb: obviously it would be better if they could provide two clouds19:36
clarkbmordred: yup! they did mention that could potentially be a possibility :) but sounded like not part of their day one planning19:37
mordredbut - given seems worthwhile to at least have a call about it19:37
fungiobviously it would be better if they could provide two clouds into the aggregate pool and just use normal resources19:37
mordredyeah - it's hard to fund two clouds when there isn't a POC of the system running on the first on eyet I could imagine19:37
fungitripleo-ci cloud syndrome19:38
corvusthe fact that these would be unavailable to other tenants of the system is a concern19:38
clarkbin any case sounds like they may have the testing running internally already but they want to push it upstream if possible and so we need to sort out where the balance is between existing test methods and existing upstream ci resources (that may be augmentable)19:38
clarkband I'm hoping the phone call gives us a better understanding of their side so that we can propose more valuable solutions19:38
mordredsounds like the call will be useful in hammering stuff out19:38
corvusyeah19:39
fungiwe already don't have the ability to prevent other users from running jobs on an environment attached to our nodepool, right?19:39
clarkbfungi: correct19:40
clarkbany restrictions there would have to be via convention at this point I think19:40
fungii'm fine with taking a hard line that tenant-scoped resources (whether enforced or merely unadvertised) is unacceptable for opendev's nodepool19:41
fungicompanies and private individuals are already donating a lot of resources which they generously allow to be used by all projects in opendev19:42
fungibut also maintaining connections to environments exclusive to specific projects is a lot of administrative overhead which only benefits that project19:43
clarkbya I lean that way too. That said I can contrive some examples in my head where restricting access comes with a reasonable argument.19:43
clarkbThings like gpu enabled hardware19:43
fungii think we can ask people not to abuse access to resources, and reach out to them about augmenting those resources if they have a need for them on a regular basis19:44
clarkbyup. Seems reasonable to set guidelines for access to more spceial resources, but then allow anyone that shows a need for the special stuff and follows the guidelines to participate19:46
funginow i just need to try to remember those points so we don't forget to raise them on the call ;)19:46
clarkbAnything else?19:47
clarkbif not I'll probably call the meeting early19:47
fungii've got nothing19:48
clarkb#endmeeting19:49
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"19:49
openstackMeeting ended Tue Jan  7 19:49:12 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)19:49
openstackMinutes:        http://eavesdrop.openstack.org/meetings/infra/2020/infra.2020-01-07-19.01.html19:49
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/infra/2020/infra.2020-01-07-19.01.txt19:49
clarkbThank you everyone!19:49
*** AJaeger has left #openstack-meeting19:49
openstackLog:            http://eavesdrop.openstack.org/meetings/infra/2020/infra.2020-01-07-19.01.log.html19:49
corvusthanks!19:49
*** Shrews has left #openstack-meeting19:49
fungithanks clarkb!19:49
*** brinzhang_ has joined #openstack-meeting19:57
*** brinzhang has quit IRC20:00
*** jawad_axd has joined #openstack-meeting20:03
*** jawad_axd has quit IRC20:09
*** bbowen has quit IRC20:18
*** slaweq has quit IRC20:19
*** Lucas_Gray has joined #openstack-meeting20:30
*** slaweq has joined #openstack-meeting20:35
*** brinzhang has joined #openstack-meeting20:40
*** pcaruana has quit IRC20:41
*** brinzhang_ has quit IRC20:44
*** brinzhang_ has joined #openstack-meeting20:46
*** eharney has quit IRC20:49
*** brinzhang has quit IRC20:49
*** ykatabam has joined #openstack-meeting20:50
*** slaweq has quit IRC20:54
*** brinzhang has joined #openstack-meeting20:54
*** slaweq has joined #openstack-meeting20:55
*** oneswig has joined #openstack-meeting20:55
*** brinzhang has quit IRC20:56
*** brinzhang has joined #openstack-meeting20:57
*** brinzhang_ has quit IRC20:58
oneswighola21:00
oneswig#startmeeting scientific-sig21:01
openstackMeeting started Tue Jan  7 21:01:01 2020 UTC and is due to finish in 60 minutes.  The chair is oneswig. Information about MeetBot at http://wiki.debian.org/MeetBot.21:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:01
*** openstack changes topic to " (Meeting topic: scientific-sig)"21:01
openstackThe meeting name has been set to 'scientific_sig'21:01
oneswigA quiet one for today I think21:01
*** trandles has joined #openstack-meeting21:01
oneswigmartial?21:01
trandlesIs there a meeting today oneswig ?21:02
oneswigHi trandles, perhaps...21:02
trandlesAlso, hello oneswig and happy new year21:02
oneswigright back at ya, hope you had a good break.  All well here.21:02
*** e0ne has joined #openstack-meeting21:02
oneswigAs a resident of the high plains, I expect you'd struggle to picture how bracing the sea is round our way on Christmas day :-)21:03
trandlesYes, thanks, good break. Luce and I spent the holidays at home and put a 3-day trip to San Francisco in between21:03
trandlesI've spent Christmas in Gosport. 2C along the sea feels a LOT colder than 2C in Santa Fe. ;)21:03
oneswigI have no doubt21:04
oneswigIn the office, we have our 6-monthly design summit next week.  Following the discussion last month I put podman onto the list for discussion21:04
trandlesexcellent21:04
trandlesI'm just about (in an hour or so) to deploy Train into production using OpenStack-Ansible.21:05
oneswigOoh, exciting.21:05
oneswigNew system or upgrade of a previous one?21:05
trandlesOther than dealing with gnocchi, it hasn't been too bad. Just the normal lack of documenting of required key variables that need to go into your config.21:05
trandlesNew system21:06
trandlesThe previous was testbed-only21:06
oneswigAs always, I'd be interested to hear how it goes.  Does OSA still build LXC containers on the hosts?21:07
trandlesYes, lxc containers.21:07
oneswigDoes it support SELinux?  Something else on the agenda.21:08
trandlesI don't believe so21:09
trandlesBut I'm likely to be wrong there. I didn't look into it.21:09
oneswigA fiddly problem, every time I get caught out by it.21:11
oneswigIIRC the LXC containers follow the lightweight OS model, right?  Do they also have private network namespaces, IP addresses, etc?21:12
trandlesYes, they have their own IPs21:12
*** bbowen has joined #openstack-meeting21:12
trandlesThey're also running systemd to init multiple processes/container21:13
oneswigMakes sense.21:13
oneswigI was hearing earlier about a new ARM CI resource - apparently it'll be all IPV6 on the control plane (for fun, perhaps?)21:14
trandlesSo there is a nova container with api, conductor, and scheduler all running for instance21:14
*** mmethot has joined #openstack-meeting21:14
trandlesI have a couple ARM nodes in our production cloud.21:15
trandlesHaven't booted them in anger yet though21:15
trandlesOtherwise, haven't done much with ARM myself. We have a couple ARM clusters from Cray...21:16
oneswigI think I heard that somewhere...21:16
*** e0ne has quit IRC21:17
oneswigThe news at this end for today has been a new project testing out Mellanox VF-LAG.  We didn't get to the point of integrating it yet but that should come in tomorrow.  Has promise I think21:18
oneswigalthough someone's dog ate all the documentation21:18
trandleslol21:18
oneswigIf it works for us, we'll write up the experience for sure.21:19
oneswigYour new deploy, is that going to be bare metal compute?21:19
trandlesYour use cases are pretty far advanced from ours. We're just going into production with a basket of internal use cases and a mob of users with pitchforks and torches screaming for "cloud" but without any ability to write a one-page proposal on what they really think they're going to use it for.21:19
oneswigWe need this because I have to exercise a filesystem with massive connectivity using as few (virtualised) clients as possible.  The tangled webs we weave.21:21
trandlesThe first use of the production cloud will be bog standard VMs. We have some internal use cases for containers on bare metal but that's stage 2. I'm still stuck as the only one tasked with getting this up and going and it's only something <40% of my time. As you can imagine, being able to execute quickly and efficiently to put a completely new class of resource into production here and only have .4 FTE to do so is highly suboptimal.21:22
trandlesWe've tried twice in the past 16 months to hire a dedicated FTE to do nothing but cloud and been thwarted both times. :(21:22
oneswigtrandles: get your logging and monitoring in good order and you'll have a better chance.21:22
oneswigsorry to hear the recruitment fell through, I'd have thought it's an excellent job (for a respectable American!)21:23
trandlesThe one thing we have boatloads of is monitoring and logging. All of my logs will be going to rabbitmq and straight into splunk. We have a team of devs that are splunk gurus and they're working up custom dashboards for me right now.21:24
oneswigI am sure that will help.  One could lose oneself without it.21:24
oneswigPerhaps we should wrap up to let you kick off that deploy!  Over here I have bioinformatics to think about.21:26
trandlesI'm hoping to get some friendlies working on it by the end of the week. If initial testing goes well we'll be quickly ramping up to ~100 hypervisors and ~1.5PB of block and object storage.21:27
trandlesHow's the meetup going in London?21:27
oneswigTwo people from our team are there (johnthetubaguy and wasaac)21:27
oneswigApparently very good discussions and lots of input, from what I've heard.  Very positive.21:28
trandlesI've not been to a meetup. Is the London one well attended? Audience from all over Europe or just mainly the UK?21:28
oneswigI gather its mostly UK, some Europe, a few Americans.21:28
oneswigI don't have numbers to make that data at all useful.21:29
trandlesHave fun with bioinformatics. That reminds me, I need to schedule a meeting with some genomics folks here at the lab who are using Charliecloud and BEE (a workflow engine) to discuss their next steps porting everything to Common Workflow Language.21:30
trandlesComing from a physics background, the bio community is whole different beast.21:30
oneswigworkflows - BEE - I've heard of a few but not that one.21:31
oneswigfor sure.  But a beast for whom cloud is a beauty :-)21:31
trandlesindeed21:32
oneswigBe interesting to see how Charliecloud gets integrated into these workflows.  Does this research become public at some point?21:33
trandlesIt's all open sourced21:33
trandlesBEE is currently being refactored, but that's a much longer conversation21:33
trandlesWe're now running extremely large production codes at scale (~15000+ MPI ranks) using Charliecloud21:34
oneswigThat's great.  When people talk about MPI in containers, it so often boils down to some abominable hack involving spawning ORTE via ssh for mapping the ranks.21:36
oneswigpossibly commented along the lines of "must fix this after graduation"...21:36
oneswigmeee-ow21:37
trandlesThe amount of FUD around MPI and containers is a major point of contention for me. It's not black magic. We run MPI applications from containers without even having MPI installed on the host21:37
*** rcernin has joined #openstack-meeting21:37
*** rcernin has quit IRC21:37
*** rcernin has joined #openstack-meeting21:38
trandlesAll of the misconceptions, and dare I say lies, about MPI and containers needs to stop. It's holding up adoption in general.21:38
trandlesORTE is deprecated anyway :P21:38
oneswigBut the containers in your case are launched within a process framework like pmix, right?21:38
trandlesYes, using something like PMI2 or PMIx solves it21:39
trandlesORTE is deprecated by Open MPI. Mpirun is deprecated by Open MPI.21:39
oneswigAnd your network fabric is based on verbs or libfabric as well?21:40
trandlesIf you're not already using PMI2 or PMIx then you're in for a very rude awakening21:40
trandlesIdeally you'd use something like UCX but I'm not sure it's production ready yet. We did have this big ugly compatibility matrix on a whiteboard of various interconnects, BTL, MCA, etc. but it just made everyone sad.21:41
trandlesThings should get much better with Open MPI 4+21:42
trandlesMVAPICH just seems to work21:42
oneswigI think the injection of host mpi libraries into the container (like shifter) raises eyebrows, but that's only for proprietary host environments, right?21:42
*** eharney has joined #openstack-meeting21:42
trandlesYeah, injection like that is for proprietary fabrics like Cray's21:42
oneswigthere was an effort a while back to create a ceph messenger class based on ucx but alas I think it didn't yield.21:43
trandlesI also think ultimate portability is overblown. Our big code teams have convinced me that at least for the workloads I need to care about, portability of a container image isn't very valuable.21:43
oneswighow does that square with udss?21:44
trandlesThe toolchains used to reliably build container images mean that they're very happy to just build a new image custom for a platform they're targeting. The benefits of containerization are provenance of application runtime and the support libraries.21:44
trandlesUDSS seems to still hold the same value in that it allows bringing in environments/runtimes that normally couldn't run at all.21:45
oneswigI just hope you never sit next to the creator of ld.so at a party :-)21:45
trandlesThings like TensorFlow, DASK, etc. where the effort to support it for a small community at the lab is way too large, and the user either can't or doesn't want to figure out how to build and run from a home or project directory.21:46
trandlesWe have a code with over 300 dependencies that get managed using environment modules, LD_LIBRARY_* hacks, etc. For the code teams to build a container image with exactly the versions of everything they need, eliminating the many opportunities to make mistakes with modules and environment variables, is a massive win.21:48
oneswigI was looking for the code using ssh and orte for mpi rank assignment I referred to (in kubeflow), but it looks like the openmpi support has been taken out.21:50
trandlesThat just sounds horribly ugly21:52
oneswigtrandles: certainly, I can see the advantages of udss in an diverse environment (which is any environment, surely!)21:53
oneswigperhaps the removed code was put out of its misery.21:54
oneswigAnyway, time to wrap up...21:56
trandlesI was challenged at a conference last year to prove my statement that it doesn't matter what version of MPI is on the host compared to what's in the container. I ran a multi-node containerized MPI job on a cluster that didn't have MPI installed. I think what you're referencing (ssh and orte, etc.) is a symptom of what Reid and I have been fighting since our rejected 2016 SC paper on Charliecloud. Everyone is working way too hard, it's not21:56
trandles difficult.21:56
trandlesAnother good SIG meeting. ;)21:57
oneswigtrandles: That specific example is due to running in k8s21:57
oneswiglacking pmi21:57
trandlesAh, yes, well, k8s is the poster child of "working too hard."21:57
oneswigI think the challenge you had might be because people assume the crafty work done by shifter is generally applicable.21:58
trandlesIt was a singularity dev who told me I was wrong about MPI21:58
oneswigwhereas what's needed is /dev/uverbs (or whatever it was)21:58
oneswighmmm21:59
oneswigdid them mean different mpis within a single application?21:59
trandlesI have no idea what the singularity folks are on about most of the time, so I'm not sure.21:59
oneswiggood god, git pull of kubeflow was 205 MB, perhaps reinforcing your point :-)22:00
trandleswoah22:00
oneswigah, time to close.22:00
oneswiggood chatting trandles, until next time22:00
trandlesLikewise! Have a good night. :)22:00
oneswigNow to have fun with the OS-A deploy!22:00
oneswig#endmeeting22:01
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"22:01
openstackMeeting ended Tue Jan  7 22:01:01 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)22:01
trandlesFingers crossed22:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-01-07-21.01.html22:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-01-07-21.01.txt22:01
openstackLog:            http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-01-07-21.01.log.html22:01
*** martial has joined #openstack-meeting22:02
martialhello :)22:02
martial#start-meeting scientific-sig22:02
martial#startmeeting scientific-sig22:02
openstackMeeting started Tue Jan  7 22:02:35 2020 UTC and is due to finish in 60 minutes.  The chair is martial. Information about MeetBot at http://wiki.debian.org/MeetBot.22:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.22:02
*** openstack changes topic to " (Meeting topic: scientific-sig)"22:02
openstackThe meeting name has been set to 'scientific_sig'22:02
martialHello everybody, and welcome to 202022:03
martiala simple agenda today22:03
trandlesAre you an hour late martial ? Stig and I just spent an hour chatting.22:03
trandleshttp://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-01-07-21.01.log.html22:03
martiallate?22:03
martialmy calendar said it was now22:03
martialokay, super confused and sorry if that is the case22:04
trandlesIt was just Stig and me, so maybe we were off by an hour...no clue22:05
martialchecking22:06
martial1100 utv22:06
*** enriquetaso has quit IRC22:06
martial2100 utc ... ah bummer22:07
martial#endmeeting22:07
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"22:07
openstackMeeting ended Tue Jan  7 22:07:39 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)22:07
openstackMinutes:        http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-01-07-22.02.html22:07
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-01-07-22.02.txt22:07
openstackLog:            http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-01-07-22.02.log.html22:07
trandlesI can't keep up with the timezones :(22:08
martialso no, you are right, my calendar invitation has to be updated it looks like22:08
martialso sorry about this22:08
trandlesMine doesn't seem to adjust for DST so I only know that in the winter, my calendar is correct and in the summer I'm off by an hour22:08
martialwell hopefully it was a good meeting, last hour :)22:09
trandlesJust a chitchat really22:09
*** yonglihe has quit IRC22:09
trandlesI do wonder if anyone knows when registration opens up for Vancouver22:09
martialhave not seen the cfp yet22:10
trandlesNope and the website just has the dates and location22:10
martiallikely this month if anything is like usual22:11
*** ykatabam has quit IRC22:11
*** ykatabam has joined #openstack-meeting22:13
*** ykatabam has quit IRC22:13
martialwell, when it does we ought to discuss options, as usual :)22:13
trandles+122:14
martialso once again, sorry for the "delay", but I guess the meeting already happened (despite the misinformation on time in the invite and the error in calendar by yours truly)22:14
trandlesNo worries martial I think many are just getting the bearings after the holiday season.22:16
*** ykatabam has joined #openstack-meeting22:17
martialkind of means the next timeslot is also 1h earlier ... so 5am so definitely not making it next week22:17
martialtalk to you soon Tim :)22:24
trandlesLater Martial :)22:28
*** trandles has quit IRC22:29
*** simon-AS5591 has quit IRC22:45
*** oneswig has quit IRC22:51
*** tesseract has quit IRC23:13
*** dviroel has quit IRC23:26
*** ociuhandu has joined #openstack-meeting23:30
*** brinzhang_ has joined #openstack-meeting23:31
*** brinzhang has quit IRC23:35
*** ociuhandu has quit IRC23:35
*** mattw4 has quit IRC23:45
*** brinzhang has joined #openstack-meeting23:49
*** brinzhang_ has quit IRC23:53
*** brinzhang_ has joined #openstack-meeting23:54
*** brinzhang has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!