Tuesday, 2019-06-18

*** mattw4 has quit IRC00:22
*** diablo_rojo has quit IRC00:35
*** markvoelker has joined #openstack-meeting00:45
*** yamamoto has joined #openstack-meeting00:45
*** jamesmcarthur has quit IRC00:46
*** markvoelker has quit IRC00:50
*** ricolin has joined #openstack-meeting01:05
*** lbragstad has quit IRC01:22
*** lbragstad has joined #openstack-meeting01:22
*** hongbin has joined #openstack-meeting01:32
*** _hemna has joined #openstack-meeting01:35
*** yamamoto has quit IRC01:42
*** yamamoto has joined #openstack-meeting01:42
*** markvoelker has joined #openstack-meeting01:46
*** iyamahat__ has joined #openstack-meeting01:46
*** yamahata has quit IRC01:49
*** iyamahat_ has quit IRC01:49
*** iyamahat_ has joined #openstack-meeting01:50
*** markvoelker has quit IRC01:50
*** rajinir has quit IRC01:52
*** iyamahat__ has quit IRC01:53
*** iyamahat_ has quit IRC01:56
*** apetrich has quit IRC01:57
*** baojg has quit IRC01:59
*** baojg has joined #openstack-meeting02:01
*** _hemna has quit IRC02:07
*** lseki has quit IRC02:22
*** _hemna has joined #openstack-meeting02:42
*** markvoelker has joined #openstack-meeting02:46
*** carloss has quit IRC02:48
*** markvoelker has quit IRC02:51
*** dviroel has quit IRC02:54
*** kevinluuuuu has joined #openstack-meeting02:57
*** whoami-rajat has joined #openstack-meeting03:01
*** mattw4 has joined #openstack-meeting03:09
*** _hemna has quit IRC03:16
*** lbragstad has quit IRC03:26
*** markvoelker has joined #openstack-meeting03:47
*** hongbin has quit IRC03:47
*** psachin has joined #openstack-meeting03:50
*** markvoelker has quit IRC03:52
*** wwriverrat has quit IRC03:57
*** imsurit has joined #openstack-meeting03:58
samP#startmeeting masakari04:01
openstackMeeting started Tue Jun 18 04:01:03 2019 UTC and is due to finish in 60 minutes.  The chair is samP. Information about MeetBot at http://wiki.debian.org/MeetBot.04:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.04:01
*** openstack changes topic to " (Meeting topic: masakari)"04:01
openstackThe meeting name has been set to 'masakari'04:01
samPHi all for masakari04:01
*** tpatil has joined #openstack-meeting04:02
tpatilHi04:03
samPtpatil: Hi04:03
tpatilSorry to join late04:03
samP#item Cirical bug or patches04:03
samPtpatil: np04:03
*** shilpasd has joined #openstack-meeting04:03
samPsorry, not item, its topic04:03
samP#topic Critical Bugs or Patches04:04
*** openstack changes topic to "Critical Bugs or Patches (Meeting topic: masakari)"04:04
samPAny critical bugs or patches need to discuss here04:04
tpatilNot from my end04:05
samPtpatil: thanks04:05
samPI think we need to remove shell scripts in masakari-monitor repo.04:06
tpatil#link : https://github.com/openstack/masakari-monitors/blob/master/masakarimonitors/hostmonitor/hostmonitor.sh04:07
samPThose are deprecated long ago04:07
tpatilYes,this script is not needed anymore as we have implemented host-monitors service04:07
samPtpatil: correct04:08
tpatilShould I upload a patch to delete this script file?04:08
samPtpatil: Please. Thank you.04:08
samP#link https://docs.openstack.org/releasenotes/masakari-monitors/ocata.html04:09
tpatilsamP: Ok, noted down, will do the needful04:09
samPtpatil: ^^ here is the reason for that.04:09
samPtpatil: thanks04:10
samP#topic Train Work Items04:10
tpatilGot it, will specify this reason in the commit message04:10
*** openstack changes topic to "Train Work Items (Meeting topic: masakari)"04:10
samPPlease share any update on Train work items04:10
tpatil1) Add devstack support to install host-monitors04:10
tpatilWork is in progress, slow progress though04:11
tpatilwe are working on understanding and configuring the Stonith part04:11
tpatil2) Operators and Developers documentation: We will start working on it from this week onwards04:12
*** ykatabam has quit IRC04:13
tpatil3) Real time rendering of recovery workflow04:14
tpatilNo progress as the fix in fasteners lib will be revoked which is a blocker to complete this task. Need to work on the work around solution of monkey patching method from fasteners lib04:15
samPtpatil: got it.04:16
samPtpatil: thanks for sharing the updates04:19
tpatilThere is one patch uploaded by Liam in masakari-monitors to fix LP bug 172852704:19
openstackLaunchpad bug 1728527 in masakari-monitors "hostmonitor can not monitor pacemaker_remote node via cibadmin query" [Undecided,In progress] https://launchpad.net/bugs/1728527 - Assigned to Liam Young (gnuoy)04:19
tpatil#link : https://review.opendev.org/#/c/647756/04:19
tpatilNeed to review this patch04:20
samPtpatil: Thanks for bring this up. I was try to test this but didn't have time to setup the environment for that.04:21
samPfrom code wise LGTM04:21
tpatilThat's all update for Train items from my side04:23
samPtpatil: Thanks04:23
samPpreviously shared patch is very important bug fix. I will review that on earliest chance I get.04:25
samP#topic AOB04:26
*** openstack changes topic to "AOB (Meeting topic: masakari)"04:26
samPAny other items to discuss?04:26
tpatilsamP: No04:27
samPtpatil: Thanks04:27
samPOK then, let's finish today's meeting. Thank you all!04:27
*** ykatabam has joined #openstack-meeting04:28
samP#endmeeting04:28
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"04:28
openstackMeeting ended Tue Jun 18 04:28:25 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)04:28
openstackMinutes:        http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-06-18-04.01.html04:28
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-06-18-04.01.txt04:28
openstackLog:            http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-06-18-04.01.log.html04:28
*** mattw4 has quit IRC04:29
*** iyamahat has joined #openstack-meeting04:30
*** JangwonLee has quit IRC04:32
*** JangwonLee has joined #openstack-meeting04:32
*** _hemna has joined #openstack-meeting04:32
*** janki has joined #openstack-meeting04:35
*** rubasov has quit IRC04:38
*** markvoelker has joined #openstack-meeting04:48
*** tpatil has quit IRC04:49
*** markvoelker has quit IRC04:52
*** pcaruana has joined #openstack-meeting04:56
*** dtrainor_ has quit IRC05:02
*** _hemna has quit IRC05:05
*** yamahata has joined #openstack-meeting05:09
*** rtjure has joined #openstack-meeting05:24
*** hochanlee has joined #openstack-meeting05:30
*** _hemna has joined #openstack-meeting05:35
*** patrickeast has quit IRC05:35
*** patrickeast has joined #openstack-meeting05:36
*** masayukig has quit IRC05:36
*** johnsom has quit IRC05:36
*** csatari has quit IRC05:36
*** hogepodge has quit IRC05:36
*** masayukig has joined #openstack-meeting05:37
*** csatari has joined #openstack-meeting05:38
*** johnsom has joined #openstack-meeting05:38
*** hogepodge has joined #openstack-meeting05:38
*** e0ne has joined #openstack-meeting05:44
*** Luzi has joined #openstack-meeting05:44
*** markvoelker has joined #openstack-meeting05:49
*** markvoelker has quit IRC05:54
*** slaweq has joined #openstack-meeting06:09
*** _hemna has quit IRC06:10
*** brault has joined #openstack-meeting06:15
*** janki has quit IRC06:15
*** vishalmanchanda has joined #openstack-meeting06:21
*** e0ne has quit IRC06:26
*** igordc has quit IRC06:32
*** e0ne has joined #openstack-meeting06:36
*** markvoelker has joined #openstack-meeting06:50
*** markvoelker has quit IRC06:54
*** trident has quit IRC06:57
*** trident has joined #openstack-meeting06:59
*** apetrich has joined #openstack-meeting07:05
*** kevinluuuuu has quit IRC07:06
*** _hemna has joined #openstack-meeting07:11
*** e0ne has quit IRC07:13
*** _hemna has quit IRC07:16
*** joxyuki has joined #openstack-meeting07:17
*** rubasov has joined #openstack-meeting07:19
*** tesseract has joined #openstack-meeting07:24
*** takahashi-tsc has joined #openstack-meeting07:47
*** _hemna has joined #openstack-meeting07:48
*** kopecmartin|off is now known as kopecmartin07:49
*** ralonsoh has joined #openstack-meeting07:50
*** markvoelker has joined #openstack-meeting07:50
*** tssurya has joined #openstack-meeting07:55
*** markvoelker has quit IRC07:55
*** tpatil has joined #openstack-meeting08:03
*** tetsuro has joined #openstack-meeting08:14
*** radez has quit IRC08:16
*** ttsiouts has joined #openstack-meeting08:16
*** _hemna has quit IRC08:22
*** e0ne has joined #openstack-meeting08:23
*** psachin has quit IRC08:24
*** ociuhandu has quit IRC08:25
*** ttsiouts has quit IRC08:29
*** ttsiouts has joined #openstack-meeting08:29
*** ttsiouts_ has joined #openstack-meeting08:30
*** ttsiouts has quit IRC08:30
*** dklyle has quit IRC08:35
*** david-lyle has joined #openstack-meeting08:35
*** priteau has joined #openstack-meeting08:39
*** ociuhandu has joined #openstack-meeting08:41
*** ociuhandu has quit IRC08:46
*** jiaopengju has joined #openstack-meeting08:50
*** markvoelker has joined #openstack-meeting08:51
*** kevinluuuuu has joined #openstack-meeting08:55
*** markvoelker has quit IRC08:56
*** ociuhandu has joined #openstack-meeting09:20
*** yamamoto has quit IRC09:23
*** yamamoto has joined #openstack-meeting09:24
*** Adri2000 has quit IRC09:26
*** yamamoto has quit IRC09:29
*** panda|off is now known as panda09:30
*** ykatabam has quit IRC09:36
*** tpatil has quit IRC09:45
*** iyamahat has quit IRC09:52
*** rcernin has quit IRC09:57
*** takahashi-tsc has quit IRC10:11
*** _hemna has joined #openstack-meeting10:18
*** ttsiouts_ has quit IRC10:23
*** ttsiouts has joined #openstack-meeting10:24
*** ttsiouts has quit IRC10:28
*** yamamoto has joined #openstack-meeting10:41
*** ykatabam has joined #openstack-meeting10:43
*** ykatabam has left #openstack-meeting10:44
*** bbowen has quit IRC10:44
*** psachin has joined #openstack-meeting10:48
*** yamamoto has quit IRC10:50
*** _hemna has quit IRC10:52
*** baojg has quit IRC10:53
*** tetsuro has quit IRC10:56
*** yamamoto has joined #openstack-meeting10:59
*** Lucas_Gray has joined #openstack-meeting11:01
*** ykatabam has joined #openstack-meeting11:11
*** njohnston has joined #openstack-meeting11:16
*** _hemna has joined #openstack-meeting11:20
*** njohnston has quit IRC11:21
*** mjturek has quit IRC11:23
*** mjturek has joined #openstack-meeting11:23
*** dviroel has joined #openstack-meeting11:24
*** ykatabam has quit IRC11:27
*** raildo has joined #openstack-meeting11:41
*** imsurit has quit IRC11:46
*** Adri2000 has joined #openstack-meeting11:46
*** yamamoto has quit IRC11:48
*** yamamoto has joined #openstack-meeting11:49
*** yamamoto has quit IRC11:53
*** _hemna has quit IRC11:54
*** rfolco has joined #openstack-meeting11:59
*** dmacpher has joined #openstack-meeting11:59
*** dmacpher_ has quit IRC12:00
*** lpetrut has joined #openstack-meeting12:05
*** bbowen has joined #openstack-meeting12:05
*** _erlon_ has joined #openstack-meeting12:17
*** artom has quit IRC12:22
*** artom has joined #openstack-meeting12:23
*** artom has quit IRC12:33
*** artom has joined #openstack-meeting12:33
*** yamamoto has joined #openstack-meeting12:35
*** lpetrut has quit IRC12:39
*** priteau has quit IRC12:49
*** markvoelker has joined #openstack-meeting12:55
*** baojg has joined #openstack-meeting12:56
*** markvoelker has quit IRC13:00
*** mriedem has joined #openstack-meeting13:06
*** ttsiouts has joined #openstack-meeting13:06
*** rubasov has quit IRC13:08
*** lbragstad has joined #openstack-meeting13:11
*** enriquetaso has joined #openstack-meeting13:16
*** priteau has joined #openstack-meeting13:19
*** _hemna has joined #openstack-meeting13:27
*** ttsiouts has quit IRC13:32
*** ttsiouts has joined #openstack-meeting13:33
*** ttsiouts_ has joined #openstack-meeting13:36
*** ttsiouts has quit IRC13:37
*** pcaruana has quit IRC13:37
*** lseki has joined #openstack-meeting13:41
*** yamamoto has quit IRC13:41
*** yamamoto has joined #openstack-meeting13:42
*** yamamoto has quit IRC13:42
*** yamamoto has joined #openstack-meeting13:43
*** psachin has quit IRC13:46
*** vishalmanchanda has quit IRC13:48
*** rubasov has joined #openstack-meeting13:51
*** liuyulong has joined #openstack-meeting13:53
*** pcaruana has joined #openstack-meeting13:54
*** mlavalle has joined #openstack-meeting13:55
*** markvoelker has joined #openstack-meeting13:56
*** lajoskatona has joined #openstack-meeting13:58
*** shintaro has joined #openstack-meeting13:59
*** _hemna has quit IRC14:00
*** tidwellr has joined #openstack-meeting14:00
slaweq#startmeeting networking14:00
openstackMeeting started Tue Jun 18 14:00:31 2019 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
*** openstack changes topic to " (Meeting topic: networking)"14:00
openstackThe meeting name has been set to 'networking'14:00
mlavalleo/14:00
haleybhi14:00
tidwellrhi14:01
amotokihi14:01
rubasovo/14:01
lajoskatonahi14:01
bcafarelhello14:01
slaweq#topic Announcements14:01
*** openstack changes topic to "Announcements (Meeting topic: networking)"14:01
slaweqMilestone is T-2 is week of July 22 - 2614:01
slaweq#link https://releases.openstack.org/train/schedule.html14:02
ralonsohhi14:02
slaweqOpen Infrastructure Summit CFP Open - Deadline: July 214:02
*** njohnston_ has joined #openstack-meeting14:02
slaweqplease remember to submit talks proposals if You want to have talk there :)14:02
slaweq#link http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006262.html14:03
slaweqand as a reminder:14:03
slaweqTrain PTG summary: http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006408.html14:03
slaweqany other announcements do You have?14:03
*** carloss has joined #openstack-meeting14:03
*** njohnston has joined #openstack-meeting14:04
slaweqok, if not lets move on to the next topic14:04
slaweq#topic Blueprints14:05
*** openstack changes topic to "Blueprints (Meeting topic: networking)"14:05
slaweqI moved blueprints from Train-1 to Train-2 today14:05
slaweq#link https://launchpad.net/neutron/+milestone/train-214:05
*** njohnston_ has quit IRC14:05
mlavalle+114:05
*** davidsha has joined #openstack-meeting14:06
slaweqand I wanted to ask if https://blueprints.launchpad.net/neutron/+spec/smart-nic-ovs  and https://blueprints.launchpad.net/neutron/+spec/event-notifier-ironic  can be maybe marked as completed14:06
slaweqmlavalle: what do You think as an approver?14:06
mlavalleyes both are complte and I am marking them so right now14:06
slaweqthx14:07
*** radez has joined #openstack-meeting14:07
slaweqso we still have 2 more on the list14:07
slaweqhttps://blueprints.launchpad.net/neutron/+spec/enginefacade-switch14:07
njohnstonthere is slow but steady progress on engine facade14:07
mlavalleon going effort14:07
slaweqthis is progressing every week thanks to njohnston, ralonsoh and mlavalle14:07
slaweq:)14:07
*** liuyulong has quit IRC14:07
slaweqthx for pushing this forward :)14:08
*** jrbalderrama has joined #openstack-meeting14:08
slaweqand the last one on the list https://blueprints.launchpad.net/neutron/+spec/multiple-segment-per-network-per-host -14:08
slaweqany updates on this?14:08
mlavalle3 things to say about this one14:08
mlavalle1) wwriverrat has updated the patch and we need to help him with answering questions he posted there14:09
mlavallehang on please...14:09
mlavallehere's the patch14:10
mlavallehttps://review.opendev.org/#/c/623115/14:10
mlavalle2) ralonsoh has been looking into this from the sr-iov perspective. His initial feedback is that it won't have any impact14:11
mlavallecorrect?14:11
ralonsohthat's correct14:11
ralonsohI  don't see any problem on this, because SRIOV is only L214:11
slaweqthere is also spec for that to review, right?14:12
mlavalle3) I am doing the same from the OVS perspective. I was going to experiment with the patch in my routed networks deployment but had tto spend some time with https://bugs.launchpad.net/neutron/+bug/183067914:12
openstackLaunchpad bug 1830679 in neutron "Security groups RBAC cause a major performance degradation" [High,In progress] - Assigned to Miguel Lavalle (minsel)14:12
*** Luzi has quit IRC14:12
mlavallebut I will come back to it later this week14:13
mlavalleand yes, here's the spec: https://review.opendev.org/#/c/65717014:13
mlavallereally the work that ralonsoh and I are doing will help to finish open questions in the spec14:13
mlavallewwriverrat asked "community" help with sr-iov and ovs14:14
mlavallethat's all we have on this topic14:14
mlavalleon this blueprint, rather14:14
slaweqok, thx mlavalle and ralonsoh for update14:14
mlavallebefore we move on....14:15
slaweqmlavalle: do we have any other blueprints which we should add to the list?14:15
*** liuyulong has joined #openstack-meeting14:15
*** markvoelker has quit IRC14:15
*** _hemna has joined #openstack-meeting14:15
mlavalleI will add today a blueprint for the work that tidwellr is doing with subnet pools14:15
mlavallewe recently approved blueprint to remove restrictions on them14:15
slaweqyes, and I think patches for that are almost ready, right?14:16
tidwellrI haven't started on that yet14:16
tidwellrI will shortly though14:16
mlavallealso want to ask rubasov if he is ok with me adding a blueprint for the work he is doing on extra routes?14:16
rubasovmlavalle: absolutely14:17
*** rajinir has joined #openstack-meeting14:17
mlavalleGreat14:17
rubasovbut it's not yet merged14:17
rubasovI mean the spec for it14:17
mlavalleI know14:17
mlavallebut we are getting there, right?14:17
rubasovI hope, there's a new patch set14:18
mlavallecool14:18
mlavalleFinally, I have to confess a miserable failure on my part....14:18
mlavalleduring the PTG we discussed the adoption of the Neutron Classifier in Neutron QoS14:19
mlavalleI have promised more than once davidsha to review https://review.opendev.org/#/c/63633314:19
mlavalleand haven't found the long enough chunk of time I need to do it14:20
mlavalleso, I want to create a blueprint to add to our dashboard to track this effort14:20
mlavalleis that ok with you davidsha?14:20
davidshamlavalle: sure14:21
mlavalledavidsha: and it is still in my pile of things to do, ok?14:21
bcafarelanything to look at on neutron-classifier side? I did not have time to check reviews on it recently either :/14:21
*** bbowen_ has joined #openstack-meeting14:21
mlavalleHere's the PoC: https://review.opendev.org/#/c/63633314:21
mlavallebcafarel: ^^^^14:22
slaweqI will also try to look at it this week finally14:22
davidshaya, it's grand14:22
bcafarelack I will push it back higher on the pile14:23
mlavallethat's all I have as far as blueprints14:23
slaweqthx mlavalle for updates :)14:23
*** bbowen has quit IRC14:24
slaweqok, lets move on14:24
*** bbowen_ is now known as bbowen14:24
slaweq#topic community goals14:24
*** openstack changes topic to "community goals (Meeting topic: networking)"14:24
slaweqwe have still one "old" goal on the list:14:24
slaweqRun under Python 3 by default (njohnston)14:24
njohnstonno progress on that this week14:25
slaweqI just wanted to say that we are tracking progress of it in https://etherpad.openstack.org/p/neutron_stadium_python3_status14:25
bcafarelnetworking-sfc is done here14:26
slaweqif someone wants to help with some stadium project, feel free to assign it to yourself :)14:26
njohnston+114:26
bcafarelthanks slaweq for the easily reusable functional job definition!14:26
slaweqbcafarel: yw :)14:26
*** ianychoi_ is now known as ianychoi14:26
slaweqalso neutron-vpnaas was done last week14:26
mlavalleby slaweq, right?14:27
slaweqmlavalle: yes :)14:27
slaweqwhy?14:27
mlavalleThanks!14:27
mlavallejust wanted to give credit to you14:27
mlavallein wasn;t the ether who did it14:28
slaweqthx :)14:28
slaweqteam did it - and that's fine :)14:28
slaweqthis week I will try to push some patches for fwaas also14:28
*** e0ne has quit IRC14:28
slaweqany updates on other goals?14:29
slaweqI don't have anything regarding 2 "current" goals14:29
*** priteau has quit IRC14:29
mlavallemaybe say we need volunteers?14:30
slaweqyes, we need :)14:30
mlavalleso one is docs in PDF14:30
mlavalleright?14:30
slaweqbut I don't think that it will be a lot of work to do on our side14:30
slaweqyes14:31
slaweqand the second is "Support IPv6-Only Deployments"14:31
mlavalleright14:31
slaweqI can take care about "Support IPv6-Only Deployments" as qa liaison :)14:31
mlavallemaybe what we need for the time being are liaisons14:31
*** jamesmcarthur has joined #openstack-meeting14:31
mlavallewith those two efforts14:31
njohnstonI believe the documentation liaison is boden14:32
slaweqso we will still need someone for this Docs in PDF one14:32
mlavalleyes he is14:32
slaweqnjohnston: yes, I was also thinking about asking him :)14:32
*** boden has joined #openstack-meeting14:33
mlavallespeaking of the devil14:33
mlavalleLOL14:33
slaweq:)14:33
bcafarelor amotoki who apparently had volunteered at ptg (from mlavalle's recap mail)14:33
mlavalleahhh, that's right14:33
slaweqamotoki: are You fine to assign this goal to You?14:34
mlavallemaybe he dropped off14:35
mlavallelet's move on14:35
slaweqok, I will ask him tomorrow14:35
slaweqlets move on then14:35
slaweq#topic Bugs14:35
*** openstack changes topic to "Bugs (Meeting topic: networking)"14:35
slaweqlast week bug deputy was mlavalle and his summary is on http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007189.html14:36
slaweqmlavalle: do You want to highlight some bugs?14:36
*** ttsiouts_ has quit IRC14:36
mlavalleI want to say that all the critical and high priority bugs are either fixed or have patches well under way14:36
*** ttsiouts has joined #openstack-meeting14:37
slaweqthat is good news :)14:37
mlavalleI want to ask ralonsoh if he is going to work on https://bugs.launchpad.net/neutron/+bug/1832925?14:37
openstackLaunchpad bug 1832925 in neutron "Class neutron.common.utils.Timer is not thread safe" [Medium,Confirmed]14:37
ralonsohmlavalle, yes but I don't see any good solution14:38
mlavallecan I assign it to you in Launchpad?14:38
ralonsohbecause using signal.alarm makes the main thread to exit with an exception when timeout14:38
ralonsohyes, please14:38
mlavallethanks!14:38
mlavalleand we need more triaging for https://bugs.launchpad.net/neutron/+bug/183202114:39
openstackLaunchpad bug 1832021 in neutron "Checksum drop of metadata traffic on isolated networks with DPDK" [Medium,New]14:39
mlavalleif anyone can help with this14:39
slaweqaccording to the last comment it was caused by using veth in dhcp agent14:40
mlavalleFinally I wanted to ask boden about https://bugs.launchpad.net/neutron/+bug/183067914:40
openstackLaunchpad bug 1830679 in neutron "Security groups RBAC cause a major performance degradation" [High,In progress] - Assigned to Miguel Lavalle (minsel)14:40
mlavalleis the improvement so far something you and your co-workers can live with while I investigate further improvments?14:40
slaweqcan we then do anything with this or maybe just docs update will be enough?14:40
bodenmlavalle thanks for your work on that.. certainly a step in the right direction, but hope we can eventually get perf back close to were it was14:40
*** ttsiouts has quit IRC14:41
bodenmlavalle yes I think we are moving in the right direction, since a revert isn't as easy as I had thought/hoped14:41
mlavalleboden: yes, that's my intent. I am just trying to get a sense how urgent is the further improvement14:42
bodenmlavalle thanks again14:42
mlavalleI definitely want to get it to what it was before14:42
mlavalleslaweq: yes, that's a valid option14:42
slaweqmlavalle: ok, so I will write it in the comment there14:43
*** baojg has quit IRC14:43
*** shintaro has quit IRC14:44
mlavalleok, I'm done with my update14:44
slaweqthx mlavalle14:44
*** priteau has joined #openstack-meeting14:44
slaweqanyone else have something to add according to bugs?14:45
slaweqok, lets move on then14:45
slaweqnext week's bug deputy is yamamoto14:46
mlavallethis week's14:46
slaweqsorry, this week, right14:46
* mlavalle sent a reminder email to him last week14:46
slaweqand next week will be rubasov14:46
rubasovslaweq: ack14:46
mlavalleyeah, so we are in good hands14:46
slaweqI think yamamoto is aware of it and he already started his bug deputy as I saw some his comments on bugs today14:47
slaweqso lets move on to the next topic14:47
slaweq#topic neutron-lib14:47
*** openstack changes topic to "neutron-lib (Meeting topic: networking)"14:47
slaweqboden: any updates?14:47
bodenhi14:47
bodenjust 1 thing quickly14:47
*** ttsiouts has joined #openstack-meeting14:48
bodenthere's a desired from folks to release neutron-lib again... including https://review.opendev.org/#/c/665658/14:48
bodenhowever we haven't even started to use neutron-lib 1.27.0 yet https://review.opendev.org/#/c/663141/14:48
bodenmaybe we should just release 1.28 and skip 1.27.0 ^14:48
boden?14:48
mlavallefine with me14:48
slaweqI just +W this patch with 1.27 but I'm fine with release 1.28 :)14:49
mlavalleI'd rather streamline folks development efforts than keep releases sequential numbers :-)14:49
*** tssurya has quit IRC14:49
bodenI will try to post a 1.28.0 release patch today; otherwise someone can feel free to do so... I will be out the reminder of the week after today14:49
mlavalleI'm anal ratentive but not that much14:50
bodenI am too, I just keep it to myself ;)14:50
bodenthats all I have for neutron-lib14:50
slaweqthx boden for update14:50
slaweqok, last topic for today14:51
slaweq#topic On demand agenda14:51
*** openstack changes topic to "On demand agenda (Meeting topic: networking)"14:51
slaweqthere is something from haleyb on agenda14:51
slaweqbut I think it was discussed last week, no?14:51
amotokisorry for dropping for a while. I walked from the station to home. I am fine to assign myself to the PDF docs goal.14:51
mlavalleyes, it was discussed last week14:51
slaweqthx amotoki :)14:52
haleybslaweq: might have forgot to remove14:52
*** e0ne has joined #openstack-meeting14:52
slaweqhaleyb: no problem :) I will remove it14:52
slaweqI just wanted to ask to not miss something :)14:52
slaweqdone for both changes14:53
slaweqanything else You want to talk today?14:53
mlavallenot from me14:54
*** mattw4 has joined #openstack-meeting14:54
slaweqif not, I will give You couple of minutes back :)14:54
*** mattw4 has quit IRC14:54
slaweqthx for attending and have a great week14:54
slaweqo/14:54
lajoskatonao/14:54
* mlavalle is happy amotoki made it to home safely :-)14:54
bcafarelo/14:54
slaweq#endmeeting14:54
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"14:54
openstackMeeting ended Tue Jun 18 14:54:24 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:54
ralonsohbye14:54
openstackMinutes:        http://eavesdrop.openstack.org/meetings/networking/2019/networking.2019-06-18-14.00.html14:54
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/networking/2019/networking.2019-06-18-14.00.txt14:54
openstackLog:            http://eavesdrop.openstack.org/meetings/networking/2019/networking.2019-06-18-14.00.log.html14:54
njohnstono/14:54
mlavalleo/14:54
amotokio/14:54
*** boden has left #openstack-meeting14:54
davidshao/14:54
*** mattw4 has joined #openstack-meeting14:55
rubasovo/14:55
*** jrbalderrama has quit IRC14:56
*** bbowen_ has joined #openstack-meeting14:57
*** priteau has quit IRC14:58
*** jamesmcarthur_ has joined #openstack-meeting14:59
*** bbowen has quit IRC14:59
ralonsoh#startmeeting neutron_qos15:00
openstackMeeting started Tue Jun 18 15:00:20 2019 UTC and is due to finish in 60 minutes.  The chair is ralonsoh. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
ralonsohHello15:00
*** openstack changes topic to " (Meeting topic: neutron_qos)"15:00
openstackThe meeting name has been set to 'neutron_qos'15:00
lajoskatonaHi15:00
rubasovhi15:00
slaweqhi15:01
davidshaHey15:01
*** mattw4 has quit IRC15:01
ralonsohHello everyone15:01
* slaweq don't knows if will be able to be on whole meeting15:01
ralonsohas commented in the neutron channel, the agenda is very short15:01
ralonsoh#topic RFEs15:01
*** openstack changes topic to "RFEs (Meeting topic: neutron_qos)"15:01
ralonsohWe have only one live item15:02
ralonsoh#link https://bugs.launchpad.net/neutron/+bug/157898915:02
openstackLaunchpad bug 1578989 in neutron "[RFE] Strict minimum bandwidth support (egress)" [Wishlist,Fix released] - Assigned to Bence Romsics (bence-romsics)15:02
ralonsohand one pending review: #link https://review.opendev.org/#/c/629253/15:02
ralonsohrubasov, lajoskatona any update?15:02
rubasovralonsoh: no update from me15:02
*** diablo_rojo has joined #openstack-meeting15:02
lajoskatonaralonsoh: the tempest test is the only remaining thing I think15:02
*** jamesmcarthur has quit IRC15:02
ralonsohthat's perfect15:03
ralonsohI saw this one merged15:03
ralonsoh#link https://review.opendev.org/#/c/629142/15:03
lajoskatonathat's has all the marks, only depends on small thing from gmann, I will ask around on qa channel to push that15:03
lajoskatonayeah, this one is hanging: https://review.opendev.org/62925315:03
ralonsohlajoskatona, thanks!15:03
lajoskatonaralonsoh: :-)15:04
ralonsohBTW, I still need to take a second look at this patch15:04
ralonsohanyway, thanks guys!15:04
ralonsohthe second topic is not currently active15:05
ralonsoh#link https://bugs.launchpad.net/neutron/+bug/156096315:05
openstackLaunchpad bug 1560963 in neutron "[RFE] Minimum bandwidth support (egress)" [Wishlist,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez)15:05
ralonsohAs commented before, I'll hold the last patch of this RFE because there are other more important topics and features15:05
ralonsohanything else in this section??15:06
ralonsohok, let's move on15:06
ralonsoh#topic Bugs15:06
*** openstack changes topic to "Bugs (Meeting topic: neutron_qos)"15:06
ralonsohwe have only one pending bug15:06
*** eharney has quit IRC15:06
ralonsoh#link https://bugs.launchpad.net/neutron/+bug/182669515:06
openstackLaunchpad bug 1826695 in neutron "[L3][QoS] cache does not removed when router is down or deleted" [Medium,In progress] - Assigned to LIU Yulong (dragon889)15:06
ralonsoh#link https://review.opendev.org/#/c/656105/15:06
ralonsohthis patch has been stuck for a while15:07
ralonsohplease, reviews are welcome, it's almost done15:07
*** gyee has joined #openstack-meeting15:07
ralonsohand nothing else in this section15:08
ralonsohAm I missing something?15:08
*** davidsha has left #openstack-meeting15:09
*** njohnston has quit IRC15:09
ralonsohok, next section15:09
ralonsoh#topic Open Discussion15:09
*** openstack changes topic to "Open Discussion (Meeting topic: neutron_qos)"15:09
ralonsohno way...15:09
ralonsohDavid left??15:09
ralonsohcome on....15:09
ralonsoh#link https://bugs.launchpad.net/neutron/+bug/147652715:09
openstackLaunchpad bug 1476527 in neutron "[RFE] Add common classifier resource" [Wishlist,Triaged] - Assigned to Igor D.C. (igordcard)15:09
ralonsohlet me ping him...15:10
ralonsohanyway, as commented in the previous meeting15:10
ralonsohMiguel is going to review the PoC15:10
ralonsoh(ok, David is telling me that he got pulled into another meeting)15:11
ralonsohas I was saying: the goal of this PoC could be to integrate the classifier inside the Neutron code15:11
ralonsohnot as a plugin but as part of the QoS code15:12
ralonsohbecause of the dependencies (API, DB, etc) needed15:12
*** markvoelker has joined #openstack-meeting15:12
ralonsohbut, as I said, Miguel is going to spend sometime reviewing this15:12
*** jiaopengju has quit IRC15:12
ralonsohIMHO, I would prefer to have the classification code (DSCP selection) integrated in the QoS service.15:14
ralonsohThat means:15:14
ralonsohyou can create a DSCP QoS rule depending on a classification groups15:14
ralonsoh*group15:14
ralonsohok, that's all related to this PoC15:15
ralonsohdo we have something else in the agenda??15:15
ralonsohok, thank you very much everyone15:16
ralonsohsee you in two weeks!15:16
lajoskatonabye15:16
ralonsoh#endmeeting15:16
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"15:16
openstackMeeting ended Tue Jun 18 15:16:34 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:16
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_qos/2019/neutron_qos.2019-06-18-15.00.html15:16
rubasovbye15:16
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_qos/2019/neutron_qos.2019-06-18-15.00.txt15:16
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_qos/2019/neutron_qos.2019-06-18-15.00.log.html15:16
*** lajoskatona has left #openstack-meeting15:16
*** _hemna has quit IRC15:19
*** whoami-rajat has quit IRC15:21
*** e0ne has quit IRC15:28
*** markvoelker has quit IRC15:31
*** ttsiouts has quit IRC15:33
*** ttsiouts has joined #openstack-meeting15:34
*** zbr|ruck is now known as zbr|brb15:34
*** ttsiouts has quit IRC15:38
*** mgagne has quit IRC15:39
*** d34dh0r53 has quit IRC15:39
*** rajinir has quit IRC15:40
*** masayukig has quit IRC15:40
*** d34dh0r53 has joined #openstack-meeting15:40
*** rajinir has joined #openstack-meeting15:40
*** masayukig has joined #openstack-meeting15:40
*** mgagne has joined #openstack-meeting15:42
*** iyamahat has joined #openstack-meeting15:42
*** dtrainor has joined #openstack-meeting15:43
*** mattw4 has joined #openstack-meeting15:44
*** mattw4 has quit IRC15:46
*** _erlon_ has quit IRC15:57
*** mattw4 has joined #openstack-meeting15:58
*** njohnston has joined #openstack-meeting15:59
slaweq#startmeeting neutron_ci16:00
openstackMeeting started Tue Jun 18 16:00:16 2019 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: neutron_ci)"16:00
openstackThe meeting name has been set to 'neutron_ci'16:00
slaweqhi (again)16:00
mlavalleo/16:00
njohnstono/16:00
slaweqAgenda for the meeting is on https://github.com/openstack/neutron/tree/master/neutron16:00
slaweq#undo16:01
slaweqAgenda for the meeting is on https://etherpad.openstack.org/p/neutron-ci-meetings16:01
bcafarelhi16:01
ralonsohhi16:01
*** pcaruana has quit IRC16:01
slaweqGrafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate16:01
slaweq#topic Actions from previous meetings16:02
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"16:02
slaweqwe have only one action from previous week16:02
slaweqmlavalle to debug neutron-tempest-plugin-dvr-multinode-scenario failures (bug 1830763) reproducing most common failure: test_connectivity_through_2_routers16:02
openstackbug 1830763 in neutron "Debug neutron-tempest-plugin-dvr-multinode-scenario failures" [High,Confirmed] https://launchpad.net/bugs/1830763 - Assigned to Miguel Lavalle (minsel)16:02
mlavalleI am currently working on this16:02
*** igordc has joined #openstack-meeting16:03
slaweqwe recently had similar (maybe same) issue in our D/S CI on release based on Stein16:03
mlavallejust 10 minutes ago, I was able to get a succesful execution of that test case in my local environment, with a pdb break at the end of the test16:03
bcafarelsorry I may have spotty connection, will try to follow16:04
mlavalleand it is easy to reproduce a failure, where I will also set a pdb break before it tears down. so now I will be able to run tcpdump and compare ip tables to see where the problem is16:04
mlavallethat's the status at this point16:04
*** liuyulong has quit IRC16:04
slaweqgreat - I may ping You about this during this week to see if You found something and if we can use it in our d/s :)16:05
mlavalleyeap, cool16:05
slaweqthx a lot for working on it mlavalle :)16:05
slaweq#action mlavalle to debug neutron-tempest-plugin-dvr-multinode-scenario failures (bug 1830763) reproducing most common failure: test_connectivity_through_2_routers16:05
openstackbug 1830763 in neutron "Debug neutron-tempest-plugin-dvr-multinode-scenario failures" [High,Confirmed] https://launchpad.net/bugs/1830763 - Assigned to Miguel Lavalle (minsel)16:05
*** baojg has joined #openstack-meeting16:05
slaweqI will assign it for next week just as a reminder, ok?16:05
mlavalleof course16:06
slaweqthx16:06
*** baojg has quit IRC16:06
mlavallekeep the pressure on that guy16:06
slaweqsure :)16:06
slaweqanything else You want to add/ask regarding actions from last week?16:06
mlavallenot me16:07
slaweqok, lets move on then16:07
slaweq#topic Stadium projects16:07
*** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)"16:07
slaweqPython 3 migration16:07
slaweqStadium projects etherpad: https://etherpad.openstack.org/p/neutron_stadium_python3_status16:07
njohnstonTrain-2 is when I was planning on getting really serious about this, so I am gearing up to work on it later this week16:08
slaweqthx njohnston16:08
slaweqI also started sending some patches and I will try to continue this during next weeks16:09
slaweqactually looking at etherpad there is no many projects left to switch16:09
mlavalle\o/16:10
slaweqthis week I will push neutron-fwaas patches16:10
slaweq#action slaweq to send patch to switch neutron-fwaas to python 316:10
slaweq^^ this way I will have more pressure to do it :)16:10
mlavalleyeah let's also keep p[ressure on that other lazy guy16:11
slaweqLOL16:11
njohnston+116:11
slaweqbut I think I'm the only lazy guy here :P16:11
slaweqok, lets move on then16:12
slaweqtempest-plugins migration16:12
slaweqEtherpad: https://etherpad.openstack.org/p/neutron_stadium_move_to_tempest_plugin_repo16:12
njohnstonI'll take another look at fwaas later this week as well16:13
slaweqhere we are good with networking-sfc (thx bcafarel) and almost done with networking-bgpvpn16:13
slaweqwe still need to do vpnaas, fwaas and networking-dynamic-routing16:14
mlavalleI have slacked off with vpnaas16:14
mlavallehoping to get back to it soon16:14
slaweqsure mlavalle - that is not urgent for sure - more like "nice to have" :)16:14
slaweqI would rather want to finish python 3 transition first if I would have to choose16:15
slaweqanything else anyone wants to add regarding to stadium projects?16:16
mlavallenot me16:16
njohnstonI wonder if once we're done we could doublecheck the migration by adding a DNM change that deletes the python 2.7 version and see how all our jobs do16:16
slaweqpython 2.7 version of what? of python binaries on host?16:17
njohnstonyes16:17
njohnstonjust to make absolutely sure we have everything16:18
slaweqnjohnston: yes, that is good idea and we can also do it for neutron jobs16:18
slaweqnjohnston: can You add such note on the top of the etherpad maybe?16:19
slaweqto not forget about it16:19
njohnstonsure thing!16:19
slaweqthx njohnston16:19
slaweqok, lets move on16:19
slaweqnext topic16:19
slaweq#topic Grafana16:19
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"16:19
slaweqhttp://grafana.openstack.org/dashboard/db/neutron-failure-rate16:19
slaweqduring last couple of days our check queue was broken (neutron-tempest-plugin-scenario-linuxbridge job) so there is not too much in gate queue graphs16:20
slaweqnow it's getting back to normal thx to fix from liuyulong16:21
slaweqalso rally job was failing quite often16:21
mlavallebecasue of server creation?16:21
slaweqyes, and that should be now fixed with https://review.opendev.org/#/c/665614/16:21
slaweqand it's going back to low numbers today16:22
mlavallecool16:23
slaweqfrom other things, networking-ovn job is almost switched to be voting - njohnston can You update dashboard when it will be merged in neutron?16:23
njohnstonwill do16:23
slaweqthx a lot16:23
njohnston#action njohnston update dashboard when ovn job becomes voting16:23
slaweqand last note from me about grafana - still functional and fullstack jobs are quite unstable :/16:24
slaweqI have couple of examples ready for latter part of the meeting :)16:25
slaweqanything else strange/worth to discuss do You see on grafana?16:25
mlavallenot from me16:25
*** njohnston is now known as njohnston_16:26
*** njohnsto_ has joined #openstack-meeting16:26
slaweqok, so lets talk about this unstable jobs then16:26
slaweq#topic fullstack/functional16:26
*** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)"16:26
slaweqfirst functional tests16:26
slaweqthe most often during last week I saw failed neutron.tests.functional.agent.linux.test_ip_lib.IpMonitorTestCase tests, like:16:27
slaweqhttp://logs.openstack.org/36/662236/3/check/neutron-functional/3214fdd/testr_results.html.gz16:27
ralonsohsorry for that16:27
ralonsohhttps://review.opendev.org/#/c/664889/16:27
slaweqbut that should be fixed with patch from ralonsoh https://review.opendev.org/#/c/664889/16:27
slaweqralonsoh: You don't need to sorry :)16:27
slaweqthx for quick fix16:28
*** markvoelker has joined #openstack-meeting16:28
slaweqother than that I saw some failures which happend once (at least I didn't found more examples)16:28
slaweqe.g. tests from module neutron.tests.functional.test_server16:28
slaweqhttp://logs.openstack.org/12/640812/2/check/neutron-functional/35dc53f/testr_results.html.gz16:28
slaweqlooking at logs from those tests, I see http://logs.openstack.org/12/640812/2/check/neutron-functional/35dc53f/controller/logs/dsvm-functional-logs/neutron.tests.functional.test_server.TestWsgiServer.test_restart_wsgi_on_sighup_multiple_workers.txt.gz#_2019-06-17_20_37_09_08516:30
slaweqdo You know if it is normal to send SIG_UNBLOCK there?16:30
*** njohnston_ has quit IRC16:30
mlavalleI don't know16:31
slaweqok, in other test (which passed) it's exactly the same16:31
slaweqany ideas about why it could fail?16:32
mlavallehard to say16:34
mlavallewithout digging deeper16:34
slaweqI know :/16:34
slaweqif that will happen again, I will open bug for it16:34
mlavallewhere is the signal coming from?16:35
slaweqit's from test: https://github.com/openstack/neutron/blob/master/neutron/tests/functional/test_server.py#L13316:35
slaweqit failed exactly in L15916:35
mlavalleahh, it's waiting16:36
slaweqyes, 5 seconds16:37
slaweqshould be enough IMO16:37
mlavalleyes, looks like plenty16:37
ralonsohit takes 8 secs to stop the process16:37
ralonsohfrom 37:09 to 37:1716:37
slaweqralonsoh: so do You think it could be "just" overloaded node?16:38
ralonsohmaybe, I'll take a look at this one16:38
ralonsohI'll open a low priority bug, just to track it16:38
slaweqthx ralonsoh16:38
*** zbr|brb is now known as zbr|ruck16:38
ralonsohok?16:38
slaweqsure, that is good idea16:38
slaweqnext one on my list is: neutron.tests.functional.agent.linux.test_keepalived.KeepalivedManagerTestCase16:39
slaweqhttp://logs.openstack.org/48/665548/4/check/neutron-functional/f6d6447/testr_results.html.gz16:39
ralonsohnot only this one but I've seen this error in other tests16:40
ralonsohthe namespace not created16:40
slaweqlogs from this one are in:16:40
slaweqhttp://logs.openstack.org/48/665548/4/check/neutron-functional/f6d6447/controller/logs/dsvm-functional-logs/neutron.tests.functional.agent.linux.test_keepalived.KeepalivedManagerTestCase.test_keepalived_spawns_conflicting_pid_base_process.txt.gz#_2019-06-17_10_02_30_82916:40
ralonsohJust a guess, but maybe is something in pyroute, I'll review the latest patches16:42
slaweqso I will report it as a bug also to track those issues16:42
slaweqok?16:42
ralonsoh+116:42
slaweqwhat is strange is that many commands in this namespace was executed earlier: http://logs.openstack.org/48/665548/4/check/neutron-functional/f6d6447/controller/logs/journal_log.txt.gz#_Jun_17_10_01_4116:43
slaweqI will open bug and try to look deeper into this16:44
slaweq#action slaweq to open bug related to missing namespace issue in functional tests16:44
slaweq#action ralonsoh to open bug related to failed test_server functional tests16:44
*** pcaruana has joined #openstack-meeting16:45
slaweqand last one on my list16:45
slaweqneutron.tests.functional.agent.linux.test_linuxbridge_arp_protect.LinuxBridgeARPSpoofTestCase16:45
slaweqhttp://logs.openstack.org/40/665640/4/check/neutron-functional-python27/e1df845/testr_results.html.gz16:45
slaweqin logs for this test there is not too much http://logs.openstack.org/40/665640/4/check/neutron-functional-python27/e1df845/controller/logs/dsvm-functional-logs/neutron.tests.functional.agent.linux.test_linuxbridge_arp_protect.LinuxBridgeARPSpoofTestCase.test_arp_protection_dead_reference_removal.txt.gz16:46
slaweq:/16:46
*** ricolin has quit IRC16:46
*** markvoelker has quit IRC16:46
slaweqand it should be more there16:46
ralonsohslaweq, maybe (I need to check better those tests cases) we should execute them in temp namespaces16:48
ralonsohslaweq, I don't think we are doing this now16:48
ralonsoh(if it's possible)16:48
slaweqralonsoh: but You mean to run "tox" in namespace?16:48
ralonsohno no16:48
ralonsohlike in the ip_lib commands16:49
ralonsohcreate everything in a namespace16:49
ralonsoha temp one, to avoid interferences16:49
slaweqahh, ok16:49
ralonsohis the best way to isolate a test case16:49
slaweqyes, we should probably16:49
ralonsoh(if possible)16:49
*** igordc has quit IRC16:49
ralonsohok I'll take this one16:49
mlavallethat makes sense16:49
slaweqbut are You talking about this linuxbridge arp tests? or the previous one?16:50
ralonsohLB arp16:50
slaweqyes, that we should run in namespaces if possible IMO16:50
slaweqralonsoh: so You will take care of it, right?16:50
ralonsohI'll create a bug for this16:51
ralonsohyes16:51
slaweqgreat, thx a lot16:51
*** e0ne has joined #openstack-meeting16:51
slaweqso that would be all regarding functional tests from me16:51
slaweqdo You have anything else?16:51
mlavallenot me16:51
slaweqok, lets quickly go through fullstack issues16:52
slaweqfirst one (I spotted it at least twice)16:52
slaweqneutron.tests.fullstack.test_l3_agent.TestLegacyL3Agent.test_north_south_traffic16:52
slaweqhttp://logs.openstack.org/11/662111/21/check/neutron-fullstack/c0f8029/testr_results.html.gz16:52
slaweqhttp://logs.openstack.org/29/664629/1/check/neutron-fullstack/e99aa1c/testr_results.html.gz16:52
slaweqthere is a lot of errors in ovs agent logs: http://logs.openstack.org/11/662111/21/check/neutron-fullstack/c0f8029/controller/logs/dsvm-fullstack-logs/TestLegacyL3Agent.test_north_south_traffic/neutron-openvswitch-agent--2019-06-18--01-52-58-123455_log.txt.gz?level=ERROR16:53
*** baojg has joined #openstack-meeting16:55
haleybCRITICAL neutron [req-c0ac0a6c-e65a-46c3-a123-82239299ec08 - - - - -] Unhandled error: RuntimeError: No datapath_id on bridge br-ethaa0e64ff916:55
slaweqand the same errors in second case http://logs.openstack.org/11/662111/21/check/neutron-fullstack/c0f8029/controller/logs/dsvm-fullstack-logs/TestLegacyL3Agent.test_north_south_traffic/neutron-openvswitch-agent--2019-06-18--01-52-58-123455_log.txt.gz?level=ERROR16:55
slaweqsorry, this is the same link as above16:56
haleybhttp://logs.openstack.org/11/662111/21/check/neutron-fullstack/c0f8029/controller/logs/dsvm-fullstack-logs/TestLegacyL3Agent.test_north_south_traffic/neutron-openvswitch-agent--2019-06-18--01-52-58-123455_log.txt.gz?#_2019-06-18_01_53_52_66516:56
slaweqhttp://logs.openstack.org/29/664629/1/check/neutron-fullstack/e99aa1c/controller/logs/dsvm-fullstack-logs/TestLegacyL3Agent.test_north_south_traffic/neutron-openvswitch-agent--2019-06-18--02-04-44-968982_log.txt.gz?level=ERROR16:56
*** baojg has quit IRC16:56
slaweqthere is log from second example16:56
*** panda has quit IRC16:56
slaweqand the same error16:57
haleybso right before that the bridge was recreated16:57
*** jamesmcarthur_ has quit IRC16:57
slaweqanyone wants to volunteer and check this issue deeper?16:57
slaweqlooks like that can be "real" bug not only test issue16:57
slaweq*might be16:58
mlavalletoo much in my plate right now. maybe next week16:58
slaweqok, so I will report this bug for now and we will see later16:58
mlavalle+116:59
slaweqmlavalle: sure, I also will not have time to work on it this week16:59
slaweq#action slaweq to report bug regarding failing neutron.tests.fullstack.test_l3_agent.TestLegacyL3Agent.test_north_south_traffic tests16:59
slaweqok, and I think we are out of time now16:59
slaweqthx for attending16:59
mlavalleo/16:59
slaweqsee You next week16:59
slaweqo/16:59
njohnsto_o/16:59
slaweq#endmeeting16:59
*** jamesmcarthur has joined #openstack-meeting16:59
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"16:59
ralonsohbye16:59
openstackMeeting ended Tue Jun 18 16:59:56 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:59
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-06-18-16.00.html17:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-06-18-16.00.txt17:00
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-06-18-16.00.log.html17:00
*** altlogbot_1 has quit IRC17:00
*** panda has joined #openstack-meeting17:01
*** altlogbot_0 has joined #openstack-meeting17:01
*** irclogbot_3 has quit IRC17:01
*** njohnsto_ has quit IRC17:03
*** irclogbot_3 has joined #openstack-meeting17:03
*** jamesmcarthur has quit IRC17:04
*** diablo_rojo has quit IRC17:04
*** yamahata has quit IRC17:06
*** iyamahat has quit IRC17:06
*** corvus is now known as thecount17:06
*** thecount is now known as corvus17:06
*** _hemna has joined #openstack-meeting17:15
*** Lucas_Gray has quit IRC17:18
*** iyamahat has joined #openstack-meeting17:20
*** yamahata has joined #openstack-meeting17:21
*** ociuhandu_ has joined #openstack-meeting17:24
*** ociuhandu has quit IRC17:26
*** jamesmcarthur has joined #openstack-meeting17:28
*** ociuhandu_ has quit IRC17:28
*** jamesmcarthur has quit IRC17:33
*** slaweq has quit IRC17:34
*** igordc has joined #openstack-meeting17:35
*** whoami-rajat has joined #openstack-meeting17:37
*** kopecmartin is now known as kopecmartin|off17:37
*** eharney has joined #openstack-meeting17:44
*** _hemna has quit IRC17:49
*** jamesmcarthur has joined #openstack-meeting17:52
*** ralonsoh has quit IRC17:56
*** enriquetaso has quit IRC18:00
*** markmcclain has quit IRC18:00
*** markmcclain has joined #openstack-meeting18:02
*** e0ne has quit IRC18:13
*** _hemna has joined #openstack-meeting18:25
*** e0ne has joined #openstack-meeting18:29
*** ociuhandu has joined #openstack-meeting18:30
*** slaweq has joined #openstack-meeting18:31
*** e0ne has quit IRC18:33
*** ociuhandu has quit IRC18:39
*** njohnston has joined #openstack-meeting18:40
*** pcaruana has quit IRC18:45
*** Shrews has joined #openstack-meeting18:45
*** mlavalle has left #openstack-meeting18:54
*** slaweq has quit IRC18:54
*** enriquetaso has joined #openstack-meeting18:57
*** _hemna has quit IRC18:59
* fungi peers around the corner19:00
ianw#startmeeting infra19:00
openstackMeeting started Tue Jun 18 19:00:29 2019 UTC and is due to finish in 60 minutes.  The chair is ianw. Information about MeetBot at http://wiki.debian.org/MeetBot.19:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:00
*** openstack changes topic to " (Meeting topic: infra)"19:00
openstackThe meeting name has been set to 'infra'19:00
ianwthe agenda for today is @19:01
ianw#link http://lists.openstack.org/pipermail/openstack-infra/2019-June/006403.html19:01
ianw#topic Announcements19:01
*** openstack changes topic to "Announcements (Meeting topic: infra)"19:01
*** eharney has quit IRC19:01
ianwclarkb obviously doing something more exciting than running this meeting :)19:01
ianwi don't think we have anything else to announce19:02
fungioh, one thing19:02
fungi#link http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006888.html Long overdue cleanups of Zuulv2 compatibility base configs19:02
mordred\o/19:03
dmsimardzuul-cloner will no longer be a thing ? :)19:03
fungi#info zuul-cloner shim and bindep fallback are being removed from non-legacy abstract jobs on Monday, June 2419:03
fungigonna follow up to that announcement on the ml with a reminder19:04
fungiand put on my flame retardant underpants when i get up monday19:04
ianw++19:04
*** pcaruana has joined #openstack-meeting19:05
*** pabelanger has joined #openstack-meeting19:05
ianw#topic Actions from last meeting19:05
*** openstack changes topic to "Actions from last meeting (Meeting topic: infra)"19:05
ianw#link http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-06-11-19.01.html19:05
ianwcorvus: work with jroll to get openstack github org self managed19:06
corvusderp19:06
corvussorry19:06
corvusdid not happen :(19:06
* mordred also supposed to make opendevadmin user - will do after meeting19:06
ianwshould we re-action or is it on todo list?19:06
ianwmordred: set up github opendevadmin account for the opendevorg org and zuul app19:07
corvuslet's re-action it, not only as a safetly valve in case i forget again, but also, it seems like something i should report back on regardless19:07
ianw#action corvus  work with jroll to get openstack github org self managed19:07
mordredsame with admin account19:07
ianw#action mordred  set up github opendevadmin account for the opendevorg org and zuul app19:08
*** jamesmcarthur has quit IRC19:08
ianwok that was it for that19:08
ianw#topic Priority Efforts19:08
*** openstack changes topic to "Priority Efforts (Meeting topic: infra)"19:08
ianw#topic Update Config Management19:09
*** openstack changes topic to "Update Config Management (Meeting topic: infra)"19:09
ianwcan talk about specific mirror stuff later19:09
fungidid we have any efforts underway on the config management side?19:10
fungii guess it's down mostly to cleanup now19:10
ianwi will have to check out the builders -- note i merged https://review.opendev.org/#/c/665014/ to stop building the control plane images19:11
fungiand some of this is intertwined with the trusty upgrades as well as the mirror updates you noted19:11
ianwlast night i noticed that nb01 had about 250 threads which i don't think was right, errors trying to delete in-use images19:11
Shrewsthat definitely does not sound right19:11
mordredit doesn't sound ideal at the very least19:12
ianwanyway will investigate more, i did py-bt dump all them but it wasn't pointing to a smoking gun inside nodepool-builder/openstacksdk19:12
*** diablo_rojo has joined #openstack-meeting19:13
ianwanything else?19:13
ianw#topic OpenDev19:14
*** openstack changes topic to "OpenDev (Meeting topic: infra)"19:14
dmsimardFor the thread about ara 1.0 on openstack-infra19:14
dmsimardsome pieces of the puzzle are falling in place -- we have an implementation of the sqlite middleware working for 1.0 that will land soon19:15
ianw#undo19:15
mordredwoot19:15
openstackRemoving item from minutes: #topic OpenDev19:15
dmsimardI somehow read that as open floor19:15
dmsimardsorry for sidetracking19:16
ianwoh, cool.  yeah, as noted by corvus i guess with swift logs we still have an issue, but at least getting status quo to 1.0 will be good19:16
dmsimardthere are some question marks around swift, yeah19:16
*** bbowen_ has quit IRC19:16
dmsimardbasically19:17
dmsimardthe equivalent middleware in 1.0 will give us arbitrary API endpoints to every job results19:17
corvusthere are question marks around middleware too.... our plan is to stop running it and rely entirely on statig generation19:17
dmsimardand ara-web is able to connect to any of those arbitrary endpoints19:17
corvusthat plan is ready to go at a moments notice (for example, the next time our filesystem is corrupted)19:18
fungiwe're overdue for one of those events19:19
dmsimardas far as full static generation goes, I have a half working proof of concept but we need some changes under the hood for it to work properly19:19
dmsimardlike, I have the actual html of the pages as if you would be browsing them but it still attempts to connect to the API19:19
dmsimardthis is tracked here: https://github.com/ansible-community/ara-web/issues/1119:20
ianw#link  https://github.com/ansible-community/ara-web/issues/1119:20
corvuscool -- i think i'd suggest that we not spend too much effort on doing ara 1.0 middleware in opendev given that we might switch to static generation at any time19:21
fungiseconded19:21
fungiwe can always pin ara<1 until we switch to object storage19:21
dmsimardworks for me19:21
ianwi suggested on list that maybe this is suitable for a single service that we sent the results to in post in the job, but i'll read through the static generation stuff to understand more too19:23
ianwi didn't think react was amenable to that, but if it is, great!19:23
ianwone other little thing is that i discussed having a vexxhost backup server with mnaser, and we agreed it was ok, so that's on my todo to bring up, and get to comments left in the review for ansible backups left last week19:24
ianwok we should move on19:24
ianw#topic OpenDev19:24
*** openstack changes topic to "OpenDev (Meeting topic: infra)"19:24
ianwi guess this was mostly the github mgmt bits which are in flight19:24
ianwany other particular updates?19:25
fungion that note19:25
fungidmsimard brought up in #zuul that we still have stale mirrors of things in the openstack-infra org on gh19:25
fungithings which didn't move to the openstack org19:26
fungiworth deciding (not necessarily in-meeting) what we want to do with those19:26
mordredI vote that we push up a retired commit with a pointer to opendev.org and then archive each of them19:26
corvusi'd guess there's quite a lot of openstack-infra/ repos in that state19:26
pabelangerianw: where can one see the github mgmt bits? I haven't really been following along as much as I would like19:26
mordredbut also agree, we don't have to agree to a plan of action here19:26
fungiwhether we push readme files to all of them or delete them and update the org description or something else entirely19:26
*** _hemna has joined #openstack-meeting19:27
* mordred can probably handle doing the retire/archive before being AFK again, if people are comfortable with that approach19:27
ianwpabelanger: sorry, yes very abstract, i was referring to the discussion started last week @ http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-06-11-19.01.log.html#l-10519:27
pabelangerianw: thanks19:28
corvusmordred: +119:28
fungii'm most ni favor of whatever someone else volunteers to do. i don't have strong feelings about our lingering repository presence on github19:28
mordredcool. I'll take a stab at $something - and will either accomplish it or give up19:28
fungiother than agreeing that stale git repositories there is probably the least desirable option19:28
mordredfungi: ++19:28
ianwok, that sounds like19:29
fungithanks mordred! and thanks dmsimard for bringing it up!19:29
ianw#action mordred look at stale github.com/openstack-infra repos and come up with retirement/removal plan/changes19:29
ianwok?19:29
mordredwhere are we keeping the rename mapping file from the Great Rename? I imagine it'll make automating the retire commits easier19:29
fungi(or jdi)19:29
mordredianw: ++19:29
fungithere were multiple phases of renames19:30
fungimay be easier to build a list by querying the gh api org object19:30
fungianything in the old openstack-infra org, retire19:30
corvuswell, we have a file from the most recent renames19:31
mordredwell - I was mostly thinking of getting the openstack-infra/zuul -> zuul/zuul but openstack-infra/puppet-zuul -> opendev/puppet-zuul mapping19:31
corvusand i bet we have the file from the great rename too19:31
corvusthose 2 should be all that's needed19:31
mordredI agree - it should be applied to all things in gh openstack-infra org19:31
fungioh, right, that file. i had already forgotten we put it all in there back to the migration batch19:31
mordredcorvus: yah19:31
pabelangermordred: I might have some code to share about adding 'archived' support to managed-projects, that is what we are using in zuul.a.c, and works well :)19:32
fungi#link https://opendev.org/opendev/project-config/src/branch/master/renames19:32
mordredpabelanger: sweet! I'll totally borrow that from you19:32
pabelangerhttps://github.com/ansible/project-config/commit/bb2d4d18ed79ef68e6f8b22b08cecd0a14f6faf519:32
mordredfungi: AWESOME - thank you19:32
corvus"unmanage-projects"19:32
pabelangerhappy to push that up19:32
mordredpabelanger: that's nice and easy to copy19:33
mordredpabelanger: I'll likely write a one-off script for this, so that's good enough for me19:33
*** panda has quit IRC19:33
pabelangerack19:34
mordred(it will almost certainly be a very ugly script)19:34
ianw#topic Storyboard19:35
*** openstack changes topic to "Storyboard (Meeting topic: infra)"19:35
fungistoryboard-api releases on pypi are a thing now19:35
fungiwe released 1.0.0 last week and 1.0.1 over the weekend19:35
fungior actually yesterday morning my time19:35
mordredneat19:36
fungiover the weekend, logins were broken because of the distname change from storyboard to storyboard-api19:36
fungifor whatever reason, the auth codepath includes asking pbr for some dist info19:36
mordredo_O19:37
fungiand we missed updatnig the string being passed into it19:37
fungithat along with a very annoying but infrequent db lock contention bug were fixed in 1.0.119:37
*** _erlon_ has joined #openstack-meeting19:38
ianwfungi: i think take a don't ask, don't tell policy with that :)19:38
ianw(the pbr stuff)19:38
*** e0ne has joined #openstack-meeting19:38
fungiindeed19:38
fungii mean, we can pickaxe the history, but it likely will take us back to commit #119:38
mordredit's probably my fault19:39
fungialso the karbor team changed their minds and decided they did want their lp bugs imported after all, so i did that yesterday19:39
ianwok, anything else?19:39
fungii don't know of any other pressing news, but maybe SotK or diablo_rojo are around?19:39
fungiif not, we can surely move on19:40
ianw#topic General Topics19:40
*** openstack changes topic to "General Topics (Meeting topic: infra)"19:40
ianwtrusty upgrade19:40
ianw#link https://etherpad.openstack.org/p/201808-infra-server-upgrades-and-cleanup19:40
fungistill poking at the wiki-dev02 xenial build as tmie allows19:41
ianwand then i think it's really just status.openstack.org ?19:41
mordredI continue to not actually do the status upgrade19:41
mordredbut I'm totally enjoying continuing to not actually do it19:41
diablo_rojoI got nothing pressing19:41
fungithe current snag is that something like 2/3 of our ~40 extensions and our half a dozen skins can't be cloned by the vcsrepo module and their .git ends up being a file instead of a directory, with just a single line like "gitdir: ../../.git/modules/extensions/Renameuser"19:42
funginot sure if that means it's some submodule magic or what19:42
fungi(does that pattern look familiar to anyone?)19:42
mordredfungi: no - but I think your reasoning that leads you to suspect submodules seems likely19:43
*** panda has joined #openstack-meeting19:43
fungiwe're cloning from https://gerrit.wikimedia.org/r/p/mediawiki/${type}s/${name}.git19:44
ianwno but i seem to remember having some issue with vcsrepo that required a much later version ... trying to remember what that was ... i think with ask.o.o ...19:44
fungi#link https://opendev.org/opendev/puppet-mediawiki/src/branch/master/manifests/extension.pp#L1319:44
fungihttps://gerrit.wikimedia.org/r/p/mediawiki/vendor.git is also getting us a clone like that19:44
fungibut if i got clone it manually i'm getting an actual repo19:45
fungis/got/git/19:45
mordredoh my.19:45
mordredgot git?19:45
*** slaweq has joined #openstack-meeting19:45
fungianyway, not going to solve it right now, bit anyone who has ideas, i'm all ears19:45
fungiianw's vcsrepo version concern is definitely worth digging into19:46
fungiwe can probably move on as there's more on the agenda19:46
ianwok19:46
ianwyeah i wanted to update on the opendev.org mirror situation19:46
ianwseveral things in flight there19:47
ianwin short, if we could review19:47
ianw#link https://review.opendev.org/#/q/status:open+branch:master+topic:kafs19:47
fungistarred19:48
ianwbasically the openafs version shipping with bionic (1.8.0~pre5) is buggy; so we either use our own built later versions, or something else like kafs19:48
ianwkafs, for those not involved, is the inbuilt kernel support for afs.  so we don't have to build any modules with dkms etc etc like with openafs.  the only issue is you need a very recent kernel19:49
ianwlike, top of tree master kernel (5.2-rcX)19:49
fungifrom what we can tell so far, one so new it's not released yet ;)19:49
ianwso that is obviously not without issue either19:50
corvusit is interesting to be on the bleeding edge of afs development.19:50
ianwwe have mirror.iad.opendev.org running this, currently setup by hand, but the reviews in the above topic make it all fairly nicely ansible controlled19:50
fungii'm just happy that there is still a bleednig edge for afs development19:51
ianwwe also have mirror.dfw.opendev.org running upstream 1.8.3 openafs taken from packages i put in our ppa19:51
ianwafaict, that has been running fine for several days now19:51
* mordred is supportive of both options19:53
ianwi feel like at this point we would want to roll out the servers with openafs 1.8.3 -- devil you know --(changes to do that in the stack above too), but opt in a few to kafs?19:53
ianwi would like to bring up a very remote to rax kafs node, to see what latency and network weirdness does to it19:53
fungiit would be nice if ubuntu could at least replace their prerelease afs package with a more updated one that isn't thoroughly broken19:53
fungibut my hopes are not high there19:53
ianwfungi: i have filed an issue ... if we know a way to push on that i'm happy to19:54
fungi(for bionic i mean)19:54
corvusthe shutdown of the kafs server is suspicious, but it still could be a random unrelated incident.19:54
ianwthere's a link in the reviews19:54
*** rubasov has quit IRC19:54
ianwneed to look into that ... i had a little background monitoring going on too after discussion in openafs last night19:54
fungiright, i'm far from blaming kafs on an instance in rackspace spontaneously (from our perspective) going into shutdown state19:54
corvusit's also probably easier to maintain our own openafs packages rather than a kernel19:54
fungithis is true19:55
fungithough the kafs implementation has an awful lot of cool factor to it19:55
fungiand that sure counts for something19:55
ianwyeah, with 1.8.3 seeming to be stable now, i think that should be our default.  but we've got great CI, and with those changes we can keep both paths in flight19:55
ianw(secretly, i'm hoping upstream notice just how cool our CI is and want to get more involved than just occasional advice ...)19:56
corvusianw: i like both of your suggestions -- maybe start thinking about resolving the current situation with openafs and continue testing kafs in more situations, since it may be good for us long-term19:56
fungii too see kafs as worthwhile investment in the future19:56
fungieven if we don't end up relying on it right now19:56
ianw++ seems like we're all on about the same page :)19:57
ianw#topic Open discussion19:57
*** openstack changes topic to "Open discussion (Meeting topic: infra)"19:57
ianwany last minute review requests, etc?19:57
fungi#link https://review.opendev.org/655783 Don't hide Zuul CI comments19:59
fungimight be nice to come to a decision around that19:59
fungijhesketh had some good suggestions20:00
ianwhide all but last seems like a nice intermediate, if the hacking required isn't too crazy20:00
*** _hemna has quit IRC20:00
*** jamesmcarthur has joined #openstack-meeting20:01
ianwok, thanks everyone, back to #openstack-infra for any other concerns!20:01
ianw#endmeeting20:01
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"20:01
openstackMeeting ended Tue Jun 18 20:01:45 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)20:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-06-18-19.00.html20:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-06-18-19.00.txt20:01
openstackLog:            http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-06-18-19.00.log.html20:01
fungithanks ianw!20:01
*** bbowen_ has joined #openstack-meeting20:13
*** _hemna has joined #openstack-meeting20:28
*** eharney has joined #openstack-meeting20:35
*** dtrainor has quit IRC20:35
*** dtrainor has joined #openstack-meeting20:36
*** e0ne has quit IRC20:36
*** whoami-rajat has quit IRC20:56
*** rfolco has quit IRC21:01
*** _hemna has quit IRC21:02
*** dtroyer has quit IRC21:09
*** dtroyer has joined #openstack-meeting21:11
*** jamesmcarthur has quit IRC21:17
*** enriquetaso has quit IRC21:19
*** ayoung has quit IRC21:38
*** mriedem is now known as mriedem_away21:47
*** tesseract has quit IRC21:47
*** markvoelker has joined #openstack-meeting21:48
*** markvoelker has quit IRC22:00
*** mattw4 has quit IRC22:00
*** mattw4 has joined #openstack-meeting22:01
*** raildo has quit IRC22:02
*** slaweq has quit IRC22:08
*** slaweq has joined #openstack-meeting22:11
*** evgenyl has quit IRC22:12
*** rajinir has quit IRC22:15
*** slaweq has quit IRC22:16
*** rajinir has joined #openstack-meeting22:19
*** evgenyl has joined #openstack-meeting22:19
*** _hemna has joined #openstack-meeting22:21
*** carloss has quit IRC22:28
*** Lucas_Gray has joined #openstack-meeting22:29
*** diablo_rojo has quit IRC22:44
*** _hemna has quit IRC22:48
*** rtjure has quit IRC23:07
*** rcernin has joined #openstack-meeting23:16
*** lseki has quit IRC23:16
*** rcernin has quit IRC23:17
*** _erlon_ has quit IRC23:17
*** rcernin has joined #openstack-meeting23:18
*** Lucas_Gray has quit IRC23:28
*** mattw4 has quit IRC23:58
*** ykatabam has joined #openstack-meeting23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!