Tuesday, 2021-05-11

*** artom has quit IRC00:28
*** hemanth_n has joined #openstack-meeting-300:41
*** hemanth_n has quit IRC00:50
*** pongboom2 has joined #openstack-meeting-301:10
*** pongboom has quit IRC01:13
*** hemanth_n has joined #openstack-meeting-302:09
*** ralonsoh has joined #openstack-meeting-305:10
*** psahoo has joined #openstack-meeting-305:21
*** bnemec has quit IRC05:27
*** bnemec has joined #openstack-meeting-305:29
*** macz_ has joined #openstack-meeting-307:02
*** macz_ has quit IRC07:07
*** tosky has joined #openstack-meeting-307:39
*** e0ne has joined #openstack-meeting-308:02
*** lpetrut has joined #openstack-meeting-308:22
*** macz_ has joined #openstack-meeting-308:45
*** macz_ has quit IRC08:50
*** macz_ has joined #openstack-meeting-309:55
*** macz_ has quit IRC10:00
*** Luzi has joined #openstack-meeting-310:03
*** lpetrut has quit IRC10:30
*** lpetrut has joined #openstack-meeting-310:56
*** e0ne has quit IRC11:01
*** lpetrut has quit IRC11:27
*** e0ne has joined #openstack-meeting-311:27
*** njohnston has joined #openstack-meeting-311:57
*** njohnston has quit IRC12:06
*** njohnston has joined #openstack-meeting-312:11
*** psahoo_ has joined #openstack-meeting-312:28
*** psahoo has quit IRC12:32
*** e0ne has quit IRC12:35
*** njohnston has quit IRC12:35
*** njohnston has joined #openstack-meeting-312:42
*** njohnston has quit IRC12:45
*** njohnston has joined #openstack-meeting-312:46
*** njohnston has quit IRC12:54
*** njohnston has joined #openstack-meeting-312:55
*** hemanth_n has quit IRC13:06
*** artom has joined #openstack-meeting-313:41
*** stephenfin has quit IRC13:56
*** obondarev has joined #openstack-meeting-313:57
*** mlavalle has joined #openstack-meeting-313:58
*** Luzi has quit IRC13:59
slaweq#startmeeting networking14:01
openstackMeeting started Tue May 11 14:01:16 2021 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.14:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:01
*** openstack changes topic to " (Meeting topic: networking)"14:01
slaweqhi14:01
openstackThe meeting name has been set to 'networking'14:01
mlavalleo/14:01
obondarevhi14:01
njohnstono/14:01
rubasovhi14:01
ralonsohhi14:01
slaweqwelcome everyone14:02
slaweqlet's start with first topic14:02
slaweq#topic announcements14:02
*** openstack changes topic to "announcements (Meeting topic: networking)"14:02
slaweqAccording to Xena calendar, first milestone is May 24th14:02
slaweqso in about 2 weeks from now14:02
slaweqnext one14:03
slaweqWe have bunch of security bugs opened http://lists.openstack.org/pipermail/openstack-discuss/2021-May/022353.html14:03
slaweqI will need some help with triaging those LPs14:03
mlavalleI'll help14:04
slaweqthx14:05
slaweqand that's all announcements from me for today14:05
mlavalleany pressing deadline?14:05
mlavallefor the triaging I mean14:05
slaweqmlavalle: no, but as those are security issues, we should probably do it in some reasonable time14:05
amotokithey are public security so there is no specific deadline but it would be nice if we can triage them.14:06
amotokiI will look into them too.14:06
slaweqthx amotoki14:06
mlavalleok, I'll try to go over as many as possible by early next week14:06
slaweqthx both of You14:06
slaweqanything else You want to share with the team today?14:06
*** e0ne has joined #openstack-meeting-314:07
slaweqif not, let's move on to the next topic14:07
slaweq#topic Blueprints14:07
*** openstack changes topic to "Blueprints (Meeting topic: networking)"14:07
slaweqBPs for Xena-1 are on the list https://bugs.launchpad.net/neutron/+milestone/xena-114:08
slaweqdo You have any updates about them?14:08
slaweqok, from my side I just want to ask for review patches related to secure-rbac https://review.opendev.org/q/topic:%2522secure-rbac%2522+(status:open+OR+status:merged)+project:openstack/neutron14:09
slaweqthere are still some of them which needs to be checked14:09
slaweqbut fortunatelly not a lot :)14:10
slaweqplease also check list of opened specs and review some https://review.opendev.org/q/project:openstack/neutron-specs+status:open14:11
slaweqI will try to review them too14:11
slaweqespecially those proposed by gibi, lajoskatona and rubasov :)14:11
gibislaweq: thanks :)14:12
rubasovthank you14:12
slaweqyw :)14:12
mlavalleany of those specs we should prioritize?14:12
mlavallethere are a few14:13
slaweqIIUC what was said during the ptg, all of them are equally important14:13
slaweqis that correct gibi, rubasov?14:13
rubasovgood question14:14
rubasovright now I don't have an order in my mind14:14
gibirubasov: do we have some dependency order between the l3 spces?14:14
gibispecs14:14
rubasovbut I'll come back if I realize some order14:14
gibiat leat the pps spec is indepented from the rest14:14
mlavallethe reason I ask is that in the list I got things like https://review.opendev.org/c/openstack/neutron-specs/+/60132014:15
slaweqmlavalle: no, I listed all opened specs14:15
mlavallewhich I don't remeber we discussed in the PTG14:15
slaweqso let's do it that way14:15
*** macz_ has joined #openstack-meeting-314:15
rubasovthat one is not from us14:16
slaweqI will tonight go over all opened specs and will set review-priority to +1 for those which are important for Xena cycle14:16
mlavallerubasov, gibi: don't worry to much on prioritize your specs14:16
slaweqmlavalle: will that be ok?14:16
gibimlavalle: ack14:16
mlavallerubasov, gibi: I'll look at your specs that we recently discussed14:16
gibithank you14:16
rubasovthank you14:17
mlavalleslaweq: it would be helpful, bit please don't sweat it too much. I just was surprsied at some of the things that came out in the list14:17
slaweqsure14:17
mlavalleI have enough with this clarification14:17
slaweqk14:18
slaweqthx for asking14:18
*** lajoskatona has joined #openstack-meeting-314:18
lajoskatonaHi, sorry for beeing late14:19
slaweqif there are no other updates about BPs, I think we can move on14:19
slaweq#topic Bugs14:19
*** openstack changes topic to "Bugs (Meeting topic: networking)"14:19
slaweqlucasgomez was bug deputy two weeks ago, report: http://lists.openstack.org/pipermail/openstack-discuss/2021-May/022168.html14:19
slaweqjlibosva was bug deputy last week but I don't see report from him14:20
slaweqfrom the bugs which lucasgomes listed, there is couple which don't have any assignment:14:22
slaweq         https://bugs.launchpad.net/neutron/+bug/1926109 - CI issue14:22
openstackLaunchpad bug 1926109 in neutron "SSH timeout (wait timeout) due to potential paramiko issue" [Critical,New]14:22
slaweq        https://bugs.launchpad.net/neutron/+bug/1926780 - CI issue14:22
openstackLaunchpad bug 1926780 in neutron "Multicast traffic scenario test is failing sometimes on OVN job" [High,Confirmed]14:22
slaweq        https://bugs.launchpad.net/neutron/+bug/1926531 - DVR issue14:22
openstackLaunchpad bug 1926531 in neutron "SNAT namespace prematurely created then deleted on hosts, resulting in removal of RFP/FPR link to FIP namespace" [Undecided,New]14:22
slaweq        https://bugs.launchpad.net/neutron/+bug/1926838 - OVN bug14:22
openstackLaunchpad bug 1926838 in neutron "[OVN] infinite loop in ovsdb_monitor" [High,New]14:22
slaweqso if anyone would have some time, would be great if You can take a look into them :)14:23
ralonsohsure14:24
slaweqany other bugs You want to discuss today?14:24
*** e0ne has quit IRC14:25
slaweqok, if not, let's move on14:26
slaweq#topic On Demand Agenda14:26
*** openstack changes topic to "On Demand Agenda (Meeting topic: networking)"14:26
slaweqralonsoh: You have one topic here14:26
ralonsohthanks14:26
ralonsohlet me find the LP14:26
slaweqhttps://bugs.launchpad.net/neutron/+bug/192678714:26
openstackLaunchpad bug 1926787 in neutron "[DB] Neutron quota request implementation can end in a lock status" [Wishlist,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez)14:26
slaweqthat one?14:26
slaweq:)14:26
ralonsohright!14:26
ralonsohand the POC: https://review.opendev.org/c/openstack/neutron/+/79006014:27
ralonsohin a nutshell: the current quota driver uses a global lock for each (resource, project_id)14:27
ralonsohas commented in the LP bug, that could lead to a lock state in the DB if number of queries > number of replies14:27
ralonsohwhat I'm proposing is an alternative driver 100% compatible with the current quota system14:28
ralonsohthe main part is14:28
ralonsoh--> https://review.opendev.org/c/openstack/neutron/+/790060/6/neutron/db/quota/driver_nolock.py14:28
ralonsohhow the reservations are made and how the resource quota is calculated14:29
ralonsohinstead of using a used_quota table, count the resources directly on the DB, along with the reserved ones and make, if quota is available, a new reservation14:29
ralonsohthat's all, thanks!14:29
obondarevralonsoh, does lock state affect all DB engines or just specific ones?14:30
ralonsohthat depends on the engine used14:30
ralonsohbut by default, if you use innodb tables, you'll have this lock14:30
obondarevgot it14:31
obondarevthanks14:31
ralonsohwhat I'm trying to do is to use the DB transactionality, instead of creating a global lock (in the BD)14:31
obondarevso this new approach gives a performance improvement as well, right ralonsoh?14:31
ralonsohfor sure!14:31
obondarevnice!14:31
mlavalle+114:31
ralonsohthanks a lot, I'll push a new patch and a periodic CI job to using this driver14:32
slaweqone question - what is the purpose of keeping both drivers in tree?14:32
ralonsohs/to using/to use14:32
ralonsohwell, I think first we need to test it14:33
slaweqdo we want to deprecate old one at some point?14:33
ralonsohI think so14:33
slaweqor keep both forever as they are maybe better/worst to some use cases?14:33
lajoskatonaIf we need to keep both for some reason a configg option could be added14:33
ralonsohwe already have it14:33
slaweqlajoskatona: there is already config option to specify quota driver which should be used14:34
lajoskatonaok, that's done:-)14:34
ralonsohbut I don't see any use case where the old one is better14:34
ralonsohwhat I can do, if you want14:34
ralonsohis to change the default one14:34
*** stephenfin has joined #openstack-meeting-314:34
slaweqmaybe we should simply replace old one with that new one?14:34
ralonsohthat will test it in any CI job14:34
slaweqof course it will be tested in all our ci jobs :)14:35
ralonsohas you wish14:35
ralonsohdo you want me to replace it?14:35
slaweqI'm just asking :)14:35
ralonsohI think we can keep both for now14:35
amotokithe config quota driver can be deprecated but our UT depends on it.14:35
ralonsohand next release, when tested, remove the old one14:35
amotokibefore we need to drop the config driver, we need to revisit our UT14:35
obondarevI'd give some time for new driver to prove it's validity :)14:35
ralonsoh^^ exactly14:36
slaweqmy main concern is that if it will not be default, nobody will really test it14:36
slaweqbut maybe I'm wrong (hopefully)14:36
ralonsohI can change the default one to the new one14:36
ralonsohthat will test this driver in all CI jobs14:36
ralonsohand in Y, deprecate the old one14:37
obondarev+114:37
slaweqIMHO that sounds like good plan - people may test our new driver and in case of issues, there is always easy way to get back to the old one14:37
slaweqif all will be fine, we can simply deprecate and later delete old one14:38
ralonsohthank you all14:38
mlavalle+114:39
slaweqralonsoh: thx for proposing that, and let's go with that :)14:40
ralonsohthanks14:40
slaweqany other topics You want to discuss today?14:40
slaweqif not, I will give You about 20 minutes back :)14:40
slaweqok, so thx for attending the meeting14:41
ralonsohbye14:41
mlavalleo/14:41
slaweqahh, one more thing14:41
slaweqsorry, I forgot14:41
slaweqbug deputy14:41
mlavalleLOL14:41
slaweqthis week it's obondarev's turn14:42
slaweqand next week we are starting new round14:42
obondarevack14:42
slaweqso please check https://wiki.openstack.org/wiki/Network/Meetings#Bug_deputy14:42
slaweqand if it's not good for You, please let me know so I can change it if needed14:42
slaweqnow, it's all done :)14:42
mlavalle\o/14:43
slaweqthx for attending the meeting and see You all online14:43
slaweqo/14:43
slaweq#endmeeting14:43
lajoskatonao/14:43
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"14:43
openstackMeeting ended Tue May 11 14:43:21 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:43
openstackMinutes:        http://eavesdrop.openstack.org/meetings/networking/2021/networking.2021-05-11-14.01.html14:43
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/networking/2021/networking.2021-05-11-14.01.txt14:43
openstackLog:            http://eavesdrop.openstack.org/meetings/networking/2021/networking.2021-05-11-14.01.log.html14:43
rubasovo/14:43
amotokio/14:43
mlavalleo/14:43
mlavallefor good now14:43
*** obondarev has quit IRC14:45
slaweq#startmeeting neutron_ci15:01
openstackMeeting started Tue May 11 15:01:21 2021 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.15:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:01
slaweqhi15:01
*** openstack changes topic to " (Meeting topic: neutron_ci)"15:01
openstackThe meeting name has been set to 'neutron_ci'15:01
ralonsohhi15:01
lajoskatonaHi15:01
slaweqGrafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate15:02
slaweqI think we can start now15:02
slaweq#topic Actions from previous meetings15:02
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"15:02
slaweqralonsoh to mark test_keepalived_spawns_conflicting_pid_vrrp_subprocess functional test as unstable15:02
ralonsohok, that won't fix the CI15:03
ralonsohbecause this test is timing out15:03
ralonsohand that cannot be marked as unstable15:03
slaweqhow often it happens? do You remember?15:04
ralonsohnot too often, but still too much15:04
ralonsohthis is because the sigterm signal does not reply15:04
ralonsoheven with shell "kill" command (same as with os.kill)15:04
ralonsohI was working in a patch for privsep15:05
ralonsohbut I never finished it15:05
ralonsohI'll continue with this one, but I'll need time15:05
slaweqsure15:06
slaweqmaybe we should simply skip those tests for now?15:06
ralonsohI think so, I'll send a patch15:07
slaweq++15:07
slaweq#action ralonsoh to skip test_keepalived_spawns_conflicting_pid_vrrp_subprocess functional tests15:07
slaweqok, next one15:07
slaweqlajoskatona to check stadium job's and what needs to be switched to ovs explicity15:07
lajoskatonaYeah, I have to run through it again, but as I remember we are ok15:08
slaweqthat's good news, thanks lajoskatona15:08
slaweqand what about networking-sfc? this one has to be switched right?15:08
lajoskatonayes15:10
slaweqk15:10
slaweqcan You send patches to switch that one?15:11
slaweqor maybe bcafarel already did that?15:11
lajoskatonayes, I can,15:11
slaweqthx a lot15:11
slaweq#action lajoskatona to switch networking-sfc jobs to ovs explicitly15:11
slaweqnext one     ralonsoh or lajoskatona will add logging of the console log, related to the https://bugs.launchpad.net/neutron/+bug/192610915:12
openstackLaunchpad bug 1926109 in neutron "SSH timeout (wait timeout) due to potential paramiko issue" [Critical,New]15:12
lajoskatonaI checked this one and as I see we log console log everywhere, but for some reason it is not always logged to subunit html report15:13
lajoskatonaonly in tempest.log15:13
slaweqahh, ok15:15
slaweqgood to know15:15
slaweqso we need to check in tempest.log in such case15:16
slaweqthx15:16
slaweqok, next one15:16
slaweqslaweq to check frequency of the multicast issue in the ovn job and report a LP bug for that15:16
slaweqI checked and opened bug https://bugs.launchpad.net/neutron/+bug/192678015:16
openstackLaunchpad bug 1926780 in neutron "Multicast traffic scenario test is failing sometimes on OVN job" [High,Confirmed]15:16
slaweqbut I didn't had time to look at it more15:16
slaweqok, next one15:17
slaweqralonsoh: to check periodic nftables jobs15:17
ralonsohdone and merged15:18
ralonsoh(I'll find the patch)15:18
slaweqthank You :)15:18
slaweqno need, we trust You ;)15:18
ralonsohhehehe thanks15:18
slaweqok, let's move on15:18
slaweq#topic Stadium projects15:18
*** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)"15:18
slaweqlajoskatona: anything regarding stadium projects ci?15:18
lajoskatonanothing, as I remmeber from the week15:20
slaweqgreat :)15:20
slaweqthat's news which I like in this meeting ;)15:20
lajoskatonafor bgpvpn (or bagpipe?) I pushed patches for stable brancehs to make l-c job non-voting15:20
slaweq#topic Stable branches15:20
*** openstack changes topic to "Stable branches (Meeting topic: neutron_ci)"15:20
lajoskatonayeah so this overlaps the next topic :-)15:21
slaweqthx, I will review that patch15:21
slaweqregarding stable branches, I have just one thing15:21
lajoskatonathis one is active, and I hope can be backported: https://review.opendev.org/c/openstack/networking-bgpvpn/+/74348715:21
slaweqjust few minutes ago there was proposed patch to go to EM phase in train15:21
slaweqhttps://review.opendev.org/c/openstack/releases/+/79076015:22
slaweqwhen this will be done we should pin neutron-tempest-plugin in stable/train jobs and remove them from the check/gate queues in neutron-tempest-plugin repo15:22
slaweqI will do it this week15:22
lajoskatona+115:22
slaweq#action slaweq to pin neutron-tempest-plugin for train after it will be EM15:23
slaweqother than that I didn't saw any issues with stable branches and ci15:23
slaweqthings are merging pretty ok AFAICT15:23
slaweqI think we can move on15:25
slaweq#topic Grafana15:25
slaweq#link http://grafana.openstack.org/dashboard/db/neutron-failure-rate15:25
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"15:25
slaweqI don't see any serious issues there recently15:26
slaweqseems like things are running pretty ok15:26
slaweqexcept failures which we already know15:26
slaweqwe should maybe go with https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/788412 to make our neutron-tempest-plugin jobs to be more stable15:27
lajoskatonaNearly offtopic: have you read the mail about stopping logstash?15:27
slaweqlajoskatona: yes, I read it today and I'm sad about that :/15:27
lajoskatonait was not perfect but better than nothing :-(15:28
slaweqyeah15:28
lajoskatonasorry for disturbing...15:28
slaweqno problem at all15:29
ralonsoh+w to this patch15:29
slaweqif You know about any alternative, I will be very happy to listen about it :)15:29
slaweqthx ralonsoh15:29
slaweqok, let's move on15:31
slaweqtbh I didn't found any new serious issues worth to discuss here today15:32
slaweqI did found few functional tests failures, but each of them were just one time failure so far15:32
slaweqI listed them in the https://etherpad.opendev.org/p/neutron-ci-meetings15:32
slaweqother than that there are only known issues which I saw in our ci15:33
slaweqdo You maybe have any issues which You would like to discuss?15:33
ralonsohwell, FT CI job is not very stable15:33
ralonsohi've seen a lot of timeouts recentlyt15:34
ralonsohbut I din't find the cause15:34
slaweqhmm15:34
slaweqI didn't saw many timeouts when I was checking that15:34
slaweqbut I will check once again, maybe I missed something15:34
slaweq#action slaweq to check functional job timeouts15:34
slaweqand that's basically all from me for today15:37
slaweqif You don't have anything else, we can finish meeting earlier15:37
ralonsohno, thanks15:37
lajoskatona\o/15:37
slaweqok15:38
slaweqthx for attending the meeting15:38
slaweqand have a great evening :)15:38
slaweqo/15:38
slaweq#endmeeting15:38
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"15:38
openstackMeeting ended Tue May 11 15:38:18 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:38
ralonsohbye!15:38
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2021/neutron_ci.2021-05-11-15.01.html15:38
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2021/neutron_ci.2021-05-11-15.01.txt15:38
lajoskatonabye15:38
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2021/neutron_ci.2021-05-11-15.01.log.html15:38
*** psahoo_ has quit IRC15:57
*** ralonsoh has quit IRC16:25
*** lpetrut has joined #openstack-meeting-317:55
*** lpetrut has quit IRC17:57
*** lajoskatona has quit IRC19:55
*** macz_ has quit IRC20:09
*** srwilkers has quit IRC20:09
*** Adri2000 has quit IRC20:09
*** macz_ has joined #openstack-meeting-320:13
*** srwilkers has joined #openstack-meeting-320:13
*** Adri2000 has joined #openstack-meeting-320:13
*** slaweq has quit IRC21:19
*** macz_ has quit IRC23:17
*** tosky has quit IRC23:24
*** mlavalle has quit IRC23:44

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!