15:00:32 #startmeeting neutron_l3 15:00:33 thanks guys 15:00:33 Meeting started Thu Aug 11 15:00:32 2016 UTC and is due to finish in 60 minutes. The chair is mlavalle. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:34 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:37 The meeting name has been set to 'neutron_l3' 15:00:38 hi 15:00:43 hi 15:00:44 o/ 15:00:50 o/ 15:01:00 #chair carl_baldwin 15:01:01 Current chairs: carl_baldwin mlavalle 15:01:24 is tidwellr around? 15:01:47 Agenda for today is here: 15:01:55 #link https://etherpad.openstack.org/p/neutron-l3-subteam 15:02:08 #topic Announcements 15:02:28 \o/ 15:02:46 The obvious reminder is the Neutron mid-cycle meeting next week in Cork Ireland 15:03:27 It is coming right up. Looking forward to seeing people there. 15:04:47 The other annoucement is we have to keep an eye on Newton-3 milestone 15:05:02 #link http://releases.openstack.org/newton/schedule.html 15:05:14 August 29th - September 2nd 15:05:24 so it is coming our way really quickly 15:05:52 any other annoucements from the team? 15:06:03 Not from me. 15:06:34 if not, let's move on 15:06:38 #topic Bugs 15:07:13 The first up is one that haleyb and jschwarz have been discussing in the Neutron channel: https://bugs.launchpad.net/neutron/+bug/1612192 15:07:13 Launchpad bug 1612192 in neutron "L3 DVR: Unable to complete operation on subnet" [Critical,Confirmed] 15:08:20 from what I got from the conversation, it might not be a Neutron issue but rather a Tempest one? 15:08:21 mlavalle: yes, there seems to be an issue in a tempest test - add-router-interface is failing, and the unwind is barfing on a port still being in the subnet 15:08:57 haleyb, mind you, I saw this happen on rally a while back as well 15:08:58 but i see a DBDeadlock on create_port() so wonder if it's an ml2 change 15:09:12 maybe it's an API change that snuck into Neutron unawares? 15:10:35 * haleyb feels like he slides down the pole info a fire station every morning :) 15:10:58 so can we say that we need to research this one further? 15:11:35 mlavalle: yes, need to look further, will scream if i need help 15:12:03 haleyb, jschwarz thank you for keeping an eye on this 15:12:31 Next up is https://bugs.launchpad.net/neutron/+bug/1540983 15:12:31 Launchpad bug 1540983 in OpenStack-Gate "Gate failures for neutron in test_dualnet_multi_prefix_slaac" [Undecided,Expired] 15:12:51 So this morning I went to logstash to try to find ocurrences of this bug 15:13:21 I am using the query at the top of the bug: message:"in test_dualnet_multi_prefix_slaac" AND voting:1 15:13:35 and couldn't find a case over the past 7 days 15:13:50 Am using a wrong query maybe? 15:14:39 mlavalle: logstash wasn't cooperating for me today either, but that one is infrequent 15:15:01 I was thinking there was another related one. Me trying to swap in memories of that. 15:15:19 I did however see a similar failure in the dvr tests, in that case dhcp failed to start, so second VM failed to get IP, and it went downhill from there 15:16:09 ok, I'll keep an eye on it daily, to make sure we don't get in trouble close to N-3 15:16:27 Could be related to https://bugs.launchpad.net/neutron/+bug/1609540 15:16:27 Launchpad bug 1609540 in neutron "Deleting csnat port fails due to no fixed ips" [Critical,In progress] - Assigned to Carl Baldwin (carl-baldwin) 15:17:16 Yeah, that's the next one in the agenda 15:17:47 and carl_baldwin and I couldn't find cases of the expected message yesterday 15:17:56 They both involve the same unit test. 15:18:13 mlavalle: Yeah, that is strange. I would expect to see that debug message. 15:18:38 I will talk to infra today to make sure logstash catches debug level messages 15:19:17 I guess all we can do for the time being is to be vigilant about these 2 bugs 15:19:40 I'll check them daily and will talk to infra 15:20:14 mlavalle: Thanks. 15:20:37 It'd be nice to know that we can search for debug messages successfully. 15:21:05 Next bugs are high importance. First one was reduced to high lately: https://bugs.launchpad.net/neutron/+bug/1562878 15:21:05 Launchpad bug 1562878 in neutron "L3 HA: Unable to complete operation on subnet" [High,Confirmed] - Assigned to Ann Taraday (akamyshnikova) 15:21:20 Thanks to jschwarz for followin up with it. any comments? 15:21:22 I tried to reproduce this one earlier this week but couldn't 15:21:46 since it's not occurring in the gate afaik, the importance can be lowered IMO 15:21:58 even lower than high? 15:22:36 Medium seems nice since if me and Ann can't reproduce this on rally, this might have been fixed already 15:23:06 ok, thanks 15:23:38 Next up is https://bugs.launchpad.net/neutron/+bug/1596075 15:23:38 Launchpad bug 1596075 in neutron "Neutron confused about overlapping subnet creation" [High,In progress] - Assigned to Kevin Benton (kevinbenton) 15:24:02 As I said last week, this is a long complicated affair, involving several potential patches 15:24:52 I pinged kevinbenton yesterday and he is still working on a couple of fixes for this. Once they are ready, he will have some more interaction with the submitter to confir it is fixed 15:25:41 Next up is https://bugs.launchpad.net/neutron/+bug/1603162 15:25:41 Launchpad bug 1603162 in neutron "Pluggable IPAM rollback fails with reference driver" [High,In progress] - Assigned to Carl Baldwin (carl-baldwin) 15:25:58 I think I've got this one fixed. 15:26:23 I was just starting to look at the multinode dvr grenade job. I doubt the failure is related. 15:26:24 #link https://review.openstack.org/#/c/348956/ 15:26:50 I wanted to be sensitive to rechecks thuogh. 15:26:54 *though 15:27:12 ++ 15:28:21 if there are no more comments, let's move on. Thanks for the update carl_baldwin 15:28:38 I'll get some reviewers on the fix today. 15:28:49 Thnaks! 15:29:02 Next up is https://bugs.launchpad.net/neutron/+bug/1610483 15:29:02 Launchpad bug 1610483 in neutron "Pluggable IPAM rollback mechanism is not robust" [High,Confirmed] 15:30:09 any comments on this one carl_baldwin? 15:30:21 This affects external drivers mostly since the reference driver (for now) uses the context DB rollback. 15:31:09 pavel_bondar: Have you guys had a chance to think about this at all? 15:31:43 carl_baldwin: yes, I agree with the issue, current rollback is not actually reliable and has to be reworked 15:32:27 So, in summary, we have no plans yet to fix this but it is an issue that we should plan for soon. 15:33:33 Thanks! 15:33:43 I would pick this task, but since I am not longer part of Infoblox openstack team (working on Infoblox another project) I don't have enough bandwith to drive it to the end. 15:34:47 Probably I could assist with comments&review, but probably what my current bandwidth allows 15:34:50 pavel_bondar: Anyone else there to pass it on to? 15:35:21 We can take this out of band, mlavalle 15:35:40 Finally we have https://bugs.launchpad.net/neutron/+bug/1599329 15:35:40 Launchpad bug 1599329 in neutron "Potential regression on handing over DHCP addresses to VMs" [High,In progress] 15:35:42 carl_baldwin: it is better to check with John B. about it, I am not sure 15:35:51 pavel_bondar: Will do. 15:36:14 We were waiting to see if a fix solved this one. Haven't heard anything 15:36:32 I will check around today about this one 15:36:41 any other comments? 15:37:00 mlavalle: that looks similar to something i noticed yesterday - VM dhcp fails 15:37:50 http://logs.openstack.org/51/337851/19/check/gate-tempest-dsvm-neutron-dvr-multinode-full/c944b3d/logs/screen-q-dhcp.txt.gz was the info i found so far, but it was a multinode failure, not strict dvr 15:38:23 ok, will take a look. will ping you if i have questions 15:38:46 tx 15:39:17 #topic Routed networks 15:39:43 Hi 15:40:04 I think we're doing pretty well here. 15:40:27 We have had some review on the create / delete segment ml2 patch. 15:40:34 yeah 15:40:44 I'm not sure if xiaohhui is in a position to handle the feedback. 15:41:15 #link https://review.openstack.org/#/c/317358 15:41:23 Looks like xiaohhui is on it. 15:41:49 yeah he uploaded a revision last night 15:42:23 Good, I had written him an email to see if he needs assistance. I hadn't heard back. 15:42:29 I'll keep watching it. 15:43:09 Do we have anything else pressing for Newton? 15:43:35 I'll push the segment ids in port patch next revision today 15:43:59 and we may need to pay some attention to docs 15:44:43 Yes, docs! My only hope now is the plane ride. :) 15:45:23 Cool! Let's move on then 15:45:35 #topic BGP Dynamic Routing 15:45:41 hi 15:45:45 It'll be nice to go over how we're doing at the mid-cycle and figure out what we need to do for Ocata. 15:45:45 tidwellr, steve_ruan you are up 15:46:03 That's the last thing about routed networks from me ^ 15:46:08 :) 15:47:01 tidwellr, to totally break the dependency 15:47:12 we've been discussing the eVPN spec this week, we're going to explore a different approach 15:47:27 you bgp will not depend on networking-bgpvpn, right? 15:47:29 however, I don't think we've ever taken the RFE to the drivers team 15:47:41 steve_ruan: I think we should explore that 15:47:53 ok 15:48:26 anyway, I don't see anything on the eVPN fron getting into Newton, but getting a start on Ocata would be good 15:48:54 carl_baldwin: has this RFE been discussed at the drivers meeting yet? 15:49:16 tidwellr: no. 15:49:28 carl_baldwin: we have a spec we've been iterating on, I assume we'd be asked for one anyway 15:50:19 The focus of the drivers meeting has shifted a bit to discussing status of Newton items. 15:50:43 carl_baldwin: ok, good to know. I don't think there's any rush to explore this with the drivers team at the moment 15:51:03 tidwellr: ok 15:51:17 I'd like to see it discussed so that maybe we can have something for Ocata 15:51:25 anything else tidwellr steve_ruan ? 15:51:31 not from me 15:51:35 no,thanks 15:51:44 Thanks for the update! 15:51:55 #topic FWaaS 15:52:02 Hi! So things are looking good for l3 agent extensions - https://review.openstack.org/#/c/339246/ has one +2 (thanks carl_baldwin!), needs another. 15:52:05 tidwellr: Let's get it teed up for discussion. We might be able to touch on it at the mid-cycle. 15:52:29 njohnston: Thanks for the reminder to revisit that one. 15:52:38 carl_baldwin: Sure thing! 15:52:54 The FWaaS side to act as an l3 agent extension is also coming along: https://review.openstack.org/#/c/337699/ 15:53:21 njohnston: Excellent. 15:53:23 We're hoping to land significant swaths of FWaaS v2 core functionality on Friday, so it's good to see these things coming together. 15:54:17 I think if I can get https://review.openstack.org/#/c/339246/ merged, all that leaves in the codebase of Neutron proper is a fullstack test, which will get very involved 15:54:30 and I am deferring that work until a little later 15:55:05 I think that's it for me 15:55:21 njohnston: thanks for the update! 15:55:53 #topic Conversion to Pluggable IPAM 15:56:51 We got that bug worked out. 15:57:00 I think we're almost in good shape. 15:57:25 I hope to get the bug fix merged and then get a few rechecks on the switch to pluggable before the mid-cycle. 15:57:33 Then, pull the trigger at the mid-cycle. 15:57:51 Great! 15:58:15 The gate is slow these days. 15:58:46 Queue max delay: 32.70 hours https://twitter.com/openstackstatus/status/763747339177717760 15:59:00 yikes 15:59:25 ok team, time is almost up 15:59:28 That's less than 500 15:59:33 Thanks, mlavalle 15:59:37 thanks all! 15:59:52 Thank you for your attendance and hard work 16:00:01 #endmeeting