14:02:12 #startmeeting networking 14:02:13 Meeting started Tue Jan 13 14:02:12 2015 UTC and is due to finish in 60 minutes. The chair is mestery. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:02:14 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:02:17 The meeting name has been set to 'networking' 14:02:21 aloha 14:02:26 #link https://wiki.openstack.org/wiki/Network/Meetings Agenda 14:02:33 #topic Announcements 14:02:44 Kilo-2 is February 5 14:02:45 #link https://wiki.openstack.org/wiki/Kilo_Release_Schedule 14:02:51 #link https://launchpad.net/neutron/+milestone/kilo-2 14:03:08 salv-orlando: hi. i'm here to discuss python 3 in neutronclient, is it the right place? 14:03:11 We have a lot of BPs targeted here, later in the meeting I want to go over the Essential and High priority ones 14:03:19 haypo: It is, can we do that later in the meeting please? 14:03:30 mestery: cool. no problem 14:03:34 \o 14:03:44 One other quick announcement: L2 Gateway API meeting is setup 14:03:47 #link https://wiki.openstack.org/wiki/Meetings#Networking_L2_Gateway_meeting 14:04:03 Any other announcements from anyone else the team should be aware of? 14:04:37 If not, we'll move on, the agenda is very thick near the bottom :) 14:04:42 #topic Bugs 14:04:48 enikanorov enikanorov__: Are you around? 14:05:02 mestery: hi 14:05:50 enikanorov__: Any bugs you are tracking the team should be aware of? 14:06:02 yes 14:06:03 https://bugs.launchpad.net/neutron/+bug/1403291 14:06:03 I know armax nailed one around a change in oslo.db last night which was causing all kinds of gate failures 14:06:04 Launchpad bug 1403291 in neutron "test_server_connectivity_pause_unpause fails with "AssertionError: False is not true : Timed out waiting for 172.24.4.64 to become reachable"" [Critical,Confirmed] 14:06:06 mestery bug #1403291 14:06:07 Launchpad bug 1403291 in neutron "test_server_connectivity_pause_unpause fails with "AssertionError: False is not true : Timed out waiting for 172.24.4.64 to become reachable"" [Critical,Confirmed] https://launchpad.net/bugs/1403291 14:06:28 (was looking for the link) 14:06:35 I reckon armax was about (or has) root caused it 14:06:37 so, what was the issue with oslo.db? 14:06:41 enikanorov__: I haven’t had a chance to look into it yesterday... 14:06:59 armax: was the that one with the DHCP loss? 14:07:03 i also didn't manage to find the root cause 14:07:12 salv-orlando: not yet, I am still fishing for ideas…from what the logs can tell everything seems to be in order 14:07:18 kevinbenton: yes 14:07:45 I have something I want to try, after being unsuccessful with https://review.openstack.org/#/c/146256/ 14:07:55 armax: the only thing I can say is that I've seen thousands of those logs, and I've never seen messages from iptables dropping DHCP packets. 14:08:16 I thought that the debug info would help get us more clue…but with all the recheck I kicked I was unable to get a single failure 14:08:27 anyway, it's good to see there are 2 people on this... I think the agenda is thick today so we should move on 14:08:30 salv-orlando: but those messages accompanying successful dhcp negotiations too 14:08:35 salv-orlando: correct, but that message is totally unrealted 14:09:12 salv-orlando: and if I look at it closely it says to drop a packet coming from the client to the server 14:09:22 and it’s a broadcast too 14:09:39 armax: But you could never trigger this with the tempest patch you proposed above to get more logs? 14:09:41 salv-orlando: I am reading it wrong? What we’re experiencing is a loss of the DHCPOFFER 14:10:16 armax: ok. then the drop of dhcp packets observed is totally unrelated to a dhcp issue manifesting. I think you're doing it right, it must be a coincidence 14:10:18 mestery: this is a typical heisenbug, as soon as you start looking at it, it’ll hide in the shadows 14:10:33 :( 14:10:50 OK, thanks for tackling this one armax, please reach out to anyone you need as you peel back the onion :) 14:10:55 armax: well if the debug happens to fix it, we should merge it anyway to stop the failures :) 14:11:05 enikanorov__: Any other high priority bugs we should be aware of today? 14:11:15 mestery: no 14:11:22 enikanorov__: Thank you! 14:11:37 #topic Docs 14:11:40 emagana: Any updates for today? 14:11:50 kevinbenton: I suspect there will be plenty of people who oppose the idea…but it might as well be that that patch put timings in a way that the bug doesn’t show up 14:12:36 In lieu of emagana not being around, lets move on, the agenda gets thick and contentious rather soon. :) 14:12:46 #topic Plugin Decomposition Status and Updates 14:12:55 I wanted this on the agenda, and we'll likely do this every few weeks or so. 14:13:03 #link http://specs.openstack.org/openstack/neutron-specs/specs/kilo/core-vendor-decomposition.html 14:13:07 As everyone knows, this spec merged. 14:13:18 mestery: some folks have taken actions, which is good 14:13:20 I know some people are in various states of implementing this for their plugins (yay!), and others are not (boo :() 14:13:25 armax: Yes 14:13:29 I wanted to do a few things here: 14:13:56 1) Allow for questions from folks. 2) Let people speak up with their success as they do this. 3) Let the leader of this effort (armax) have a moment to give us an overview of where he thinks things are at. 14:14:03 armax: You want to go first? :) 14:14:23 I am no much of a leader, but ok 14:14:24 :) 14:14:52 as I said we’ve had a number of folks starting 14:15:19 at the moment the things that I am looking at when I do reviews is to ensure that bug fixes or changes do not step on each others toes 14:16:05 armax: I know you and I have been reviewing stackforge creation reviews as well. 14:16:11 armax: One question I wanted clarified here is this: 14:16:15 so when fixing or changing things, people should keep in mind that ultimately the code they’re touching may end up elsewhere and they should act accordingly 14:16:23 Are we now letting people create stackforge without the branches? 14:16:33 *stackforge projects 14:16:57 mestery: are you referring to stable/x branches? 14:17:00 mestery: imo stable branches can be optional, but since the effort in mirroring them to the stackforge repo is minimal 14:17:03 kevinbenton: Yes 14:17:14 mestery: I don’t see why people leave them behind 14:17:16 armax: I agree with that sentiment 14:17:33 armax: Me either, but I've noticed more and more people are, and I just wanted to understand the official position from the spec author ;) 14:17:34 mestery: especially if people follow the instructions in the docref as a bible 14:17:44 that said, stackforge is one option, but not the only one 14:17:55 midonet is going without stackforge 14:18:15 mestery: https://review.openstack.org/#/c/145792/ 14:18:15 #info When creating a stackforge repository for your plugin/driver backend to comply with Plugin Decomposition, bringing along stable branches is optional. 14:18:34 and that is perfectly valid too 14:18:55 Yes, cool! I like to see multiple options here. 14:18:59 if we could have folks acknowledge in the commit message their choice, that would help 14:19:03 we are requiring that it be open source somewhere, right? 14:19:03 #link https://review.openstack.org/#/c/145792/ 14:19:11 kevinbenton: Yes 14:19:13 as in "we choose to leave the stable branch behind" 14:19:16 anteaya: you beat me to it, I was going to say! 14:19:22 then reviewers know they are aware 14:19:26 armax: :D 14:19:33 as infra ultimately are the ones who approve the stackforge patch 14:19:40 if one goes the stackforge route 14:19:46 armax: if you're an atheist you would not follow anything as a 'bible' ;) 14:19:47 so it’s best to be proactive and tell them 14:19:48 based on what you want 14:19:50 #info When creating your stackforge project, please indicate in the commit message your intent to carry stable branches. 14:19:53 we just need to know 14:19:59 salv-orlando: sorry, rulebook 14:20:06 anteaya: Thanks for bringing that up here! 14:20:10 mestery: or lack thereof 14:20:12 sure 14:20:33 armax: true 14:20:39 Any questions from anyone undergoing plugin decomposition that is hitting a wall somewhere? 14:20:47 mestery: I still need to complete the docref with a few other aspects 14:20:57 armax: True, and thanks! 14:21:06 mestery: I’ll get to it asap 14:21:23 what is the preferred way to track status per plugin? filing a bug? 14:21:23 I should note that armax and I made some good progress on the ODL decomposition yesterday, I expect today we should have the ODL ML2 driver decomposed. 14:21:33 amotoki: Yes, a wishlist bug for the neutron commit 14:21:54 mestery: thanks for the clarification. i swa ODL filed a bug. 14:21:55 #info To track the neutron side of your plugin decomposition, please file a bug. It will be marked as Wishlist and you can use that for the commit in neutron. 14:21:58 amotoki: see https://review.openstack.org/#/c/145792/ 14:22:01 too 14:22:41 Thanks armax for your work here! 14:22:49 If anyone has quiestions, please reach out to armax or myself. 14:23:04 Like I said, we'll do this every week so people ahve a standing slot to discuss any issues they are facing. 14:23:04 mestery: for what is worth we have the decomposed plugin working. We're just waiting to make sure the CI is up and running also on the other vmware plugin we added and then maybe we'll just leave about 10 lines of code for integration 14:23:05 hi, joining late :/ 14:23:11 or perhaps 10 thousands :) 14:23:29 salv-orlando: It's a small difference either way ;) 14:23:34 salv-orlando: you’re always the funny one 14:24:10 OK, lets move on if there is nothing else here. 14:24:16 The thick part of the agenda awaits! 14:24:33 #topic Kilo-2 Specs of "Essential" and "High" Priority 14:24:44 I wanted to walk through these now. 14:24:49 #link https://launchpad.net/neutron/+milestone/kilo-2 14:25:08 To get an idea of where they are at, given we have 3 weeks left and the gate is getting more and more backlogged with each passing day. 14:25:15 First one: 14:25:16 #link https://blueprints.launchpad.net/neutron/+spec/wsgi-pecan-switch 14:25:24 kevinbenton: This one is yours, you lucky guy ;) 14:25:47 Have you had a chance to start on this one yet this week ? 14:25:48 mestery: yes, this depends on the internal refactor 14:26:01 kevinbenton: You mean the next one on the list https://blueprints.launchpad.net/neutron/+spec/plugin-interface-perestroika 14:26:04 #link https://blueprints.launchpad.net/neutron/+spec/plugin-interface-perestroika 14:26:08 yes 14:26:13 salv-orlando: That one belongs to you (lucky guy #2)! 14:26:15 :) 14:26:24 mestery I am the proposer 14:26:38 I think the main author is still markmcclain, with me in the role of code monkey 14:26:50 salv-orlando: Ack, just looking for a status update. 14:27:02 salv-orlando: how is the code monkeying coming? 14:27:05 For these two specs, if we don't see code landing by next week, I fear they will miss Kilo-2 14:27:10 Indeed I have not done absolutely any work on it. I hope there will be some when kevinbenton and markmcclain do their syncup 14:27:22 Per discussions with marun, I'd like to see these land in Kilo-2 so we can have Kilo-3 to sort any issues out. 14:27:32 mestery: I don't know what to say. did kevinbenton speak with makr? 14:27:36 * mark 14:27:49 I think the plan is now for kevinbenton to just go to it 14:27:53 So we can make progress. 14:27:56 salv-orlando: no, i haven't been able to land a meeting slot with him 14:28:16 kevinbenton: You have my authorization to move forward at this point, let me know what you need :) 14:28:34 mestery: +1 14:28:36 sounds reasonable 14:28:47 there’s no point in keep on waiting now 14:28:58 mestery: ack, so salv-orlando, did you want to start putting something together for the internal plugin interface? 14:29:03 kevinbenton: marun has volunteered to work with you here as well, especially in bouncing ideas, etc. 14:29:53 OK, lets move on to the next one. 14:29:55 #link https://blueprints.launchpad.net/neutron/+spec/reorganize-unit-test-tree 14:29:57 kevinbenton: I will. I won't be afraid of pushing pointless and terrible code so we can work together on it to make it proper code 14:30:07 That one is assigned to marun, armax, do you by chance know if he's started on this yet? 14:30:09 salv-orlando: sounds great! 14:30:10 besides, that what I reguarly do 14:30:15 salv-orlando: lol 14:30:29 mestery: I spoke with marun the other day 14:30:53 mestery: the effort is pretty mechanical for the reorganize-unit-test-tree bp 14:31:08 armax: Ack, sounds like it's under control then, thanks! 14:31:08 mestery: it’s a matter to get some focussed time and get it done 14:31:10 mestery: I believe it was his intention, but not sure if started. 14:31:18 yes, just moving files around 14:31:20 ajo: thanks! 14:31:23 OK, next one 14:31:25 #link https://blueprints.launchpad.net/neutron/+spec/restructure-l2-agent 14:31:35 rossella_s owns this one, I haven't seen any code for this yet. 14:31:43 Unfortunately not much progress on my side. Anyway completing the blueprint will be my priority for the next weeks, I will send a first patch in the next days. If anybody is interesting in contributing, please contact me. 14:31:56 rossella_s: Yay! Thank you for the update! 14:32:07 Next up (moving quickly): 14:32:08 #link https://blueprints.launchpad.net/neutron/+spec/rootwrap-daemon-mode 14:32:16 otherwiseguy: This one is yours now I believe. 14:32:35 otherwiseguy: Looks like you've made a start on this one now, thanks for that! 14:32:55 Next up 14:32:56 that's what I know, mestery :) he is focused on that 14:33:03 ajo: Cool! 14:33:05 #link https://blueprints.launchpad.net/neutron/+spec/retargetable-functional-testing 14:33:17 I spoke with marun on this yesterday, he has a plan for this and is documenting it on the wiki 14:33:23 #info I spoke with marun on this yesterday, he has a plan for this and is documenting it on the wiki 14:33:31 Next one: 14:33:39 #link https://blueprints.launchpad.net/neutron/+spec/refactor-iptables-firewall-driver 14:33:48 ajo: This one is assigned to you I believe? 14:33:50 mestery: have some functional tests going for the ovs agent related to the spec rossella_s mentioned https://review.openstack.org/#/c/140042/ 14:33:54 mestery: we’re still shaking some issues with infra 14:34:02 Hmm, mestery , I need to split the patch in three 14:34:03 Bleah! Sorry marios :) 14:34:09 ajo: OK, cool! 14:34:19 mestery, will do during this week to have it ready for merge 14:34:19 marios: Thanks! 14:34:20 mestery: re the retargetable-functional-testing 14:34:26 armax: Ack 14:34:41 Thanks ajo! 14:34:43 Next up: 14:34:47 #link https://blueprints.launchpad.net/neutron/+spec/vsctl-to-ovsdb 14:34:53 mestery: once that’s cleared, it should be downhill… :) 14:34:56 otherwiseguy has some patches out for this one as well, so it's looking good 14:35:16 hmm mestery 14:35:21 ajo: ??? 14:35:39 about the refactor-iptables-driver bp, It's assigned to me, but the work I have done is not referenced to that spec 14:35:47 it was a general cleanup before that 14:35:53 not sure who's working on that specific spec 14:36:04 jakub? 14:36:05 I'm just looking at it 14:36:06 so what is your work related to? 14:36:08 jlibosva ^ 14:36:16 seems like the specs what I proposed 14:36:27 jlibosva: Shall I move that one to you then? 14:36:37 mestery: yes please, thanks 14:36:50 salv-orlando my patch is to the current iptables firewall driver, to cleanup lots of small inconsistencies and code which could be refactored but without rearchitecturing 14:36:54 I'll sync with ajo after meeting 14:36:57 ack 14:37:01 ajo: thanks 14:37:14 jlibosva: Done, thanks! 14:37:33 OK, moving on 14:37:33 #link https://blueprints.launchpad.net/neutron/+spec/lbaas-api-and-objmodel-improvement 14:37:50 dougwig has informed me he and blogan and team are on this and expect to get this one done by Kilo-2 14:38:03 That work is all being done in neutron-lbaas 14:38:15 Next up 14:38:17 #link https://blueprints.launchpad.net/neutron/+spec/restructure-l3-agent 14:38:23 carl_baldwin: You have a lot of working going on here :) 14:38:42 We’re actively working the meaty part 14:38:47 carl_baldwin: Yay! 14:38:53 yeah, amazing work carl_baldwin 14:39:07 Kilo-2 will be tight for completion but we should be over the hump 14:39:43 carl_baldwin: Excellent, we can deal with the logistics of kilo-2 vs. kilo-3 when we get there, but sounds like the majority of the work will make kilo-2. great! 14:39:54 There are a number of contributors doing good work. 14:39:59 ++ 14:40:03 Excellent! 14:40:11 OK, moving on (mestery keeps his eye on the clock ...) 14:40:13 #link https://blueprints.launchpad.net/neutron/+spec/neutron-ovs-dvr-vlan 14:40:33 I spoke to Vivek, and he has pushed the patch which implements this, his next patch will add the unit tests for this change 14:40:35 So, this appears to be in good shape. 14:40:53 mestery: that one is gettign there, I need to do a code review 14:40:59 And with that, we're done reviewing all Essential and High priority BPs! 14:41:03 armax: Thanks! 14:41:17 but I’ll tail Vivek, in case he’s getting lost in the weeds :) 14:41:23 I encourage all reviewers to really focus review time on these BPs, especially if your review bandwidth is limited in the coming weeks 14:41:44 Thanks to all the contributors here, this is all outstanding work! 14:42:03 Any questions on BPs at this point? 14:42:24 I would be grateful for some review on the ProcessMonitoring stuff, I need to rebase it on the last l3-agent refactors today... 14:42:40 I'd love it not to slip over another release :) 14:42:47 yeah, that one took too long already 14:42:51 ajo: yes, sorry about that. 14:42:51 ajo: Cool, we'll make sure to land that 14:42:57 Thanks salv-orlando! 14:43:10 OK, lets move on then. 14:43:11 salv-orlando, no problem, lot of moving grounds with the refactors, that also slow me down doing rebases all the time :) 14:43:25 #topic nova-network to neutron migration 14:43:33 anteaya: This is your part of the meeting I believe :) 14:43:41 it is 14:43:42 I also hope obondarev is here today as well ;) 14:43:42 hello 14:43:49 yep 14:43:58 we have a meeting 14:44:02 #link https://wiki.openstack.org/wiki/Meetings/Nova-nettoNeutronMigration 14:44:09 and we met 5 hours ago 14:44:19 #link http://eavesdrop.openstack.org/meetings/nova_net_to_neutron_migration/ 14:44:26 and it was productive 14:44:44 anteaya: Yay, excellent! 14:44:56 the summary is that obondarev and belmoreira and markmcclain and angus will all be working on the spec 14:45:26 and we expect a new patchset before the end of thursday new zleand time 14:45:42 anteaya: Double yay, nice! 14:45:55 and nova and mark have agreed that we need to stop ceasing work around releases 14:46:12 anteaya: I don't follow that, what does that mean? 14:46:14 this needs to continue to be ap ont of focus is we don't make kilo 14:46:40 that means that so far we work if we think we can make the realease and then stop working on this effort if we cna't 14:46:41 anteaya: I wish we could just stop addressing the ‘general’ problem and just get these big players that do need to migrate step up and lead technically themselves 14:46:42 anteaya: OK, got it 14:46:46 until the next realease 14:47:07 so mikal suggested we just barral through and focus on getting the work done 14:47:12 and there was agreement 14:47:21 we had a face to face on monday 14:47:33 * mestery thinks anteaya must be at LinuxConfAu 14:47:42 mikal took nots on an ehterapd wil the intention of making it public 14:47:44 I am 14:47:51 and that hastn'happened yet 14:48:03 so hopefully the link will be posted to the mailing list today 14:48:10 I'll remind mikal 14:48:21 I think that is it unless there are questions 14:48:46 armax: well cern is doing what they think we expect of them 14:49:04 thanks anteaya, please make sure an email with that etherpad comes out, I really want the CERN folks to review that, they are the only people interested in this who I am aware of that have come forward 14:49:09 anteaya: forgive my question, but what is cern doing, exactly? 14:49:13 Without their input, we'll be missing hte mark for the target user 14:49:22 mestery: yes 14:49:38 well belmoreira is going to outline their thoughts on testing in teh spec 14:49:45 so it can make it into the next iteration 14:50:02 armax: cern has a developer attending meetings and offering testing resources 14:50:16 and reviewing the spec, answering questions and asking them too 14:50:16 Thanks anteaya 14:50:20 np 14:50:26 Lets move on now, there are two items left and only 10 minutes in the meeting. 14:50:41 #topic Compiling a list of Neutron upgrades implementation status and known problems 14:50:47 hi, first I wanted to ask, what is the status of oslo.versionedobjects adoption in Neutron? 14:51:03 xek: Is this your section, did you add this to the agenda? 14:51:09 anteaya: thanks, I am still not convinced this is going to get where we want to be, but ok 14:51:12 mestery, yes 14:51:21 armax: acknowledged 14:51:23 versioned-objects deal with DB schema being at a different version than the code expects and allows for rolling upgrades 14:51:24 xek, afaik it was not even discussed at that point 14:51:24 xek: Cool, thanks. salv-orlando, being our oslo liasion, can you answer xek? 14:51:33 ihrachyshka: thanks :) 14:51:42 (with different code bases running at the same time) 14:51:47 mestery: ihrachyshka is the oslo guy now 14:51:48 #link https://blueprints.launchpad.net/heat/+spec/versioned-objects 14:51:56 I'm the guy for all the pointless things 14:52:21 currently, work on versioned-objects adoption is done in Heat 14:52:26 http://specs.openstack.org/openstack/heat-specs/specs/kilo/versioned-objects 14:52:30 this can be a reference implementation for Neutron 14:52:45 xek, so I have the library in my todo list, but I haven't checked it, yet 14:52:47 xek: This looks interesting, but at this point, we'd have to do this work in Lxxx 14:53:04 xek: Once Heat's work lands, that could be a model for us. ihrachyshka, does that sound reasonable to you? 14:53:12 yes, a lot of oslo work just to keep up with oslo team inventions in existing libs 14:53:26 yes, we'll return to that in L 14:53:30 I will submit a spec for this and see how it goes, ok? 14:53:48 xek: That's fine, but please note we won't open up Lxxx specs in neutron until we're near the end of Kilo. 14:54:03 xek: So the spec may sit dormant with a -2 from someone until then, FYI. 14:54:10 xek, there is no subtree for L yet in neutron-specs, but yeah, we can discuss it for now. 14:54:10 mestery, ok, thanks for the info 14:54:29 I'm also looking for pointers to efforts of implementing live upgrades neutron->neutron (whithout downtime and additional hardware) 14:54:33 #info xek to file spec around oslo.versionedobjects for Neutron for inclusion in Lxxx 14:54:39 I did push a review req to specs to create a "drafts" in the tree, that could be useful for future looking specs 14:54:52 https://review.openstack.org/145545 14:54:53 sc68cal: Thanks for the reminder there! 14:55:04 sc68cal++ 14:56:09 OK thanks xek! We'll talk more about live upgrade in Vancouver for Lxxx I suspect. 14:56:12 #topic Open Discussion 14:56:15 4 minutes left now 14:56:34 lets move on 14:56:37 Someone wanted to discuss python 3.3 for the client? 14:56:51 oh 14:56:53 yes 14:56:57 haypo: ^ 14:56:57 haypo: This was you? 14:57:02 and I :) 14:57:08 We got the client working with Pyhon 3 14:57:10 Steap: :) 14:57:14 we'd like to enable the gate now 14:57:20 salv-orlando: You had comments here? 14:57:24 see https://review.openstack.org/#/c/145430/ 14:57:29 i think we should do that. I just wanted to announce publicly we were doing it 14:57:38 sort of "speak now or shut up forever" 14:57:49 Works for me. Anyone not in favor of this? 14:57:56 #link https://review.openstack.org/#/c/145430/ 14:58:11 #info We plan to enable Python 3 support in the gate for python-neutronclient 14:58:14 That was easy :) 14:58:18 yes :) 14:58:19 Thanks Steap and haypo! 14:58:26 my -1 became a +1 14:58:28 +1 thanks! 14:58:37 salv-orlando: Cool! 14:58:40 OK, thanks everyone! 14:58:48 We have lots of great work going on, keep it up! 14:58:56 come on say it say it 14:58:59 end .... ing 14:59:03 We'll see you next week and on IRC and the ML! :) 14:59:03 #endmeeting