14:00:57 #startmeeting kuryr 14:00:58 Meeting started Mon Jun 20 14:00:57 2016 UTC and is due to finish in 60 minutes. The chair is apuimedo. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:59 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:01:01 The meeting name has been set to 'kuryr' 14:01:15 Hi! Who's here for the kuryr meeting? 14:01:49 Hi, Ton Ngo from the Magnum team. 14:01:54 O/ 14:01:57 o/ 14:04:05 irenab: ping 14:04:11 devvesa: ping 14:04:16 limao: ping 14:04:25 just checking if we have extra people 14:04:39 hi 14:04:41 hi 14:04:57 :-) 14:05:12 hi 14:05:32 #info tango, vikasc, hongbin, irenab, limao devvesa and apuimedo present for the meeting 14:05:51 #topic kuryr-libnetwork 14:06:42 We had some issues in the past weeks with the tests 14:07:00 and some questions about the container images 14:07:07 let's start with the testing issues 14:07:27 For those that did not see, the fullstack breakages were caused by our dependencies 14:07:52 #info flask 0.11 caused some fullstack breakages 14:08:17 #link https://review.openstack.org/#/c/323895/12/ 14:08:23 apuimedo: issues are fixed and merged? or there arestill pending patches? 14:08:47 #info pyroute2 0.3.x caused some breakages on trusty kernels 14:09:00 #link https://review.openstack.org/#/c/331333/ 14:09:17 irenab: the latter is pending a merge by the infra team 14:09:37 #action apuimedo to nag the infra team so that the change is merged and the bot sends an update to our repo 14:10:39 about the kuryr/libnetwork image now 14:10:53 tango brought up this question the last week 14:11:32 there was a question whether that is the official upstream image 14:11:56 #info https://hub.docker.com/r/kuryr/libnetwork/ is the official upstream image 14:12:28 which means that you can open bugs in launchpad for any possible fault it may have, like the one mentioned in the previous meeting, logging 14:12:43 Thanks apuimedo for confirming, we are integrating Kuryr with our Swarm bay, and the image is a good way to do this. 14:12:51 tango: did you check if with `docker logs` it showed the logs? 14:13:08 tango: that's great to hear! 14:13:26 I believe docker log show the log from the web server 14:13:45 but so far I have not been able to get the log from Kuryr 14:13:52 it still is a bug if it did not go to /var/log/kuryr, so I'd appreciate if you can report the bug in launchpad 14:14:25 ok, I will double check a few more things to make sure I am not missing anything, and open a bug 14:14:41 #info the container is built from contrib/docker/libnetwork/Dockerfile 14:14:46 apuimedo: about the image 14:14:49 thanks tango 14:14:57 right now, the build system is terrible 14:15:12 should not the source repo be the upstream kuryr? 14:15:20 irenab: it is 14:15:28 in an indirect way 14:15:30 :P 14:15:31 looks like some fork … 14:15:36 it relies on a private server I have making a mirror of the changes 14:15:47 :-) 14:15:55 because to create the automated build in docker hub we must have permissions over the repo in github 14:16:00 which only infra has 14:16:23 apuimedo: so eventuleyy it will go directly? 14:16:27 #info infra is planning on a generic build system for project's containers that will then be able to push images 14:16:46 however, due to a lot of requirements in scale for kolla 14:16:49 that may take a while 14:16:54 that's why I did a workaround 14:16:55 apuimedo: I guess soimilar needs as Kola has 14:17:06 kolla has many more needs :P 14:17:07 apuimedo: got it, thanks 14:17:29 another issue with the container image is that it pulls from github when doing docker build 14:17:56 it would be much better if it pulled from the repo, so you could edit the code, do docker build and you immediately got a new container with your changes 14:18:10 that would require us to move the Dockerfile to the root of the git repository 14:18:36 and I do not think we can do that until we move the libnetwork parts to the new kuryr-libnetwork repository 14:18:55 but we will get there 14:19:29 That's helpful to know. 14:19:34 #info apuimedo proposes to move the Dockerfile to /Dockerfile as soon as we move to the kuryr-libnetwork repository 14:19:44 tango: any more questions about the container? 14:20:22 I am good for now. One other related question: 14:20:31 tango: go ahead 14:20:56 I guess we need to run the openvswitch agent on each node along with the Kuryr image 14:21:37 tango: that's right. The image contains ovs-vsctl, but the neutron ovs agenst need to run on each node 14:21:42 *agents 14:21:45 I am looking at running the agent as a container also, so I wonder if anyone has tried this or if there might be any issue. 14:21:58 tango: it should just work 14:22:11 if both are on the host networking namespace 14:22:20 if it does not. Time to ping us in #openstack-kuryr :-) 14:22:28 Sounds good, Kolla has this image, so I am hoping it will work. 14:22:45 so do I 14:22:53 I did not look at that kolla image recently 14:22:59 but let me know how it goes 14:23:07 moving on to other libnetwork things 14:23:08 Thanks apuimedo, that's all I have for now. 14:23:33 #action vikasc to rebase https://review.openstack.org/#/c/331050/ 14:23:54 I think the infra went to a dark place today early in the morning and it was failing everything 14:24:07 Hi, got delay 14:24:15 irenab: did you have time to review that patch about the overlapping stuff? 14:24:27 and by stuff, I meant cidrs :P 14:24:35 apuimedo, done 14:24:35 devref ? 14:24:50 irenab: the short term fix 14:24:52 https://review.openstack.org/#/c/331050/ 14:24:57 #link https://review.openstack.org/#/c/331050/ 14:25:07 Ithe above one is LGTM, I was waiting to see Jenkins 14:25:32 yes, that's why I asked for a rebase 14:25:37 vikasc: thank you for adressing my comments 14:25:44 let's see if jenkins runs again, otherwise let's re-trigger a re-check 14:25:51 apuimedo, i rebased this morning 14:26:02 apuimedo, i will do recheck 14:26:12 irenab, thanks for reviewing 14:26:49 thanks vikasc 14:27:18 #action apuimedo irenab to try to get https://review.openstack.org/#/c/331050/ merged 14:27:22 apuimedo, irenab , please review delay_neutron_ext_check also 14:27:30 link? 14:27:36 https://review.openstack.org/#/c/326303/ 14:27:41 #action apuimedo to review https://review.openstack.org/#/c/326303/ 14:28:28 apuimedo, I observed that currently config file does not work 14:28:51 because neutron client gets created before starting server with config file 14:28:57 vikasc: I am aware :P The config file is in bad state, diga filed a blueprint to fix it 14:29:10 to fix the config file, that is 14:29:15 the initialization is separate 14:29:23 and you are working on that, vikasc, right? 14:29:30 yes 14:29:50 good 14:30:01 it's https://review.openstack.org/326303 14:30:08 apuimedo, above patch is addressing this as well 14:30:17 apuimedo, yes 14:30:18 do we have the mitigation policy in general for the case that neutron API server is not responding? 14:30:22 I also saw, limao_ that https://bugs.launchpad.net/kuryr/+bug/1593906 was filed 14:30:22 Launchpad bug 1593906 in kuryr "Remove mox from testing infrastructure " [Undecided,New] - Assigned to bailin.zhang (bailin-zhang) 14:30:49 i.e kuryr restart or just failing the called method? 14:30:51 irenab: the change to delay neutron checks until the first interaction should mean that you can start kuryr before Neutron is ready 14:31:12 apuimedo: this is only part of the handling 14:31:21 it will only fail once you ask kuryr to translate docker net into neutron if Neutron is not up at that moment 14:31:21 neutron API server may stop responding 14:31:33 or its config can be changed and restarted 14:31:41 irenab: in these cases we fail operations 14:32:06 devvesa: could you check https://bugs.launchpad.net/kuryr/+bug/1593906 ? 14:32:06 Launchpad bug 1593906 in kuryr "Remove mox from testing infrastructure " [Undecided,New] - Assigned to bailin.zhang (bailin-zhang) 14:32:11 so when neutron server is responding again, there is no check for supported extensions 14:32:23 or the model inconsistency, right? 14:32:26 we are using mox with python3 in our prototype, so I wonder what the nova people have found 14:32:27 irenab, hmm 14:32:35 I will fill a bug to trace it 14:32:41 irenab: good idea 14:32:59 #action irenab to file a bug about reconnection and extension re-checking 14:33:25 apuimedo, :) 14:33:35 irenab: please review limao's https://review.openstack.org/#/c/319524/ 14:33:43 okie 14:33:46 I think it is practically ready 14:34:02 vikasc: you're welcome to repeat your +1 as well ;-) 14:34:14 apuimedo, sure :) 14:34:40 irenab: apuimedo : I am setting up my environment on fresh server, I think I should try testing all the scenarios 14:34:50 diga: good 14:35:14 Cool 14:35:16 anybody else has any question about libnetwork can ask it later in #opentsack-kuryr or in the mailing list 14:35:26 #topic kuryr-kubernetes 14:36:07 irenab: IIUC, you have a minor objection to https://review.openstack.org/#/c/328106/ before we merge it, is that right? 14:36:30 checking 14:37:20 just a suggestion to be more explicit about DeLETE event. but maybe it was not clear only to me, so I am fine to proceed as is 14:37:54 irenab: so please +2 :P 14:37:56 I think the meaning is that DELETE event is not generated for the endpoits 14:38:06 yeah. I also think so 14:38:20 I hoped to get a responce and confirmation on the patch :-) 14:38:21 irenab: I'd rather we get this merged and you send a patch to clarify afterwards 14:38:31 apuimedo: good idea 14:38:45 #info there's been work in the kubernetes devref 14:39:01 #link https://review.openstack.org/#/c/328106/ 14:39:18 this one addresses the endpoint translation 14:39:43 #info further detail into the watcher and translator https://review.openstack.org/#/c/330282/ 14:40:01 apuimedo: any one working on this stuff ? 14:40:17 I want to thank irenab, devvesa, tfukushima and vikasc reviews to my patch 14:40:19 after config file, I will take this in 14:40:28 you are getting it much better than I originally had it :-) 14:40:35 both in wording and in concept 14:40:52 diga: we have quite a bit of work done in it in our prototype repo 14:41:15 the most urgent task is to move kuryr libnetwork part to the new kuryr-libnetwork repo 14:41:27 apuimedo: okay 14:41:45 diga: but you are welcome to help with kuryr-kubernetes as well ;-) 14:41:45 apuimedo: I will take a look at it 14:41:59 thanks apuimedo 14:42:06 #action apuimedo to address the final comments to the patch 14:42:14 hopefully we can merge it this week 14:42:44 devvesa: am I forgetting anything about kuryr-kubernetes? 14:42:54 Anybody else has any questions? 14:43:37 #info once we have these two specs I think we can start bringing the code to kuryr-kubernetes repository 14:43:47 apuimedo: is there anyone working on moving the libnetwork code to kuryr-libnetwork? 14:43:58 gsagie: not really, no 14:44:09 gsagie: is that a volunteering? 14:44:16 :P 14:44:19 you can write it on me :) 14:44:26 Yay! 14:44:30 i do wonder what is going to stay at the main kuryr repo thought 14:44:38 maybe the devstack installation 14:44:46 and the tests 14:44:47 gsagie: the library, specs 14:44:51 sorry, got disconnected 14:44:55 unti tests for the library 14:45:06 apuimedo: what is "the library" ? 14:45:15 from the code that we have now 14:45:17 #action gsagie to work on splitting kuryr into kuryr (having the common library and binding) and kuryr-libnetwork 14:45:24 the binding, utils 14:45:28 part of the controller 14:45:34 apuimedo: gsagie : any update on the nested containers? 14:45:37 so, binding and common neutron ops 14:45:44 hey, hey, hey 14:45:44 Need to talk with fawad 14:45:53 irenab: we are on kubernetes topic :P 14:45:57 let me swap it 14:46:08 #topic containers-in-vms 14:46:09 apuimedo: ok, think i have an idea but we might need to do some refactoring in the main repo first then i can move it easly 14:46:16 but i will work on that 14:46:31 gsagie: yes, some refactoring (specially of controllers) is in order 14:46:47 gsagie: it would be good if we could use python namespaces 14:47:05 so that in the end we had kuryr.libnetwork kuryr.kubernetes kuryr.cni, etc 14:47:17 in pypi 14:47:34 yes 14:47:37 np 14:47:44 fawad is not present, but if there are questions about the nested case 14:47:52 now is the time to record them 14:47:54 ;-) 14:47:55 well i do know that Omer was trying to reach him 14:47:56 gsagie: I think there is some launchpad issue to trace it 14:47:59 to take some tasks 14:48:05 apuimedo, why do we need kuryr server and kuryr agent 14:48:37 vikasc: the idea is that whatever makes the requests to Neutron should not be in a VM that belongs to the user 14:48:45 (or tenant) 14:49:03 the agent is a more lightweight version that is mostly incharge of the binding at this point 14:49:18 the agent should have only minimal knowledge in order to do the binding, that's right 14:49:47 apuimedo, gsagie .. got it 14:49:53 :-) 14:49:56 apuimedo, gsagie thanks 14:50:01 okay, moving on 14:50:01 i do feel it might be used when we start consider policy, but we will need to talk about it (esp for the security case) 14:50:15 the agent that is 14:50:18 gsagie: yes, if we ever want to do policy in the VMs 14:50:22 using bpf or the like 14:50:26 yes 14:50:29 then the agent will have to smarten up 14:50:38 #topic open discussion 14:50:57 irenab presented about kuryr in the openstack-israel days 14:51:06 Back to the nested containers topic 14:51:20 hongbin: yes plese, go ahead 14:51:24 apuimedo: its Networking Day 14:51:41 irenab: too many things ending up on "Day" 14:51:41 I remembered someone volunteer to split the BP into several subtasks 14:51:42 xD 14:51:53 hongbin: thats fawad, he isnt here 14:51:58 right 14:52:03 he is suppose to be in charge of the nested containers 14:52:03 OK 14:52:30 For the task spliting, is any chance to do it by another person? 14:52:42 hongbin: yes 14:52:44 (if fawad is busy) 14:52:45 hongbin: do you have any volunteer on your end? 14:52:49 I think at this point anybody can help 14:53:07 apuimedo, gsagie , I would be happy to volunteer 14:53:08 I might find contributors if tasks is splitted 14:53:10 shooting an email to the mailing list presenting a possible breakup would be great, for example 14:53:16 vikasc great :) 14:53:35 i have some level of magnum understanding also 14:53:42 I think taking it to the ML for a round of discussion would raise awareness too 14:53:45 not sure if hongbin remember me :P 14:53:50 Right now, it is hard for contributors to figure out how to contribute 14:53:52 vikasc: maybe you can start with splitting the work into tasks and then you can take some and maybe someone else can help with the others 14:53:56 vikasc: yes, I do :) 14:53:58 hongbin: right 14:54:15 that's why I think that a few emails in the ml can help 14:54:24 gsagie, sure 14:54:36 great thanks! 14:54:39 hongbin, thanks :) 14:54:41 gsagie: do you mind getting the ball rolling sending a first email? 14:54:53 vikasc can jump in afterwards :P 14:54:59 apuimedo: ok i will 14:55:01 gsagie: apuimedo : I can put some efforts in there if vikasc needs any help 14:55:03 great! 14:55:24 thanks gsagie 14:55:35 anything else in these last 5 minutes? 14:55:38 diga, thanks diga.. :) 14:55:51 vikasc: you are welcome! :) 14:56:10 going once 14:56:21 \o/ 14:56:23 going twice 14:56:35 100$ 14:56:35 sold 14:56:40 gone! 14:56:47 thank you all for your participation! 14:56:53 have a nice rest of day 14:56:53 cya 14:56:56 #endmeeting