18:01:03 #startmeeting container-networking 18:01:04 Meeting started Thu Dec 10 18:01:03 2015 UTC and is due to finish in 60 minutes. The chair is daneyon_. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:05 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:08 The meeting name has been set to 'container_networking' 18:01:14 Agenda 18:01:18 #link https://wiki.openstack.org/wiki/Meetings/Containers#Agenda 18:01:33 I'll pause a minute for everyone to review the agenda 18:01:49 o/ 18:01:49 #topic roll call 18:02:08 o/ 18:02:09 o/ 18:02:37 Thank you hongbin Tango dane for attending the meeting. 18:02:45 #topic Flannel host-gw patch 18:02:50 #link https://review.openstack.org/#/c/241866/ 18:03:04 Tango I'll defer to you 18:03:17 Doesn't seem like much has happened with the patch 18:03:20 So I tried a few things 18:03:39 Can you share a few details? 18:03:50 The patch that Gus added didn't work 18:04:14 I tried turning off port security to allow the node to route traffic 18:04:39 I can see that the packet does get rerouted correctly to the target host 18:05:05 However once on the target host, the packet could not get to its destination 18:05:38 I notice that the flannel0 TUN interface is not created in the host 18:06:01 I am not sure if this is how it should work, or it's an error 18:06:26 I sent email to Angus but have not heard back from him. 18:06:41 Tango have you tried pinging the flannel or coreos irc channel? 18:06:54 Tried a few times pinging on IRC, but the time difference is tricky 18:06:57 I have never tried the host-gw mode 18:07:29 but it should work since L2 is the only requirement. 18:07:37 I am thinking about trying to create the flannel0 interface manually, seems like we need this for the flanneld to pick up the packet 18:07:59 Another thing to try is a later flannel version 18:08:23 I'll take a look through the flannel code too. If I see anything that pops out at me, I'll ping you on irc. 18:08:32 I couldn't build with the package from kojipkg, but I can just overwrite the binary 18:08:33 Thx for working on this. 18:08:57 A little slow going, I haven't been able to put a lot of time on it. Too many other things going on. 18:09:22 Tango, If I recall correctly, our flannel version is super old. I think it's a good idea to update flannel no matter what. 18:09:32 then at leat the vxlan bug will be fixed ;-) 18:09:42 until we can run flannel in a container 18:09:52 Understandable 18:09:56 I know the feeling 18:09:59 Yeah. I think I almost get the fedora image without atomic to work, so this would be easier 18:10:24 Agreed.. Atomic is a PITA ;-) 18:10:33 Yep :) 18:10:58 #topic Build Fedora Atomic with DIB 18:10:59 daneyon: What is the vxlan bug? 18:11:04 #link https://blueprints.launchpad.net/magnum/+spec/fedora-atomic-image-build 18:11:18 Yolanda added this topic to the agenda 18:11:29 dane: the vxlan backend doesn't work on Flannel 0.5.0 18:11:44 Tango: Thanks! 18:11:54 yolanda are you in attendance? 18:12:30 Seems like Yolanda is not here. 18:12:39 I will ping her offline 18:12:56 Maybe we'll address this another time 18:13:21 Does anyone else know about this BP? 18:14:04 It's been a while since I used diskimagebuilder, but i do remember it being somewhat painful. 18:14:09 I only had a discussion with her when she kicked off the BP 18:14:17 Anyone else familiar with DIB? 18:14:23 I have been using it 18:14:35 to build fedora, ubunu images 18:14:55 we have a few elements to build ironic and mesos images 18:15:00 has is worked well for you? 18:15:04 Tango 18:15:30 It seems OK 18:15:44 Documentation can be a little sparse 18:16:11 AFAIK, the ironic kubernetes image was built by Dib 18:16:33 It is a standard fedora image, not atomic 18:16:49 with several elements added to it. Works very well 18:16:53 right 18:17:01 Hello 18:17:04 The coreos ISO is 200+ MB... I think we need to focus on coreos and put atomic on the backburner 18:17:12 I know a thing or two about dib if you all want any info 18:17:27 right now yolanda is working on adding lvm support to dib which is a requirement for atomic AIUI 18:17:47 The documentation shouldn't be *that* sparce 18:17:53 greghaynes: ah, here a question: for the install-package element, can you specify a version othe package? 18:17:57 greghaynes thanks for the add'l info 18:18:29 greghaynes: Like if I want docker 1.9.1 and not 1.8.2 18:18:34 Tango: There are many ways to install packages, that one doesn't really but you could either install the package using standard tools or add that to pkg-map 18:18:48 greghaynes Do you happen to know why yolanda added #link https://blueprints.launchpad.net/magnum/+spec/fedora-atomic-image-build to today's agenda? 18:18:56 As in - there is no reason you cant just use apt-get or yum to install whatever you want 18:19:20 daneyon_: She has a goal of using DIB to generate the atomic images so we can greatly decrease their size, among other things 18:19:30 daneyon_: I dont know of any specific topics she wanted to discuss 18:19:46 Here are teh DIB docs FWIW http://docs.openstack.org/developer/diskimage-builder/ 18:19:48 greghaynes can you expand on the among other things? 18:20:28 daneyon_: Sure - a *lot* of projects use DIB in testing for image creation - infra also uses it for their testenv creation. As a result we have a lot of knowledge around using it to make images as part of our testing pipeline 18:20:44 it also is a lot more efficient because we have a lot of caching set up to make dib fast when run in our testenv's 18:21:04 I like it :) 18:21:34 greghaynes I see. Thanks. 18:21:53 np 18:21:54 greghaynes has anyone used DIB with coreos? 18:22:06 I am not sure what that means - do you mean to make a coreos image? 18:22:12 yes 18:22:35 Not that I am aware of, but the tool itself is pretty general purpose - adding new 'distros' tends to be really easy 18:22:50 OK 18:23:20 greghaynes: How about the ubuntu-minimal or fedora-mininal elements? 18:23:29 Tango: How about them? :) 18:23:52 That is just a distinction of whether we use debootstrap or a cloud-image (regular ubuntu element) 18:23:53 I had Tango create an image for me ~6 weeks ago for adding a new network-driver... We needed too add some DIR's and the ovs pkg to Atomic 18:24:05 greghaynes: I tried those but kept getting errors. I am guessing I have to include other dependent elements? 18:24:07 I have yet to test it b/c I have been working on a few other things 18:24:14 I will be going back to it soon. 18:24:26 Tango: You shouldnt have to, if you let me see the error I could probably help/fix 18:24:32 I wonder if I should use DIB instead 18:24:39 * greghaynes would highly recommend 18:24:54 That's what Yolanda BP is about 18:25:05 The current process is manual 18:25:07 greghaynes Is their an ETA on when yolanda's work will be ready to test? 18:25:31 there are system package requirements on the build host for -minimal elements 18:25:32 I have no idea.. 18:25:39 deboostrap for example 18:25:40 I just know about it because I was helping her review some things 18:26:05 clarkb: Are they documented somewhere? 18:26:06 OK 18:26:15 I'll ping her directly then. 18:26:17 Tango: should be in the element 18:26:23 Tango: http://docs.openstack.org/developer/diskimage-builder/elements/ubuntu-minimal/README.html 18:27:11 #topic Review Action Items 18:27:19 * daneyon_ Tango to implement option 2 in the flannel host-gw patch https://review.openstack.org/#/c/241866/ 18:27:26 Tango already provided this update 18:27:27 Thanks. 18:27:33 * daneyon_ danehans to create a network-driver support matrix 18:27:41 One sec and I'll provide the links 18:28:05 #link https://wiki.openstack.org/wiki/Magnum 18:28:14 From ^, go to Resources 18:28:26 oops References 18:28:43 and you will see Network Driver Support Matrix and Labels Support Matrix 18:29:08 Pretty basic so far, but should be a good starting point as other network drivers get added 18:29:22 Feel free to contribute to improve. 18:30:08 I have also been working on a doc that goes through magnum networking details 18:30:10 #link https://wiki.openstack.org/wiki/Magnum/Networking 18:30:29 When you have time, pls review and feel free to contribute to improve. 18:31:09 ^ doc will also get updated as we add network drivers. 18:31:20 I will add a link to the doc from the main M wiki page 18:32:34 * daneyon_ danehans check with wanghua on implementation status of https://blueprints.launchpad.net/magnum/+spec/run-kube-as-container 18:32:59 I did sync-up with wanghua ~ 2 weeks ago 18:33:24 I will give a brief update on behalf of him 18:33:28 We discussed a few details re implementation options. We're on the same page re the details. 18:33:38 great, thx hongbin 18:34:01 He put a review for the bootstrap docker daemon 18:34:34 A bootstrap docker daemon is a separated docker daemon for running etcd flannel 18:34:52 Once that patch landed, we are ready to containerize flannel 18:35:07 great 18:35:08 #link https://review.openstack.org/#/c/250999/ 18:35:30 I believe Egor had some concerns regarding the bootstrap docker daemon. 18:36:06 Yes, he worried the extra complexity of introducing the additional docker daemon 18:36:15 i would like to take 5 min to review the patch 18:36:16 eghobo_: yt? 18:36:26 hongbin: yep 18:36:49 eghobo_: we are talking about the docker bootstrap daemon 18:37:34 yes, I saw it. honestly I don't see the value 18:38:15 flannel is just 4 files you can just copy them ;) 18:38:50 also even coreos folks don't run flannel in contaner 18:39:18 From #link https://github.com/kubernetes/kubernetes/blob/release-1.0/docs/getting-started-guides/scratch.md#selecting-images 18:39:31 "For etcd, kube-apiserver, kube-controller-manager, and kube-scheduler, we recommend that you run these as containers, so you need an image to be built." 18:41:01 hongbin do you have a kube doc reference that states why the docker bootstrap is needed? 18:41:16 let me find it 18:41:33 ;) don't believe everything what you read, very often it's outdated 18:41:49 It seems like the bootstrap is not a flannel-specific requirement. 18:41:58 we can easy run 'etcd, kube-apiserver, kube-controller-manager, and kube-scheduler' 18:42:20 i have PR for everything except etcd 18:42:35 eghobo_ agreed, but i think it's important that we understand the recommendations from the kube community and use that as data for our decision 18:43:03 #link http://kubernetes.io/v1.1/docs/getting-started-guides/docker-multinode.html 18:43:19 bootstrap is very flannel specific, you need to create bridge before real docker start 18:43:24 It's my understanding that the M community is trying to ensure that our COE's are following upstream best practices. I want to make sure we're aligned from a networking standpoint. 18:43:54 If CoreOS, Kube, etc.. are saying flannel should not be containerized, then I will heavily lean on those recommendations 18:44:42 taking a few minutes to review #link http://kubernetes.io/v1.1/docs/getting-started-guides/docker-multinode.html 18:45:15 daneyon_: actually they are not saying it officially ):, but they just don't use 18:46:07 all #link http://kubernetes.io/v1.1/docs/getting-started-guides/docker-multinode.html does not provide much background on the benefits to this 2 daemon approach. 18:46:42 Is someone willing to address this further on the kube irc channel? 18:47:00 I do have concerns about over complication 18:47:46 daneyon_: I think you are the best candidate :) or I can ask wanghua to do that 18:47:57 ATM I don't see the benefits outweighing the concerns of over complication. 18:48:38 we can ask if others have tried this and have any suggestion 18:48:39 #action danehans to address the 2-daemon approach on the kube irc and provide add'l info through Magnum ML 18:48:51 I'll take it on 18:49:03 daneyon_: thx 18:49:06 Until then, I would like to see the other kube service containerized. 18:50:19 So how do we select a version with the kube service containerized? 18:50:32 like 1.1.1 ... 18:50:41 I would think it would be through an image tag 18:51:53 ok 18:52:15 i'm trying to find the doc 18:52:22 i'll shoot it to you if i can find it 18:52:29 #topic Open Discussion 18:53:11 On the last meeting, we had a brief discussion on networking for mesos cluster 18:53:23 Their is a meetup re: Docker Trusted Registry tomorrow. Check it out if you can #link http://www.meetup.com/Docker-Online-Meetup/events/227287361/ 18:53:50 Yes Tango. I believe we agreed not to iomplement Flannel for Mesos 18:54:00 at least until etcd is implemented in mesos 18:54:14 Does anyone know if tenant isolation is supported within mesos? 18:54:25 the general concensous was that not every network driver will be applicable for every coe 18:54:38 In other words, can we allow different tenants on the same mesos cluster? 18:54:52 i don't believe it is. 18:55:13 OK, so we will still have to isolate by private neutron network 18:55:13 I don't think so either 18:55:58 i believe multi-tenancy is addressed in some of the mesos frameworks 18:56:07 #link https://engineering.twitter.com/university/videos/spark-on-mesos 18:56:25 or by running mesos in magnum :-) 18:56:29 lol!!! 18:56:56 One of the concern about containers is different tenants running containers on the same host 18:57:17 we solve that by VM's and networking 18:57:51 Mesos claims to run on the whole data center, so I wonder how they address this 18:58:13 Tango take a look at the Selecting Images section #link https://github.com/kubernetes/kubernetes/blob/release-1.0/docs/getting-started-guides/scratch.md#software-binaries 18:58:24 All, we are down to our final 2 minutes. 18:59:25 In general, mesos is focused on pooling all compute resources 18:59:46 the different frameworks that run on mesos uses the pooled resources for different purposes 19:00:08 Sounds like Magnum has the right approach for multi tenancy then 19:00:11 i.e. the kube-mesos framework uses the pooled resources for container clustering engine purposes. 19:00:19 we can talk more on the M irc channel. 19:00:26 Thanks all for attending!!! 19:00:34 #endmeeting