16:02:41 <adrian_otto> #startmeeting containers
16:02:41 <openstack> Meeting started Tue Sep  1 16:02:41 2015 UTC and is due to finish in 60 minutes.  The chair is adrian_otto. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:02:42 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:02:44 <openstack> The meeting name has been set to 'containers'
16:02:48 <adrian_otto> #link https://wiki.openstack.org/wiki/Meetings/Containers#Agenda_for_2015-09-01_1600_UTC Our Agenda
16:02:53 <adrian_otto> #topic Roll Call
16:02:56 <mfalatic> o/ redux
16:02:57 <adrian_otto> Adrian Otto
16:02:59 <xek> o/
16:02:59 <daneyon_> o/
16:03:00 <ctrath> o/
16:03:01 <muralia> murali allada
16:03:01 <Drago> o/
16:03:01 <dane_leblanc> o/
16:03:01 <rlrossit> o/
16:03:03 <wanghua> o/
16:03:03 <thomasem> Thomas Maddox
16:03:04 <vilobhmm1> 0/
16:03:08 <Tango> Ton Ngo
16:03:10 <eghobo> o/
16:03:12 <travisn> Travis Nguyen
16:03:13 <hongbin> o/
16:03:15 <juggler> o/ Perry Rivera   hello all!
16:03:39 <apmelton> o/
16:03:47 <diga> o/
16:04:18 <harshs> o/
16:04:55 <wznoinsk> o/
16:05:29 <adrian_otto> hello mfalatic, xek, dane_leblanc, daneyon_, Drago, ctrath, muralia, rlrossit, wanghua,thomasem, vilobhmm1, Tango, eghobo, travisn, hongbin, juggler, apmelton, diga, harshs, and wznoinsk
16:05:34 <adrian_otto> great group here today
16:05:58 <adrian_otto> welcome sdake/sdake_
16:06:07 <adrian_otto> #topic Announcements
16:06:13 <sdake_> yo
16:06:15 <sdake_> morning
16:06:22 <sdake_> lotss of cats :)
16:06:27 <adrian_otto> 1) OpenStack Silicon Valley event went great.
16:06:51 <adrian_otto> check out www.openstack.org/containers for the new containers whitepaper
16:07:19 <adrian_otto> There is video of the event online, for those of you unable to attend
16:07:25 <adrian_otto> anyone want links to that?
16:07:34 <vilobhmm1> yes please :)
16:07:35 <daneyon_> sure
16:07:35 <apmelton> definitely
16:07:36 <sdake_> yup
16:07:41 <ctrath> yes
16:07:48 <juggler> yes please
16:07:56 <thomasem> undoubtedly
16:08:03 <apmelton> we're sitting on the edge of our seats here!
16:08:10 <thomasem> I fell out of my chair.
16:08:22 <juggler> somebody say no just for fun lol
16:08:34 <mfalatic> no... I won't.
16:08:40 <mfalatic> (does that count?)
16:08:42 <adrian_otto> 1:18:00 - 1:23:12 http://livestream.com/iTechSherpa/OpenStackSV/videos/97301358
16:08:52 <juggler> mfal heh
16:08:52 <adrian_otto> 00:02:00 - 00:36:00 http://livestream.com/iTechSherpa/OpenStackSV/videos/97409605
16:09:01 <adrian_otto> ok, so those are the ones I appeared in
16:09:18 <juggler> great thanks
16:09:31 <adrian_otto> tpics are basically OpenStack Maturity, the OpenStack Innovation Center (OSIC), and Magnum (contrast to Virtual Machines)
16:09:40 <adrian_otto> there were also press interviews
16:09:49 <adrian_otto> I gave credit for the new things we added to Magnum as a team
16:10:11 <adrian_otto> and a little kerfuffle with James Waters
16:10:23 <adrian_otto> you might want to watch the panel to the end if you care about that
16:10:37 <adrian_otto> ok, so we are closing in on Tokyo
16:10:54 <adrian_otto> sdake_: I understand last week we put topics on an etherpad
16:10:59 <adrian_otto> for the design Summit
16:11:02 <sdake_> right
16:11:15 <sdake_> ttx asked for session count
16:11:18 <sdake_> so i got us kicked off
16:11:22 <sdake_> we need to trim the count
16:11:32 <adrian_otto> 2) Dates for the Tokyo OpenStack Summit are Oct 26-30
16:11:34 <sdake_> i saw an email this morning on the topic - did yu catch it?
16:11:41 <adrian_otto> Design Summit is the last three days, right?
16:11:50 <sdake_> iirc
16:11:58 <adrian_otto> okay, we will plan to revisit that today
16:12:11 <adrian_otto> any other announcements form team members?
16:12:51 <tcammann> o/ sorry I'm late
16:13:01 <apmelton> adrian_otto: looks like design summit is tuesday through friday
16:13:08 <juggler> tcammann no worries...thanks for coming :)
16:13:10 <adrian_otto> tx apmelton
16:13:14 <vilobhmm1> etherpad link for topics discussed last week : https://etherpad.openstack.org/p/magnum-tokyo-summit-sessions
16:13:21 <adrian_otto> welcome tcammann
16:13:26 <sdake_> friday is contributor meetup day
16:13:33 <apmelton> ah gotcha
16:13:36 <adrian_otto> #topic Container Networking Subteam Update
16:13:43 <adrian_otto> daneyon_: ?
16:13:55 <adrian_otto> I think the link to the previous meeting in the agenda is outdated
16:14:00 <adrian_otto> I will correct that now
16:14:23 <daneyon_> oops
16:14:45 <adrian_otto> #link http://eavesdrop.openstack.org/meetings/container_networking/2015/container_networking.2015-08-27-18.00.html Previous Network Subteam Meeting
16:14:58 <daneyon_> we spent most of the meeting reviewing and adding to the kuryr integration etherpad and kuryr design spec
16:15:13 <daneyon_> #link https://review.openstack.org/#/c/213490/
16:15:20 <daneyon_> #link https://etherpad.openstack.org/p/magnum-kuryr
16:15:50 <daneyon_> i took an action to clean up the integration ep and work with the kuryr team to create actionable steps
16:15:53 * adrian_otto updated the agenda with the revised link to the subteam meeting minutes
16:15:58 <daneyon_> to move the integration planning forward
16:16:17 <daneyon_> their was also an action for everyone to test the WIP patches that i have posted
16:16:29 <daneyon_> I have recently updated the client patch to include labels
16:16:41 <daneyon_> that's about it
16:16:44 <daneyon_> any questions?
16:16:47 <adrian_otto> yes, di all the reviewers get clarity on that?
16:17:01 <adrian_otto> I noticed there were questions in the comment thread on that patch
16:17:14 <adrian_otto> let's find the link to that daneyon_
16:17:23 <daneyon_> adrian_otto I am still working through addressing all the comments in all the patches
16:17:41 <daneyon_> a link to each of the WIP patches?
16:17:49 <adrian_otto> #link https://review.openstack.org/215260 WIP: Adds Container Networking Model Support
16:17:51 <daneyon_> or just the client?
16:18:17 <adrian_otto> Manjeet raised a question there, but I think we have that covered
16:18:21 <daneyon_> this is the client:
16:18:23 <daneyon_> #link https://review.openstack.org/#/c/215260/
16:18:58 <adrian_otto> ok, anything our networking subteam needs from the other contributors at this time?
16:19:10 <daneyon_> I still need to address the feedback from Jay Lau and Ton Ngo for the link you provided
16:19:46 <adrian_otto> ok, thanks
16:19:52 <daneyon_> adrian_otto no, just for someone else to test the patches. I believe Dane Leblanc is taking that on
16:19:55 <daneyon_> thx
16:20:06 <adrian_otto> daneyon_: great, thanks. I'll advance topics now
16:20:11 <daneyon_> ok
16:20:16 <adrian_otto> #topic Magnum UI Subteam Update (bradjones)
16:20:23 <adrian_otto> I think Brad is absent today
16:20:28 <adrian_otto> I'm a little worried about these:
16:20:46 <adrian_otto> #link https://review.openstack.org/211750
16:21:21 <adrian_otto> I have new contributors interested in that work, but we don't even have a readme file merged into the repo yet
16:21:34 <adrian_otto> so what can we do to move that along?
16:22:06 <adrian_otto> will a volunteer follow up with Brad Jones, and offer to adopt the patchset if needed?
16:22:26 <hongbin> adrian_otto: I think there is a readme file
16:22:49 <Tango> The patch that follows has a readme file.  I tried it out but it doesn't work yet
16:23:00 <hongbin> adrian_otto: it seems to have necessary instruction to get started
16:23:06 <adrian_otto> #link https://github.com/openstack/magnum-ui Our magnum-ui repo
16:23:23 <adrian_otto> looks empty to me, maybe I missed something?
16:23:32 <hongbin> #link https://review.openstack.org/#/c/211750/10/README.rst
16:23:38 <Tango> you have to download the patch
16:23:40 <adrian_otto> right, we need to merge that
16:23:56 <adrian_otto> if it's imperfect, fine, let's iterate on it
16:24:03 <adrian_otto> but let's get something into the repo
16:24:41 <sdake> adrian_otto one action from last week fory ou
16:24:49 <sdake> is to ride people in the magnum-ui-core to actually do reviews
16:24:52 <adrian_otto> sdake, oh?
16:25:06 <hongbin> I think they did the reviewers
16:25:06 <sdake> because they aren't happening ;)
16:25:11 <adrian_otto> yeah, that action should be carried forward
16:25:25 <adrian_otto> so look, either we produce code, or we rip out the project
16:25:38 <sdake> agree
16:25:51 <adrian_otto> because putting down a flag on something prevents others from innovating on it
16:26:59 <adrian_otto> #topic Review Action Items
16:27:18 <adrian_otto> #action adrian_otto to follow up with magnum-ui cores to inspire faster progress
16:27:55 <adrian_otto> 1) sdake to walk yuanying-alt through how gating works
16:27:56 <adrian_otto> Status?
16:28:11 <sdake> h ehas been walked
16:28:21 <adrian_otto> completed, great!
16:28:22 <adrian_otto> 2) tango to create a blueprint for the kuberntees 1.0 port and include devstack, image creation, k8sclient regen, heat templates in it
16:28:27 <adrian_otto> Status?
16:28:35 <adrian_otto> I think I have 2 BPs on this topic
16:28:44 <Tango> Done, we have 6 patches against the BP
16:28:47 <adrian_otto> one commented as redundant, and marked for death
16:28:52 <Tango> Yes one is a duplicate
16:29:03 <Tango> I didn't notice
16:29:19 <adrian_otto> ok, so is there more work remaining?
16:29:32 <adrian_otto> or should the BP be marked as Implemented?
16:29:43 <Tango> Yes, the next step is to debug the V1 API code that Hongbin generated
16:30:09 <Tango> I am also debugging a new image
16:30:19 <adrian_otto> ok, cool. Action item is complete, BP work continues. Thanks for the update Tango.
16:30:20 <hongbin> I debug the generated client with *-4 image. Waiting for the *-v5 image to debug further
16:30:33 <Tango> travisn volunteered to help with functional test
16:30:38 <adrian_otto> 3) x3k to start ml thread on versioned objects review objectives
16:30:38 <adrian_otto> Status: Complete
16:30:46 <adrian_otto> I will revisit this topic in a moment
16:30:57 <adrian_otto> 4) adrian_otto to ping all magnum-ui core reviewers to review bradjones magnum ui patches
16:31:01 <adrian_otto> (carried forward)
16:31:19 <adrian_otto> #topic Versioned Objects
16:31:36 <adrian_otto> we have a non-unanimous consensus on adding support for this feature.
16:32:00 <adrian_otto> I'm tempted to just call tie break on this, but I'm open to hearing further input before then
16:32:28 <adrian_otto> hongbin: I know you opposed this, and I do recognize your reasoning
16:32:35 <hongbin> As I started in the ML, I agree to merge the code, but don't like to start bumping object version.
16:33:01 <hongbin> Yes, so -1 from me
16:33:08 <adrian_otto> I do agree that it is one more thing to deal with as we evolve our data model
16:33:12 <eghobo> I think it's good idea, my only concern there is not enough how to do it
16:33:14 <rlrossit> so my baseline request is to have this assertion be made before the start of M
16:33:47 <rlrossit> because at that point, we have a "release" that we need to be able to (eventually) do rolling updates from
16:33:53 <xek> rlrossit, +1
16:34:06 <dims> rlrossit: +1
16:34:10 <eghobo> hongbin: after we become familiar with the process it will become routine as unit tests ;)
16:34:10 <rlrossit> if it's missed now, you have to wait 6 months to even have the decision again
16:34:16 <ctrath> rlrossit has been working on it in Nova… rlrossit, do you have the cycles to overlook it in Magnum?
16:34:35 <adrian_otto> s/overlook/oversee/
16:34:45 <rlrossit> yes, as long as I have that unit test to help me out :)
16:34:46 <ctrath> +1
16:35:09 <rlrossit> It was a struggle keeping an eye on the Nova changes without the unit tests I was attempting to add
16:35:17 <adrian_otto> worst case if we try this and it's a serious impediment, we can take it back out
16:35:24 <rlrossit> adrian_otto: +1
16:35:28 <dims> totally, +1
16:35:40 <adrian_otto> but in all honesty our data model changes are not frequent enough for this to be a significant burden
16:35:54 <rlrossit> (don't misunderstand initial learning time as a mistake though)
16:36:45 <eghobo> adrian_otto: +1
16:36:48 <dims> another aspect is if we don't learn what's needed now (what the steps needed are for using o.vo effectively), it will take a lot more pain later
16:36:55 <adrian_otto> ok, so are there other viewpoints to oppose this change?
16:37:08 <Tango> +1  We do need it, so we pay now or pay later
16:37:21 <hongbin> The last thing is that we need to have a clear document on how to do that (how to bump version, etc.)
16:37:26 <vilobhmm1> +1 agree
16:37:31 <rlrossit> hongbin: I can totally do that
16:37:32 <dims> hongbin: totally agree
16:37:33 <adrian_otto> hongbin: , agreed 100%
16:37:33 <wanghua> We need it sooner or later
16:37:44 <adrian_otto> rlrossit: can I assign you an action item for that?
16:37:45 <rlrossit> let me know where you want it, and I can write it up
16:37:54 <adrian_otto> is by next Tues enough time?
16:37:56 <tcammann> docs would be great! +1
16:38:00 <dims> rlrossit: we can add it to o.vo itself
16:38:15 <juggler> +1
16:38:19 <xek> +1
16:38:21 <rlrossit> adrian_otto: I think I can do it in a week
16:38:32 <rlrossit> depends on how fine-grained we want me to get
16:38:38 <adrian_otto> ok, so let's do that, and make a good compromise here
16:38:43 <dims> go rlrossit!!
16:38:54 <juggler> thanks rlr
16:39:06 <rlrossit> adrian_otto: if it pleases more folks, we don't have to merge my test until we are happy with my docs too
16:39:31 <Tango> Lots of review on your docs
16:39:38 <adrian_otto> #action rlrossit Add documentation that clearly defines how to bump object versions, and reference the location of those instructions in related unit test errors so they can be easily located.
16:39:43 <adrian_otto> rlrossit: is that action fair?
16:39:44 <vilobhmm1> rlrossit : will be happy to help
16:39:58 <rlrossit> adrian_otto: totally!
16:40:02 <hongbin> yes
16:40:02 <adrian_otto> ok, cool
16:40:15 <adrian_otto> and look, if this ends up being a huge mess for us, we will revisit it
16:40:34 <adrian_otto> #topic Blueprint/Bug Review
16:40:46 <adrian_otto> #link https://blueprints.launchpad.net/magnum/+spec/objects-from-bay Obtain the objects from the bay endpoint (sdake)
16:40:51 <adrian_otto> is sdake still on this BP?
16:40:57 <vilobhmm1> i am working on this
16:41:04 <sdake> vilo should own it
16:41:09 <adrian_otto> ok, adjusting
16:41:41 <adrian_otto> reassigned.
16:41:50 <vilobhmm1> proposed the WRITE path here https://review.openstack.org/#/c/213368/ working on READ path…this needed lot of thought behind it…should have it ready by next meeting
16:42:23 <vilobhmm1> making sure with this approach keep the existing functionality and usage intact
16:42:39 <vilobhmm1> adrian_otto : ^^
16:42:45 <adrian_otto> thanks vilobhmm1
16:42:54 <adrian_otto> need anything from team members to assist you?
16:43:18 <vilobhmm1> adrian_otto : nothing…may be reviews that I will request later :)
16:43:21 <adrian_otto> ok, advancing to the next one.
16:43:23 <adrian_otto> #link https://blueprints.launchpad.net/magnum/+spec/secure-kubernetes Secure the client/server communication between ReST client and ReST server (madhuri)
16:43:36 <adrian_otto> madhuri is not in attendance today
16:43:39 <apmelton> I can give an update here
16:43:45 <adrian_otto> tx apmelton
16:44:03 <apmelton> I've taken over her patch (https://review.openstack.org/#/c/214179/) while she's away
16:44:19 <adrian_otto> great
16:44:22 <apmelton> I had that entire patch tree working in devstack yesterday
16:44:36 <apmelton> just need to rebase it, and finish up test coverage then it should be out of WIP
16:44:37 <adrian_otto> whoot!!
16:44:46 <adrian_otto> ok, looking forward to that.
16:45:06 <adrian_otto> reviewers, please earmark a chunk of time to get a good look at this code
16:45:18 <adrian_otto> this is the most important addition to Magnum in this cycle
16:45:43 <adrian_otto> because without it, Magnum is not suitable for production workloads
16:46:02 <adrian_otto> any other remarks, apmelton?
16:46:12 <apmelton> nope
16:46:19 <adrian_otto> sweet, tx. Advancing…
16:46:21 <adrian_otto> #link https://blueprints.launchpad.net/magnum/+spec/external-lb Support the kubernetes service external-load-balancer feature (Tango)
16:46:36 <adrian_otto> I think this one is close, per remarks from earlier
16:46:56 <Tango> I tested out several more scenarios to narrow down the failure
16:47:34 <Tango> I did build a customer Kubernetes based on 1.0.3 with the 3 patches that Gus suggested, but it shows the same error
16:48:10 <Tango> I also verified that creating the load balancer using curl from the master node works, so the infrastructure is OK within the cluster
16:48:36 <Tango> Gus mentioned he is building his own devstack with magnum to try to replicate the error
16:48:52 <Tango> Haven't heard from him for a week though
16:49:24 <adrian_otto> ok, time to ping him again
16:49:40 <adrian_otto> please let me know if follow up from me may help
16:49:47 <mfalatic> Right now that LB is active in the code, right?
16:49:48 <Tango> So debugging continues, we are getting closer
16:49:58 <adrian_otto> thanks Tango. Advancing…
16:50:00 <adrian_otto> #link https://blueprints.launchpad.net/magnum/+spec/external-lb Support the kubernetes service external-load-balancer feature (Tango)
16:50:05 <adrian_otto> whoops
16:50:08 <adrian_otto> ignore
16:50:13 <adrian_otto> #link https://blueprints.launchpad.net/magnum/+spec/secure-docker Secure client/server communication using TLS (apmelton)
16:50:30 <adrian_otto> this is related to the previous update, but specific to our swarm bay
16:50:36 <apmelton> no new update there, but as soon as I get the other patch out of WIP, I'll begin this
16:50:42 <adrian_otto> ok, thanks.
16:50:50 <adrian_otto> #topic Open Discussion
16:51:15 <wanghua> I has something to discuss
16:51:24 <wanghua> for docker registry v2
16:51:28 <adrian_otto> ok
16:51:39 <wanghua> If we use trust, which user we should trust
16:51:56 <wanghua> I think we should create a user for registry v2
16:52:21 <wanghua> all the bay use the user and trust id to communicate with swift
16:52:24 <adrian_otto> ideally, we would trust the user who generates the image
16:52:36 <adrian_otto> but registry is not multi-tenant in that way
16:52:48 <adrian_otto> so we'll probably want to define a system user for that
16:53:09 <wanghua> yes, we should add a user in config
16:53:16 <adrian_otto> and acknowledge that storage consumed by that user is not accounted for by the individual Magnum user
16:53:36 <adrian_otto> we can iterate on that to avoid abuse
16:53:39 <eghobo> wanghua: could you elaborate what was changed in v2 related to user, sorry I don't remember any changes in this area
16:54:14 <hongbin> eghobo: The VM needs a OS credential to talk to swift for storing docker image
16:54:25 <wanghua> registry v2 is used to construct a local docker image repo
16:55:15 <adrian_otto> step one is to simply have a private repo capability per bay for performance and manageability reasons
16:55:23 <eghobo> you mean you want to run docker registry at each master and use swift as backend, right?
16:55:28 <wanghua> it needs to put the image to swift, so credentials are needed
16:55:35 <adrian_otto> then we can look at that to approach a more ideal usage and security posture for that feature
16:55:37 <wanghua> right
16:55:38 <notmyname> adrian_otto: you might find http://docs.openstack.org/developer/swift/overview_auth.html#openstack-service-using-composite-tokens interesting. originally for stuff like glance. probably useful here too
16:56:11 <mfalatic> I have a topic I'd like to briefly mention here. On recent devstack installs, who else is running into the problem of etcd dying on the master node after a bay is created? It seems to underpin the bug I've been working on: https://bugs.launchpad.net/magnum/+bug/1481889
16:56:12 <openstack> Launchpad bug 1481889 in Magnum "pod-create fails ('NoneType' object has no attribute 'status')" [Undecided,Confirmed] - Assigned to Martin Falatic (martinfalatic)
16:56:13 <adrian_otto> notmyname: yes, that's exactly the sort of enhancements that should follow our initial naive implementation
16:56:22 <notmyname> tl;dr objects owned by the tenant, but only editable if both the tenant and service tokens are available. makes accounting easier and prevents the user from messing up external indexes
16:56:27 <mfalatic> (If nothing else, PM me)
16:56:41 <adrian_otto> thanks for raising that mfalatic
16:57:35 <adrian_otto> we'll need to wrap up in a few minutes
16:58:06 <mfalatic> This is definitely a new issue. I dug up an older devstack tarball I had and so far it's only an issue in new devstack deployments with magnum
16:58:14 <adrian_otto> mfalatic: are the OOM messages in the dmeg buffer when that happens?
16:58:26 <adrian_otto> s/dmeg/dmesg/
16:58:32 <mfalatic> Digging into that now. Out of memory
16:58:34 <mfalatic> ?
16:58:35 <adrian_otto> or turn on syslog and check there
16:58:46 <adrian_otto> yes, that's one common reason for an unexpected process termination
16:59:20 <adrian_otto> another way to debug that is to attach an strace to the etcd process while it is still running
16:59:21 <mfalatic> I will try that, but not quite sure how to trap it when it fails on bay create.
16:59:30 <adrian_otto> and get clues about the outcome by watching the syscall trace
16:59:34 <mfalatic> ok
16:59:45 <adrian_otto> time for us to wrap up.
16:59:49 <mfalatic> Thank you
17:00:05 <Nikolay_St> hi all
17:00:10 <Nikolay_St> it's murano time
17:00:21 <adrian_otto> thanks everone for attending. Our next meeting is 2015-09-08 at 2200 UTC
17:00:24 <adrian_otto> #endmeeting