21:01:02 #startmeeting containers 21:01:03 Meeting started Tue Oct 16 21:01:02 2018 UTC and is due to finish in 60 minutes. The chair is strigazi. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:01:04 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:01:07 The meeting name has been set to 'containers' 21:01:30 o/ 21:01:33 #topic Roll Call 21:01:39 o/ 21:01:42 o/ 21:01:43 o/ 21:01:52 \o/ 21:02:10 \o/ 21:02:38 #topic Announcements 21:03:21 thanks eandersson for fixing the magnum gate https://review.openstack.org/#/c/610740/ 21:03:27 anytime 21:03:41 Do you know why it used work or why ot broke? 21:03:48 Do you know why it used work or why it broke? 21:03:59 I was looking at some recent changes with oslo.messaging that affects the defaults 21:04:23 Which I think caused it with 9.0.1 21:04:25 that's bad 21:04:27 o.m version bump then, thanks 21:05:10 https://github.com/openstack/requirements/commit/29f8a80a71111ea59141c1aeab94bae9cf3f30a9#diff-0bdd949ed8a7fdd4f95240bd951779c8 21:05:16 introducing changes in base components and breaking everybody 21:05:31 flwang: it is better than it used to be I think :) 21:05:44 better for what? 21:05:54 less breaks/ time unit 21:06:03 haha 21:06:11 Yea - it's usually 3rd party libraries now that break things 21:06:13 like evenlet 21:06:15 O/ 21:06:20 it's true, but i think it's because its losing contributors :D 21:07:04 canori02: o/ 21:07:18 #topic Stories/Tasks 21:08:39 I don't think I have anything from last week to discuss, flwang do you want to discuss the CT rename? 21:09:03 flwang: i found a solution to making a single LB scenario 21:09:15 huge cut in resource usage is nice :) 21:09:54 imdigitaljim: use a single LB for many services? 21:10:02 for etcd/api 21:10:12 oh, right 21:10:27 imdigitaljim: is there a patch? 21:10:33 i think our plan is to submit a v2 driver that is parity with the first one's features 21:10:44 and huge improvements all around 21:10:57 cluster speed/feature rich/compatibility 21:11:44 imdigitaljim: cool, we can discuss the single lb later 21:11:50 yeah definitely 21:12:03 i can push up a WIP for you guys soon if you want to see where things are 21:12:03 strigazi: yep, i'd like to know you guys opinions about CT rename 21:12:09 imdigitaljim: go for it 21:12:41 as you know, Catalyst Cloud just released magnum on our prod 21:12:50 Nice! 21:12:55 congrats 21:12:56 and we're starting to get the pain of change/update public templates 21:13:04 cbrumm__: hah, thanks 21:13:32 that's great, how did the deployment go? 21:13:59 colin-: we got some issues, but so far, most of the things are ok 21:14:27 a big lesson learn is don't use docker_storage_size with overlay/overlay2 21:14:42 we are considering a redesign (part of the v2) 21:14:46 that decouples that entirely 21:14:50 and magnum/heat-container-agent does have the multi region bug 21:15:01 imdigitaljim: nice 21:15:04 that sounds rough, glad it worked out though 21:15:12 what is the issue with overlay? 21:15:14 so you dont have to worry about kube-tag or other quick changing data 21:15:22 ^ 21:15:28 +1 eandersson 21:15:34 what is the issues with overlay2? 21:15:35 imdigitaljim: that's interesting 21:16:24 with overlay2 and docker_volume_size(external volume for docker), we have seen some issue with docker, which cause pods can't be created 21:16:31 it's not always repeatable 21:16:56 we just realized it yesterday with strigazi's help, still need testing though 21:16:59 but just FYI 21:17:17 btw, our storage backend is ceph 21:17:18 cgroup issue? 21:17:25 Interesting - feel free to share more when you have had time to dig deeper 21:17:32 we're using cgroupfs already 21:17:35 which cgroup driver? just curious 21:17:38 ok 21:18:23 I still don't know the root cause, may need help from storage expert or docker expert 21:18:33 but i will keep you guys posted if there is any progress 21:18:43 Sounds good - thanks 21:18:48 flwang: sorry for shiming in, regarding the mentioned multi-region bug in heat-container-agent is that addressed or is there a bug tracked somewhere with more info? 21:18:54 we are planning to deploy magnum in another region soon 21:19:49 eandersson: thanks for the review 21:19:56 anytime 21:20:37 tobias-urdin: it is addressed, one moment 21:21:27 tobias-urdin: https://review.openstack.org/#/c/584215/ 21:21:56 tobias-urdin: included here: https://hub.docker.com/r/openstackmagnum/heat-container-agent/tags/ rocky-stable 21:21:57 it has been fixed 21:22:20 tobias-urdin: this patch is pending: https://review.openstack.org/#/c/609579/ 21:22:53 tobias-urdin: you also need you update your existing deployment 21:23:00 i mean Mangum 21:23:06 for existing cluster, should be fine 21:23:09 flwang: and heat to queens, right? 21:23:23 strigazi: yes, at least Queens 21:23:54 I fixed it in Rocky, and I think i have backported all necessary changes into queens 21:24:10 ok thanks! so if I understand correctly when 609579 is merged and backported to stable/rocky we are all good? 21:24:21 tobias-urdin: yes 21:24:33 got it, thanks 21:25:59 all good for Magnum 21:26:14 tobias-urdin: also need to upgrade Heat 21:27:24 ack 21:28:18 tobias-urdin: me and strigazi will have a topic at Berlin to share what we know ;) 21:31:40 flwang: cool, i'll keep an close eye out for the video on that, unfortunately won't attend berlin :( 21:32:06 no problem 21:32:20 strigazi: i'm good now 21:32:33 strigazi: imdigitaljim: anything else you want to discuss? 21:33:27 just throwing it out, were working on inplace upgrades currently 21:33:35 strigazi: has some work thats very close 21:33:39 to what we started with 21:33:59 we're adding new mechanisms but i should also be adding it to the v2 driver 21:34:21 but id like to maybe discuss some basic approaches next time 21:34:22 imdigitaljim: will you push upstream something for v1? 21:34:44 probably not for v1 since its too far back from the updates 21:34:54 it perhaps can be backported 21:35:59 we *are* keeping features of the v1 driver introduced on the v2 21:36:11 so when were done with v2 it should be a minimum everything v1 has 21:36:46 people can still choose v1 for their needs/legacy but because its a driver they can also choose v2 21:36:57 :P 21:37:00 I think we should have a spec or another document for these changes 21:37:14 makes sense? 21:37:39 im sure i can come up with something :) 21:38:18 maybe a spec to sync with nodegroups 21:38:30 yeah i plan to watch your nodegroups changes and improve/port that 21:38:53 (were not using it so i might need you to test it) 21:39:15 sure 21:39:23 i will still be pushing a few for common 21:39:33 that can help any/all drivers 21:40:11 flwang: btw congrats on the prod deploy! 21:41:03 imdigitaljim: thanks, it wouldn't happen without you guys help 21:41:12 strigazi: +1 for spec for v2 21:41:32 I assume it a big change, so better to have a spec 21:41:51 or at least doing the code and spec in parallel, so that reviewer can understand the design correctly 21:41:52 yeah totally understandable 21:42:11 some of the goals of the v2 was improved readability as well 21:42:33 imdigitaljim: quick question, does the v2 involve any API change? 21:42:39 no 21:42:42 nice 21:42:57 core magnum is mostly untouched 21:43:33 it's great 21:44:46 there are a some constraints heat >= queens/rocky?, octavia (if used) >= rocky, kubernetes >= v1.11.X 21:45:11 i think heat can be pike if not using octavia though 21:45:50 octavia >= queens* 21:46:01 queens is a reasonable requirement 21:49:34 Anything else for the meeting? 21:49:51 strigazi: i'm good 21:50:10 others? 21:50:40 the issue we talked about earlier today is fixed in https://review.openstack.org/#/c/611097/ 21:51:06 nothing fancy :) 21:51:18 thanks +2 21:51:33 should be safe for backports as well 21:52:49 thanks everyone for joining the meeting, see you 21:53:10 #endmeeting