21:00:42 <strigazi> #startmeeting containers 21:00:43 <openstack> Meeting started Tue Aug 28 21:00:42 2018 UTC and is due to finish in 60 minutes. The chair is strigazi. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:47 <openstack> The meeting name has been set to 'containers' 21:00:53 <strigazi> #topic Roll Call 21:01:03 <strigazi> o/ 21:01:05 <colin-> hello 21:01:15 <tobias-urdin> hello o/ 21:02:02 <imdigitaljim> o/ 21:02:40 <strigazi> Thanks for joining the meeting colin- tobias-urdin imdigitaljim 21:02:56 <strigazi> #agenda https://wiki.openstack.org/wiki/Meetings/Containers 21:03:11 <strigazi> #topic Announcements 21:03:41 <strigazi> Magnum 7.0.0 provides certified kubernetes 21:04:00 <strigazi> thanks to flwang that pushed the results to cncf 21:04:14 * strigazi is fetching the path 21:04:14 <colin-> awesome 21:04:18 <imdigitaljim> :] 21:04:21 <cbrumm> Congrats all 21:04:24 <tobias-urdin> cool, nice work 21:04:45 <strigazi> #link https://github.com/cncf/k8s-conformance/commit/263ea91efef9f1decd3c5f5437ee704f0a1af261 21:05:16 <strigazi> FYI, magnum queens was passing the conformance tests as well, we just didn't posted the results to cncf 21:05:29 <canori02> o/ 21:05:38 <strigazi> Now it is more official 21:05:42 <strigazi> hi canori02 21:05:46 <imdigitaljim> are these just sonobuoy runs? 21:06:01 <strigazi> yes, nothing more nothing less 21:06:14 <imdigitaljim> SUCCESS! -- 143 Passed | 0 Failed | 0 Pending | 853 Skipped PASS 21:06:26 <imdigitaljim> i feel it a goal to improve the tests skipped 21:06:32 <imdigitaljim> (if applicable) 21:06:58 <imdigitaljim> it skips some tests when features arent present 21:07:08 <imdigitaljim> some it skips with alternative configurations 21:07:10 <imdigitaljim> so it depends 21:07:26 <strigazi> let's see if we can run more 21:08:29 <imdigitaljim> anyways nice job o/\o 21:08:35 <strigazi> 2nd announcement: magnum 7.0.1 is released, we will to more point releases in rocky 21:08:45 <strigazi> 2nd announcement: magnum 7.0.1 is released, we will two more point releases in rocky 21:09:15 <strigazi> Of course we can do more later if we find bugs 21:09:39 <strigazi> #topic Blueprints/Bugs/Ideas 21:10:00 <strigazi> I have pushed the patch to put kubelet on the master node 21:10:08 <imdigitaljim> looks good 21:10:29 <strigazi> And since I was trying to make it work with flannel on the master node 21:10:40 <imdigitaljim> when flwang finishes his part ill be pushing any of my remaining changes 21:10:47 <strigazi> I proposed a patch for making flannel self hosted 21:10:54 <strigazi> similar to calico 21:10:54 <imdigitaljim> :) 21:11:15 <strigazi> What is flwang's part? 21:11:19 <strigazi> on kubelet? 21:11:33 <imdigitaljim> not on kubelet, he was working on some remaining changes i believe 21:11:41 <strigazi> ok 21:12:29 <strigazi> If the flannel patch passes the conformance tests, I don't see a reason not to take, it cleans a lot of files. 21:13:06 <strigazi> thoughts? ^^ 21:13:26 <strigazi> *take it in rocky 21:13:55 <colin-> i can't think of anything about flannel that would prevent it from working in that way; sounds good to me 21:14:48 <strigazi> I like the fact that it doesn't mess with docker config a lot like before. let's take this discussion in the review: 21:15:02 <strigazi> #link https://review.openstack.org/#/c/597150/ 21:15:20 <colin-> i'll take a look there 21:15:28 <strigazi> and two more items: 21:16:03 <strigazi> You can give input in flwang's patches for cluster health monitoting here: 21:16:20 <strigazi> #link https://review.openstack.org/#/q/status:open+project:openstack/magnum+branch:master+topic:story/2002742-24593 21:16:38 <strigazi> I posted sample output of some possible cases here: 21:16:57 <strigazi> #link https://review.openstack.org/#/c/572897/ 21:17:36 <strigazi> I think we can take it this week. We will just need a client patch as well. 21:18:52 <strigazi> And last item, is regarding upgrades, I haven't pushed the UTs yet but the API will not change: 21:18:59 <strigazi> #link https://review.openstack.org/#/c/514959/ 21:20:17 <strigazi> Next, I think tobias-urdin wants to bring up some items 21:20:29 <tobias-urdin> sure 21:20:54 <tobias-urdin> i've pretty much went through what i through is missing to make it easier to run magnum in production 21:21:19 <tobias-urdin> magnum-ui is missing functionality, we cannot discover COE, volume drivers, network drivers etc 21:21:49 <strigazi> We adddress this issue by using public cluster templates ^^ 21:22:07 <tobias-urdin> two other things I think is, two config options, one to select allowed coe engines and allowed volume drivers 21:22:13 <tobias-urdin> same like was done for the network drivers 21:23:13 <strigazi> We have an option in magnum conf to enable only some drivers 21:23:27 <strigazi> for volumes, let me check 21:24:01 <tobias-urdin> having something like glance implements, a discovery endpoint that then magnum-ui can use to populate causing it to not be hardcoded 21:24:12 <tobias-urdin> i think would benefit a lot 21:26:02 <strigazi> For disabling COE's we have http://git.openstack.org/cgit/openstack/magnum/tree/magnum/conf/drivers.py#n40 21:26:08 <tobias-urdin> another example would be swarm-mode completely missing in magnum-ui :) 21:26:31 <strigazi> and for ops we have magnum list-drivers 21:27:06 <strigazi> So, you proposal is to discover the configuration options with an API? 21:27:10 <tobias-urdin> ok, how would the disabled_drivers look if specified? 21:28:01 <strigazi> magnum has these drivers enabled by default: http://git.openstack.org/cgit/openstack/magnum/tree/setup.cfg#n60 21:28:27 <tobias-urdin> thanks 21:28:29 <strigazi> in the magnum controller node the admin can do: 21:28:35 <strigazi> magnum list-drivers 21:28:46 <strigazi> and see which are enabled 21:29:13 <strigazi> using disabled_drivers the operator can opt-out from using some of them 21:29:42 <tobias-urdin> then that is exactly what i'm looking for, i then think that is something we should expose using API so that the UI can consume it 21:30:59 <strigazi> we decided not give the end users this option, one it adds complecity and second the clients don't need access to such low level info about magnum. 21:31:22 <strigazi> What we actually tried in the past was 21:31:38 <strigazi> to allow operators intriduce more drivers more easily 21:32:13 <strigazi> but continue to provide the users with public cluster templates 21:33:03 <strigazi> what we could invesigate is, having magnum-ui consume some of the configuration options in the magnum.conf? 21:33:23 <imdigitaljim> magnum-ui may not always be on the same location as magnum.conf 21:33:37 <tobias-urdin> yeah :( 21:33:38 <strigazi> imdigitaljim: 99% is not in the same place 21:33:41 <imdigitaljim> so i'd say magnum-api should still provide it and ui consume it 21:34:02 <imdigitaljim> by same please i mean even same machine* 21:34:31 <strigazi> I think end users should only consumes the cluster API 21:34:36 <strigazi> I think end users should only consume the cluster API 21:34:52 <strigazi> not the clustertemplate api 21:35:08 <imdigitaljim> we also agree with that sentiment 21:35:25 <imdigitaljim> we have modified our ui to only allow cluster creation and not template creation 21:35:34 <strigazi> the massive hole we have on the cluster template API is labels 21:35:37 <imdigitaljim> not just by policy but we've simplified the ux 21:35:57 <imdigitaljim> yeah labels are a feature-gate but eventually need to be turned into actual features :P 21:36:05 <strigazi> labels can not be consumed by the API atm 21:36:15 <imdigitaljim> i think we should convert to consuming config files 21:36:29 <imdigitaljim> have a config file map to a template 21:36:35 <tobias-urdin> my users is mostly ui based, so we'll be doing the same, which is what i'm wanting to prevent 21:36:42 <imdigitaljim> instead of openstack coe cluster template create --amillionthings 21:36:55 <tobias-urdin> dont want to modify downstream, instead have it supported upstream the ways operators want to use it 21:37:16 <imdigitaljim> we're looking for ways to make the new content upstreamable 21:37:24 <imdigitaljim> but still just consuming it as downstream for now 21:37:41 <strigazi> tobias-urdin: if you provide users with public cluster templates? Doesn't this help? 21:38:57 <tobias-urdin> yes but it doesn't address all pain points that the ui currently has, you never know which coe, volume/network drivers is supported (and recommended) by the operators 21:39:18 <imdigitaljim> this is how simple our create panel is 21:39:19 <imdigitaljim> https://imgur.com/a/Y7acH9X 21:39:39 <strigazi> but these parameters are not exposed in the cluster api 21:40:14 <imdigitaljim> id like to add some kind of configurable traits to simplify the user experience on the ui 21:40:31 <imdigitaljim> as an option at least 21:41:16 <strigazi> ours look like this: https://imgur.com/a/Sqk56Zw 21:41:20 <tobias-urdin> i think we've come to the conclusion that we have something to look at atleast :) i'm very satisfied disabling drivers was possible, which i probably should have checked better myself 21:42:06 <strigazi> user need only to select the cluster template and the size of the cluster, flavor, num_nodes, docker-volume-size 21:42:23 <strigazi> s/user/users/ 21:42:59 <tobias-urdin> when we are on the magnum-ui subject, i would like to propose a change to add swarm-mode to the hardcoded COE list in magnum-ui, backport it 21:43:23 <strigazi> https://imgur.com/a/xx5DnnO 21:43:23 <tobias-urdin> and request a new magnum-ui release for queens to release that change and bug fix https://review.openstack.org/#/c/595245/ 21:43:38 <strigazi> we will do both 21:44:28 <strigazi> tobias-urdin: imdigitaljim what we discussed in the community before was: 21:45:05 <strigazi> have cluster template or another api object that operators use for configuration parameters and to set the allowed values 21:45:58 <strigazi> the end users would consume the cluster templates, fork them and edit them but being restricted by the operator 21:47:13 <strigazi> it required a lot of effort on the API and we lacked man power to do it. 21:47:31 <tobias-urdin> which is fine and working good, i'm simply talking about the population of the fields for cluster templates 21:47:52 <strigazi> So, at least at CERN, we decided to have a set of supported Cluster Templates and tell users consume those 21:48:00 <tobias-urdin> it's no big deal really, but a simple thing like the swarm-mode COE is not even present in the list there 21:48:25 <strigazi> advanced users that want more features should open tickets or experiment with the api. 21:48:49 <strigazi> tobias-urdin: swarm-mode should be there, we missed that. 21:49:16 <tobias-urdin> which takes me back to :) 21:49:16 <tobias-urdin> 23:33 < tobias-urdin> when we are on the magnum-ui subject, i would like to propose a change to add swarm-mode to the hardcoded COE list in magnum-ui, backport it | 21:49:21 <tobias-urdin> 23:34 < tobias-urdin> and request a new magnum-ui release for queens to release that change and bug fix https://review.openstack.org/#/c/595245/ 21:49:50 <strigazi> please cherry-pick it 21:49:58 <strigazi> to rocky and queens 21:50:14 <strigazi> I'll approve and release 21:50:47 <strigazi> for swarm-mode propose a patch and will take it, backport it and release it 21:50:52 <strigazi> sounds good? 21:51:09 <tobias-urdin> yeah, thanks 21:51:41 <strigazi> Anything else for today? 21:52:00 <tobias-urdin> i have one more, unless somebody else have anything 21:52:12 <strigazi> tobias-urdin: where are you based? Europe? Sure, go ahead 21:52:33 <tobias-urdin> yes gmt+1 21:52:58 <canori02> Just my coreos patch for me. I believe I have the cert issue sorted out. Just need reviews again 21:53:03 <tobias-urdin> i've been testing around to see what works, are there any support matrix COE vs OS version etc 21:53:16 <tobias-urdin> kubernetes pretty much worked on first attempt, seems the most maintained 21:53:35 <strigazi> Fedora Atomic 27 works with kubernetes and swarm-mode 21:53:52 <tobias-urdin> docker swarm is broken on fedora 27, etcd is not in fedora atomic but is run as container, which i saw a commit 21:54:05 <strigazi> os_distro=fedora-atomic coe=swarm-mode|kubernetes 21:54:27 <tobias-urdin> docker swarm-mode on fedora 27 almost works, it's missing tcp port 2377 so minion node cannot join swarm cluster 21:54:47 <tobias-urdin> is fedora 28 working at all? 21:55:04 <strigazi> it does 21:55:08 <canori02> Works with k8s for me 21:55:19 <tobias-urdin> to fix swarm-mode we need to push port 2377 for security group 21:55:22 <tobias-urdin> here i think https://github.com/openstack/magnum/blob/master/magnum/drivers/swarm_fedora_atomic_v2/templates/swarmcluster.yaml#L244 21:55:27 <strigazi> got it 21:56:16 * strigazi is thinking how swarm-mode works in devstack 21:56:23 <tobias-urdin> same there, would appreciate a backport for such a fix 21:56:30 <strigazi> +1\ 21:57:01 <tobias-urdin> without even googling my guess is 2377 is TLS swarm endpoint and 2375 is non-tls which perhaps, unfoutunately, the testing uses? 21:57:22 <strigazi> 2375 is behind tls 21:57:30 <tobias-urdin> i added 2377 to sec group after I saw that was executed in the magnum-swarm-manager script 21:57:34 <strigazi> AFAIK 2377 is not 21:58:08 <strigazi> 2377 is the default port for the swarm-mode inter-node communication 21:58:58 <strigazi> tobias-urdin: in your cloud, magnum creates a private network per cluster? 22:00:35 <tobias-urdin> unsure 22:00:51 <strigazi> tobias-urdin: just push a patch for the security group and we take it. Just add a story in storybaord 22:00:54 <tobias-urdin> think that example used an existing private network 22:01:18 <strigazi> we can take the discussion there 22:01:25 <tobias-urdin> sure 22:01:28 <strigazi> anything else? it's time 22:01:48 <tobias-urdin> sorry for the lengthy meeting, would like to be more involved in magnum but don't have the time currently :( 22:01:51 <tobias-urdin> i have nothing 22:01:59 <colin-> thanks for hosting strigazi 22:02:21 <strigazi> canori02: I haven't forgotten you, I'll leave comments in your coreos patch 22:02:23 <tobias-urdin> yea, real thanks for clearing a lot of stuff up! 22:02:31 <strigazi> thanks colin- 22:02:46 <strigazi> Thnaks for joining colin- tobias-urdin canori02 imdigitaljim 22:03:04 <strigazi> See you 22:03:14 <strigazi> #endmeeting