09:01:44 <jakeyip> #startmeeting magnum
09:01:44 <opendevmeet> Meeting started Wed May  1 09:01:44 2024 UTC and is due to finish in 60 minutes.  The chair is jakeyip. Information about MeetBot at http://wiki.debian.org/MeetBot.
09:01:44 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
09:01:44 <opendevmeet> The meeting name has been set to 'magnum'
09:01:51 <jakeyip> #link https://etherpad.opendev.org/p/magnum-weekly-meeting
09:01:57 <jakeyip> #topic Roll Call
09:01:59 <jakeyip> o/
09:02:02 <jakeyip> ping dalees
09:02:04 <dalees> o/
09:02:12 <jakeyip> #topic Review
09:02:23 <jakeyip> Update control-plane nodes taint
09:02:30 <jakeyip> Update control-plane nodes taint - https://review.opendev.org/c/openstack/magnum/+/917407
09:03:05 <jakeyip> dalees: that's ok?
09:03:46 <jakeyip> my motivation is to get the heat driver working with v1.28, passing sonobuoy, updating docs, etc, for D cycle
09:04:10 <dalees> it looks okay to me if we can't have master taint's anymore. A shame we didn't overlap with control-plane taint for a while, but this should only affect freshly built clusters
09:04:32 <jakeyip> yeah we all missed that
09:04:36 <dalees> and if it breaks old k8s <1.20, then that's okay :D
09:05:01 <dalees> on a similar topic - I'm updating wording from `master` to `control plane` in magnum-ui. Any issues with that in general?
09:05:30 <dalees> it'll be a bigger effort to update the internal variables and api; I'm not tackling that yet. But we can do some user facing text
09:05:36 <jakeyip> no we should go with that everywhere, since it's the word upstream uses now
09:05:58 <dalees> cool
09:06:52 * dalees notes the reno in that patchset. Yep operators will need that.
09:07:45 <jakeyip> feilong did minion to node a while back - https://review.opendev.org/c/openstack/magnum/+/608799
09:09:06 <jakeyip> one concern is new cluster for existing template will have a behaviour change too
09:09:30 <jakeyip> I think I'll add that sentence to be more clear to operators
09:10:19 <jakeyip> I think that's prob all we should do
09:10:39 <dalees> yeah, it will. That'll be a change that some might not expect - hard to make it fully backwards compatible though. easier to roll forwards for those folk
09:11:20 <jakeyip> yeap
09:11:51 <jakeyip> any other concern with this review? I will update reno
09:12:14 <dalees> no, LGTM. just my comment about that duplicate `toleration`
09:13:12 <jakeyip> yeap I think I've deleted that just haven't sent it up
09:13:34 <jakeyip> next. Change network driver test to use non-default driver https://review.opendev.org/c/openstack/magnum/+/905632
09:13:41 <jakeyip> still needed? I've rebased
09:14:11 <dalees> i recall it improves test coverage
09:14:14 <dalees> which we need
09:14:30 <dalees> will wait and see zuul coverage results and compare.
09:15:03 <jakeyip> ok
09:15:12 <jakeyip> next:  Update autoscaler clusterrole permissions to support 1.22 https://review.opendev.org/c/openstack/magnum/+/892846
09:15:25 <jakeyip> another one of yours :)
09:16:50 <dalees> not too much to say, it allows using a recent autoscaler for Heat driver clusters
09:16:58 <dalees> do you enable cluster autoscaler?
09:18:18 <jakeyip> no we didn't, do y ou?
09:18:35 <dalees> yep, some of our customers do.
09:18:55 <dalees> we carry that patch locally, otherwise autoscaler doesn't run :)
09:20:05 <dalees> on that topic, there's an interesting problem with CAPI driver and cluster autoscaler. Node counts won't update in Magnum currently - only in CAPI.
09:20:22 <jakeyip> ok I'll rebase, if it passes test I'll merge.
09:20:27 <opendevreview> Dale Smith proposed openstack/magnum master: Update autoscaler clusterrole permissions to support 1.22  https://review.opendev.org/c/openstack/magnum/+/892846
09:20:54 <jakeyip> :D ha you beat me to it
09:21:03 <dalees> :)
09:24:33 <jakeyip> any ideas for the CAPI node count mismatch?
09:24:56 <jakeyip> maybe driver can update
09:25:44 <dalees> yeah - but it requires lots of changes to the magnum provider in cluster-autoscaler (kubernetes project). Right now it reaches into Heat Stacks, because Magnum API cannot yet return a list of node group members.
09:26:42 <dalees> driver could poll and update, but that feels the wrong way around. Might be simpler tohugh. cluster autoscaler should probably just talk to Magnum API to do the job.
09:27:41 <dalees> I raised a bug here https://github.com/stackhpc/capi-helm-charts/issues/317 - so it's tracked *somewhere* :)
09:30:43 <jakeyip> yeah ok let's see how it goes, maybe someone from there will pick it up :D
09:32:56 <jakeyip> I'm not familiar with that code so can't help much
09:34:26 <jakeyip> dalees: on the topic of capi-helm-charts, when do you think we'll be ready for openstack/magnum-capi-helm-charts ?
09:35:44 <dalees> What are the blockers? CI pipelines?
09:37:14 <jakeyip> we will fork so we don't have to bring in all their CI
09:37:27 <jakeyip> how are you handling the chart now for catalyst?
09:39:11 <dalees> we forked it locally, and publish it to our OCI registry for Magnum to use. We have several modifications like ignoring the keypair, and Calico BGP (which I do need to submit upstream)
09:39:50 <dalees> however, we will continue to sync with upstream, and push changes that would be useful to others
09:41:20 <jakeyip> once Magnum forks it to openstack/magnum-capi-helm-charts, your upstream should then be this repo?
09:41:32 <dalees> and the management loadbalancer - which would be really useful to others who want to allow private clusters... but that requires CAPI and CAPO builds.
09:41:56 <dalees> yeah, we would switch to that - as i understand stackhpc would sync with it too.
09:43:26 <jakeyip> yeah matt will take care of openstack <-> stackhpc, we are aware some things might clash cos they use it for Azimuth
09:43:35 <jakeyip> will sort that out when we get to it
09:46:09 <opendevreview> Jake Yip proposed openstack/magnum master: Update control-plane nodes taint  https://review.opendev.org/c/openstack/magnum/+/917407
09:46:38 <dalees> I've got a question about your usage of magnum-ui
09:47:17 <jakeyip> sure
09:47:29 <dalees> have you updated magnum-ui to Antelope(?), and have you tried ricolin's "Get Cluster Config" button?
09:49:04 <dalees> I rebased onto 2024.1 today, and that button made the browser download 3 certificate files and a kubeconfig - but the kubeconfig doesn't reference the cert files. I'm a bit puzzled by this. Did it ever work?
09:49:29 <jakeyip> I'll have check and get back to you, I believe our dashboard is at Bobcat but we tear out a bunch of panes.
09:50:11 <jrosser_> ^ we came across the same thing, not knowing what to do with the downloaded files
09:50:15 <jakeyip> I don't have an existing cluster to check now
09:51:01 <jakeyip> if you have a link to the patch that'll be helpful
09:51:20 <dalees> my other question is - do many others use keystone auth? It's useful for us to provide a button for Kubeconfig with Keystone Auth, and a button for Admin Kubeconfig.
09:52:09 <jakeyip> it doesn't work out of the box for us and I haven't patched it yet to make it work
09:52:13 <dalees> jrosser_: thanks, useful to know I'm not the only one. I think I'll propose a change to embed the certs inline in the kubeconfig. That will make it the same as the CLI `openstack coe cluster config`.
09:52:51 <jakeyip> basically because our role names are different from keystone. `Member' instead of 'member', etc.
09:53:25 <dalees> ah righto. One day we'll catch up with these role names...
09:53:40 <dalees> I think we still have `_member_` ;)
09:54:07 <dalees> but `k8s_admin`, `k8s_viewer`, `k8s_developer` are used mostly in keystoneauth.
09:55:22 <jakeyip> yeah the good ole _member_, we have some clouds with that too :D
09:55:53 <jrosser_> adding an implied role making _member_ and member equivalent is a handy way to migrate out of that
09:56:17 <jakeyip> nice :)
09:56:45 <jakeyip> caveat is implied roles don't work well with app cred, there's an open bug
09:56:48 <jrosser_> there were recent fixes to keystone to make that also work for existing app creds i think
09:56:55 <jrosser_> ahha snap :)
09:57:06 <jakeyip> _member_ -> member is easier than Member -> member.
09:57:51 <jakeyip> keystone says names are not case sensitive (so you can't have two names with different cases), but some places are case sensitive so a wrong case won't work
09:58:00 <jakeyip> :q
09:58:50 <jakeyip> dalees: your keystone-auth issue is with CAPI driver?
10:00:06 <jakeyip> jrosser_: :D heee I remember cos I was just looking at the keystone reviews
10:00:58 <dalees> jakeyip: it applies to magnum-ui, so it's not driver specific. I'll propose two buttons in the UI: "Download KeystoneAuth Kubeconfig" and "Download Admin Kubeconfig". If I can make the KeystoneAuth one only appear for those clusters with it enabled, that'll be ideal.
10:01:22 <jrosser_> https://review.opendev.org/c/openstack/keystone/+/910337
10:02:38 <jakeyip> dalees: sorry I mean, keystoneauth is working for you now? for clusters spun up by CAPI or Heat?
10:03:00 <dalees> jakeyip: yes, we use it for both Heat and CAPI(helm)
10:04:04 <dalees> though there's a snag in v1.29 which travisholton is working on.
10:04:11 <jakeyip> jrosser_: I like this one more https://review.opendev.org/c/openstack/keystone/+/893737
10:04:39 <jrosser_> ah yes that is a patch from my team
10:04:48 <jrosser_> but adding tests is just soooo hard /o\
10:05:08 <jakeyip> oh nice! I'll comment on this :P
10:08:32 <jakeyip> dalees: I think that sounds good. give it a go
10:09:43 <dalees> alright, incoming magnum-ui patchsets soon.
10:13:10 <jakeyip> so if memory serves, the files that you downloaded are actually from the certificates endpoint.
10:14:23 <jakeyip> python-magnumclient grabs them and formats them for kubeconfig
10:14:25 <dalees> the CA is, the key and CSR(not downloaded) are generated, and posted to the certificates endpoint, yeah.
10:14:27 <jakeyip> you may know this already...
10:14:39 <dalees> I was looking at this code today ;)
10:16:22 <dalees> anything else for meeting?
10:16:29 <jakeyip> ok I'll leave you to it then.
10:16:32 <jakeyip> nothing
10:16:54 <jakeyip> let's call it then, we are over time
10:17:11 <jakeyip> #endmeeting