16:01:33 <lamt> #startmeeting openstack-helm
16:01:33 <openstack> Meeting started Tue Feb 11 16:01:33 2020 UTC and is due to finish in 60 minutes.  The chair is lamt. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:34 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:36 <openstack> The meeting name has been set to 'openstack_helm'
16:01:46 <lamt> #topic roll call
16:01:51 <srwilkers> o/
16:01:58 <lamt> \o
16:02:20 <lamt> how's things going srwilkers?
16:03:05 <gagehugo> o/
16:03:44 <srwilkers> not too bad lamt - howre things out that way?
16:03:58 <lamt> giving people a few more minutes to join. portdirect is busy again, so I am hosting.
16:04:01 <stevthedev> Good morning everyone
16:04:11 <lamt> srwilkers: same old same old
16:05:49 <lamt> let's get going then
16:06:08 <lamt> #topic Endpoint Auth Secrets
16:06:24 <lamt> looks like there is a lot of typing in the etherpad regarding this:
16:06:35 <lamt> #link https://review.opendev.org/#/c/706181/
16:06:46 <srwilkers> nothing like a good etherpad flame war
16:06:48 <srwilkers> ./s
16:06:56 <lamt> so may well make that a topic for discussion
16:07:06 <lamt> stevthedev: you have the floor
16:07:28 <stevthedev> hahaha
16:08:13 <stevthedev> So with development of Loki upstream, and other things downstream, it'd be cool if fluentd could be more dynamically configured
16:08:58 <stevthedev> Currently and ES endpoint is hard coded, and we have a toggle for a Kafka endpoint, but I'm not sure if this pattern is extensible
16:09:43 <stevthedev> I think if we had some functions to read what's defined in .Values.endpoints, the fluentd chart would become more flexible
16:10:39 <stevthedev> So I started working on a pair of HTK functions, to parse all of the auth: credentials under .Values.endpoints
16:12:19 <songgongjun> Hi, everyone, can i ask a question about overrides?
16:12:37 <srwilkers> My only concern with that approach is that it ends up creating a secret that has credentials for every default endpoint defined in the fluentd chart, which means we've got to burden operators with overriding any credentials defined in the outputs that aren't used
16:13:04 <lamt> songgongjun: sure - lemme put that on https://etherpad.openstack.org/p/openstack-helm-meeting-2020-02-11 - and will discuss after the current topic - is that okay?
16:13:05 <srwilkers> With regards to dynamic configuration, we already support overriding the entirety of the configuration file, including support for environment variables
16:13:57 <srwilkers> So ultimately, this is just duplicating a feature that already exists in the chart, while condensing the two secrets for elasticsearch and kafka we have today into one secret
16:14:49 <srwilkers> With regards to the current functionality that fluentd uses for dynamic secret creation - that's something that should be extended to every chart, as the other charts should also support dynamic environment variables that are defined in the clear
16:14:51 <songgongjun> Ok,  thanks!
16:16:51 <stevthedev> I know there are concerns about removing functionality, but is there another reason why the elasticsearch endpoint must remain in values? Why not define everything, endpoints and conf, by overrides?
16:17:36 <stevthedev> Let the operator decide how the application will work, where it will send logs, which logs it collects, etc. Maybe I am thinking too generally here, as this is OS helm after all
16:19:14 <srwilkers> That's a pretty broad sweeping change
16:20:09 <srwilkers> The reason Elasticsearch is a default endpoint for fluentd is that the EFK stack is pretty established as the CNCF standard for logging, and our opinionated stance has been that we'll provide a mechanism for logging as part of the project that include those two together
16:21:56 <srwilkers> Also, if this is a path that's decided on, I don't think this belongs in helm-toolkit as it seems pretty tailored to the fluentd chart.  All our other charts have been standardized to use static secrets for auth credentials
16:22:12 <srwilkers> It'd probably need to be a fluentd specific helper template
16:22:22 <srwilkers> But that's, like, my opinion man
16:22:34 <srwilkers> I'll let others weigh in
16:23:04 <stevthedev> Yeah, I'd like to hear from others too. That's not a bad idea though. I specifically had fluentd in mind while working on this
16:23:30 <lamt> I agree such change would be a large sweeping change across all charts
16:24:52 <lamt> I am not against it though, but for now I think it may be more appropriate for it to be a fluentd specific helper template
16:26:05 <gagehugo> agreed on it being more fluentd specific, rather than helm-toolkit
16:26:52 <stevthedev> Thanks for the feedback, I'll move it in that direction then
16:27:30 <lamt> cool, thanks for everyone's feedback - anything else on this topic?
16:27:53 <srwilkers> yeah, i hate fluentd
16:27:54 <srwilkers> that's all
16:27:59 <lamt> lol
16:28:10 <lamt> if not, moving on
16:28:18 <lamt> #topic Overrides
16:28:25 <lamt> songgongjun: the floor is yours
16:29:32 <songgongjun> I am doing the work of ovs per-host overrides support  (https://storyboard.openstack.org/#!/story/2006965), and need the functionality of overrides (https://github.com/openstack/openstack-helm-infra/blob/master/helm-toolkit/templates/utils/_daemonset_overrides.tpl) to update daemonset parameters.
16:29:47 <songgongjun> However, before using overrides in the daemonset file, the $daemonset_yaml variable(Take neutron as an example, https://github.com/openstack/openstack-helm/blob/master/neutron/templates/daemonset-ovs-agent.yaml----line 294 ) has been generated, and new parameters generated by overrides can’t be passed into the daemonset file to generate the
16:29:47 <songgongjun> specified daemonset.yaml.
16:30:27 <songgongjun> Why not put the $daemonset_yaml parameter in the overrides file and what is the reason for this design.
16:34:48 <lamt> looking at the history of those lines, it looks like it was place in 2 years ago, so I can't quite recall the reason for the design.
16:35:02 <lamt> perhaps srwilkers and others can chime in
16:35:35 <srwilkers> honestly, i can't weigh in here - im still convinced the daemonset overrides foo is black magic
16:35:41 <lamt> but then again, it is probably not perfect 2 years ago.
16:36:21 <songgongjun> If you choose to place the $daemonset_yaml parameter in the overrides file, you need to modify the parameters passed in by overrides, but this will affect the files that previously used overrides and need to be modified accordingly. For example, you need to pass in the $ serviceAccountName parameter.
16:38:19 <lamt> I agree. I think we can improve the daemonset overrides. songgongjun do you mind submitting a patch set for it?
16:40:21 <songgongjun> Ok, i'm planning to do this.
16:41:48 <lamt> Thanks. A lot of the stuff was created awhile ago, and should probably be revisited (but not due to capacity).
16:43:47 <songgongjun> Thank you very much for your help!
16:44:44 <lamt> Np - we can review this once a patch set is up - and thank you for your help.
16:45:04 <lamt> #topic Open floor
16:45:41 <lamt> Opening the floor for questions/discussions
16:47:49 <lamt> If there's nothing, we can wrap up and give everyone back 13 minutes. Have a good rest of the day.
16:47:57 <lamt> #endmeeting