15:02:45 <portdirect> #startmeeting openstack-helm
15:02:45 <openstack> Meeting started Tue Jan 15 15:02:45 2019 UTC and is due to finish in 60 minutes.  The chair is portdirect. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:02:46 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:02:49 <openstack> The meeting name has been set to 'openstack_helm'
15:02:52 <powerds0111> o/
15:03:02 <portdirect> #link Agenda https://etherpad.openstack.org/p/openstack-helm-meeting-2019-01-15
15:03:21 <portdirect> lets give it a few lins for people to arrive who happen to be even later than i :)
15:03:26 <srwilkers> got the room wrong
15:03:37 <mattmceuen> o/
15:03:54 <georgk> o/
15:04:07 <mattmceuen> this channel feels so much more... 4-like than it used to.  totally different vibe here.
15:04:29 <jayahn> fantastic-4....
15:04:46 <portdirect> can i be thing?
15:04:51 <portdirect> anyway - lets begin
15:05:03 <portdirect> #topic Support cinder multi rbd backends
15:05:10 <portdirect> jayahn: think this is yours?
15:05:15 <jayahn> powerds0111
15:05:21 <jayahn> dan will talk about it
15:05:33 <powerds0111> cinder & libvirt charts cannot support multiple rbd backends but some sites need the feature. so i described what issues in charts and made tasks on the story https://storyboard.openstack.org/#!/story/2004777
15:06:10 <powerds0111> first task is implemented on https://review.openstack.org/#/c/630810/ and others are  on going
15:06:36 <portdirect> the timing of this is somewhat great - now we have support for multiple ceph clusters (and gating for them) with the ceph chart.
15:07:00 <yongiman> o/
15:07:40 <powerds0111> you mean it is implemented already?
15:07:41 <portdirect> powerds0111: could you explain the rationalle with `ceph configuration generation chart for supporting multiple cluster` that you link to above?
15:08:33 <portdirect> powerds0111: not the consumption of them by libvirt - but that the ceph chart now supports multiple deployments within a single cluster - so we can simply test the work you are undertaking
15:09:00 <powerds0111> multi rbd backends need multiple ceph.conf so the PS make cepf.conf files in ceph-etc configmap
15:10:48 <portdirect> perhaps - rather than a ceph specific chart for this use case, a generic: configmap chart would make sense?
15:11:11 <portdirect> alternativly you could just turn all manifests other than the configmap off for any other ceph chart
15:11:28 <portdirect> i'm not fully aligned with supamatts line of thinking
15:11:52 <portdirect> but am on the other side concerned that if we end up with many 'pico' charts then we will quickly end up in a mess
15:12:16 <portdirect> the work you did for the 'bring your own service' chart was great here
15:12:31 <portdirect> as its equally applicable for ceph, as it is for mariadb
15:15:22 <powerds0111> could you explain more about the idea to support multiple ceph.conf?
15:15:58 <portdirect> i dont have any ideas there yet :)
15:16:24 <portdirect> the point im making though is that having a chart dedicated to rendering a ceph.conf is a very heavyweight solution for a single task
15:16:42 <jayahn> portdirect: you are suggesting to think a bit more about if there is any possible case to make it more generic feature?
15:17:08 <jayahn> however, dose not have any concrete suggestion yet. ;) am i understanding correctly?
15:17:20 <portdirect> when the same end result could be achived by turning all of the lines here to 'false' https://github.com/openstack/openstack-helm-infra/blob/master/ceph-client/values.yaml#L438-L450 with the expetion of 441
15:17:59 <portdirect> jayahn: pretty much - i think if we go down the 'chart to produce configmap' approach, then it would be nice to see it able to produce *any* configmap that htk supports
15:18:07 <portdirect> rather than one for each
15:18:59 <mattmceuen> ah, agree - because the only thing ceph-specific about the chart is the values themselves, which could be overridden for all kinds of handy non-ceph things
15:20:16 <portdirect> yes - as it stand the chart is a 'helm-toolkit.utils.to_ini' chart with prepopulated values for ceph
15:20:28 <powerds0111> :) i agree i need to check 1) how to fix ceph-client to support multiple cepf configs 2) make new chart to generate general configmap
15:21:21 <portdirect> awesome - thanks powerds0111, really looking forward to seeing this shape up
15:21:42 <mattmceuen> ++ good stuff powerds0111
15:22:03 <portdirect> also - lets sync sooner rather than later on getting gating for this in place
15:22:25 <portdirect> as it should be a simple addition to the tenant ceph job that was added recently
15:22:40 <portdirect> ok to move on?
15:22:45 <powerds0111> okay
15:23:06 <portdirect> #topic: referencing work outside of openstack-helm repo
15:23:24 <portdirect> jayahn: this one is yours i think?
15:23:41 <jayahn> it was a simple question, all of your today's comment on https://review.openstack.org/#/c/630810/  are great!
15:24:12 <jayahn> but on the comment with "-1", i see someone actually put things happening outside of openstack-helm repo (attdev), then gave -1.
15:24:33 <portdirect> i think i can take this one and draw some lines in the sand (hopefully)
15:24:54 <jayahn> okay. thanks,. :)
15:25:38 <portdirect> I strongly disagree with the -1, not least of which is that the work referenced there is not happening in openstack hosted infra (which is the important point here)
15:26:10 <portdirect> I do fortuantly have some insight into the project that supamatt is refering to
15:27:13 <portdirect> ATT has been operationalizing airship based osh clusters - which is great for us as a community as it really exposes the gaps we have
15:27:32 <portdirect> att-comdevs porthole is a great example of this
15:28:22 <portdirect> I think its got tot the stage where it would be a great thing for the community at large to benefit from
15:28:33 <jayahn> okay, porthole. i need to learn this word first.
15:28:52 <portdirect> its the round windoes on the side of a boat
15:28:55 <portdirect> *windows
15:29:22 <portdirect> the idea being they let an operations team look inside a cluster without having to be in the cluster itself
15:29:28 <mattmceuen> hooray for clever names
15:29:39 <portdirect> (att does spend a long time coming up with names for things it seems)
15:30:17 <portdirect> this week I'm going to work with mattmceuen and alanmeadows to try and free porthole from its current home
15:30:25 <portdirect> so it can set sail out into the world
15:30:35 <jayahn> ah, okay, now i am getting what you are talking about.
15:30:37 * portdirect i'll run out of these soon...
15:30:54 <jayahn> took sometime to understand... :)
15:31:26 <portdirect> most likley i think it would have a good home in airship - whihc would let us leverage it a lot easier within osh if there is desire to do so
15:31:47 <portdirect> but . thats something we would need to propose to the airship community
15:31:50 <jayahn> I am just opposing a practice to give -1 with outside of osh work. that is all. but absolutely support brining porthole into the world.
15:32:31 <portdirect> jayahn: i could not agree with your more on the -1 in this instance
15:32:59 <portdirect> sometimes though - it may make sense - eg there is no point re-inventing a wheel that already exists ;)
15:33:17 <jayahn> for the porthole thing, if you need any supporting words, I will do it.
15:33:21 <portdirect> but in this instance - thankfully it helps rip a scab off that was frustrating a lot of people.
15:33:37 <jayahn> yeap. that is true. it was just -1.. not message contents itself.
15:33:42 <portdirect> jayahn: that would be really appreciated - I'll add it to the meeting for airship next week.
15:33:55 <jayahn> sure, I will be in airship meeting next week then
15:34:45 <portdirect> #topic New charts: pod security policy
15:35:17 <portdirect> mattmceuen added a chart for pod security policy recently - and it was one of the topics in this weeks airship meeting
15:35:40 <portdirect> mattmceuen: you oke go give a breif over-view of your work here for those that were not there?
15:36:02 <mattmceuen> Sure!
15:36:42 <mattmceuen> in a nutshell - the podsecuritypolicy chart adds k8s RBAC-based policy that restricts what particular roles are allowed to do with pod specifications
15:37:04 <mattmceuen> e.g. restricting whether pods are allowed to have elevated permissions, host-level access, etc
15:37:36 <jayahn> that would be really great one to have
15:37:38 <mattmceuen> I will be lazy and cut and paste :)
15:37:42 <mattmceuen> So you may or may not be familiar with k8s PodSecurityPolicies
15:37:42 <mattmceuen> 8:04 AM If you configure your k8s to use them, then the k8s api server will only allow you an actor to schedule pods that meet certain criteria/policy
15:37:42 <mattmceuen> 8:04 AM Based on k8s RBAC
15:37:42 <mattmceuen> 8:05 AM This is a security feature we wanted to add into airship, but in a way that doesn't break all the privileged actions that are taken across airship and openstack-helm
15:37:42 <mattmceuen> 8:05 AM This helm chart was added to openstack-helm-infra: https://github.com/openstack/openstack-helm-infra/tree/master/podsecuritypolicy
15:37:42 <mattmceuen> 8:06 AM It specifies by default an incredibly (100%) permissive podsecuritypolity, and sets it up as a default for the cluster
15:37:42 <mattmceuen> 8:06 AM It was also added into the Treasuremap reference yamls: https://review.openstack.org/#/c/629686/
15:37:43 <mattmceuen> 8:07 AM You can do a couple things through the chart:
15:38:04 <mattmceuen> You can use the chart to:
15:38:04 <mattmceuen> 8:08 AM 1) add whatever additional podsecuritypolicies you want for your cluster, programmatically, letting helm manage lifecycle
15:38:04 <mattmceuen> 8:08 AM 2) change the default(s)
15:38:04 <mattmceuen> 8:09 AM Over time, we want to tune the defaults in the chart to be a reasonable non-fully-open set of policy, as much as possible.  However, the intent is also that operators fully customize it for their workloads,
15:38:13 <mattmceuen> You can set up defaults in the chart individually for serviceacounts, authenticated users, and unauthenticated users via the chart, and/or associate the PSPs to other roles outside of the chart itself
15:38:28 <mattmceuen> Give folks a minute to read that - let me know if any questions!
15:40:06 <srwilkers> i think this could add value down the road once things are fleshed out.  my only request would be to have this run in some sort of job just to validate that `helm install foo ./podsecuritypolicy` results in a successful installation
15:40:21 <portdirect> personally i love it, though i am biased
15:40:24 <srwilkers> then the post run jobs can help sanity check this as we move to provide a reference policy that makes sense
15:40:41 <portdirect> srwilkers: good point
15:40:52 <portdirect> mattmceuen: what does the default psp permit/deny?
15:41:06 <mattmceuen> srwilkers - will do!
15:41:58 <mattmceuen> portdirect:  the default psp in the values allows everything!  basically the most permissive settings for each factor.  By default the psp is bound to all serviceaccounts and all authenticated users (but nothing is bound by default to unauthenticated users)
15:42:18 <mattmceuen> although we can and should definitely tune those defaults down over time
15:43:10 <portdirect> mattmceuen: nice, in that case a simple gate test could be as easy as deploying a simple pod in host nework, deploying the chart with this flipped https://github.com/openstack/openstack-helm-infra/blob/master/podsecuritypolicy/values.yaml#L38, and trying to deploy another
15:43:22 <portdirect> if the 2nd fails on psp - then take it as a win?
15:43:51 <mattmceuen> I like it
15:43:53 <mattmceuen> will do
15:44:13 <portdirect> woot - would also be great over time to add docs around this
15:44:26 <portdirect> as it highlights some areas were we could maybe improve our posture
15:44:48 <mattmceuen> agree, will add to the todo list :)
15:44:54 <portdirect> can you apply psps to only select workloads in a namespace, or is it a blanket?
15:46:09 <portdirect> if the latter - should longer term we start to consider splitting the control plane away from the data plane?
15:46:58 <portdirect> as apis'/engines' we could really lock down, though agents (eg nova compute) will require much more permissive rules
15:47:36 <mattmceuen> it's all done through roles / rolebindings
15:47:50 <mattmceuen> so we can RBAC it however we want
15:48:15 <mattmceuen> the PSPs themselves are defined at the cluster level only
15:48:47 <mattmceuen> oh I think I misunderstood you
15:48:54 <portdirect> oh i'll need to read up more then - as we dynamicly generate service accounts/roles/bindings for each pod we may need some accounting mechanism
15:49:07 <mattmceuen> yes, let me chew on this as well
15:49:52 <portdirect> nice - thanks for the intro mattmceuen
15:50:12 <portdirect> we also have another addition to the fold this week:
15:50:31 <portdirect> #topic New charts/images: mini-mirror
15:51:06 <portdirect> dwalt: can you give a similar breif into for this work?
15:51:13 <dwalt> o/
15:51:17 <dwalt> Gladly!
15:51:59 <dwalt> One of the challenges Airship was facing in larger scale deployments was the coordination of packages on the host -- i.e. package versions changed between the time that we ran test deployment pipelines t to the time of the actual deployment.
15:52:36 <dwalt> Mini-mirror exists as a way to combat that. By mirroring Debian/Ubuntu package repositories and deploying them into the cluster, we can control what packages exist on the host for a deployment.
15:53:04 <dwalt> Currently, you can utilize your own mini-mirror by building an image and specifying the sources/packages you would like to mirror.
15:53:10 <dwalt> #link https://github.com/openstack/openstack-helm-images/blob/master/mini-mirror/README.rst
15:53:19 <dwalt> #link https://github.com/openstack/openstack-helm-addons/tree/master/mini-mirror
15:53:26 <portdirect> not just an airship problem, but a 'hey i need to manage an airgapped set of hosts' problem - so really nice to see this work in osh-addons/images
15:53:42 <dwalt> portdirect: definitely
15:53:50 <dwalt> Also, as discussed in the #airshipit meeting earlier, this can easily be expanded to include other types of repos (e.g. rpm)
15:54:12 <jayahn> rpm! :)
15:54:24 <powerds0111> sounds good :) do you have a plan to support yum and pip?
15:54:31 <jayahn> that would be a good addition, i agree!
15:54:41 <portdirect> ++
15:55:05 <portdirect> that should be simple to add dwalt ?
15:55:15 <portdirect> would just be a case of a new image?
15:56:41 <dwalt> portdirect: indeed! There isn't a fleshed out design for it yet, but it would be as simple as copying the existing Dockerfile and modifying it to use a tool to mirror the specific type of repository. OSH images already supports multiple distributions
15:57:07 <portdirect> nice - jayahn / powerds0111 if you have the bandwidth it would be awesome to see this reliased
15:57:40 <portdirect> I suspect evrardjp would also have interest as well
15:57:58 <portdirect> s/reliased/implemented
15:58:55 <portdirect> would also be great to see some basic gating around this if possible
15:58:56 <jayahn> yeah, will surely discuss internally, we really need this. thanks dwalt
15:59:16 <jayahn> dwalt: thanks for a good work :)
16:00:31 <portdirect> ok - out of time :(
16:01:16 <portdirect> lets move over to the osh channel - though next week id really like to discuss the work that has been going on in docs - lets get it 1st thing on the schedule
16:01:29 <jayahn> okay
16:01:49 <portdirect> and thanks for all the work everyones been putting in over the last few weeks - really nice to see things ramping back up again :D
16:01:55 <portdirect> #endmeeting