16:00:02 <lbragstad> #startmeeting keystone
16:00:03 <openstack> Meeting started Tue Jun 26 16:00:02 2018 UTC and is due to finish in 60 minutes.  The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:04 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:05 <lbragstad> #link https://etherpad.openstack.org/p/keystone-weekly-meeting
16:00:06 <openstack> The meeting name has been set to 'keystone'
16:00:07 <lbragstad> agenda ^
16:00:11 <knikolla> o/
16:00:17 <lbragstad> ping ayoung, breton, cmurphy, dstanek, gagehugo, hrybacki, knikolla, lamt, lbragstad, lwanderley, kmalloc, rodrigods, samueldmq, spilla, aselius, dpar, jdennis, ruan_he, wxy, sonuk
16:00:36 <wxy|> o/
16:00:37 <lamt> o/
16:01:03 <kmalloc> lbragstad: you should ping in -keystone too :P
16:01:10 <hrybacki> o/
16:01:31 <hrybacki> anyone else using irccloud sometimes realize their web client hasn't been connected for at least a couple of hours?
16:01:38 <kmalloc> hrybacki: all the time
16:01:47 * hrybacki shakes fist at paid for service
16:01:51 <kmalloc> irccloud is not that great
16:01:58 <knikolla> i switched back to firrre last week, free znc for freenode
16:01:58 <hrybacki> I wish there was /anything/ better
16:02:08 <kmalloc> weechat is awesome
16:02:19 <hrybacki> yeah, I've got weechat on my new dev machine
16:02:20 <kmalloc> but i don't want to pay for a VM in AWS or <insert cloud here>
16:02:26 * hrybacki nods
16:02:43 <kmalloc> and my home lab is not... setup fully atm
16:02:48 <lbragstad> ok - we have a lot on the agenda today
16:03:05 <lbragstad> real quick announcement though
16:03:10 <lbragstad> #topic announcements
16:03:13 <gagehugo> o/
16:03:21 <lbragstad> if you haven't noticed, our docs job is failing
16:03:42 <lbragstad> this is due to breaking warning that we treat as errors in Sphinx 1.7.5
16:03:52 <lbragstad> we have a patch in the gate to fix the issue
16:03:57 <kmalloc> the fix is +2/+A and pending check then will gate
16:04:00 <lbragstad> but it's really just a work around
16:04:04 <lbragstad> #link https://review.openstack.org/#/c/578121/
16:04:15 <lbragstad> the commit message gives the details if you're interested in them
16:04:41 <lbragstad> TL;DR we can remove those docstrings once oauthlib is compatible with Sphinx 1.7.5
16:05:01 <lbragstad> so - if you have something tripping over docs, ^ that should fix it
16:05:14 <lbragstad> #topic Default roles next steps
16:05:16 <lbragstad> hrybacki: o/
16:05:34 <hrybacki> o/
16:05:48 <hrybacki> yes, so we are now ready to start doing audits of our APIs
16:06:04 <hrybacki> I've gone ahead and done an initial mapping (kmalloc)
16:06:12 <hrybacki> #link https://docs.google.com/spreadsheets/d/14c2pAPeX14RR2EZU89fuSx5yW3Q9pjdyaKkD0cyFzaU/edit?usp=sharing
16:06:20 <hrybacki> that should be public, but please let me know if it's not
16:06:31 <knikolla> it's not
16:06:34 <hrybacki> We've also begun doing a similar audit in barbican
16:06:39 <hrybacki> knikolla: sec, will fix
16:06:45 <hrybacki> https://wiki.openstack.org/wiki/Barbican/Policy
16:07:27 <hrybacki> okay, permissions are whack because I pulled from my Red Hat google drive
16:07:38 <hrybacki> I'll fix and submit a new link
16:07:42 <kmalloc> welcome to google docs unfun =/
16:07:51 <hrybacki> aye
16:07:55 <kmalloc> i've been battling that for weeks, keeps defaulting to the wrong account
16:08:07 <hrybacki> anyways, there is a complete mapping of each route->policy
16:08:18 <hrybacki> along with other details. Pretty verbose.
16:08:45 <hrybacki> Next step is to sit down and walk through these laying down where we /want/ these to fall in line WRT the new default roles and scopes
16:09:07 <kmalloc> I highly recommend performing the audit after a given API path is moved to flask, it will make it much easier to see everything under (e.g. /users)
16:09:15 <kmalloc> rather than having to find what routers append where
16:09:21 <hrybacki> lbragstad: there are also quite a few notes for work that needs to be done in order to enable scope
16:09:35 <hrybacki> kmalloc: I'm concerned about the feature freeze in < weeks (correct?)
16:10:27 <lbragstad> July 27th
16:10:32 <lbragstad> #link https://releases.openstack.org/rocky/schedule.html
16:10:39 <kmalloc> it is a concern. but the bulk of the API moves should be proposed by then.
16:10:43 <lbragstad> we have 4 weeks
16:10:55 <lbragstad> i'm wondering if we can do the audit in review?
16:10:56 <kmalloc> audit should be fine post FF.
16:11:04 <hrybacki> here
16:11:06 <hrybacki> #link https://docs.google.com/spreadsheets/d/1kd3OJCLMsIkPgULN31WFw9PA9-3-X99yaDnjWDGOvm0/edit?usp=sharing
16:11:09 <hrybacki> that should be viewable
16:11:11 <lbragstad> versus auditing in google docs then porting everything to gerrit
16:11:42 <lbragstad> also - however we do things, we should be sure to update the specification with the table once the audit is done
16:11:45 <hrybacki> lbragstad: well, the doc has full context where a single file within gerrit may not
16:12:04 <hrybacki> I'm trying to lay down a pattern for helping the other services
16:12:06 <kmalloc> unless we're building a full context file in gerrit.
16:12:07 <hrybacki> barbican was a nightmare
16:12:27 <lbragstad> hmm
16:13:06 <kmalloc> hrybacki: my only concern with the audit pre-flask is just that the structure (and controllers) are all being re-worked and routes are handled very different
16:13:20 <kmalloc> i just don't want to have to re-audit if an api is moved.
16:13:38 <kmalloc> just because the code shuffled a ton.
16:13:53 <hrybacki> I think it may be good practice tbh (jsut from a community goal perspective)
16:13:54 <kmalloc> otherwise ++ we should do it.
16:13:59 <hrybacki> but don't want to burn others out
16:14:47 <hrybacki> okay, so (not wanting to eat the whole meeting). We are opting to wait until Flask stuff is implemented
16:14:58 <hrybacki> who will want to do the audit at that point with me? kmalloc ?
16:15:08 <kmalloc> hrybacki: i can help some.
16:15:11 <kmalloc> i can't do it all ;)
16:15:36 <hrybacki> I'll do the implementation -- I just need core help doing the initial audit :)
16:15:42 <lbragstad> i can help with the audit stuff
16:15:47 <hrybacki> ack
16:16:10 <lbragstad> let's start with a couple easy APIs if we can
16:16:20 <hrybacki> so action: harry to set up audit appt kmalloc and lbragstad after Flask stuff has merged?
16:16:30 <hrybacki> I don't care where we start
16:16:40 <hrybacki> is it realistic for this stuff to begin landing in rocky?
16:16:45 <kmalloc> likely
16:16:50 <lbragstad> depends on the reviews of the flask work
16:16:55 * hrybacki nods
16:16:57 <kmalloc> the flask stuff will be focused on easy APIs first e.g. limits
16:17:09 <hrybacki> ack
16:17:10 <kmalloc> stuff that is isolated, where things like /users is likely to be towards the end.
16:17:24 <kmalloc> just because ... a lot of things touch /users
16:17:45 <hrybacki> ack. I can work 'behind' your flask progress with the audit
16:18:01 <hrybacki> that's all on that from me lbragstad
16:18:08 <kmalloc> expect APIs to start moving as soon as the RBAC enforcer stuff has eyes
16:18:27 <lbragstad> sounds good
16:18:37 <lbragstad> #topic keystone federation testing
16:18:39 <lbragstad> knikolla: o/
16:18:42 <knikolla> o/
16:19:03 <knikolla> so first, there's a lot of interest lately from the edge computing group in keystone and keystone federation
16:19:16 <knikolla> me and lbragstad attended the edge computing meeting today and answered a few questions
16:19:24 <kmalloc> lbragstad: lets punt the oslo_policy bit to another meeting. [so skip it today, netx week or the following[]
16:19:34 <knikolla> tomorrow there's a meeting with the OPNFV group, who is interested in helping out with keystone federation testing
16:19:42 <kmalloc> yay more testing!
16:19:43 <knikolla> there's an etherpad to track the effort https://etherpad.openstack.org/p/ECG_Keystone_Testing
16:19:58 <knikolla> with that announcement out of the way, i had a technical question
16:20:27 <knikolla> does it make sense to test with Shibboleth as a SAML ECP idp, if keystone in keystone to keystone can act as a SAML ECP Idp?
16:20:58 <lbragstad> does Shib buy us anything k2k doesn't?
16:21:21 <knikolla> lbragstad: the fact that it's shibboleth
16:21:49 <knikolla> so it sort of does integration testing, but they're using the same protocol
16:22:06 <lbragstad> with k2k we have the ability to exercise the idp paths more exhaustively, yeah?
16:22:29 <knikolla> yeah, keystone currently acts as an SP only
16:22:43 <knikolla> the original plan was to deploy a local shibboleth
16:22:48 <knikolla> shibboleth-idp*
16:22:51 <knikolla> and also use k2k
16:23:09 <lbragstad> oh - got it
16:23:13 <knikolla> i'm questioning whether a local shibboleth-idp makes sense at all
16:23:29 <lbragstad> seems like it would be a good sanity check
16:23:40 <knikolla> and whether we should just do k2k, since keystone as an idp uses the same protocol that shibboleth as an idp uses
16:23:41 <lbragstad> but i'm also not familiar with the maintenance cost of doing each
16:24:57 <knikolla> switching over from testshib.org (which we're currently using) to keystone-as-an-idp should be easier than setting up an idp in the gate.
16:25:16 <lbragstad> ok
16:25:31 <knikolla> do people have strong opinions? kmalloc?
16:25:47 <kmalloc> i am a fan of dropping testshib
16:25:53 <kmalloc> as a dep for our testing
16:26:03 <lbragstad> IMO; anything is better than what we have today, but aiming for a lower bar is fine, too
16:26:07 <kmalloc> i would still like to see a shib setup to test federation.
16:26:29 <kmalloc> but that said, any testing is good as long as it is representative of real use
16:27:03 <lbragstad> with how much k2k has been talked about recently, specifically for edge related things, test coverage would be useful there
16:27:30 <knikolla> ok, so i will update the testing plan to have k2k instead of testshib, for now. And then later have shib as a local idp.
16:27:44 <kmalloc> ++
16:27:45 <lbragstad> makes sense
16:27:58 <lbragstad> knikolla: anything else on that front?
16:28:04 <knikolla> awesome, that's all i had.
16:28:10 <lbragstad> kmalloc: did you say you wanted to do the next topic?
16:28:16 <lbragstad> oslo-policy check types?
16:28:19 <kmalloc> skip that one
16:28:24 <kmalloc> until the end or a future meeting
16:28:24 <lbragstad> ok
16:28:42 <lbragstad> #topic pluggable policy
16:28:47 <lbragstad> hrybacki: o/
16:28:49 <hrybacki> o/
16:28:50 <lbragstad> you're up again
16:28:57 <hrybacki> i wish ozz was here to speak to this
16:29:27 <hrybacki> okay, so there are operators rolling their own centralized policy solutions
16:29:42 <hrybacki> I am hearing asks from some of our customers as well.
16:29:58 <kmalloc> this is related to the pluggable enforcer bit in oslo_policy?
16:30:04 <gagehugo> centralized rbac is a topic that keeps coming up at summits/ptgs
16:30:08 <kmalloc> being built?
16:30:18 <hrybacki> to that end, after returning from KubeCon, ozz was pretty amped about Open Policy Agent
16:30:20 <hrybacki> kmalloc: yes
16:30:38 <lbragstad> #link http://jaormx.github.io/2018/rewriting-openstack-policy-files-in-open-policy-agent-rego-language/
16:30:46 <hrybacki> he wrote about his idea of using OPA for policy checks within openstack
16:30:47 <hrybacki> ^^
16:30:49 <hrybacki> thanks lbragstad
16:30:59 <hrybacki> a first step is making Enforce plugable
16:31:04 <kmalloc> so, i 100% support pluggable enforce
16:31:19 <hrybacki> #link https://review.openstack.org/#/c/577807/
16:31:22 <hrybacki> and my mouse just died
16:31:31 <kmalloc> and i 100% support centralizing policy in a smart way (by default, say... etcd)
16:31:49 <kmalloc> but not dumping the policy DSL for the "out-of-the-box" experience [yet]
16:31:52 <hrybacki> that is ozz's WIP
16:32:18 <kmalloc> i think that code is headed the right way.
16:32:26 * hrybacki nods
16:33:01 <lbragstad> so - this isn't using http_check
16:33:04 <hrybacki> One question ozz raised was, should operators be able to override scope in the same way they can rules for policy? e.g. in policy files
16:33:25 <kmalloc> hrybacki: scope, iirc is not overridable
16:33:28 <kmalloc> and shouldn't be
16:33:39 <lbragstad> i'm inclined to say no, but that could be a knee jerk reaction
16:33:54 <kmalloc> scope is intended to be communication from the service that this works with project vs system scope
16:34:04 <kmalloc> so, start with no, not overridable
16:34:09 <kmalloc> we can expand if needed
16:34:13 <lbragstad> it's also intended to be set by the people writing the API
16:34:23 <hrybacki> both sound reasons
16:34:26 <kmalloc> easier to add funcitonality than take it away
16:34:46 <lbragstad> so if i write an api that does things on a system level, i probably don't want to expose things that allow people to bypass that
16:34:53 <kmalloc> ++
16:35:11 <kmalloc> we can always re-evaluate and allow overrides
16:35:23 <lbragstad> hrybacki: what was ozz's use case for overriding them?
16:35:48 <hrybacki> lbragstad: so OPA has its own lingo for policy (and other stuff) 'rego'
16:36:05 <hrybacki> he wrote a parser that ports default policy files to rego for use with OPA
16:36:12 <hrybacki> using the defaults we have built out so far
16:36:19 <hrybacki> but scope type isn't indicated in them
16:36:30 <hrybacki> so he's trying to get around parsing the policy-in-code files
16:36:37 <lbragstad> hmmm
16:36:41 <hrybacki> we are thinking about customers that already custom policy files
16:36:51 <hrybacki> s/operators/customers/
16:36:57 <hrybacki> damnit, reverse that
16:37:20 <lbragstad> can he load the custom policy files, fill in the gaps with the missing policys, covert them to RuleDefault instances, and then inspect the .scope_types attributes?
16:37:35 <kmalloc> and we expose the list_rules stuff, right?
16:37:39 <lbragstad> yeah
16:37:49 <kmalloc> so it is possible to just python-introspect the objects
16:37:50 <hrybacki> we can do that, I just wanted to verify the simple solution wasn't the correct answer
16:37:52 <lbragstad> that's how we munge custom policy files today
16:38:18 <lbragstad> usualyl something like
16:38:22 <hrybacki> ack. thanks for the input :)
16:38:50 <lbragstad> 1.) give me the default set of rules 2.) override the ones i need to for my deployment
16:38:50 <hrybacki> but yes kmalloc I like the Enforcer patch he's posted. I'll look at your work (that I saw in the agenda) too
16:38:57 <lbragstad> but keep everything as a set of RuleDefault objects
16:39:35 <lbragstad> everything in that set should maintain the .scope_types attributes we set from step #1, but include the overrides the customer needs from step #2
16:40:01 <hrybacki> okay
16:40:16 <lbragstad> i feel like i just waved my hands everywhere
16:40:23 <lbragstad> but hopefully that helps
16:40:27 <hrybacki> lbragstad: I have more than enough info for now :)
16:41:01 <lbragstad> and if it doesn't we should find a way to expose that information so that scope_types is consumable but not overrideable
16:41:09 <kmalloc> ++
16:41:32 <lbragstad> open/closed principle ya know?!
16:41:33 <kmalloc> call out for policy check, but still check scope internally
16:41:35 <kmalloc> easy
16:41:51 <kmalloc> outside of the pluggable backend
16:42:03 <hrybacki> interesting approach
16:43:00 <lbragstad> if ozz is going to be around this week, maybe we can sit down and see what he thinks
16:43:06 * hrybacki is being quiet so we cover more :)
16:43:23 <hrybacki> lbragstad: he's here through Thursday, just a +7hr timeline so couldn't make this meeting
16:43:29 <lbragstad> ah
16:43:42 <lbragstad> maybe early in the day for us then and we can catch him
16:43:47 <hrybacki> he lives in Finland for context
16:43:56 <lbragstad> hrybacki: anything else on this?
16:43:57 <hrybacki> sure, that would be good. I'll send him an email
16:44:01 <hrybacki> nope!
16:44:07 <lbragstad> cool - thanks!
16:44:22 <lbragstad> #topic shiny new RBAC enforcement with Flask
16:44:23 <lbragstad> kmalloc:
16:44:26 <lbragstad> kmalloc: o/
16:44:38 <kmalloc> so. it turns out i can't move an API to flask without providing RBAC enforcement
16:45:00 <kmalloc> @protected sucks... and has next to no documentation and testing is limited to "do we enforce what we think we enforce"
16:45:20 <kmalloc> #link https://review.openstack.org/#/q/topic:bug/1776504+(status:open+OR+status:merged)
16:45:39 <kmalloc> that is the stack of changes. at the current end there is a NEW RBACEnforcer object
16:46:15 <kmalloc> it only works with flask, but it eliminates @protected as a decorator with the idea enforcement should happen within the method at the right time [e.g. after loading the object to be enforced on]
16:46:34 <kmalloc> so, enforcer.enforce_call(<args>) is used instead of just letting the decorator do magic
16:46:44 <lbragstad> this should make our API enforcement logic more like business logic instead of stuffing it all into a decorator, right?
16:46:48 <kmalloc> yes
16:47:05 <lbragstad> awesome
16:47:37 <kmalloc> in the simplest form, you can decorate the method (e.g. .get()) with @enforcer.policy_enforcer_action(<action, e.g. identity:get_user>)
16:47:45 <kmalloc> and then call enforcer.enforce_call()
16:48:01 <kmalloc> with no args, assuming everything is setup
16:48:42 <kmalloc> you can also call .enforce_call() with a number of variations to get the same functionality @protected, @filterprotected, and the callback versions without needing to duplicate the check_protection code into a callback
16:48:52 <lbragstad> cool
16:48:59 <lbragstad> i'll review that chain today
16:49:04 <kmalloc> we also ahve full unit tests that build Creds, Target, data directly
16:49:08 <lbragstad> that's going to have a big impact on the default roles work for sure
16:49:21 <kmalloc> instead of assuming it is right if the .enforce() call is working
16:49:36 <kmalloc> last of all, i am sorry it's a 1000+ line change
16:49:41 <kmalloc> but 50% of the change is tests.
16:50:11 <kmalloc> in flask, we also have an explicit check to see if .enforce_call was made
16:50:20 <lbragstad> it's also going to help us fix stuff like this
16:50:23 <lbragstad> #link https://bugs.launchpad.net/keystone/+bug/1750660
16:50:23 <openstack> Launchpad bug 1750660 in OpenStack Identity (keystone) "The v3 project API should account for different scopes" [High,Triaged]
16:50:37 <kmalloc> if it wasn't called, keystone will error unless the method is explicitly exempted
16:50:48 <kmalloc> and there is a decorator to exempt the methods
16:50:57 <kmalloc> that will be easier to show in a followup.
16:51:08 <kmalloc> as i move APIs.
16:51:37 <kmalloc> all the assertions that .enforce_call is made is done via "assert" it is not meant to block use in production if python -O is used
16:51:44 <kmalloc> it is meant to be caught in dev.
16:52:01 <kmalloc> so no one implements an unenforced (unintentionally) API
16:52:13 <lbragstad> good deal
16:52:18 <kmalloc> last note.
16:52:39 <kmalloc> there is a weird bug in how we extract subject-token data now
16:52:40 <kmalloc> https://review.openstack.org/#/c/577655/
16:53:09 <kmalloc> if you are doing .get_user, the subject-token data will be in target_dict['target']['user']<token data>
16:53:19 <kmalloc> if project it will be in ['project'] not user
16:53:26 <kmalloc> because it uses the "controller" to populate
16:53:46 <kmalloc> so we have weird data that isn't target.token it is target.<whatever member name controller supplies>
16:53:56 <kmalloc> that patch fixes it for @protected
16:54:05 <kmalloc> i have rolled the same fix into the RBACEnforcer
16:54:14 <kmalloc> since they use different code paths
16:54:16 <kmalloc> ...
16:54:22 <lbragstad> hmm
16:54:28 <lbragstad> good to know
16:54:32 <lbragstad> anything else on that kmalloc?
16:54:36 <lbragstad> just need eyes?
16:54:50 <kmalloc> The RBACEnforcer also still just leans on oslo_policy, so all pluggable enforcers will work
16:54:53 <kmalloc> it needs eyes.
16:54:57 <lbragstad> awesome
16:55:06 <kmalloc> this chain has taken weeks to unroll what @protected does
16:55:16 <kmalloc> hopefully enforcer.enforce_call is easier to understand
16:55:23 <lbragstad> yeah - i hope so
16:55:26 <lbragstad> sounds like it will be
16:55:41 <kmalloc> this also implementes the first example of the new REST testing
16:55:49 <kmalloc> which is *way* cooler than the use of WEBTEST
16:55:50 <kmalloc> :)
16:56:07 <kmalloc> with self.test_client() as c:
16:56:09 <kmalloc> ^_^
16:56:26 <lbragstad> only a couple topics left
16:56:31 * kmalloc is done.
16:56:41 <lbragstad> thanks kmalloc!
16:56:45 <lbragstad> #topic unified limits
16:56:49 <lbragstad> wxy|: o/
16:56:55 <wxy|> hi
16:57:09 <lbragstad> wxy|: looks like you have some things that needs reviews, too
16:57:09 <wxy|> it's just a reminder.
16:57:15 <wxy|> yeah
16:57:56 <wxy|> nothing else, just need some eyes
16:58:12 <lbragstad> #link https://review.openstack.org/#/c/557696/
16:58:29 <lbragstad> that's the implementation of the strict two level spec ^ ?
16:58:45 <lbragstad> #link https://review.openstack.org/#/c/576025/
16:58:50 <wxy|> part of
16:58:57 <lbragstad> #link https://review.openstack.org/#/c/577751/
16:59:15 <lbragstad> those should address the things the kmalloc brought up last week (nice job on the turn around time)
16:59:23 <wxy|> yes.
16:59:47 <lbragstad> i'll review those too
16:59:55 <lbragstad> if anyone else wants to test out the unified limit stuff manually
16:59:58 * hrybacki slides into next meeting
17:00:01 <lbragstad> ksc now supports unified limits
17:00:02 <wxy|> thanks.
17:00:12 <lbragstad> i'm working on some patches to get support into python-openstackclient
17:00:26 <lbragstad> i should have some patches up for oslo.limit this week, too
17:00:34 <lbragstad> so - no shortage of stuff for review
17:00:45 <lbragstad> wxy|: anything else you need on unified limits except reviews?
17:01:15 <wxy|> no, that's all
17:01:20 <lbragstad> thanks for all the hard work everyone
17:01:22 <lbragstad> #endmeeting