18:00:36 #startmeeting keystone 18:00:36 Meeting started Tue Jan 31 18:00:36 2017 UTC and is due to finish in 60 minutes. The chair is stevemar. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:37 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:00:40 The meeting name has been set to 'keystone' 18:00:43 o/ 18:00:53 o/ 18:01:15 o/ 18:01:17 hi 18:01:33 wow, there is a bknudson here! 18:01:36 :) 18:01:52 o/ 18:01:56 a rare sighting in the wilds of IRC! :) 18:02:10 o/ 18:02:33 a bknudson! 18:02:36 * stevemar finishes switching desks 18:03:15 #topic announcements 18:03:23 we're in the RC week, RC must be tagged by thursday, don't expect anything to merge unless it's a priority for the release 18:03:29 oops, agenda link! 18:03:31 #link https://etherpad.openstack.org/p/keystone-weekly-meeting 18:03:46 priorities for RC are listed here: #link https://etherpad.openstack.org/p/keystone-sprint-to-ocata 18:03:53 #link #link https://etherpad.openstack.org/p/keystone-sprint-to-ocata 18:04:10 we approved per-user-auth (MFA) for RC 18:04:37 see bp here: https://blueprints.launchpad.net/keystone/+spec/per-user-auth-plugin-reqs 18:05:33 thanks morgan for working so hard on it 18:05:37 its coming along really nicely 18:05:38 ++ 18:05:54 also a bug in some logs found in the process https://review.openstack.org/#/c/427004/ 18:06:03 needs a second +2/A 18:06:07 but it is not directly related 18:06:16 yeah, easy one to review ^ 18:06:22 release note was just pushed for MFA 18:06:27 please look for it. 18:06:38 #link https://review.openstack.org/#/c/427328/ 18:06:38 ack 18:07:02 yowza, thats a heck of a release note 18:07:13 more in depth docs will come post RC likely, pending stevemar's resource option docs 18:07:29 morgan: may have to trim it, just so it doesn't differ too much from the other notes 18:07:33 current rendered version: http://docs-draft.openstack.org/28/427328/2/check/gate-keystone-releasenotes/ad8ed97//releasenotes/build/html/unreleased.html 18:07:47 i like the detail though, but i prefer consistency, if that makes sense. 18:07:50 stevemar: but... it's an accurate release note 18:08:02 * morgan will leave that trimming to stevemar 18:08:05 ;) 18:08:12 :) 18:08:50 any questions about RC? we have some identified bugs but that's the next topic, have some more announcements 18:09:09 +2A 18:09:22 next announcement 18:09:29 gyee is stepping down from core 18:09:41 due to inactivity and a change in responsibility i asked him to step down 18:09:53 Not a huge surprise. 18:10:11 yeah, i wanted to give the next PTL a clean slate to start from 18:10:15 thanks gyee for your service! 18:10:17 * dolphm salutes gyee 18:10:41 thanks gyee :) 18:10:45 * morgan cheers for gyee and thanks him for his service 18:10:51 though... he's not here to hear it atm 18:10:57 the ether knows 18:11:10 s/ether/etherpad 18:11:13 *shiftyeyes* 18:11:15 you can email him at his personal address if you want, PM me if you want it 18:11:23 he responds quickly there 18:11:32 yeah he typically does. 18:11:36 not sure if hes going to write a note to the ML 18:11:52 stevemar: make sure you announce the Core change to the ML sometime 18:12:02 if he doesn't write a note. 18:12:05 will do 18:12:07 (sorry I’m late) 18:12:10 * morgan blinks 18:12:13 henrynash o/ 18:12:13 henrynash: heyo! 18:12:15 we have a henrynash too! wow. 18:12:30 W00t! 18:12:33 henrynash too! is so much better than the old one 18:12:41 it is a redletter day 18:12:47 all the rascally ibm'ers are showing up 18:12:56 gettin' the band back together 18:13:03 stevemar: i don't see topol 18:13:04 (dog eat my homework etc.) 18:13:05 stevemar: :P 18:13:07 morgan: zing 18:13:18 last announcement 18:13:21 thanks gyee! 18:13:23 19 PTG tickets left 18:13:25 stevemar: or jamielennox 18:13:40 i think they started with 200? 18:13:55 * morgan needs to book an airplane 18:13:55 thats a huge drop from 2 weeks ago, i think they have >100 left 18:13:58 I have one, too, that I am not going to use 18:14:11 Was going to go to my backup, but backup is not going 18:14:33 Think the deadline for transfer is soonish. Let me know if you need it. 18:14:42 * stevemar is pretty sure there are hourly flights going from toronto to atlanta, will book later 18:15:22 i'm sure the 19 will end up being bough 18:15:23 bought 18:15:26 o/ 18:15:32 perhaps more slowly 18:15:47 I will be in ATL Sunday night 18:15:57 * morgan will be flying in on Monday night 18:15:59 I'll be there Wednesday morning 18:16:05 Tues night 18:16:13 so i'll be missing day 1 18:16:13 (see, we all synced that so well) 18:16:17 Tuesday night 18:16:21 and staying until late Friday 18:16:28 * breton will be from Sunday to Saturday 18:16:40 and i am either saying until friday mid-day or early sat 18:16:44 likely leaving midday friday 18:16:56 schedued for sun-sat 18:16:58 * topol topol had to come late to this meeting. Still distraught over morgan's incredibly ugly bagels... still shaking :-) 18:17:16 topol: it's ok, i gave you a giant release note to review. 18:17:19 i'm there sunday to saturday too 18:17:22 topol: because you were late 18:17:42 stevemar i assume organization of everything will start soon? 18:17:54 thanks 18:17:58 I'll be there Sunday night too, in the case someone wants to hang out for food or something 18:18:02 i'm finally here! 18:18:18 lbragstad: yeah, as soon as we cut RC1 we can start planning 18:18:25 stevemar ok - cool 18:18:37 3 weeks is plenty of time to plan it all out :P 18:20:22 okay, next topic 18:20:22 * topol who will join me for a tearful reunion with my Ga Tech Ph.D. advisors? Havent seen them in 18 years... 18:20:36 topol: you're on your own there bud 18:20:40 :-) 18:20:53 #topic bugs we can fix during RC 18:21:20 i plan on cutting RC1 when morgan's MFA stuff merges 18:21:40 but we can cut RC2 if we deem some bugs are worth backporting 18:21:51 i think these two qualify 18:21:55 Federation: federated users can't log into Horizon - https://bugs.launchpad.net/keystoneauth/+bug/1660436 18:21:55 Launchpad bug 1660436 in python-novaclient "Federated users cannot log into horizon" [Undecided,New] 18:21:56 Federation: cannot use Trusts with federaed users - https://bugs.launchpad.net/keystone/+bug/1589993 18:21:57 Launchpad bug 1589993 in OpenStack Identity (keystone) "cannot use trusts with federated users" [High,In progress] - Assigned to Boris Bobrov (bbobrov) 18:22:07 i can talk about the trusts issue 18:22:14 crinkle is working on the first, and breton is working on the second, are either of you around? 18:22:14 I was actually working on that bug downstream for some time. 18:22:18 o/ 18:22:27 breton, you first, crinkle afterwards? 18:22:36 stevemar: ok 18:22:40 The problem with the whole thing is that with federated users we cannot really know what the user can unless they have authenticated 18:23:03 can.. do? 18:23:13 because, for example, with trusts, at the moment of trust creation the user might have already lost his group, that provided a role, because something has changed in the IDP, and we don't know about it 18:23:21 can do, yes 18:23:55 breton: you can use shadow mapping to create users 18:24:11 for us, downstream, the solution now is to recreate user group membership for federated users each time the user authenticates. We just know that the customer will not add shadow users to groups directly and can live with the possible gap between group removal and next authentication 18:24:19 rderose: that won't help with groups necessarily 18:24:39 breton: can you assign the role necessary to the user directly? 18:24:51 stevemar: no. There can be very many users. 18:25:00 ah dang 18:25:19 breton: so is the issue that A trusts B, A never logs in again (and meanwhile as lost their group in teh idp), meanwhile B can carry on usingthe trust for ever? 18:25:32 breton: that's how federated mapping works by design right? re-evaluate group membership for each auth 18:25:50 henrynash: and that sucks 18:26:01 henrynash: ah, that makes sense 18:26:08 if the trust is no longer valid due to a group memebership change, they cannot use it 18:26:18 so you have to evaluate A's group membership for each B use of a trust? 18:26:20 so the problem is that trusts don't have the opportunity to stay in sync with whatever is in the idp? 18:26:21 dstanek: kinda. But that group membership never persists inside the database 18:26:21 breton: and we can’t tell from our side that the group has been lost unless A choices to reauthenticate 18:26:34 ayoung: the group membership can be ephemeral 18:26:39 henrynash: which may not happen 18:26:46 dstanek, then the trust will never work 18:26:50 stevemar: yep 18:26:53 breton: exactly. but you can evaluate it anyway right? 18:27:05 dstanek: no 18:27:12 i think the answer here is simple: with shadow users allow explicit group additions 18:27:13 if the trust code cannot link the original user to the roles in the trust, they trust execution fails, no token 18:27:14 if you know the user and the idp of the trustor can't you lookup the mapping? 18:27:15 to the shadow user 18:27:19 dstanek: groups might come from assertion 18:27:21 it is only allowed in that case 18:27:26 not the "assertion" groups 18:27:42 breton: ah, right. good call 18:27:44 i can't get behind supporting trusts on the values that come from an assertion 18:27:46 you'd need the assertion, wouldn't you? 18:27:50 we need to record all membership information used in a trust 18:27:55 ayoung: ++ 18:28:10 ayoung: therefore the shadow user, if added to a group, could explicitly be granted the values 18:28:15 it's "known" 18:28:15 morgan, ayoung: but how would the group memberhsip info ever be deleted 18:28:30 ok, so no trusts based on federated ephemeral group membership? 18:28:32 henrynash: you'd purge the shadow user 18:28:43 morgan: that won't work 18:28:44 morgan: agreed 18:28:51 dstanek: yeah, i'd -2 adding trusts based on ephemeral group membership 18:28:52 morgan: i dislike it, because it requires operators manage users manually, not in the idp 18:29:00 if a change in the IdP puts them in a different group then they'd get different ephemeral groups 18:29:10 and we have code that relies on it -- heat uses it for delayed operations 18:29:13 morgan: but an idp (or whoever is creating users in the idp) would have to know to do that 18:29:16 dstanek: right, this is why we only support in the case of explicit group additions in keystone 18:29:17 if you put that in keystone then any change in the idp would need a change in keystone 18:29:22 we could try fixing heat to use allow_expired though. 18:29:28 aka shadow users. 18:29:42 breton: that might be the better call here 18:29:45 or the local user w/ mapped values 18:29:50 morgan: how do we differentiate between explicit and implicit groups? 18:29:57 morgan: a column in the db? 18:30:00 operator assigned in the db 18:30:16 it is loaded from the db. vs loaded from the assertion 18:30:34 we have the ability to allow a local user to auth with federated creds, we have shadow users now 18:30:47 if you grant an explicit membership in either case, that carries forward 18:30:50 regardless of assertion 18:30:56 that is an explicit assignment 18:31:03 implicit from the IDP can never have trusts 18:31:08 that was the whole point of federation -- to manage users not in openstack, but outside of it 18:31:10 because of the issues you've outlined 18:31:25 that is a step back to moving users around in keystone 18:31:28 the IDP provides authentication 18:31:38 the IDP never, ever, ever has provided authorization in keystone 18:31:41 keystone has done that 18:31:46 trusts are authz 18:31:48 not authn 18:32:09 if the idp is providing authz (not keystone), we have a bigger issue. 18:32:22 morgan: but the group mappings from teh idp ARE used to “generate” authz 18:32:25 we have the same issue if using trusts and LDAP users right? 18:32:31 henrynash: that is now what i mean 18:32:33 stevemar: no 18:32:38 henrynash: keystone has to explicitly map that. 18:32:45 morgan: agreed 18:32:58 ah the groups are mapped back, duh 18:33:00 henrynash: this is no different except an operator who wants trusts needs to explicitly add federated users. 18:33:07 to a group 18:33:14 no implicit based on trust 18:33:25 unless the trust expiry is a known value 18:33:34 groups are an attribute that are used to assign authz. Those groups may come from the IdP origianlly, but must be recorded in Keystone in order to have a non-ephemeral effect 18:33:39 aka it expires about when the assertion (or similar view) is expired 18:33:45 13:24 stevemar: breton: can you assign the role necessary to the user directly? 18:33:46 13:24 breton: stevemar: no. There can be very many users. 18:33:46 13:24 stevemar: ah dang 18:33:46 ayoung: ++ 18:34:01 morgan: it's not the operator wants trusts. Users want. 18:34:01 ayoung in order to have a non-ephemeral effect on users* 18:34:16 lbragstad, in order to have any non-ephemeral effect 18:34:24 morgan: we can't make each user ask operator when they want to use a trust 18:34:36 ayoung aha - yeah 18:34:38 you can create a trust, but if the info used to create that trust is not recorded, the trust is non-executable 18:34:44 breton: yes we can. 18:35:06 ayoung: ++ 18:35:36 ayoung: so you're saying... shadow groups! :D 18:35:36 breton: i'm sorry but no creating trusts based on ephemeral idp groups 18:35:51 stevemar, probably should be something explicit 18:35:52 if the data is exclusively controlled by the IDP, no trusts 18:35:58 morgan: so the explict support is obvioulsy one approach (and we should support it imho..and I assume we do already), 18:36:00 would short term trusts work? something that expires at the same time as the token of the trustor 18:36:08 henrynash: i think it needs it 18:36:11 operators will be able to create federated users via API and can then explicitly make trusts 18:36:24 *operators/admins 18:36:25 dstanek: well i'd do a expiration of the assertion 18:36:34 dstanek: but yes, i'd be ok with fixed life trusts 18:36:36 stevemar, something in the mapping that indicates that users in IdP I via protocol P should be added to group G 18:36:42 rderose but then if something changes in the idp - the trust can still be valid 18:36:49 dstanek: as an option provided we track the data from the idp 18:36:51 morgan: this is a huge step back in federation 18:36:52 when it might not be 18:36:56 rderose: operators could also assign roles individually. breton is saying he doesn't want his operator/admin to deal with all the requets 18:37:02 if users have to be greated by the operator in this case them mapping buys them nothong and the are using a glorified ldap 18:37:07 lbragstad: what idp change would cause it to be invalid? 18:37:18 rderose: many things. 18:37:27 rderose depends on the assertion/idp setup 18:37:37 (case-by-case problems are *so* fun!) 18:37:41 morgan: it becomes a thing that doesn't support part of keystone functionality 18:38:01 the whole point of federation is that we don't have to have all the users/groups in keystone and they are dynamically provided by the IdP 18:38:03 we should look at what OpenIDC does here. This is a pattern larger than Keystone and OpenStack 18:38:09 breton: the only way you're gettiing trusts based on idp info is if the trust expires at the expiry of the assertion (or something similar) 18:38:11 morgan: lbragstad: but if you explicitly give a federated user a role, that role assignment should exist regardless of changes to the idp 18:38:19 rderose exactly 18:38:23 ayoung: they map it to a local user 18:38:25 We had only looked at OAUTH1a when Trusts were implemented 18:38:28 ayoung: then the local user is granted access 18:38:33 if you manually provision, then you should manually deprovision 18:38:33 ayoung: in almost every-single-case 18:38:36 rderose: exactly. and that;s the problem 18:38:37 rderose what if something in the idp changes and that trust should no longer be valid? 18:38:39 morgan, I mean for management of delegation 18:38:47 lets than document that trusts don't work with groups from federated token and stop advertising federation as first-class citizen 18:38:54 ayoung: it's checked on each authn 18:38:54 ayoung: or fixed life 18:38:55 morgan, is there an analogue to trusts there? 18:39:11 ayoung: no. the assertion (token?) is the trust 18:39:20 ayoung: so i auth with say google, to APP 18:39:31 i give access to the google things for the life of the token/assertion 18:39:42 breton: everything you can do with a local user, you should be able to do with a federated user 18:39:55 if the app has specific attrs/authz it is given explicitly to the user object that google auth links to 18:40:10 rderose: i absolutely agree with that. But the reality is different. 18:40:14 rderose: thats breton's issue, he can't right now :P 18:40:19 morgan: exactly what i was thinking with short term trusts 18:40:26 rderose, you can. The problem is that you still need to record the group membership for the Federated user. Which gets us back into the "the user needs to log in first before they can be admin'd" 18:40:41 dstanek: so ... if we do time-limited trusts and store the information, i'd be ok with it. 18:40:42 rderose, which is why it would be better if the group memebership was triggered by the mapping process 18:40:52 gonna cut this conversation off in a minute or two 18:40:54 dstanek: i just wont support indefinite trusts on ephemeral data 18:40:58 i don't think we're going to solve it here 18:41:07 morgan: ++ 18:41:15 this is interesting though - should we save it for -keystone later? 18:41:19 ayoung: or, provision your federated users via the API; deprovision when their access should change 18:41:21 someone write up the use case as a spec please 18:41:33 ayoung: the bug exists :P 18:41:43 short-lived trusts won't work for Heat 18:41:45 rderose: there is a mechanism to say assertion is expired...but that feedback is not well supported in most IDPs 18:41:46 rderose but then we're managing federated users in the IDP and in keystone 18:42:07 lbragstad: ++ that doesn't make sense 18:42:08 lbragstad: we're managing their authorization in keystone 18:42:10 morgan: I would also support the time-limiting option for a trust (which should be applicable to regualr suers or federated users) 18:42:11 (because it explicitely requires long-lived trusts) 18:42:21 henrynash: we already support that feature 18:42:28 henrynash: it's implemented :). 18:42:37 lbragstad: at HP, user requests access, some system calls keystone and provisions access 18:42:38 breton: what is a long-lived trust? 18:42:43 breton: indefintie? 18:42:47 He’s smart, that morgan guy, ya know 18:42:55 1-day? 2-days? 16-days? 18:43:05 rderose: you are no longer using federation 18:43:07 morgan: indefinite i guess. 18:43:16 * antwash being the newbie sucks... 18:43:19 dstanek: you are using federation for authentication 18:43:23 hehe 18:43:26 #link https://bugs.launchpad.net/keystone/+bug/1589993 18:43:26 Launchpad bug 1589993 in OpenStack Identity (keystone) "cannot use trusts with federated users" [High,In progress] - Assigned to Boris Bobrov (bbobrov) 18:43:26 breton: that is breaking how security in federation works then 18:43:26 antwash :) 18:43:33 breton: this is *not* secure. 18:43:49 rderose: you don't get any of the benefits of what federation offers 18:43:51 antwash patience young grass hopper, patience 18:43:55 okay, i'm gonna cut this one off, we have a lot more on the agenda 18:44:04 morgan: i agree. Then lets stop calling federation first-class citizen and advertise that it can everything local backend can. 18:44:06 breton, grab me after and we can talk through it 18:44:14 breton: works for me 18:44:20 I'll hang out in -keystone after the meeting to continue this discussion with folks if they want 18:44:23 ayoung morgan ++ 18:44:26 cool 18:44:29 anyway lets talk in -keystone 18:44:30 same here 18:44:35 * topol I remember when lbragstad was the grasshopper... feelin old 18:44:42 crinkle: o/ 18:44:46 o/ 18:44:55 breton: i have a fix btw... just want to be very deliberate about how we implement it 18:45:00 Federation: federated users can't log into Horizon - https://bugs.launchpad.net/keystoneauth/+bug/1660436 18:45:01 Launchpad bug 1660436 in python-novaclient "Federated users cannot log into horizon" [Undecided,New] 18:45:02 topol lol 18:45:03 so 1660436 - what's happening is after a federated user successfully authn's, horizon then tries to load one of the nova panels with the novaclient, and it's trying to create a new token session with the federated token, and it fails because it's not persisting domain data, so keystoneauth's discovery returns a v2 endpoint, which a federated user can't use 18:45:19 crinkle: euuw. 18:45:20 I was digging into it and more or less ran into a wall at that point because it involves multiple projects, but I put everything I found out into the bug 18:45:25 we should fix that :P 18:45:27 morgan: yeah it is super gross 18:45:50 this is unexpected side effect 18:45:52 to me, it feels like the fix belongs in how horizon doess stuff... 18:46:04 stevemar, probably not 18:46:09 robcresswell said he would look into it 18:46:20 stevemar, Horizon is pretty simple here: get a Federated unscoped token, convert it to a scoped token 18:46:22 we could persist the domain info in the token, no? 18:46:27 crinkle so it's pulling the domain from the token - but failing to look things up with it? 18:46:30 and let horizon consume that? 18:46:30 crinkle: yeah, but we probably need to help the horizon team out here 18:46:35 we'd still have no groups 18:46:52 ayoung: oh ugh, and with fernet that becomes icky 18:46:56 hmmm. 18:46:56 yeah 18:46:56 lbragstad: no, it's not pulling the domain info 18:47:10 morgan, again, shadow all the info we need for future use 18:47:13 lbragstad: that's kind of the problem, with no domain info keystoneauth says use v2 18:47:20 perhaps we time-limit group membership? 18:47:21 ayoung: pretty much how other apps do it 18:47:43 ayoung: we can overwrite the data if the assertion is changed (from the same IDP) 18:47:47 ayoung: that is sane 18:47:52 and no different than how local users work 18:48:07 ayoung: so a single record per-user-per-idp for that 18:48:13 morgan, assuming that a new assertion has the complete set of groups, then, yes, remove group membership for missing groups 18:48:26 is that a safe assumption? 18:48:35 the new assertion would *have* to have the value of all groups for that ID from that IDP 18:48:39 it is authortitative 18:48:45 i don't think you get partial assertions ;) 18:49:06 i would say worth saying it's safe and have someone do a bit of research before impl 18:49:19 morgan, you could use different profiles, and get different sets of groups, but...I think that in that case, the profile used would be static, and should be authoritative...so still remove 18:49:26 exactly 18:49:31 again, per-idp-per user 18:49:39 per protocol 18:49:40 if you authed with a different idp for example FB vs Google 18:49:41 yeah 18:49:52 i'd expect it to be different 18:49:56 same IdP, different protocol.... 18:50:01 different groups? 18:50:03 should result in the same data? 18:50:03 might happen 18:50:12 i'd start with per-IDP and expand if needed 18:50:34 mapping is per protocol. I'd do it per protocol 18:50:39 ok sure 18:50:43 er...per mapping, really. 18:50:44 anyway 18:50:48 per mapping 18:50:56 shadow the data, 18:51:02 you cabn technically reuse the mapping for different protocols, but that would be unlikely to work 18:51:16 so would still end up per protocol, I think 18:51:21 * ayoung is done 18:51:29 anyway 18:51:44 crinkle: fixable we probably need to shadow the user info in keystone 18:52:06 morgan: i don't think we have time to implement all that? 18:52:07 crinkle: and we can update that as assertions come in. 18:52:12 stevemar: not in Ocata 18:52:25 morgan: right, but right now federated users can't get into horizon 18:52:25 going to go on record and say, it wont happen in Ocata (sadly) 18:52:34 it's an architectural issue 18:52:35 like, any of them 18:52:41 we'd need to store groups too 18:52:51 because the token has to be reconstructed 18:53:06 crinkle: can we revert the change that novaclient made? 18:53:09 we could make a temp fernet token formatter that stores a ton extra data 18:53:21 right now - federated fernet tokens store groups in the token itself 18:53:31 lbragstad: oh ok then we should be ok 18:53:36 and we can just persist domain data 18:53:37 stevemar: well not really, the bug it fixed was a pretty nasty bug too 18:54:01 crinkle: ok, so solution: persiste the domain data 18:54:07 crinkle: what about catching the exception on the horizon side? 18:54:17 morgan https://github.com/openstack/keystone/blob/master/keystone/token/providers/fernet/token_formatters.py#L596 18:54:24 crinkle: much easier and Ocata possible if that is the issue. it's just storing the info in the fernet format 18:54:57 crinkle: it will be a new fernet formatter that is then used (we can't change the formatters we have as they are tied to specific format) 18:55:15 lbragstad: ^ sound right to you? 18:55:26 stevemar: ^cc 18:55:38 dstanek, lbragstad weren't one of you working on creating resources on demand for Federated users? 18:55:46 proejcts was the big one, but also role assignments etc? 18:55:49