18:01:49 #startmeeting keystone 18:01:49 Meeting started Tue Feb 16 18:01:49 2016 UTC and is due to finish in 60 minutes. The chair is stevemar. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:51 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:53 The meeting name has been set to 'keystone' 18:01:58 #topic mitaka-3 release countdown 18:02:03 not much on the agenda today 18:02:16 #link https://launchpad.net/keystone/+milestone/mitaka-3 18:02:25 we've got about 2 weeks before mitaka-3 is cut 18:03:13 we have 4 major blueprints that still need to get wrapped up 18:03:33 henrynash: you've got domain scoped roles, which i think is in good shape 18:03:42 henrynash: do you have folks actively reviewing the work? 18:03:48 I am 18:03:49 stevemar: yep, just fixing up the trust interaction 18:04:01 henrynash: anyone else, i poked at it a bit 18:04:03 stevemar: yep, I, ayoung and henrynash are working on getting dsr and reseller (phase 1) 18:04:10 I am keeping an eye on them as well 18:04:18 stevemar: I am also looking at project tree ops 18:04:39 samueldmq: i punted reseller to N, there were a bunch of patches still outstanding 18:04:42 stevemar, ayoung: next set of patches on dsr and reseller are looking for final +2/A 18:05:02 samueldmq: project tree ops are still on target 18:05:02 stevemar: even phase 1 ? 18:05:16 stevemar: I didn't know about that decision, thanks for the update 18:05:30 samueldmq: i think so, last i checked on friday, there were about 7 patches to go, in various WIP and merge conflict states 18:06:04 stevemar: there isn't that many patches, and as henrynash said, most of them had +2 18:06:31 stevemar: yes, I agree, perhaps we can consider looking at them again if we have good progress this week ? 18:06:48 htruta: i count 10 here: https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:bp/reseller 18:06:55 stevemar: not all those are required, there are 5 that re needed for it to be actualy implented, the rest is cleanup….first 4 aer all good to go, the final one is waiting on a cinder fix 18:07:08 stevemar: I was/am putting a lot of effort reviewing then (still at the top ones) 18:07:13 stevemar: those include phase1 and phase 2. We're only interested in phase 1 in M 18:07:28 htruta: so 5 ? 18:07:33 henrynash: ^ 18:07:50 samueldmq: yes 18:07:54 henrynash: htruta then update the bp accordingly? remove the patches from the discussion section that are unrelated 18:07:55 samueldmq: yes, the first 5 on that list. If we do that, then projects are acting as domains 18:08:13 stevemar: ++ 18:08:30 stevemar: let them update the bp with the *5* patches to implement it 18:08:47 stevemar: and reconsider that to M :) 18:08:52 samueldmq: just remove ones that are unrelated 18:08:56 samueldmq: yep 18:09:09 stevemar: I will keep working with henrynash and ayoung and htruta to get them through this week (ideally) 18:09:14 henrynash, at least +1 the reviews that you are feel you've done too much on to +2, to show that you approve the additions 18:09:16 samueldmq: sounds good 18:09:23 stevemar: perfect 18:09:26 https://review.openstack.org/#/c/264533/ for example 18:09:32 ayoung: ok 18:09:57 next up is the totp stuff 18:10:01 #topic totp 18:10:18 #link https://review.openstack.org/#/c/274901/ 18:10:18 ayoung: done 18:10:40 i've been extra harsh on it since the spec took so long, and the we had to give it a FFE 18:10:57 i'm not seeing any docs or tests, and we're 2 weeks away 18:10:59 at this point with lack of docs, testing, etc 18:11:06 i am voting push to N 18:11:08 stevemar we still have 2 weeks :) 18:11:11 strategy with keystoneauth is undefined 18:11:23 and i was a champion of it at the midcycle assuming it would move faster than it did 18:11:46 gyee: it's more than that, it's diverted resources from other bps, and also maintaining the code 18:12:14 stevemar, I am just trying to help, is the Werner in the house? 18:12:25 I think totp can make it 18:12:30 i'm pessimistic on this one, if i ned a reality check, then i'm asking others to let me know 18:12:41 it really is just a documentation change, to distinguish it from standard password 18:12:46 it looks like it has tests? 18:12:50 it should not be doing anything really wacky 18:12:52 lbragstad: 2 tests 18:13:02 ayoung: and needs a ksa/auth-plugin story 18:13:10 ayoung: and tests for tha 18:13:12 t 18:13:31 notmorgan, ksa can happen after M3 though 18:13:33 ksa requirement is a bit harsh IMO 18:13:37 ayoung: no it can't really 18:13:45 client libs final release is before M3 18:13:47 we don't require keystoneclient code complete with the new APIs 18:13:48 ayoung: we will be past the mitaka deadline soon for non-client libs 18:13:48 stevemar: i'm going to be helping out with that one i think 18:13:54 notmorgan, ok 18:13:56 stevemar: and if we vote for keeping it, we should have super-champions to help getting a great progress until the end of this week ? 18:14:02 the totp one that is 18:14:06 * ayoung needs to finish implied roles then 18:14:20 and client libs will be 1 week past m3 18:14:22 ftr 18:14:29 Final release for  non-client libraries: Feb 24 18:14:35 yes, client libs are coming out very soon 18:14:35 Final release for client libraries: Mar 2 18:14:38 yep 18:14:49 i'm going to propose keystoneauth and keystonemiddleware today actually 18:14:54 that was another topic 18:14:58 samueldmq, it will be in good shape by the end of week :) 18:15:08 stevemar: hold please till closer to the deadline. 18:15:10 gyee: OK, i'll give it til end of week 18:15:16 so no server side feature will be approved without client libs? 18:15:18 stevemar: for ksm/ksa 18:15:20 clientside impl 18:15:27 notmorgan: why? anticipating something? 18:15:42 stevemar: there is something jamie was working on i want to see igf we can land 18:15:49 the authplugin -> oslo.context bigs 18:15:51 bits* 18:16:04 marekd, we can make it a new rule if you want 18:16:06 notmorgan: okay, get it lined up quickly 18:16:08 marekd: we need a "what does this look like" story. 18:16:19 stevemar: just propose on the 22 or so ;) 18:16:20 marekd: gyee no no no, it's not a requirement 18:16:29 marekd: not a "this is implemented" 18:16:47 but with an auth-type i expect especially with the low amount of code to have that complete 18:16:54 marekd: gyee -- i am asking "how will totp work with ksa?" how am i expected to use it 18:16:57 gyee: i absolutely don't want that. 18:17:00 especially with how slow that code is moving on updates/responses to comments 18:17:10 notmorgan: understood. 18:17:29 marekd: gyee for most work items, it's just new APIs, which are an existing pattern in keystoneclient 18:17:52 i know how much work is expected in keystoneclient when we add a new route 18:18:02 stevemar: yeah, i just got scared that we now must ship server + client code and somehow I wasn't aware of that. 18:18:05 i have no idea how much work is needed in keystoneauth with totp 18:18:15 auth plugins are also tiny amounts of work really 18:18:39 stevemar, yeah, like notmorgan said 18:18:41 but the keystone side has dragged a lot. 18:18:52 notmorgan: ++ 18:18:56 it's a mix of a lot of things 18:19:01 stevemar: what is the minimum i would need to do to ksa? a proof-of-concept show how it would work? 18:19:05 but yeah I agree the main author need to be a bit more urgent 18:19:24 we're here to help, but ... 18:19:42 same with shadow users :( 18:19:43 dstanek: poc works for me, even a straw man dictating how you expect it to work 18:19:46 dstanek: a POC in KSA [and i bet you'd fine the POC will be almost ready to go] and full testing/docs on the keystone side 18:19:50 right now it's all hand waves 18:20:01 stevemar: if we wnt to see a great progress until the end of the week, I'd like to propose we find champions for this 18:20:11 stevemar: otherwise it won't get the attention it needs 18:20:15 samueldmq: sounds like dstanek and gyee are on it 18:20:39 i mean, an auth plugin that just bundles the stuff in the right way to auth will likely be able to land before m3. 18:20:41 stevemar: perfect then, I just wanted to make sure we're on the right way :) 18:20:48 stevemar: yeah, it was written by someone on my team so i have no issue picking it up 18:21:04 since marek brought it up... 18:21:12 #topic shadow users 18:21:24 * stevemar waves at rderose 18:21:43 you found your way to irc land! 18:21:44 shadow users has 3 main parts, separate user identities, shadow federated users, shadow ldap users 18:22:01 separate user identities is ready for review 18:22:08 rderose link? 18:22:19 should be closer to finishing shadow federated users by the end of the week 18:22:24 lbragstad: https://review.openstack.org/#/c/278570/ 18:22:26 https://review.openstack.org/#/c/278570/ 18:22:43 shadow ldap users probably push to N 18:22:43 rderose: hm i only see some extra layer in the db 18:22:44 #link https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:bp/shadow-users 18:22:51 for all the types of users 18:22:53 dles t 18:22:53 ^ branch in case anyone is interested 18:22:59 rderose i'm really happy with the first patch 18:23:05 does it make sense to make shadow users partially in a release ? 18:23:10 vs pushing everything to N ? 18:23:26 samueldmq: we can live without the ldap stuff i think 18:23:29 dolphm: ^ 18:23:31 ++ 18:23:43 marekd separating user identites is really only in the db layer 18:23:47 scaffolding is a find thing to land in M 18:23:48 I think it's going to be confusing to deployers if it only partially works. 18:23:49 dolphm: what were you ++'ing, there were 5 messages at the same time 18:23:55 stevemar: cool, we should be fine with 2 first parts then 18:23:55 stevemar: yours 18:24:02 bknudson_: ++ 18:24:04 as long as it doesn't expose the confusing things to deployers or end users 18:24:05 dolphm: ++ 18:24:10 stevemar: ++ 18:24:16 if you need to just land some tables and some internal restructures, thats fine. 18:24:25 rderose: yeah, but are you going to add some logic apart db layer? 18:24:33 bknudson_: there's nothing partial about it, it's an internal move of our data 18:24:34 just try and avoid leaking info for un-implemented things to deployers/end-users 18:24:44 thats all i ask 18:24:52 and if that is the feature in it's entirety, great 18:25:00 separating user identities and shadowing federated users are independent features of shadowing ldap users 18:25:04 We're not going to get this all done, are we? 18:25:08 rderose: how shadow users were advertised in Tokio was I think much more complicated stuff. 18:25:20 rderose: that's why i am asking 18:25:20 marekd: the benefits are big 18:25:28 marekd: and the mental model is substantially different 18:25:39 dolphm: ++ 18:25:46 how is it less confusing without the ldap stuff? 18:26:00 dolphm: so i should only expect anything more than extra layer in the db. 18:26:28 dolphm: rderose: will the driver contract eventually change? 18:26:32 marekd: just for remote identities to be shadowed as necessary (federation, ldap, maybe remote user) 18:26:48 dstanek no, the driver contract will not change 18:26:51 gyee: adding ldap users to the overall database is the same as adding federated users 18:27:00 gyee: but adding federated is more important, and the main use case 18:27:11 dstanek creating a new driver for shadowing users 18:27:17 gyee: since now we can assign roles to them, and more easily audit bits 18:27:29 dolphm: the question i think i never got any response is whether the process of shadowing (assigning unique ids for remote users) will be handled by keystone with some smart algos or it will be somewhat leveraged for e.g operators creating mapping rules for federation? 18:28:02 stevemar, that's fine, so as long we get all the users from list users API, I was under the impression that we only get a partial list 18:28:13 if it will still be repsonsibility of ops then yes, i think it's just a db layer. 18:28:13 gyee: nope 18:28:16 we actually have something like shadow users for ldap. It's called id_mapping_api. 18:28:27 breton: ++ 18:28:46 breton: yeah, we need to merge that back into the new schema - the new schema follows that model a bit 18:28:59 it is quite a pain in terms of performance though 18:29:13 dolphm: do you have time to review this one with me this week? 18:29:28 stevemar: have time this afternoon? 18:29:33 dolphm: yep 18:29:38 stevemar: done 18:29:40 marekd: (i don't know the official train of thought) but i pictured this as social auth. as a user i have an account in keystone and i attach another account to it 18:29:41 i'll make time 18:29:52 okay, shadow users is still on the table 18:29:56 breton: dstanek yyy? 18:29:59 rderose: i have to imagine that you'd be adding driver methods at some point though, right? 18:30:01 stevemar: tell me how to make it later :) 18:30:26 https://review.openstack.org/#/c/279162/17/keystone/identity/core.py adds a new driver 18:30:53 dstanek new methods, but part of a new shadow users driver. what are you thinking? 18:31:42 does shadow users have a different table for each type of user? I thought there'd be one table to rule them all 18:32:04 bknudson one local identity table to rule them all 18:32:06 rderose: i think no contrllers are touched at all, for instance for federated/remote users, right? 18:32:11 bknudson_: the "identity" table links back to "user" 18:32:29 marekd yes, correct 18:32:52 is the current shadow users spec accurate? 18:33:11 ayoung it should be 18:33:15 rderose: not sure. do you plan on getting the federated stuff in for m3? 18:33:29 dstanek: i'd like that 18:33:38 #link https://github.com/openstack/keystone-specs/blob/master/specs/mitaka/shadow-users.rst 18:33:41 dstanek yes, the plan is to shadow federated users for m3 18:33:41 #link http://git.openstack.org/cgit/openstack/keystone-specs/tree/specs/mitaka/shadow-users.rst 18:33:47 damnit 18:33:59 breton: curious about your performance pains, btw 18:33:59 ayoung :) 18:34:05 stevemar: ok then i think we need to figure out what that driver should look like. i'm not fond if it's current impl 18:34:52 yes, rderose said it will be fully ready for review by the end of the week, looks like we can focus on the first patch until then ? 18:34:53 dstanek: i think rderose is open to suggestions 18:34:55 dstanek: ^ 18:34:56 and guidance 18:35:31 dstanek stevemar absolutely, have gone back and forth on the implementation 18:35:38 dolphm: #openstack-keystone 18:35:41 last BP that needs discussion... 18:35:49 #topic service-provider-filtering 18:35:57 #link https://blueprints.launchpad.net/keystone/+spec/service-provider-filters 18:36:05 * the implementation for shadowing federated users that is 18:36:07 yeah, so i noticed that davechen put some -1s 18:36:09 marekd: i apologize for not reviewing this 18:36:19 separating user identities should be ready for review 18:36:22 marekd: i think there have been very few reviews 18:36:28 I never liked extending *endpoint* filterting to *service providers*, which are different things 18:36:29 but in general i think it's more or less complete +/- some comments from other reviewers. 18:36:33 stevemar: no worries 18:36:38 but I guess everyone know that already, as I've been saying from the start 18:36:45 stevemar: and yes, not so many reviews but Dave was very helpful. 18:36:50 esp with json schema stuff 18:37:08 maybe we need to talk about why no one has reviewed this? 18:37:16 do folks not like the idea? 18:37:44 stevemar, just haven't found the time 18:37:45 samueldmq has voiced concerns about existing ep-filter 18:38:10 stevemar: lot's of people do that now...but it was discussed 2-3 times in the meetings 18:38:40 and I've been always hoding the same opinion 18:38:47 stevemar: i actually don't care that much whether it would be under /OS-FEDERATION/service_providers_filtering/ or /OS-EP-FILTER 18:38:47 stevemar: sounds like something i should be interested in, but i have not reviewed 18:38:50 because endpoint != service provider 18:39:11 and configuring service provider filtering undersomething called [endpoint_filter] soudns bad to me 18:39:30 we had this discussion awhile back, whether SP should be part of service catalog, but that ship has sailed 18:40:08 SP is Federation, not servicea catalog term. Keep them separate 18:40:17 marekd: i'll confess that i am in no rush to get this in, i think our SP story is a bit wonky 18:40:24 going to be really easy to confuse people with these terms 18:40:31 ayoung, people is treating SP as a region right now 18:40:35 so why not part of SC 18:40:43 ayoung: ++ 18:40:49 at least the IBM patch is making it a virtual region 18:40:57 gyee, it has nothing to do with regions or SC. 18:41:02 from horizon dropdown 18:41:05 It leads in to authorization 18:41:10 gyee: is region already formalized? because afair it was all and nothing at the same time. 18:41:14 gyee, its wrong then/ but I hadn;'t seen it 18:41:31 ayoung, take a look at Doug Fish's patch 18:41:39 link 18:42:09 stevemar: so your only concert is namespacing? 18:42:12 concern 18:42:55 marekd: no, i just think how we handle SPs is a bit weird and maybe it needs more thought 18:43:26 i think SP-filtering (what you have proposed for m3) is a symptom of how we handled SPs along side the catalog 18:43:29 ayoung, https://review.openstack.org/#/c/159910/ 18:43:33 stevemar: ++ 18:43:53 stevemar: so i think jamie was always repeating that only endpoints that you can reach/use with the same token should be in a catalog 18:44:06 SPs are clearly not usable without extra token 18:44:20 marekd: gyee was on that, don't know if jamie was sold on it 18:44:34 marekd: i'm not saying we put the SPs in the catalog 18:45:01 notmorgan: i'd say jamie, but might be wrong. 18:45:01 SC is a nice place for it as they also need to be "discoverable" 18:45:31 gyee: but SPs will differ from {project,user} to {project,user} 18:45:34 lets not wedge more in the SC 18:45:36 please. 18:45:51 make it a /service-providers call or whatever 18:45:59 /v3/auth/service_providers 18:46:13 bknudson_: sure. 18:46:41 and what should be under that route? 18:46:42 anway, like I said, that ship has sailed, so lets look ahead :) 18:46:47 and create a separate API/internal code for filtering it 18:47:28 samueldmq: that was not even proposed. 18:47:42 unless you want to amend the spec 18:48:11 i propose we read the spec again and raise some comments (again). 18:48:12 the specs aren't written in stone 18:48:26 marekd: yes, and that's my concern, re-use a thing that is a separate concern 18:48:30 marekd: i just think we need to think about it more before we take on more code 18:48:40 i see the problem recurs and is more architecture, not implementation. 18:49:04 SPs are only use for k2k, I'm assuming there aren't many people using this, and they won't be upset if we don't land this for mitaka 18:49:29 I don't think we ever got the auth plugin 18:49:35 stevemar: ++ 18:49:46 bknudson_: we did, i believe 18:50:25 marekd: i'd rather wait and see if we can't come up with a better way to do this, sorry if this comes as a surprise 18:50:26 bknudson_: i think we did 18:50:38 It's not documented: http://docs.openstack.org/developer/keystoneauth/authentication-plugins.html ? 18:50:56 bknudson_: that's a different story 18:50:57 bknudson_: nothing is ever documented 18:51:07 so nobody can use it 18:51:11 :) 18:51:23 that's the way we like it 18:51:33 marekd: we'll chat in a bit 18:51:49 #topic bugs! 18:52:08 stevemar: well, it does suprise me a little bit, but i am not upset or something, so don't worry 18:52:15 there are a bunch of bugs open against mitaka-3: https://launchpad.net/keystone/+milestone/mitaka-3 18:52:47 lots of good work being done on that front 18:52:52 marekd, but you'll be sending stevemar christmas card still right? :D 18:52:57 summary: 3 New, 1 Triaged, 12 In Progress 18:53:34 i've marked all new-ish bugs that have a patch as mitaka-3 candidates... some of these are not important enough to block mitaka-3, but they are actively being worked on 18:53:39 gyee: and hugs 18:54:26 K2K is not Federation. 18:54:31 the only bug i'm confused about it one of the trusts ones 18:54:35 Federation includes K2k...gah 18:54:36 ayoung: stay on topic 18:54:40 sorry 18:55:06 henrynash: you've got 2 related to domain configs 18:55:10 stevemar: which one? reported by henrynash earlier today? 18:55:14 stevemar: yep 18:55:18 those aren't blockers right? 18:55:26 samueldmq: nope 18:55:34 k 18:55:34 they didn't block liberty, so i assume they won't block mitaka 18:55:40 stevemar: nope, if I can fix them I will, but not blockers 18:55:48 henrynash: splendid 18:56:11 lbragstad: ayoung there's also https://bugs.launchpad.net/keystone/+bug/1539766 18:56:11 Launchpad bug 1539766 in OpenStack Identity (keystone) "trust redelegation allows trustee to create a trust (with impersonation set to true) from a redelegated trust (with impersonation set to false)" [High,In progress] - Assigned to Jorge Munoz (jorge-munoz) 18:56:35 i am having trouble keeping track of this and whether or not it's valid 18:57:05 lbragstad: also, are any of your fernet token patches blockers for mitaka-3? 18:57:15 remimder: 3 minutes left 18:57:22 i know we had hoped to make fernet the default for mitaka, but i'm not so sure that's possible? 18:57:25 stevemar i don't think they are blockers 18:57:26 reminder* 18:57:35 actually the oauth one might 18:57:38 lbragstad: only if we want fernet to be the default? 18:57:45 fernet is still not default??? 18:57:53 breton: correct, uuid 18:57:55 why 18:57:59 breton no - it's been in a string of patches 18:58:20 stevemar, I have -1 on that right now 18:58:33 gah, we're out of time 18:58:38 breton: because the default is not just a philosophical choice 18:58:47 please review review review 18:58:52 +++ 18:58:58 we've got 2 weeks left 18:59:05 let's keep the momentum going 18:59:23 thanks everyone for their hard work (now and in advance) 18:59:28 if you care about policy usability -- I posted a spec to switch policy.json to YAML -- https://review.openstack.org/#/c/279748/ -- it's essentially a one-line change. 18:59:29 #endmeeting