18:00:21 #startmeeting keystone 18:00:22 Meeting started Tue Dec 6 18:00:21 2016 UTC and is due to finish in 60 minutes. The chair is stevemar. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:23 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:00:26 The meeting name has been set to 'keystone' 18:00:34 o/ 18:01:13 o/ 18:01:14 hi 18:01:19 o/ 18:01:25 #topic announcements 18:01:38 hello all! 18:01:41 Next week we are cutting Ocata-2 -- Week of Dec 12-16 -- This is also spec freeze week, specs must merge by then. 18:02:06 Any 'contentious' feature already approved, but not completed, will get bumped to Pike if it doesn't merge. 18:02:10 Contentious meaning: touching a critical path (token validation / issuance), affecting many components, potentially changing output of API calls. 18:02:36 hey 18:02:37 I don't think theres anything contentious that risks getting bumped (maybe the policy stuff?) but most of it is isolated 18:02:59 most approved features* are isolated 18:03:05 Oyez oyez 18:03:51 i think i started too soon? did i lose anyone? 18:04:08 stevemar i think you're good 18:04:16 okie doke 18:04:17 stevemar: so 'contentious' features (the impl) must be merged by o-2 ? 18:04:36 samueldmq i think the spec needs to be merged 18:04:40 by o-2 18:04:49 yes, i don't think there are any, the contentious ones were the token refactoring and the expiry window stuff jamie was working on 18:04:57 links? 18:04:59 those are all merged 18:05:10 agenda link: https://etherpad.openstack.org/p/keystone-weekly-meeting 18:05:21 stevemar: okay, but we still talking about the specs, right ? 18:05:24 that is the link I was looking for 18:05:41 lbragstad: kk just confirming, thanks 18:05:56 samueldmq: yes. all specs must be approved by EOW next week 18:06:04 * samueldmq nods 18:06:16 otherwise they get the boot 18:06:46 * stevemar waits for more questions 18:06:57 Trust Scope Extensions should not go forward. It is a good idea, but untenable as written. It can't be Trust based, needs to be at the token itself... 18:07:18 Extend user API to support federated attributes looks pretty good 18:07:27 the ones I am worried the most are the trust ones 18:07:32 ayoung: ^ 18:07:47 samueldmq, Spec: Trust Scope Extensions [jgrassler] 18:07:48 https://review.openstack.org/#/c/396331/ you mean? 18:07:51 #link https://review.openstack.org/#/c/396331/ 18:07:57 #link https://review.openstack.org/#/c/396634/ 18:08:04 ayoung: there are two trust ones 18:08:16 samueldmq, yeah, I had a long talk with jgrassler this morning. My position is unchanged 18:08:28 ayoung: theres also "lightweight" trusts: https://review.openstack.org/#/c/396634/ 18:08:52 stevemar, that is oauth 18:09:03 ayoung: https://review.openstack.org/#/c/396331/ is solved with our OATH in your opinion? 18:09:04 we can do those today but should merge trusts and oauth 18:09:14 samueldmq, no the other one is 18:09:20 396634/ is oauth 18:09:42 ayoung: ok, and what about "Added trust-scope-extensions" 18:10:26 samueldmq: that one also has negative feedback 18:10:49 anyway, some specs are close, others are not. keep reviewing them 18:10:57 * samueldmq nods 18:11:18 moving along 18:11:20 stevemar: we might want to have a deeper discussion in next meeting on those that haven't been merged by that time 18:11:27 stevemar: ++ 18:11:27 samueldmq: yep 18:11:32 #topic Introduce Annapoornima Koppad [annakoppad] 18:11:40 raildo, rodrigods ^ 18:12:00 stevebaker, thanks 18:12:02 she have an irc nick? 18:12:03 so... welcome annakoppad_ 18:12:03 stevemar, * 18:12:16 ah.. 18:12:17 are you saing 396634 we're calling OAUTH instead of calling it trust something? 18:12:20 welcome 18:12:21 Thanks, raildo! 18:12:30 chrisplo, it is already implemented 18:12:32 annakoppad_ o/ 18:12:35 has been for years 18:12:42 she is our new outreachy intern, her project starts today! 18:12:49 ayoung: chrisplo let's continue that in -keystone, we've changed topic already. thanks 18:13:07 the idea is to add a new job to our CI to handle LDAP scenarios 18:13:08 welcome annakoppad_ 18:13:18 that's great! 18:13:18 rodrigods, ++ she will be working with us (at least) march 6 :) 18:13:19 Thanks! stevemar! 18:13:26 raildo and rodrigods as mentors ? 18:13:30 yep 18:13:31 samueldmq: yep! 18:13:32 samueldmq, yes 18:13:36 annakoppad_: welcome aboard! 18:13:41 annakoppad_: good choice of mentors :) 18:13:45 hopefully, more as well, as I do not intend to stop coding ever at all! 18:13:52 nice nice, good to see it continuing, congrats on the initiative rodrigods and raildo 18:13:54 annakoppad_: :D 18:13:57 at raildo! rodrigods 18:14:03 samueldmq, ++ 18:14:08 samueldmq, your welcome. 18:14:13 thanks stevemar and samueldmq 18:14:25 so... her project is to help keystone, but involves lots of devstack, project-config, etc 18:14:39 annakoppad_: welcome 18:14:48 hello dstanek! 18:14:49 annakoppad_: looking forward to reviewing your patches, please don't be shy and ask questions in #openstack-keystone if you need a hand 18:14:59 yes stevemar! 18:15:02 stevemar, ++ 18:15:09 thanks rodrigods and raildo for doing this 18:15:22 morgan: you around? 18:15:25 annakoppad_, so, any doubts, you can ask for help in the #openstack-keystone, and I know that you will find someone to help :) 18:15:34 stevemar, no problem :) 18:15:53 great, raildo! 18:16:13 i don't think morgan is around, so we can skip the next topic, taskmanager in keystoneauth 18:16:28 plus, the work is well under way in https://review.openstack.org/#/c/362473/ 18:16:28 think it is not only my opinion that having the LDAP job is a great advance for us :) 18:16:37 annakoppad_, BTW, it turns out the devstack LDAP code is not currently working 18:16:49 just tried it last week... 18:16:58 ayoung: yeah, its bit rotted a bunch 18:17:07 I was trying out running the devstack..will try it a little bit 18:17:11 ayoung, out first step will be fix it 18:17:17 our* 18:17:34 raildo, we can discuss. Lets set up a time to talk it through 18:17:41 ayoung: i have no doubt that raildo and rodrigods have a plan ;) 18:17:47 ayoung, ++ 18:17:49 stevemar, and ayoung, never mind, I will help raildo and rodrigods fix whatever is broken 18:17:49 that works too 18:17:52 stevemar, I['ve been talking with them about it 18:18:00 ayoung: cool cool 18:18:18 time for our super fun topic 18:18:26 #topic Custom project IDs 18:18:28 RBAC? 18:18:33 stevemar, AH 18:18:36 ok so I -2 that 18:18:40 and here is why 18:18:41 agrebennikov, ayoung, but on your boxing gloves 18:18:59 stevemar, so the argument that agrebennikov gave me is he is trying to do multi site 18:19:17 and syncing stuff at the database layer (Galera) puts a lot of load on him 18:19:18 isn't the intent behind shadow mapping to fix that? 18:19:24 rodrigods, not quite 18:19:26 let me go on 18:19:26 and I was very surprizrd to get -2 on the nexd day after we agreed on everything telling the truth 18:19:35 #link https://review.openstack.org/#/c/403866/ 18:19:36 theres a good mailing list thread going on: http://lists.openstack.org/pipermail/openstack-dev/2016-December/108466.html 18:19:37 agrebennikov, that was because we did not think it through 18:19:51 what you are proposing is going to put some very had to debug errors into the system 18:20:01 we have an assumption right now that a token has a single project ID in it 18:20:14 and from that project ID, we deduce the set of roles in a token 18:20:31 now, ignoring all the work that everyone else is doing around this, lets thinkg that through 18:20:35 say we have 2 sites 18:20:47 and we add a project to one, and give the user some role assignemtnes there 18:20:59 then, we used this approach to create the project on the second site 18:21:02 ayoung- I would like to propose a differnt use case. Where in I would have many independent keystones but many central reporting, finance and billing apps that only understand one project_Id I also believe that doing this in the DB would be akin to a hack and the API can and IMHO should support it. Also if we do DB replication and that gets out of sync the user experance is quite horable as well. ;) 18:21:02 o/ (i'm late) 18:21:09 and...we say that tokens issued at one site are valid at the other 18:21:15 which we could do via fernet 18:21:21 but...we screwed up on the second site 18:21:31 and gave the user a different set of roles 18:21:42 now, the token validation response will depend on which keystone you go to 18:21:59 but the tokens will be valid at all service endpoints 18:22:00 ayoung that's was a big part of breton's concern 18:22:13 lbragstad, oh, we totally have breton to thank for seeing this issue 18:22:31 lbragstad: ayoung won't you end up with those error if you choose to create projects with id? 18:22:35 cbits: so you're saying that you trust hand rolled replication over DB replication? 18:22:37 what agrebennikov is asking for (and is reasonable) is a way to sync multiple Keystones. But it is a Hard problem 18:22:40 NP-Hard 18:23:02 stevemar well - you can.. it depends on how you create the role assignments in each site 18:23:09 so with this just remove the "one token for all" usecase 18:23:15 i still don't see why federation is not an option 18:23:16 keys are different 18:23:25 breton, ++ 18:23:30 dstanek: I trust the orchestration layer I will put in place to make this work over DB replication 18:23:32 tokens are only valid in one region 18:23:42 db replacation to 30 sites is a problem 18:23:54 I may only need to place a project in 4 of the 30 18:23:57 agrebennikov, remove the project ID from the equasion and you have the same solution: make sure everything matches between two keystone 18:23:59 s 18:24:05 tokens are meant to be valid only inside a single keystone installation 18:24:17 you must restrict yourself to using only names, no ids, when requesting tokens 18:24:24 In my use case we are talking about > 30 regaions 18:24:41 Names always change. thats why we love id's 18:25:03 cbits, then you need IDS, and complete replication forthe entire data store 18:25:05 cbits: imo partial replication makes it worse. hard to track down bugs and an increased chance of running into collisions 18:26:04 breton, I keep struggling to understand how federation helps over here.... 18:26:12 If my orchestration creates the ID's and we have basic validation at the API point whow wil these independent keystones colide? 18:26:15 but lets move it aside, I'll ask you in person 18:26:46 cbits, and then when someone makes a change at one site outside of your orchestration? 18:26:50 agrebennikov if we're going to talk about federation why no do it here, in the open? 18:26:54 not* 18:26:56 i say we make this a config option disabled by default, and let agrebennikov try it out, if it blows up we deprecate and remove the option -- document the limitations and state it's experimental 18:27:11 the orchestration will have the admin creds. 18:27:18 cbits, and only the admin? 18:27:24 I mean, and only the orchestration? 18:27:27 no other admins? 18:27:30 yes 18:27:31 stevemar: the problem for me is that it will blow up over time not immediately 18:27:53 cbits, then make your databse read only on the regions and do it at the replication level 18:27:54 because in a complex system not only do we need to create the project but we need to also bind it to internal budjets, cmdb, etc 18:28:01 stevemar, its a nightmare 18:28:04 cbits: how do you keep track of used ids and names? 18:28:13 AD Grups are used. 18:28:20 no UID's in openstack. 18:28:22 just groups 18:28:36 cbits: and that is replicated (not at the API layer) 18:28:53 theres a good argument between trying to replicate and sync at the various layers 18:29:25 no, If say project X needs access to site 1, 2, and 3 we will simply add the project and role (with ad group there) Ad is centeralized in the company 18:29:33 mfisch had a nice reply here: http://lists.openstack.org/pipermail/openstack-dev/2016-December/108499.html 18:29:49 agrebennikov: federation won't completely help until shadow mapping exists 18:30:05 DB replication works well with 3 or 4 sites. it gets really nasty when you have lots of sites. 18:30:06 stevemar, this is one of the sites that was using Assignment in LDAP 18:30:13 dstanek: yes it will 18:30:25 We only use AD for auth and autz 18:30:29 even that... federation is for something else. I don't see any application of federation in my usecase 18:30:32 user / password and group membership 18:30:36 breton: there's still no way to create projects right? 18:30:52 neither projects nor groups 18:30:57 if I'm not mistaken 18:31:00 dstanek, I think what he's saying is they want to have the same kind of behavior with LDAP as you are proposing via Federation: 18:31:05 agrebennikov: federation is how separate keystone installations collaborate 18:31:07 I would love to have hte project_ids be unique per site. But I live in a wold where the cloud is intergrated with many systems. That does not chage over night. 18:31:21 first time you hit a Keystone, create the resources there. But they want to make sure the resources match multi-site 18:31:41 dstanek: projects will be created the same way agrebennikov is going to do it now. He suggests to create 10 projects, each in every location 18:31:46 ayoung: isn't that the goal of shadow mapping? 18:31:51 dstanek: just with static id 18:31:54 Just the project_id needs to be the same so reporting and other systems can see it as one thing. 18:32:12 dstanek, is that sufficient, though? It is also the "create the assignment on the fly" part 18:32:21 not because its cool. but because other systems that will use that data and intergrations are not ready for project_id[] 18:32:40 dstanek: and then manually create assignments in each location 18:32:51 cbits, what you are saying, I think, is you need a way to correlate projects between multiple deployments? 18:32:54 dstanek: which can already be done with federation, but cleaner and more predictable 18:33:22 ayoung: ++ 18:33:24 agrebennikov: cbits have you guys tried this already? do you have results? can you use a custom interface? 18:33:37 they also tried K2K and found it too slow 18:33:52 ayoung: optimize it ? 18:33:53 Multi-site is going to be a real issue 18:34:02 yes for many products, OpenBook, ceilometer, custom CMDB, etc. 18:34:03 samueldmq, perhaps. 18:34:06 how slow is too slow? 18:34:18 breton, "16 times" was the agrebennikov quote 18:34:29 I have been working with agrebennikov: to find a good way to do this. That is why I like his patch 18:34:30 ayoung: the assignments should also be part of that or automatically creating a project is useless 18:34:32 but..again, not a complete solution anyway 18:34:46 cbits, it seems to me that what you 18:35:09 16x for obtaining the initial token? 18:35:12 need is a way to syncronize the remote sites and perform "eventually consistent" semantics 18:35:19 dstanek, ++ 18:35:28 wow 18:35:46 please, lets remove my emotional paragraph from the topic please :/ 18:35:47 IE. I usually work with region A but today I am working with region B and I need everything over there to mirror Region A for me 18:36:53 dstanek: I agree with you. user_id must be the same too. assignments must be the same too. 18:36:55 agrebennikov: if you're referring to the etherpad, you can edit that. 18:37:01 cbits, are new projects and assignments only every going to be written via your orchestration tool and in a centralized location, and then you need to sync them down to the remote Keystones? 18:37:12 I mean 16x 18:37:20 ayoung: yes 18:37:39 agrebennikov: cbits what's preventing you from creating a custom interface for create_project? 18:37:48 cbits, ok...so it sounds like what you need is along these lines: 18:38:00 1. disallow local writes on a certain set of tables: 18:38:07 projects, assignements, to start 18:38:27 i can't see this landing upstream given the concerns, it's way too contentious. but i don't want to stop you from attempting it yourself 18:38:41 we keep going around in circles 18:38:55 agrebennikov did you experience a 16x performance degradation in your testing? 18:39:03 2. push data from a central orchestration hub (say a keystone server, but that is limited to just accessfrom the orchestration engine) down to the remote keystones for that data 18:39:13 AD is external and used for authN 18:39:26 stevemar: agreed... it would be nice to come back with some experience on this being deployed and discuss again in the future 18:39:30 stevemar: is the ML discussion still flowing ? 18:39:37 samueldmq yeah 18:39:43 With the exception of Token data (which now should be ephemeral) I think you can get away with a completely read-only Keystone everywhere but in the auth Hub 18:39:54 will not allow for self service of anything, though 18:40:02 agrebennikov: how often did you need to do the operation that took 16x? 18:40:07 lbragstad: stevemar so perhaps that's the right way to continue the discussion 18:40:16 We can patch the code, thats easy. however it seems that an optional pram that is validated seems resonable. 18:40:17 dstanek, he was just saying K2K took a lot longer than getting a token 18:40:23 and I don't think it solves his needs anyway 18:40:26 I did not of course. But keeping in mind you always have to make inter-services calls between different geo locations, and since I tried it before with one of the external auth system ldap/federation manner I can say for sure that it's way slower 18:40:54 agrebennikov so 16x is arbirtrary 18:41:05 cbits: unfortunately not :( 18:41:24 cbits: once we open up the possibility, others will start using it 18:41:38 yes, I get where you are coming from. 18:41:45 yes sir. also another usecase which I described yesterday and nobody pointed an attention - replicated images should be always replicated across zoned into the same projectIDs 18:41:49 I was hoping to solve my use case and not carry a patch. 18:41:51 and this is not a problem of glance 18:41:54 ayoung: yeah, i was just trying to get a handle on how slow it actually was 18:42:35 cbits: i think this is one of the few instances where i'd advocate that you folks try it out first and pass along any results you come up with 18:42:45 agrebennikov, I really think you need to do this at the database level. Sorry. There is just too much of a data integrity problem otherwise. It is more than just projects. 18:43:03 ayoung: + 18:43:17 agrebennikov: i don't know anything about glance, but from the thread i thought that the image ids were different 18:43:44 i meant to follow up with sigmavirus on that 18:44:07 nope. replicator needs to make sure that whenever the image appears in regionA - it moves it to the regionB. Obviously it may only rely on the project 18:44:48 OK...I don't think we are moving ahead with this one. Sorry folks. 18:44:52 dstanek, and obviously not the project name 18:45:04 ayoung, got for it.... :( 18:45:07 yeah, we're just going around in circles at this point 18:45:07 agrebennikov: but you're not comparing apples to apples. if they don't guarantee the ID and we are saying we can't then you already have that ability 18:45:09 :\ 18:45:29 agreed, and still have 3 topics :( 18:45:32 Thanks for your help. 18:45:38 think it's better to continue on the ML 18:45:40 and review 18:45:41 sorry agrebennikov and cbits 18:45:54 for sure we will. Thanks everyone 18:46:09 agrebennikov: cbits thanks 18:46:19 Last up ... RBAC 18:46:32 ayoung: lol, thats not even on the agenda :P 18:46:40 stevemar, yes it is 18:46:45 ehehe 18:46:51 oh..last weeks 18:46:54 ayoung: ^_^ 18:47:05 it's on last week's agenda 18:47:13 quick vote on whether we think it can go in? RBAC in MIddleware [ayoung] 18:47:13 https://review.openstack.org/#/c/391624/ 18:47:21 since we are talking specs 18:47:38 i need to look at the latest revision 18:47:38 i haven't looked into it enough, abstaining 18:47:44 If the group opinion is to support it, I'll push. If not, I'll back off. 18:47:53 biggest difference is naming 18:48:21 instead of URL_PAttern, calling the main entity: access_rule 18:48:51 making role checks in middleware is technically possible and I like it 18:48:55 also added in a default scheme 18:49:00 not sure about the steps to get there. like splitting the policy etc 18:49:08 { pattern: "ANY", verb: "ANY" role: ""} 18:49:09 ayoung: I also owe you a review 18:49:29 times running out before spec freeze. I think this moves us ahead a lot. 18:49:51 It is based on, quite simply, years of discussions, trial, and feedback 18:49:59 ayoung: so you'll have a bunch of predefined responses for URL patterns ? 18:50:20 stevemar, I don't quite understand the question 18:50:25 responses? 18:50:31 10 minute mark 18:50:47 ayoung: i'll talk to you about it after the meeting 18:50:49 we can figure out what the URL patterns them selves are from the api-regs 18:50:50 refs 18:50:53 ok... 18:51:03 I'll stick around. surrendering the conch 18:51:09 ayoung: does it define a brand new syntax for the new RBAC-only policy for the middleware ? 18:51:14 ok, later on -keystone 18:51:37 ayoung: it looks decent though, and isolated 18:51:47 yes...like the routes in keuystone: 'pattern': '/v2/images/{image_id}/reactivate', 18:51:53 ayoung: i'm just looking for stuff that doesn't touch critical paths at this point 18:52:13 we can default just about everything to "Member" with no loss of security, too 18:52:42 spilla: around? 18:52:50 yup 18:52:59 ayoung: switching to spilla's topic 18:53:07 gotta give the new folks the floor! 18:53:14 #topic PCI-DSS Query Password Expired Users 18:53:33 spilla: sounds like you're asking advice on an implementation detail 18:53:41 correct 18:53:53 I like /v3/users?password_expires_at=gt:{timestamp}, which is what was in the spec and is consistent with the guidelines: 18:53:53 http://specs.openstack.org/openstack/api-wg/guidelines/pagination_filter_sort.html#filtering 18:54:38 That will work 18:54:44 any driving reason not to do it? 18:54:48 rderose: well the gt and lt stuff is more for quantity i think? 18:54:59 I am not sure filtering at an exact timestamp is useful for anyone 18:55:11 /v3/users?password_expires_at={timestamp} 18:55:16 /users?pasword_expires_after={some time stamp} seems a bit clearer 18:55:21 ah... 18:55:25 the reason I went with /v3/users?password_expires_before={timestamp} was to make it easier to under stand if you start chaining them, or even in general 18:55:36 but that doesn't make it more right or wrong 18:56:01 at the API level, let us implement the set of them 18:56:01 wanted some input to head the best direction 18:56:21 if there is a rationale for each, we can give the choice 18:56:23 and we can combine them, right ? with ?password_expires_before={timestamp1}&password_expires_after={timestamp0} 18:56:29 ++ 18:56:36 that makes sense 18:56:49 expires AT seems low value, but if there is a need for it, include it as well 18:57:05 ayoung ++ both is always an option 18:57:08 again, making a perfect match ?password_expires={timestamp} does not make sense to me 18:57:17 this reminds me of http://developer.openstack.org/api-ref/compute/?expanded=list-servers-detailed-detail#list-servers-detailed 18:57:19 the gt: is what the api-wg is recommenting 18:57:19 not sure if anyone agree with me on that 18:57:20 I just think lt gt solves it and it's consistent with other filtering 18:57:31 dstanek: but that's more for quantities 18:57:41 stevemar: no according to their examples 18:57:52 spilla, that enough to go on? 18:57:53 stevemar: no 18:57:57 "GET /app/items?finished_at=ge:15:30&finished_at=lt:16:00" 18:58:07 dstanek: ++ 18:58:26 oh did i miss that? 18:58:36 http://specs.openstack.org/openstack/api-wg/guidelines/pagination_filter_sort.html#time-based-filtering-queries 18:58:46 dstanek i like that 18:58:52 oh there it is: GET /app/items?finished_at=ge:15:30&finished_at=lt:16:00 18:59:02 i assume dates would be handled the same way, but that's not explicit 18:59:06 spilla: there's your answer ^ 18:59:11 so gt lt seems to be it 18:59:24 yessir 18:59:27 wrapping up! 18:59:30 #endmeeting