18:01:32 #startmeeting keystone 18:01:32 Meeting started Tue Jan 28 18:01:32 2014 UTC and is due to finish in 60 minutes. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:33 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:33 o/ 18:01:36 #topic Reminder: Icehouse feature proposal freeze February 18th (features must be in code review) 18:01:36 The meeting name has been set to 'keystone' 18:01:38 i was zoned out reading an article, would have missed the meeting w/o the ping 18:02:05 similar to the havana cycle, but with more advance notice, so the date is set a week earlier than havana-m3 :) 18:02:18 dolphm, I thought Feb 18 was for blueprints? 18:02:38 topol, bug fixes and non-feature work will continue afaict. 18:02:40 topol: ++ ? 18:02:43 topol, code needs to be submitted for review 18:02:44 morganfainberg: ++ 18:02:54 ayoung: ++; bp's not in Code Review will be bumped to juno-m1 18:02:58 topol, code need to be able to compile 18:03:08 k code submitted by Feb 18th. got it 18:03:08 gyee: it needs to be feature complete 18:03:19 gyee: half baked features that compile will be bumped ;) 18:03:25 ah 18:03:32 submit a change "TODO: implement the feature" ... then we -1 it 18:03:48 things still heavily in flux near the deadline should be bumped as well, but we can go case by case if necessary 18:04:02 bknudson: or -2 in this case 18:04:18 So after Feb 18...start working on your features for Juno 18:04:22 similarly, feature freeze is March 4th -- the actual merge deadline 18:04:32 so that gives us two weeks to fight the gate 18:04:43 ayoung: ++ 18:04:45 hopefully we will spend some time fixing bugs 18:04:52 and maybe improving logging? 18:04:53 "I fought the gate and the gate won." 18:04:57 dolphm, whats the relase date. Is it in April or May? 18:04:58 dolphm, thats nice, gives a couple weeks to massage code/argue about any details (not-feature specifc), pass gate 18:05:04 Gate always wins 18:05:14 topol: April 17th https://wiki.openstack.org/wiki/Icehouse_Release_Schedule 18:05:18 cancel your valentine party people, code freeze on the 18th! 18:05:31 gyee, heh 18:05:54 any questiosn? 18:06:00 Nope 18:06:00 St. Valentines day Massacre! 18:06:05 or questions, depending your spelling ability 18:06:18 #topic Hackathon retrospective 18:06:27 Sorry wife, dolphm, says I have to spend valentines day night coding. No fancy dinner 18:06:28 dolphm, so for extensions, spec can still get in after I2 right? 18:06:30 i don't want to spend too long on this, given that we have two more interesting topics on the agenda as well 18:06:42 Hackathon was awesome. Learned a lot 18:06:56 gyee, externsion, defaulting off, can go in post I2, yes 18:07:02 sounds like everyone wants to make this a regular thing 18:07:07 dolphm, ++ 18:07:09 Yup 18:07:15 it was great. 18:07:15 ayoung, cool 18:07:19 in SA, in July! 18:07:23 Um...wait 18:07:24 dolphm, ++ 18:07:45 dolphm: ++ 18:07:54 Aw, hell ,we wnever really got outside durnig the day anyway, SA in July! 18:07:57 but -- unless someone wants to jump in and host, i'm going to go ahead and start planning for same time (week before end of juno-m2)? 18:08:11 we like going to San Antonio 18:08:18 topol: easy enough for me 18:08:26 dolphm, that sounds like a good plan. 18:08:32 Enjoyed the free m&ms 18:08:40 i'm going to look into Geekdom vs using the Customer Experience Center again -- geekdom has the advantage of no transportation necessary 18:08:53 hotel, food, and workspace will all be walking distance 18:09:02 that'd be cool 18:09:06 and entertainment? 18:09:07 dolphm: geekdom would be great 18:09:20 dolphm, yeah, geekdom sounds like a better option overall (not that the CEC was bad) 18:09:29 what's geekdome ? 18:09:29 topol: i'll bring you a bag of m&m's next time 18:09:36 http://www.geekdom.com/san-antonio 18:09:39 dolphm: claco says it's moving about 3 blocks away from the Valencia 18:09:43 #link http://www.geekdom.com/ 18:10:07 * topol had my physical when I got back from San Antonio. Thanks to all the great food dr put me on a diet 18:10:26 topol, heh 18:10:42 topol: there's your excuse for v-day 18:10:57 jamielennox, put in your request now for the mid-Juno Hackathon 18:11:36 getting dates nailed down would be good so we can start working on approvals as well 18:11:47 ayoung: heh, still got to put in for summit 18:11:56 topol, need to know the Juno schedule 18:11:58 guessing based on the release schedule, i'm going to tentatively say it'll be the last week in july 18:11:59 topol, likely specific dates can't happen until ATL 18:12:15 but, we can give a estimate window and nail it down right after the summit 18:12:23 dolphm, ooh...good to know. Much later than that and I will have a conflict 18:12:31 but there won't be a final release schedule (or hackathon date) until the Juno release schedule is chosen at the summit in atlanta 18:12:59 k thats fine 18:13:16 topol: say, plus or minus a week at this point 18:13:36 dolphm, it almost seems like early in Juno 2 would have the biggest impact, we don't need to do it right before API freeze 18:13:58 ayoung: that's also VERY close to the summit (within a couple weeks) 18:14:00 ayoung, great idea, start with a bang 18:14:06 ayoung, but aiming too early means we wouldn't have had some of the federation work/convos 18:14:15 ayoung, and usually m1 is short compared to m2 18:14:22 ayoung: i also found the first day of the summit to be the most productive -- focusing on landing already-implemented code 18:14:39 dolphm ++ 18:14:48 m1 is halfway over by the time we leave the summit 18:15:06 i mean, maybe we aim for 2wk before m2 deadline? 18:15:06 feels like it, anyway 18:15:14 instead of the week before? 18:15:27 I think having 2 weeks afterwards would be better. 18:15:35 we try to shove too many big changes into the last week. 18:15:42 a little more room to get things done after hackathon vs. rushing 18:15:49 Summit ends May 16. July 18 is 2 months after. I think that would be a good planning number 18:15:57 bknudson: ++ more time to review and iterate 18:16:05 16-18? 18:16:14 i'm good with that. 18:17:06 that means we'll have more code to write, less code to land, and likely less discussion overall? 18:17:40 dolphm, i don't think 1 week will change things massively. 18:17:42 I find it hard to believe this group would have less discussion. 18:17:47 I guess the real question is the purpose of the hackaton 18:17:54 if it is to plan it should be 2 weeks 18:17:55 are we having a discussion about having discussions? 18:18:05 if it is to deliver, probably dolphm is right 18:18:10 fabiog, to push along the essential features prior to API freeze 18:18:11 joesavak, nah, discussion about discussions in meetings 18:18:32 IOK...we have fun things to talk about for realz 18:19:20 i'm definitely happy to try a different week -- i was just personally happy with the timing in icehouse 18:19:25 seems like a lot of the code written at the summit would have been written at the summit no matter when it was. 18:19:36 * topol Next hackathon plz dont ditch me when I try and meet up at the mexican restaurant.... uh they just left... 18:19:42 i think there is opportunity for discussing the design of features as they are being written 18:19:46 topol: you didn't show 18:20:17 next time I'll take a selfie in front of the restaurant with my watch 18:20:17 Speaking of Summit...do we know hotels etc yet? 18:20:38 ayoung, don't think so 18:20:48 I'm assuming the summit will be at the convention center in atl. 18:21:01 #action dolphm to aim for 2 weeks before juno-m2 deadline 18:21:35 ayoung: http://www.openstack.org/summit/openstack-summit-atlanta-2014/ 18:21:53 Georgia world congress center 18:22:01 no hotel announcement yet, but it says to expect it in january *shrug* 18:22:14 "We'll be posting links to the call for speakers, registration, and discounted hotel room block rates in January so please follow the OpenStack blog and twitter for more updates. " 18:22:26 Federation? 18:23:01 ayoung: ++ 18:23:05 yes please 18:23:11 * dolphm sidetracked 18:23:12 #topic Federation with Apache 18:23:26 #link http://lists.openstack.org/pipermail/openstack-dev/2014-January/025570.html 18:23:27 #link http://lists.openstack.org/pipermail/openstack-dev/2014-January/025570.html - i posted this yesterday 18:23:32 dolphm: ++ 18:23:38 marekd, /OS-FEDERATION/tokens/identity_provider/(.*?)/protocol/(.*?) works ? That is kool 18:23:50 ayoung: yes and no :) 18:23:58 what doesn't work? 18:24:19 marekd: from your email, i was a little lost on what didn't work / what you were asking from the community? 18:24:49 ayoung: if i hit that URL apache matches the URL and fires the wsgi url, but it stores PATH_INFO to "/" and this is what kestone (routes in particular) try to match as a URL. 18:25:11 marekd: at the end of your email, you mentioned that "the initial input is lost" -- what input are you referring to? 18:25:12 dolphm: it was my idea, i was expecting to get some comments, maybe questions or just general approval. 18:26:02 but it stores PATH_INFO to "/" what does that mean? That you lose all of the /OS-FEDERATION/tokens/identity_provider/ URL? 18:26:03 dolphm: i thought we could use a static URL, instead of dynamic with IdP name and used protocol, but then you would have to somehow send what IdP and what proto you are going to use - this could be normally done in a HTTP POST, right? 18:26:25 is it just the POST operation to get a token ? (like /v3/auth/tokens) 18:26:32 oooh, i think i understand 18:26:41 ayoung: yes. when you hit keystone it tries to match urls with the value from env variable PATH_INFO. 18:26:57 ayoung: when you NORMALLY hit keystone with your request. 18:27:08 sorry guys, i am having too many questions now.... 18:27:09 is there an apache config option to pass the whole path to mod_wsgi? 18:27:16 why isn't the full URL in PATH_INFO? 18:27:30 this is how apache sets that env var. 18:27:33 marekd, dstanek OK, something to research 18:27:46 dstanek: apache is serving an app at the full path, so it discards it? 18:28:03 lets assume it is a solvable problem and move on 18:28:14 ayoung: well, it's a major blocker if it's not 18:28:31 marekd: send me the apache config you are using when you have a chance - i'd love to poke at this 18:28:32 but it's got to be solvable -- anyone with deep httpd experience want to lend marekd a hand? :D 18:28:41 yay 18:28:51 #action dstanek to help marekd with httpd + mod_shib config 18:28:55 dolphm: when i configure my apache to match path say.... /a/b and then hit it with /a/b/c/d the apache will match it correctly, but set PATH_INFO to /c/d. that's why when you run keystone w/ apache you always set WSGIAlias / /path/to/keystone.wsgi 18:29:01 dolphm: cool! 18:29:13 marekd: ++ that's a good explanation 18:29:19 I can pull in some of the FreeIPA type people, too 18:29:20 dstanek: pasting ocnfig. 18:29:28 dstanek: nothing fancy though... 18:29:31 marekd: i'm familiar with this behavior, but i've never had to work around it 18:29:43 post the config for all to see, please 18:30:27 i wonder if httpd exposes the original request URL in another variable somewhere 18:30:38 dolphm: 18:30:38 yes 18:30:41 REQUEST_URI 18:30:46 but it doesn't matter 18:30:56 since you will not even touch right pipeline 18:31:05 i though about making dummy middleware 18:31:05 oh, yikes. 18:31:18 the WSGI binding code can handle that 18:31:20 but it would be something between apache and routes library. 18:31:36 https://github.com/openstack/keystone/blob/master/httpd/keystone.py 18:31:41 http://pasteraw.com/aumfv8jkjt9gabxtf4dp9poojv7rai3 18:31:44 marekd: sounds like the actual federation endpoint needs to be it's own application, rather than in middleware 18:32:16 we can change anything from https://github.com/openstack/keystone/blob/master/httpd/keystone.py on down 18:32:42 ayoung: if you have any idea, just let me know. i have already some workarounds, but not too clean... 18:33:12 marekd, so...this is a detail we can solve. Anything else? 18:33:13 we might need to append a /auth or something to the full federation endpoint to distinguish between CRUD and auth as well 18:33:27 dolphm, that might not be a bad approach 18:33:48 ayoung: if you are fine with my idea, i'd be happy to implement it this week. that's what i actually was expecting. 18:33:54 i will later ping ayoung and dstanek 18:34:01 i think we understands the problem, at least -- enough to follow up after the meeting 18:34:07 marekd, heh, I thought it was *my* idea. So, yeah, I'm happy with it 18:34:11 or in open discussion if we have time 18:34:27 #topic If can't call identity from assignment... 18:34:30 bknudson: o/ 18:34:34 ayoung: okkkk, execution. 18:34:43 marekd, +1 18:34:44 I posted a bunch of places where we call identity from assignment 18:34:45 ayoung: my implemenation..whatevs. 18:34:46 there's a bunch of meeting notes on the agenda for this one https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting 18:34:52 bknudson, those should not be a problem 18:34:56 ayoung: not trying to steal anybodys ideas :P 18:34:59 and what we can't do if we can't call identity 18:35:00 it is OK to call assignment from identity 18:35:17 so move them into identity? 18:35:26 marekd, nah, you are rocking hard. I love what you are doing, and there are a lot of details to dust up here. Keep on keeping on 18:35:50 ayoung: correction... 18:35:50 bknudson, list X for users is do-able from the identity side, assuming we know the groups for that user. 18:35:52 ayoung: :-) 18:36:03 it's okay to call identity from assignment (and vice versa) at the controller layer 18:36:08 bknudson, list_projects_for user should be doable w/o calling ideneity 18:36:18 at the driver layer especially it's entirely not acceptable 18:36:20 It is its the cases where we have never seen the user before, and do not know their groups that will bne problematic\ 18:36:20 what's the difference between controller layer and backend? 18:36:34 dolphm, maanger or controller? 18:37:08 regardless of where the logic lives, the problem is the data 18:37:30 if we don't know about the user, for example, a federation use case where we've never seen the user before, list projects for user is going to come up with Bupkis 18:38:01 morganfainberg: managers are a grey area IMO :-/ i can go both ways 18:38:16 with Federation, always use groups, using user is asking for trouble 18:38:21 always map user to a group 18:38:32 So...those APIs are going to be useless. Now, more interesting is that we can still do get_roles_for_group 18:38:34 dolphm, i think cross-system talk should be manager -- regardless. 18:38:39 gyee, ++ 18:38:54 but groups are coming out of the identity backend, too 18:38:56 gyee++ 18:38:59 morganfainberg: that's fine -- but let's do it more consistently then 18:39:07 we don't have groups in Assignment, and I think that is OK 18:39:08 OpenStack should really think about the persona approach 18:39:14 for now...someone will want them eventually 18:39:17 how to come up with pre-canned groups 18:39:26 dolphm, sure, that was that business logic refactor i did earlier on that allowed us to break v2/v3 controller interdeps 18:39:31 dolphm, anyway..back on topic here 18:39:44 dolphm, i'll continue with that cleanup as we go on. 18:39:48 morganfainberg: ++ 18:39:59 totally forgot about that :) 18:40:48 are there going to be some operations that are ok to call on identity backend? 18:40:57 like list_groups_for_user? 18:40:57 bknudson: from where? 18:41:12 i think groups_for_user is a hard one short of caching. 18:41:24 dolphm: they could be called via REST API or they could be called from some other backend. 18:41:25 We need to rework the token provider code into a pipeline. I think that some of the ugliness that bknudson is tripping over comes from doing Identity calls in that token provider, and they need to be early on in the pipeline to clean up the distinction 18:41:44 bknudson: ? backends should not cross-talk with other backends 18:41:50 bknudson: but we're saying that managers can 18:41:52 bknudson, only for the "embedded IdP" 18:41:55 bknudson: does that clarify things? 18:42:06 ayoung: out of scope here? 18:42:12 so, if a domain is stored in the Keytone Identity backend, then "list_groups_for_users" is OK 18:42:13 dolphm: ok, this is more of an architectural decision 18:42:29 but for federation, it will not work. 18:42:34 but if I can call identity from assignment then I could just move the current calls in the backends up to the manager. 18:42:39 bknudson: agree; but does that distinction make sense / answer your question? 18:42:46 so the keystone server could continue to validate that the user exists, for example 18:42:56 bknudson: basically, yes 18:43:11 well, what's missing is that keystone can't tell if a user exists anymore or not 18:43:13 assignment shouldn't care if the user exists for those calls, it could... in general just report the state of the mapping, right? 18:43:21 do you have to define all users in keystone with federation? 18:43:34 bknudson: no 18:43:48 bknudson, i don't see a need to validate existence of a user. in fact i think we discussed at one point exactly that, assignment doesn't care 18:43:50 it just maps 18:43:51 bknudson: users don't exist in keystone with federation 18:44:04 morganfainberg: ++ 18:44:11 do the group assignments exist in keystone? 18:44:20 group assignments yes 18:44:22 it's a bit of a special case, architecturally assignment could try to validate users, but we don't want it to 18:44:26 if the group assignments don't exist, then self.identity_api.list_groups_for_user(user_id) doesn't work. 18:44:53 so you can create a group and add a bunch of non-existant user-ids to it? 18:45:02 yea, list the direct assignments for a user id, or, if you know the groups, list the assignments by way of the group_ids. That is completely within the Assigment backend. But list roles for user assumes that you do the lookup by way of the groups 18:45:30 bknudson: that we might need to refactor a bit, or it won't/shouldn't be in the code path for federated users, and the mapping engine will output a list of groups to be used 18:46:15 identity backend also does self.identity_api.list_users_in_group(group_id,domain_id) -- so if we have groups in keystone then that would work find. 18:46:40 oops, first identity there should have been assignment... 18:46:46 it's used in delete_grant. 18:46:46 bknudson: that's not a call that makes sense for federation either 18:46:56 there are no users to get 18:47:00 correct 18:47:07 so, it would be an empty list, at best 18:47:18 ok, so we're going to keep the call in there. 18:47:24 bknudson: if that's all it's used for, it'll be replaced with a revocation event 18:47:29 grants are always made to a single entity. Delete grant should be no problem 18:47:31 looks like for federation we won't be able to revoke tokens (by list) 18:47:49 bknudson, federation likely needs revocation events 18:47:50 bknudson: correct 18:47:50 no reason to confirm tha the entity exists. If it doesn't, the grant is just latent and bogus 18:47:51 tokens revocation would probably have to use ayoung's events. 18:47:56 :) 18:48:04 it's all coming together. 18:48:05 bknudson: ++ 18:48:07 :D 18:48:16 :) 18:48:17 All my plans are coming together.... 18:48:26 ayoung: i just hope the implementation does! 18:48:31 ok, you've given me enough to think about. 18:48:33 cool 18:48:38 #topic Possibly make domain role inheritance default/non-optional 18:48:39 morganfainberg: o/ 18:48:42 death or glory is where we are at :-) 18:48:47 so, i was talkign to nova folks 18:48:47 dolphm, it might possibly be the coolest code I have written 18:48:50 this sounds like an API backwards compatibility? 18:48:54 incompatibility* 18:49:10 and they want to do some heirachical work e.g. instances are domain aware based on projects 18:49:26 so a domain admin can look at other project instance lists w/o rescoping 18:49:39 is it possible then for a latent grant to be hanging around and then some entity does exist with that ID and gets the grant by chance? 18:49:55 morganfainberg: that breaks the domain / project abstraction 18:50:12 dolphm, currently they have API calls to list instances across all projects 18:50:18 dstanek, "chance"? 18:50:25 dstanek: assuming uuid's, the chance is low 18:50:31 Well, that is why I want the Id scheme that we discussed at the hackathon 18:50:39 dstanek: that's something to think about before pursuing user-defined ID's system-wide 18:50:41 morganfainberg, isn't that inherited roles are for? I am lost 18:50:43 dolphm, and they want to include domains, but a rescope seems silly just to access the instances (internal to nova) 18:50:53 user_id needs to be one part from Keystone (domain id) and one part from the IdP 18:50:53 morganfainberg: i'm not following the need 18:51:09 dolphm: if there is even a chance we should probably document it somewhere 18:51:26 so basically, to do domain-wide actions in nova, the thought was to assert against a known domain role 18:51:40 that would be inherited to all projects so you know it's there 18:51:43 e.g. "domain admin" 18:51:52 so if I grant a role to userid="abc123@456fed" it will only ever possibly match other users from domain_id 456fed 18:51:56 rahter than having to have domain admin assigned in each and every project 18:52:31 why do you need domain admin? 18:52:35 morganfainberg: "list all instances regardless of project" is a use case for "admin" API / service-scoped tokens, IMO -- it's inherently not a multi-tenant or cross-tenant call at all 18:52:35 the crux is that w/o OS-INHERIT becoming a default (or non-optional) you can't guarantee a role like that could be created 18:52:46 as projects are owned by a domain doesn't a domain scoped token let you access them anyway 18:53:13 (guess, never tried that) 18:53:19 dolphm, ok, what about list all instances in a domain 18:53:21 jamielennox: not by default 18:53:26 i'm a domain admin, not a cloud admin 18:53:34 i want to know everything goin on in my domain 18:53:49 not all of nova either 18:53:52 are they going to have somethink like GET v3/instances/{domain_id} ? 18:53:59 morganfainberg: then i think you need to follow the basic rules of multitenancy 18:54:04 bknudson, they are just starting conversations about it 18:54:19 GET /v3/instances?domain_id=xxx 18:54:30 dolphm, ok so the answer is.. you can't do it? 18:54:31 seems more like a container. 18:54:57 it's about knowing what makes someone a domain admin 18:55:03 morganfainberg, is there a wiki/bp where the use cases are documented? 18:55:06 morganfainberg: well you can, but i think the client should be generating a bunch of tokens (or you need v4 multi-project tokens) 18:55:26 dolphm, i think we're going to get hammerd on this one then 18:55:27 dolphm, tbh. 18:55:44 dolphm, but thats fine i brought it up to see our view. 18:56:14 dolphm, i think we can solve this with a token version change (not api change) but token versions independant of api versions will be Juno at best 18:56:40 morganfainberg: if you're adding a field that doesn't require a new version? 18:56:45 operations are enforced at the point of contact. If you want a new policy on Nova that says "project must be in a domain X and user has role R on daomin X" you can do that 18:56:46 morganfainberg: i think multitenancy is a poorly understood problem :( everyone wants it at face value, and then doesn't understand the implications... 18:57:05 dolphm, fair enough im happy to bring that argument back to them. 18:57:09 We can't treat policy as something fixed and jump through hoops to do all of the magic in Keystone 18:57:35 morganfainberg, that make sense? 18:57:39 ayoung, sure. 18:57:42 if nova has the right resources they should be able to do this with policy 18:57:47 ++ 18:57:49 morganfainberg: let's map out the user cases and options 18:57:50 bknudson, but they don't 18:57:50 ayoung: my argument is that nova has no reason to be domain aware -- domains are not providing multi-tenancy to nova... projects are. 18:58:32 dolphm: ++ 18:58:33 dolphm, actually, domains as constructed are poorly understood and imply a secondary level of multi-tenancy 18:58:37 * ayoung gobsmacked 18:58:44 domains effectively provide multitenancy to keystone, quotas, billing, etc 18:59:14 (time is about up) 18:59:15 If I had known that Nova was not planning on supporting domains, I might have pushed back harder on them 18:59:16 ok, so what if nova wants to do domain quotas now as well? 18:59:23 but as a project is registered in a domain there should be certain things that a domain scoped token can do to projects 18:59:56 ayoung: it sounds like they do -- and i'm sort of in favor of an actual quota service to solve that problem 18:59:56 a domain = customer, and domain can have 1000 instances regardless of number of projects 18:59:58 jamielennox, assuming Nova can match the project to a domain. If not, we have to do the majik 18:59:58 (but i can't see why nova needs to know any of them) 19:00:29 we can circle back up on this in -dev and discuss outside of the meeting before i go back and chat w/ nova folks 19:00:58 ++ 19:01:00 #endmeeting