18:03:13 #startmeeting keystone 18:03:14 Meeting started Tue Jul 7 18:03:13 2015 UTC and is due to finish in 60 minutes. The chair is stevemar. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:03:15 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:03:17 The meeting name has been set to 'keystone' 18:03:23 #link https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting 18:03:32 Means in can be the peanut gallery today :P 18:03:38 I* 18:03:49 morganfainberg: don't heckle too much 18:04:00 the agenda is small today 18:04:23 #topic Start restricting "/" in project names in Liberty 18:04:29 htruta_: rodrigods ^ 18:04:30 +2A 18:04:31 next 18:04:35 lol 18:04:37 ayoung: lol 18:04:39 lol 18:04:49 so, morganfainberg asked us to send an email to ml 18:04:49 yeah, i can't image there is much push back on this one 18:04:53 they are already restricted 18:04:57 including operators 18:05:02 since we build the URL out of them, they will break 18:05:03 Fwiw I like this plan. But i dont want to break people (operators) 18:05:04 no one answered 18:05:20 ayoung: we dont do urls by project name anywhere afaik 18:05:22 not everybody reads the mailing list 18:05:34 morganfainberg, not it Keystone, just everywhere else 18:05:39 oh, wait, no you are right 18:05:40 the only guy who answered said "Do you mean project names or project IDs?" 18:05:40 that is ID 18:05:45 heh 18:05:47 Crud 18:05:49 there is no better broadcast AFAIK, bknudson 18:05:55 Just do it 18:06:04 anyone putting slashes in there is sick in the head anyway 18:06:06 we tried to give them voice, bknudson 18:06:26 We could probably just escape the / in the names (with a migration) and make clients smart about handing escaped variables 18:06:32 And/or horizon 18:06:39 morganfainberg, ++ 18:06:42 morganfainberg: that's not very restful 18:06:50 htruta_: rodrigods this is for displaying heirarchy? 18:07:00 dstanek: we still dont do anything by project name in urls 18:07:09 stevemar, this is for passing hierarchy in the future 18:07:11 with the slash 18:07:26 htruta_, ++ 18:07:26 in token body, env variables 18:07:28 new rule. All names must be URL safe 18:07:30 dstanek: not very rest impacting 18:07:41 ayoung ++ 18:07:50 ayoung: base64 encode all the things /s 18:07:50 all names must be english 18:07:59 NONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONONO 18:08:08 if we're not talking about the URL then we should be fine and don't need any encoding 18:08:13 let me find the technical term 18:08:23 ayoung: awesome? 18:08:26 ayoung: url safe isnt good for utf-8 unless urlencoded. 18:08:31 Just fyi 18:08:38 wouldn't a global delimiter work better? and then check if the name includes the delimiter? 18:08:49 https://tools.ietf.org/html/rfc3986#page-10 18:08:55 dstanek: we need to expose a get-project-by name api. But that is something we can handle. 18:09:02 the slash would be it, stevemar 18:09:36 morganfainberg: hmmm....i think you can actually match a '/' in the routes regex 18:09:36 let's use slash 18:09:39 linux uses it 18:09:46 "All names must pass through URL encoding unchanged" maybe? 18:09:46 i think swift uses it too 18:09:52 everybody likes slash: http://bigread.mojo4music.com/2014/01/slash/img/slash-portrait.jpg 18:09:54 Ok so i think a combo of two things solves the issue. 18:10:01 (sorry, I had to do this) 18:10:03 htruta_: ++ 18:10:03 This is from two proposals 18:10:12 htruta_: ++ 18:10:13 Slash is default 18:10:23 Always preceed the hierarchy with the delimiter 18:10:33 so 1st character is the delimiter 18:10:39 Slash is default. Clapton is God 18:10:43 We really do need a global standard for the delimiter. Old school Path Separator worked ok on an isolated machine, broke with distributed file systems 18:10:44 morganfainberg, beautiful 18:10:45 ayoung: +++ 18:10:48 And the delimiter can be configured if needee 18:10:50 ayoung ++ 18:10:53 ayoung: fact 18:11:18 "first character is delimiter" is guranteed to confuse when someone gets it wrong 18:11:35 geoffarnold: we cant arbitrarily restrict a delimiter from 18:11:36 so we have this new config that says the "prohibited" char? morganfainberg 18:11:37 Names. 18:11:40 i'm in favor of a configurable delimiter 18:11:42 morganfainberg, hell yes we can 18:11:55 if we use the first character will all project names from now on need to be specified with the delimiter? 18:11:56 i'm anti a configurable delimiter 18:11:58 ayoung: no we cant. We cannot break current deployments. 18:12:13 We should use the ascii bell character instead >.> 18:12:16 it's another switch that makes one deployment different to another for no real benefit 18:12:17 dstanek: no, we'll only pass the parent_id as we do today 18:12:24 please dont break current deployments 18:12:26 "Names must be usable as a segement of URLS. Any Names that do not currently comply are not guarenteed to work with all future APIs" 18:12:45 topol, we don't want to... we sent an email to -dev and -operators 18:12:46 htruta_: how do we know when to use the first character as a delimiter? 18:12:49 if any existing deployments use "/" in a project name, it will break 18:13:01 we can deprecate / in project names and switch in 6 months 18:13:02 names as they exist now will be grandfathered in, but will not work with HMT 18:13:05 dstanek, it is not in the name, but in an extra field 18:13:10 or other, future APIS 18:13:10 to specify the hierarchy 18:13:19 rodrigods ++ 18:13:19 the names remains like the "relative path" 18:13:34 Oh 18:13:36 Wait 18:13:39 deeeerrrp 18:13:42 the "absolute path" would be an extra field 18:13:42 ayoung: yeah, just can't use them as part of HMT 18:13:43 now i'm really confused 18:13:53 Here's a test: how many people ever, anywhere, configured the Unix/Windows path sep to anything other than the default? 18:14:03 In the hierarchy just replace (when construtong) / with \/ 18:14:09 geoffarnold: you can do that? 18:14:13 Backslash slash 18:14:25 geoffarnold, I ofetn configure the Windows one with the Unix one. Breaks all over the place. I do this on systems for people I don't like 18:14:26 It works like posix would 18:14:38 morganfainberg, slashes are by default for HMT too? 18:14:58 So are we going to introduce POSIX-style configurability, or mandate "/" 18:15:11 Just make / the delimiter. But in the heirarchy doing \/ if / is in the name 18:15:15 morganfainberg, oh, right - "projects" 18:15:32 morganfainberg: ++ 18:15:33 Or quote the names. 18:15:37 morganfainberg +++ 18:15:43 Easy 18:15:45 I'm strongly against "first char is the sep". Imagine inadvertantly typing "abracadabra" and getting "a" as the sep 18:15:50 morganfainberg ++ 18:15:52 morganfainberg, ++ 18:16:01 let's be very restrictive here 18:16:09 geoffarnold I agree 18:16:12 ayoung: slash is the delimiter. 18:16:18 "names must pass through URL Encoding unchanged" 18:16:29 +1 18:16:31 ayoung: if slash is in the name, escape it when buildong the hierarchy 18:16:41 That solves the grandfathered in issue 18:16:49 +1000 18:16:56 And doesnt change anything for the operator 18:17:05 morganfainberg, works for me. 18:17:08 so what morganfainberg suggests is easy to understand and follow 18:17:10 morganfainberg: just to make sure, if we already had \/ in the project name, that still works, right ? 18:17:16 morganfainberg, should we change the current project names? or just in M? 18:17:26 samueldmq: you'd end up with \\/ 18:17:35 we should advise that names will be used this way, and should not include characters that will be escaped 18:17:46 rodrigods: talk about being more restrictive in M with more operator feedback 18:17:54 so, in Liberty we'll just warn people? 18:18:00 I want to eliminate slash. But i want to avoid breaking anyone. 18:18:03 I'd like a "safe" mode in CLI and Horizon, rejecting "/" in non-HMT contexts 18:18:04 or will we already make this restriction? 18:18:06 Dont change anything in L 18:18:30 geoffarnold: we cant because everyhing is HMT soon 18:18:35 we'll need identty API v4 to support HMT 18:18:58 I mean any context where a token is meant to represent a node, not a path 18:19:01 So in specifying a hierarchy \/ == / in a prkect name 18:19:17 But onlu in the hierarchy represenataiton 18:19:47 Very similar to filesystem referencing special characters/spaces/etc 18:19:49 practical concern: will our webobjectwhatever support that? 18:20:29 ayoung: since we dont do anything by project name as part of the url, (query string aside) we shpuld be ok 18:20:40 Yet. 18:20:53 morganfainberg, ...lets face it UUIDs are abusive 18:20:54 can't we filter something by the name? 18:20:56 people want names. 18:21:08 ayoung: so in m we talk about being more restrictive 18:21:31 We need more feedback before hard locking that out. Or knowing we need urlencoding 18:21:34 fair enough. Works for me. Although , I would argue we should be restrictive for new resources now... 18:21:46 This isnt really "new though" 18:21:46 ayoung, ++ 18:21:54 but that might put the hurt on upgrades that do a full export-import 18:21:57 And all projects are part of a hierarchy in l 18:22:09 ayoung: that too 18:23:13 Most cases we wont ever hit issues 18:23:26 So let's be clear: what will be in "L" and what in "M"? Is HMT usable in "L" (without APIv4? 18:23:28 so we decided to not do project names are unique in a domain? 18:23:29 But we address where people put slashes into the names. 18:23:45 jamielennox, not yet 18:24:02 geoffarnold: we will not be doing an api v4 in L. And i dont want to talk about thst for M at this point 18:24:09 Meiji! 18:24:15 So is HMT usable in L? 18:24:20 geoffarnold: yes. 18:24:39 明治 18:24:58 That is the plan. The only edge case is: if a project has a / in the name, do \/ in the hierarchy definition. Dont change the name 18:25:03 Think of it like a path 18:25:04 Good 18:25:06 geoffarnold: it's already usable in K, for organizating the projects in a hierarchy (let's say representing a departmental organization, for ex) 18:25:26 OK...anything that needs formal approval for this htruta_ ? 18:25:33 spec or anything 18:25:38 ayoung: just an update to a spec 18:25:55 Unless anyone has concerns with the proposal? 18:26:10 20 minutes on slash, good thing we only have 1 other topic 18:26:11 so the plan will be if i create a new project and i name it "A/B/C", i'll automatically be creating a hierarchy? 18:26:17 morganfainberg: who has to do that escaping? the keystone servers or the clients? 18:26:23 it's not backwards compatible 18:26:27 dstanek: clients 18:26:32 so it violates the stability guidelines 18:26:42 bknudson: we dont require specification of the heirarchy today 18:26:49 So this doesnt break anything 18:26:55 jamielennox: i don't think so 18:27:06 is "A/B/C" absolute or relative? 18:27:07 morganfainberg: is there anyway we can do it on the server side? 18:27:10 The escape is only in the hierarchy representation. 18:27:18 jamielennox, we are not doing this, at least, not now. I think we should restrict using slash in create projects 18:27:25 it should be relative, right? 18:27:27 dstanek: well the server will hand back the hierarchy like that if asked 18:27:45 jamielennox, we need parent_id (not name), a domain and a name 18:27:45 dstanek: but if they havent asked for the hierarchy, the client would need to construct it 18:28:04 It should be relative to the current project scope. 18:28:18 htruta_: create and set 18:28:18 if the server can has here is your project name and the hierarchy under which it sets they wouldn't have to - i don't like the idea of putting this kind of logic in the clients 18:28:54 dstanek: if you have the info from the server its fine. The part that the client needs to know is *if* you dont already have that info 18:29:01 if we are going to support creating a hierarchy, it should be a new API call. like mkdir -P 18:29:04 Same as today. 18:29:29 morganfainberg: can't it ask for it? 18:29:32 If you know your project id, from previous query etc, you can use it 18:29:40 ok, i didn't mean to derail that onto creation but i'm just trying to figure out where you would use this 18:29:47 Depends on auth by name 18:29:57 i worry that as we make changes we can't do it as easily is we have clients that depends on doing the logic themselves 18:29:57 why using a slash is better than listing subprojects of the current project then doing things by id 18:29:59 jamielennox, it's on token request by project name 18:30:00 You may not be able to know it dstanek 18:30:46 Uuid/uuid/uuid is awful and borderline unusable from a ux 18:30:49 htruta_: if i'm authenticating with project name, do i have to supply the whole 'a/b/c' or just 'c' ? 18:31:06 so because we decided we're not going to change token request to use a list rather than a string now we're forced down this path of restricting project names to not have / 18:31:09 stevemar, whole thing if it is ambiguous 18:31:13 that was pretty sneaky 18:31:28 bknudson: we didnt restrict / from project names 18:31:28 stevemar: the whole thing if there is another project with the same name, i think 18:31:47 ayoung: we can't say if it's ambiguous, if it works today where name is unique and suddenly tomorrow name is not unique then everything breaks 18:31:55 morganfainberg: that's the proposal -- don't allow / in project names 18:32:01 bknudson: we just require escaping / in the hierarchy specification thats all that is being proposed now 18:32:04 Start restricting "/" in project names in Liberty? 18:32:20 bknudson: right and weve shifted from that as incompatible 18:32:28 +1 bknudson 18:32:29 stevemar, I'd be fine if you passed just C when there is no conflict. if there is another C in the domain, you'd need the hierarchy 18:32:37 Though I would like to restrict that. We cant atm 18:32:39 bknudson, +1 18:33:05 stevemar: htruta_: that was/is one of my concerns about what the user has to know 18:33:09 so is htruta and rodrigods fine with that change? 18:33:28 So what will "GET /v3/projects" yield from the root of the HMT? 18:33:34 i'd be happier if they always had to provide an unambiguous project name 18:33:41 htruta, rodrigods ^^ bknudson's question 18:33:44 dstanek, ++ 18:33:50 dstanek: ++ 18:33:51 bknudson, restricting / ? 18:34:09 Re the change to escape / vs restrict / 18:34:25 And escape only in the hierarchy representation 18:34:37 jamielennox, nothing else is backwards compatible 18:34:51 dstanek: yeah, that'll be clunky when authenticating i think 18:34:57 morganfainberg, I'd be glad to follow any of them 18:35:10 just looking for a consensus :) 18:35:25 i'd be okay with any of them, but I'd prefer restricting 18:35:25 project:{ name: {"a/b/c/"}} - i guess that works 18:35:43 doesn't work since it's not valid json 18:35:53 bknudson: it's valid 18:36:17 stevemar: uh. {"thing"} isnt 18:36:31 project: {root: 'a/b', name]' 18:36:36 ugg... 18:36:53 dstanek: still needs escapes but.. Sure? 18:37:05 project: {root: 'a/b', name: 'c'} --or-- project: {name: 'a/b/c'}? 18:37:22 morganfainberg: oops, yeah, i was more concerned about slashes in the value part 18:37:25 dstanek: both need to handle / in the names. 18:37:33 But either work for me 18:37:35 dstanek, {root: '/a/b', name: 'c'} 18:37:38 I'd prefer 18:37:38 project: ['a','b','c'] 18:37:40 so we could 18:37:40 --or -- project: {root: ['a', 'b'], name: 'c'} 18:37:45 dstanek, {root: '/', name: 'a'} 18:37:45 both should work 18:37:52 ++ 18:37:57 ayoung: also an option. And id be fine with that 18:37:58 dstanek, the one with the root 18:38:20 so, we could keep passing just the name when no conflict happens :D less people broken 18:38:36 htruta_: how do we know when there will be a confilct? 18:39:03 dstanek: until you try you dont. 18:39:11 And you can suddenly have a conflict 18:39:21 So a valid request could suddenly fail 18:39:24 yep 18:39:31 morganfainberg: and then how do they get the list of hierarchies? 18:39:38 and that means that if you are worried about that, use the full path version 18:39:47 if you can get a list the clients don't have to know how to encode 18:39:47 dstanek: we should always pass the full path 18:39:57 dstanek, we can include the "root" attribute in projects info 18:40:14 maybe we should use URLs for project IDs 18:40:15 but this is the thing i didn't like before. working code can suddenly break 18:40:20 If you want deeper than 1 level below the domain. 18:40:45 bknudson: we should do that for all resources in openstack 18:40:46 bknudson, I think we should work towards that 18:40:56 Not just in keystone. 18:41:11 bknudson: ++ 18:41:12 so users will have OS_PROJECT_NAME and OS_PROJECT_ROOT defined when using the clients? 18:41:12 dstanek, that is the price we pay for making this feature work 18:41:32 dstanek, yes 18:41:35 dstanek: I'll put it in my clouds.yaml 18:41:43 dstanek, that will make people sad 18:41:46 need to change all the auth plugins 18:41:52 ayoung: i disagree this is the price we pay for having the is_domain project point to itself 18:42:05 I guess this is another discussion 18:42:06 Always pass a hierarchy if you want anything below the first layer of the domain 18:42:13 using / has the advantage for SAML assertions generated by keystone too 18:42:22 If you dont pass a hierarchy it works like today. Nothing nested. 18:42:29 otherwise we would include another attribute to identity projects/domains 18:42:31 ? 18:42:35 Why do we do this to ourselves? 18:42:38 so i say this is divisive and nobody really knows how this will work because HMT really isn't out there. I think we initially restrict login by name to that top level of existing projects, require login by id for others and we figure out the rescoping process better in future 18:42:42 Oh, wait, we inherited this mess.... 18:43:06 ayoung, lol 18:43:24 jamielennox, and add in a new rule that says that project names under the root must not contain / 18:43:48 OK, that took 43 minutes 18:43:52 ~16 minutes left 18:43:52 Ok this is going in circles 18:43:56 We need to move on 18:43:56 anything else we want to talk about? 18:44:08 ayoung: morganfainberg dynamic policies .. 18:44:10 ok, morganfainberg... so... nothing changes in L? 18:44:12 should this be a topic for the midcycle? 18:44:12 Discuss this more offline 18:44:13 OK...so 18:44:15 The worst thing would be if we created a horrible solution for working with API v3 and then felt constrained to perpetuate the bad bits in v4 18:44:17 dstanek: likely 18:44:18 dstanek: ++ 18:44:20 "Should "Policy Fetch & Cache + Endpoint Constraint Enforcement" live in their own Middleware ?" 18:44:20 ayoung: i don't really care. I think we want to enforce the unscoped token, then scoped token flow and we have the client search for project name as a convenience and do auth by id 18:44:27 short answer "yes and no" 18:44:32 yes in that it would be cleaner 18:44:39 and that we should be able to deploy that way 18:44:42 no in that it is a PITA 18:44:45 so...we do both 18:44:54 #topic Should "Policy Fetch & Cache + Endpoint Constraint Enforcement" live in their own Middleware 18:44:56 make ATM work as a stack of separatable middelwares 18:45:03 sooo which one is yes and which one is no? 18:45:21 gyee, we make ATM work as a pre-canned pipeline 18:45:30 so, first of all, Spec Freeze Exception email from last week 18:45:37 #link https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg57416.html 18:45:38 but make it so we can deploy the middleware's separately as well 18:46:01 jamielennox, you have any nasty thing to say to ayoung? 18:46:10 for making it separate middleware? :) 18:46:23 one of the topics for dynamic polcieis today is whether we create a separate middleware filter for dynamic fething of policies + endpoint constraint enforcement 18:46:43 i think it should be seperate, i don't really see the difference in the auth_token middleware pipeline to being in the regular pipeline after auth_token middleware 18:46:50 lets make it "separatable" and then the question is whether we make it possible to link into ATM as well 18:47:18 what do you mean by link into? 18:47:19 jamielennox, modifying every single puppet deployment script for each service would be painful 18:47:52 ayoung, it is separatable right now, _enforce_global_target is a separate class 18:47:54 dstanek, Ideally, ATM would be something like a list of middlewares 18:48:05 trivial to move it to separate middleware 18:48:06 gyee, we are on the way to it, yes 18:48:12 exactly, 18:48:15 just put a shim around it 18:49:06 so make it explicit: Auth token middleware contains no code other than managing a list of other middlewares executed in order, and we add policy to that to start. 18:49:24 messing with paste means that this will go nowhere 18:49:44 we want to minimize the impact on other projects 18:49:47 mo refactoring :) 18:49:55 but I like it 18:50:07 we already made them all update to run oslo as a library. Let's hold it there 18:50:18 yes make sense .. although auth_token could be called anything else then 18:50:29 but if we change its name, we fall into the same issue 18:50:38 authnz 18:50:53 the trickier parts of getting the other projects to agree is still in the future 18:50:53 * gyee is stealing from apache mod_authnz 18:51:05 gyee: i read that as auth new zealand 18:51:05 and it has to do with unifying the approach to the policy content itself. 18:51:12 jamielennox: hehe 18:51:27 ~9 minutes left 18:51:39 How to identify the endpoint at Middleware ? Endpoint URL vs Endpoint ID vs Policy custom ID 18:51:41 so auth_token would be refactored to be a middleware 'wrapper', containing the actual auth_token + the policy fetch thing 18:51:42 jamielennox, nice 18:51:43 short answer. URL 18:51:43 ayoung: gyee ^ 18:52:10 URL must map to at least one endpoint's URL value in order to work 18:52:29 ayoung: long answer "U R L"? :P 18:52:36 if there are multiple "endpoints" at the same URL, but with different endpoint_ids...we force them all to use the same policy file 18:52:38 samueldmq, didn't we agree on custom endpoint ID? 18:52:43 gyee, nope 18:52:48 morganfainberg: you're enjoying the peanut gallery too much today 18:52:53 whahhh? 18:52:53 gyee, here is the goal: 18:53:07 stevemar: hey im not heckling much. 18:53:09 we need something that is pre-calculated but that maps to an endpoint 18:53:22 if we do a custom name, that requires a whole new mapping 18:53:22 ayoung: that could be a custom endpoint id 18:53:34 samueldmq, nope 18:53:38 ayoung: custom endpoint id != custom policy id 18:53:49 "cuystome endpoint id" means something that we need to map to a new endpoint 18:53:55 we need these to resolve upwards 18:54:02 we don't expect policy to be deployed per endpoint 18:54:04 ayoung, endpoint group then? 18:54:10 dynamic endpoint group 18:54:10 gyee, lets not 18:54:19 please...keep it simple 18:54:26 you are making this much harder than needs be 18:54:33 simpler than endpoint groups? 18:54:42 gyee, we don't have those in every deployment 18:54:42 just a filter 18:54:47 No 18:54:48 No 18:54:49 No 18:54:54 just the end point 18:55:03 you "fetch" based on the most specific 18:55:03 endpoint what? 18:55:09 you resolve to the most general 18:55:23 I would do endpoint ID, but we want something that is precalculatable 18:55:26 URL is too ambiguous 18:55:34 no it is not 18:55:46