18:02:10 #startmeeting keystone 18:02:11 Meeting started Tue Feb 11 18:02:10 2014 UTC and is due to finish in 60 minutes. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:02:13 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:02:15 The meeting name has been set to 'keystone' 18:02:22 o/ 18:02:22 #topic Reminder: Icehouse feature proposal freeze February 18th (features must be in code review) 18:02:22 (under the wire!) 18:02:25 o/ 18:02:28 o/ 18:02:37 we're in *really* good shape for feature freeze, GREAT JOB EVERYONE! :D 18:02:49 dolphm, you probably should add morganfainberg_Z to your notify line to wake him up 18:03:05 ayoung: what time does he have now? 18:03:09 ayoung, i see pings for all variations on my names 18:03:26 mrgnfeignbern: good to know 18:03:36 lol 18:03:39 nice 18:03:41 * morganfainberg adds that now as well 18:03:50 dolphm, Yay! 18:04:03 so we're a week out from feature proposal freeze, with three bp's not "in review" yet -- and i'm not at all worried about any of them at the moment 18:04:05 so let's move on! 18:04:13 dolphm, I snuck in one last item on the agenda. 18:04:13 go team 18:04:19 #topic Spread Domains for other OS services (Nova) 18:04:28 tellesnobrega: i assume this is your topic? 18:04:33 \o 18:04:40 how is nova going to use domains? 18:04:50 dolphm: yeap 18:05:01 and this also directly conflicts with hierarchical multitenancy, so i'm interested in what you're thinking 18:05:32 I believe the auth_token middleware sets the domain info in the env 18:05:51 if a domain-scoped token is used 18:05:51 bknudson: it is not in the nova context 18:05:52 HTTP_X_DOMAIN_ID 18:05:53 bknudson: it dies 18:06:01 tiamar: it's in the environment, though 18:06:04 it does, i mean! 18:06:10 nova is just one of the ideas, i was also thinking about glance 18:06:17 tellesnobrega: what's the use case? 18:06:50 for example if we want to have an image that belongs to coca-cola and an image that belongs to pepsi, we dont want them put together 18:07:02 nova is more related to quotas 18:07:07 tellesnobrega, hierarchical multitenacy 18:07:27 tellesnobrega: so, that doesn't require domain-scoped authorization 18:07:29 do something at one level of the tree, and only have it vbisible to subnodes 18:07:47 i see 18:07:54 I'm guessing you can't do anything with a domain-scoped token outside of keystone 18:07:59 ayoung: really? you mean by having a single top level project in each domain that contains the images? 18:08:05 some other actions may require domain-scoped auth 18:08:13 henrynash, domain *is* the top level project 18:08:14 bknudson, correct 18:08:22 tellesnobrega: but rather user- or project-domain based quota enforcement, which are passed to nova as x-user-domain-id and x-project-domain-id 18:08:26 lets accept two things, one, domains are unnecessary 18:08:34 two we are not going to b e able to get rid of domains 18:08:41 tiamar: like what? 18:08:42 the domain (top-level) would contain the most public images 18:08:45 ayoung: how do you have namespaces for users without domains? 18:08:46 ok...its a language thing, not a "best design" thing 18:09:00 bknudson, ah...let me rephrase 18:09:09 tiamar: "may": sure. do we have a specific use case to discuss? 18:09:15 ayoung, ++ 18:09:26 for assignments, domains are unnecessary, but we won't be able to remove them becvause they are embeddd in the collective brain of our users 18:09:47 lbragstad: that steps into hierarchical multitenancy; it's not a use case for our existing definition of "domains" which do NOT own openstack resources 18:10:11 dolphm: mainly to big deployments of Openstack, where a top level administrator may set things i.e images, quotas to a entire domain and projects a domain owns 18:10:18 ayoung, this is related to our conversation yesterday... 18:10:19 makes sense 18:10:33 ayoung: we need *some* administrative construct that allowes the creation of levels of administration scope 18:10:39 do domains own projects? 18:10:50 bknudosn: today, ues 18:10:54 mrgnfeignbern, so I would say that root, domain, and nested projects are the terms we use. implementation wise, they can all be the same thing 18:10:55 today, yes 18:10:57 henrynash: hierarchical multitenancy :) 18:11:08 ayoung, aye, 18:11:12 root namespaces domains 18:11:16 +1 18:11:17 domains namespace projects 18:11:27 projects namespace subordinate projects 18:11:42 we use different terms for Hysterical Raisins 18:11:46 +1 18:12:31 ayoung, you saying domains contains projects and project contains subprojects? just make sure I understand 18:12:33 tiamar: so the use case is distributed domain-wide quota enforcement; what do you need from the keystone community? 18:12:43 tellesnobrega: ^ 18:12:44 gyee, that is my view, yes 18:12:52 dolphm: and that's fine…..as long as we have the support for how we define users such that hierarchical multitenacy can be used to infer scope of administration 18:13:25 henrynash: if by "infer" you mean "explicitly define" -- then yes! that's the goal 18:13:26 dolphm: not only that, but a domain administrator that could do anythign with a single token in Nova for example 18:13:35 ayoung, I don't have a firm opinion right now, but I hear ya 18:13:38 dolphm: that'll do too ! 18:13:39 tiamar: +1 18:13:56 now...a domain is also a concept in identity, so the domain object will have one more, optional field, which is "what IdP do domain users live in" 18:14:11 tiamar: that conflicts with our existing concept of multitenancy; you need hierarchical multitenancy for that, and it's an *entirely* separate discussion 18:14:17 you have to squint a little to see it, but, feh 18:14:26 tiamar: probably best saved for the friday meeting, as it evolves 18:15:02 dolphm: ok. but domain isolation is also necessary for multitenancy in Nova, glance... 18:15:10 tiamar, it will be there 18:15:15 tiamar: explain? 18:15:23 ayoung: how? 18:15:38 tiamar, a domain is a top level project with no named parent. It will come right off of "root" a majic project-type-thingy 18:15:55 tiamar: nova and glance have no need to care about domains to provide any aspect of multitenancy whatsoever 18:15:59 ayoung: with the nested projects? 18:16:03 tiamar, replace the term domain with project, and say that projects are hierarchical 18:16:23 and that a resource in a parent node is visible to all children 18:16:30 dolphm: if we want to have an image that belongs to a specific domain? 18:16:42 tellesnobrega: that currently makes no sense, as domains do not own resources 18:16:52 tellesnobrega, then it is owned by that domain and visible to all subprojects in that domain 18:17:09 ayoung: yes, but doesn't this replacement requires a lot of code rewrite in keystone? does it affect about policy.v3cloudsample.json? I love this file 18:17:13 tellesnobrega: again, you're providing use cases for hierarchical multitenancy, so i'd suggest bringing up the use case in that meeting / pursuing hierarchical multitenancy first 18:17:20 and if it is owned by dom1->p1->p2 it is visible to dom1->p1->p2->p3 18:17:37 tiamar, it requires code, yes 18:17:52 that is why we are discussing it. I think we should table until Friday and move on. 18:18:02 dolphm: sure 18:18:10 ayoung: ok 18:18:28 ayoung: i think so too 18:18:32 #topic Fixing multi-domain LDAP support 18:18:34 henrynash: o/ 18:18:46 henrynash: i can provide some background here if you'd like 18:18:46 see the above line about the domain table 18:19:20 henrynash: (although i don't have any links in front of me) 18:19:25 ok so the plan for this had been to fix the assignment tables, and then layer a fix for the multi-domain LDAP on top of it (as discussed in San Antonio) 18:19:35 we need to be able to safely generate userids based on LDAP source. 18:19:39 Same problem as Fedration has 18:19:42 yesh. 18:19:54 morganfainberg, put away the booze 18:20:17 henrynash: ah, i missed that discussion 18:20:18 ayoung, but :( 18:20:21 most of the code for multi-domain LDAP is OK…the bit that isn't is where we try and infer the domain_id (to ayoung's point) 18:20:47 henrynash, I don't know if we need to infer it, so long as the generation is safe 18:20:58 but we do need to have larger fields to incorperate 18:21:12 ayoung: Oh, I agree…but the CURRENT code tries to infer it, which is the bit that is broken 18:21:19 sure 18:21:34 where is the explicit domain id if we don't infer it? 18:21:38 henrynash, so...I think the hack goes, for now, in the LDAP driver id_to_dn 18:21:42 and the reverse 18:21:50 ayoung: agreed 18:21:51 we need to support the existing approach 18:22:02 si if CONF.multi_domain_backends.... 18:22:38 dolphm: so basically, I'l like to driver this to closure now the assignment tables are ready for review…in conjunction with ayoung 18:22:58 henrynash: is it realistic to ship icehouse with this fixed? 18:23:17 dolpm: I think so…young" 18:23:21 henrynash, so I had a SQL question 18:23:27 i'd rather not ship another release with documentation for a broken feature 18:23:33 is there something magic about 64 chars long keys? 18:23:36 dolphm: ++ 18:23:41 ayoung: no 18:23:44 ayoung, no 18:23:51 if we up the key length to...I guess 130 chars, do we have the same support? 18:23:57 Is there some number we need to stay under? 18:23:59 ayoung: up to 255 18:24:01 512? 18:24:02 or varchar255 18:24:03 OK. 18:24:13 len(uuid.uuid4().hex) == 32 18:24:15 ayoung, > 255 has implications in mysql 18:24:33 ayoung: so we either do that or we store it in two fields in the new assignment table and we return a composite key from the driver…. 18:24:36 bknudson: yeah, i don't know why they were ever 64 to begin with 18:24:38 morganfainberg, and pg and DB2 are OK with <255 as well? 18:24:40 bknudson: essex-ism 18:24:44 ayoung, pg will be 18:24:46 ayoung: yep 18:24:48 bknudson, db2?^ 18:24:55 oh, don't know about db2 18:25:01 I don't think db2 is going to have a problem with a longer key 18:25:04 morganfainberg: it will truncate on varchar255, won't it? 18:25:15 lbragstad: how convenient! 18:25:19 lbragstad, thats the implication w/ mysql, 18:25:19 it's the total length of all the fields in the column that might have an effect on the pagesize. 18:25:29 lbragstad, i think pg is less... picky 18:25:40 morganfainberg: doesn't pg barf? 18:25:47 OK, so lets expand the key size to the max safe length, explain why we are doing it, and have a comment in there saying "we can't go any larger" 18:25:56 dolphm, doesn't matter, 255 is what we're setting it at ;) 18:26:02 sure 18:26:08 dolphm, oh if value > 255, yeah 18:26:13 then, for user and groups IDs from LDAP, it is ldap-attribute@@domain_id 18:26:43 two @@ or one? 18:26:45 henrynash: if we don't have a "fix" for multi-ldap by feature freeze, i'd at least like to ship icehouse with the relevant docs commented out or removed 18:26:46 two 18:26:55 topol, to avoid tripping on people using an email address 18:26:58 topol, so 18:27:10 ayoung@redhat.com@@abcd1234 is OK 18:27:12 dolphm: You'll have a fix in code review by the 18th 18:27:16 henrynash: the "guess my domain" code needs a giant docstr explaining how broken it is as well, and not to use it as-is 18:27:28 henrynash: alright 18:27:37 henrynash: is there a bug or something we can track against i3? 18:27:46 the fix is not going to be guessing domains, I hope? 18:27:50 dolphm: yes, there is…let me find it 18:27:56 bknudson, no no guessing 18:28:03 its also essential that the code that we don't break existing LDAP deployments, or timbell will be sad 18:28:16 We don't need to "guess domains" 18:28:24 we just need to be safe in generating the userids 18:29:04 Oh...wait, for the list users stuff...we need a field in the domain table 18:29:49 ayoung: i'm just talking about the code that's currently in master 18:30:22 dolphm, yeah...so looking up a user by userid...can we just say that for multi domains, user domain id needs to be specified in the token request etc? 18:30:42 ayoung: that's an API change, but you can propose it 18:30:55 ayoung, i don't think that is unreasonable. but yeah.. api change (but it's not incompatible) 18:31:18 morganfainberg: actually, it's an additional required attribute, so it is backwards-incompatible 18:31:21 get/users/ayoug@@edhat.com@@abcd1234 ? 18:31:32 oops GET /v3/users/ayoug@@edhat.com@@abcd1234 ? 18:31:38 dolphm, no it's only incompatible if you enable multi-domains 18:31:45 dolphm, oh oh wait.. 18:31:53 dolphm, this would include the sql backends? 18:32:03 yeaaah nvm i'm wrong 18:32:04 dolphm: I can't find the explicit defect (I thought there was one), there's this: https://bugs.launchpad.net/keystone/+bug/1226171 18:32:05 GET /v3/users/ayoug@redhat.com@@abcd1234 18:32:06 Launchpad bug 1226171 in keystone "When using per-domain-identity backend, user_ids could collide" [Medium,Triaged] 18:33:00 seems like we'd break everybody if we changed the user ID format. 18:33:05 ok...so is userid parsing acceptable? 18:33:33 either way (explicit domain id or userid parsing) we need to map from domain id to LDAP backend 18:33:44 which means we need a way to enumerate the backends 18:33:45 henrynash: that works for now 18:34:14 ayoung: as long as keystone owns ID's, it's acceptable IMO 18:34:20 bknudson, we just say that userid is a blob, but the LDAP folks have dealt with the assumption of UUID for user id already 18:34:27 dolphm, ++ 18:34:40 OK, so how do we go from domain table to the LDAP config? 18:34:56 bknudson, if we change the id format we can also do "if it's not the 'new-format' handle it in a specific backwards-compatible way" 18:34:57 or, more correctly, can we make it so that this works for Federation as well 18:35:16 ayoung: the multi-domain config files are "indexed" by domain name (or ID, one or the other) 18:35:29 just a quick note about the above user ids - we need to make sure @ is url safe or bknudson is right we could break things 18:35:35 in the face of hierarchical multitenancy, i should put my patch back up for configurable UUID lengths (so we don't hit that 255 limit immediately) 18:35:57 I'd like to treat the SQL backend, any LDAP backens, and the Federated "backends" as different forms of the same thing. 18:36:07 ayoung: ++ 18:36:16 henrynash: that should be replaced with one or the other 18:36:21 henrynash: the indexing by name or id 18:36:24 is the @ sign going to make things break>? 18:36:26 dolphm, ? configurable uuid lengths? 18:36:29 henrynash: just pick one; i think names are terrible but that's just me 18:36:36 dolphm: it is one ptr the other, I just can't remember which! 18:36:44 I know people are using email addresses for userids already 18:36:50 I can't see that @@ is any worse. 18:36:56 henrynash, files are indexed by name afaik for the external configs 18:37:01 dolphm: I think it is name so that it is human readable 18:37:04 morganfainberg: i replaced all the uuid.uuid4().hex calls with something in keystone.common that did uuid.uuid4().hex[:CONF.uuid_length] 18:37:08 henrynash, it uses domain-name.conf and reads it in 18:37:31 ayoung: i would guess yes since the @ is usually before the domain portion of a url, but if we aleady have some... 18:37:44 morganfainberg: i got shot down due to increased risk of collision, but it sounds like a more interesting tradeoff now 18:37:51 dolphm, hm.... 18:38:11 dolphm, because we have uuid@@uuid possibly? 18:38:26 but aren't our uuids 64 in len as is? 18:38:38 dolphm, i'm confused on the benefit of trimming them down 18:39:01 morganfainberg: uuid@@openstack.domain_uuid.project_uuid.subproject_uuid 18:39:05 risk of ^ 18:39:20 NO! 18:39:29 dolphm, we should move to another column type 18:39:32 dolphm, if we did that 18:39:34 lol 18:39:36 project Ids are not domain scoped. They are assigned by Keystone 18:39:37 dolphm, not truncated uuids 18:39:50 unless they want to pull them in from LDAP (in the future) 18:40:07 in which case it would be, at most LDAPattribute@uuid 18:40:13 but not more nesting than that 18:40:27 ayoung: i'm referring to hierarchical multitenancy (which now needs an acronym badly because i don't want to type that anymore), where they might be exposed like that 18:40:38 HMT 18:40:40 HMT 18:40:42 yay 18:40:52 HMT4U 18:40:58 topol ++ 18:41:00 PMT? 18:41:01 ;) 18:41:01 topol: ++ 18:41:04 * dolphm just tried to type that and muscle-memoried HTML 18:41:06 Aitch Emm Tee 18:41:10 Heh 18:41:33 Heavy MeTal? 18:41:33 dolphm, you'll manage 18:41:48 ayoung: very good 18:41:48 ok, so if that is legitimately a concern, we should look at alternate column types 18:42:07 should be hierarchical multiprojectcy since we changed the term. 18:42:08 but, they'll need other work for indexing properly then (PK isn't viable on some of them) 18:42:10 in juno+1 we can rename HMT to HMP (hierarhical multiprojectsy) 18:42:18 dolphm, ++ 18:42:29 anyway... what else is on the agenda... 18:42:32 keys need to stay short, as they are used in indexes, you don't want them going into the big text blob that most RDBMSes do for freeform text 18:42:41 #topic Specifying format for compressed tokens 18:42:43 ayoung: o/ 18:42:49 OK...so this is pretty cool 18:42:59 ayoung: is this referring to the static prefix? 18:43:03 I've been learning a little bit more about CMS 18:43:09 dolphm, at the end have a quick review / bp to bring up as possible i3 target 18:43:14 (code is already up) 18:43:28 I need a way to specify that a token is in comporessed format 18:43:31 morganfainberg: (?) 18:43:43 the current brokenness is look for MII at the start of the token 18:43:52 dolphm, let ayoung do his topic and i'll then jump in with the quick bit after 18:43:53 where does MII come from? 18:43:56 that is length specific, so with compression, it will not be MII 18:44:06 I was going with {cmsz} as a prefix 18:44:14 but...that i s kindof made up 18:44:17 an explicit prefix makes more sense 18:44:24 bknudson, ++ 18:44:28 it would be nice if it were something that said "here is how you deal with the data" 18:44:32 i like explicit - known prefixes 18:44:34 bknudson: it's a quirk of encoding the start of a CMS token into base64 18:44:39 and the order of ops most people need to do is: 18:44:47 bknudson, that is base64 encoded asn.1 thing 18:44:48 ayoung: you know the zlib output contains gzip headers, correct? 18:44:56 so, it's already clear that it's compressed 18:45:00 base64 decode, uncompress, verify sign, parse JSON 18:45:24 dolphm, but not everything has headers like that. 18:45:24 dolphm, yeah...and that is also an option: just say that it is BASE64 and then deduce at each step 18:45:34 the formats we need to deal with are 18:45:35 or something gzip compatible; i'm not clear on what zlib does different from plain "gzip" 18:45:38 dolphm, might make sense ot just say "this is what it is" 18:45:45 base64, gzip, der, JSON 18:46:04 morganfainberg: i'm not opposed to something human-readable, i'm just pointing out that it's not entirely machine-useful 18:46:12 rather, it's redundant 18:46:29 now, there are not an clear content types in HTML land for this, mainly because crypto, comoppression, etc are done as separate HTML headers 18:46:52 I was thinking 18:46:53 ayoung: it's also in a header by itself, and should be opaque to clients 18:47:01 ayoung: you shouldn't expect clients to work with headers for this 18:47:04 (other) headers 18:47:11 application/JSON+CMS+GZIP+BASE64 18:47:19 dolphm, right 18:47:26 ayoung, I though CMS offer compression as well, lemme check the spec 18:47:29 so I was strwamanning using ^^ as the prefix 18:47:41 gyee, I am pretty sure it does 18:47:49 ayoung: why not use that? 18:47:59 gyee: i've never seen that 18:48:06 https://tools.ietf.org/html/rfc3274 18:48:07 dolphm, well we still need a way to detect the token type 18:48:19 ayoung: that's also giant, and starts to bulk the token back up! 18:48:20 gyee, beat me to it 18:48:30 not sure if its been superseded though 18:48:43 ayoung: if CMS handles compression already, then it wouldn't be our problem 18:48:43 dolphm, yep...I am aware, 18:48:52 gyee: awesome 18:48:55 dolphm, CMS format might, not sure if the tools expose it 18:49:02 Let me see... 18:49:07 so, does openssl support that spec? 18:49:30 dolphm, should, but need to double check 18:49:38 that spec makes a good argument for compressing before signing 18:49:43 the lib does 18:49:45 http://www.openssl.org/docs/crypto/CMS_compress.html 18:49:49 not sure if the CLI does 18:49:55 I don't see a switch for it 18:50:11 -compress 18:50:21 okits not inthe man page, but on the site. let me play around with that 18:50:32 http://www.openssl.org/docs/apps/cms.html 18:50:49 so...what to do about detecting token format, then? 18:51:14 ayoung: can you hit -dev with a clear breakdown of the options? 18:51:17 ayoung, might be a new(er) release than the cli 18:51:19 then you'll need --uncompress on the other side? 18:51:27 ayoung: we have several viable paths at this point, it seems :) 18:51:31 bknudson, I think that might be done by default 18:51:33 ayoung, we should use a header to indicate the token format 18:51:42 I think that's the proper way to do this 18:51:43 ayoung: nothing? it means we keep a CMS packet and we can just detect whether its compressed or not 18:51:51 plus it'd be fun to advertise support for this to a broader audience 18:52:03 dolphm, yep...and there is one other nastiness I was hoping to clean up, which is base64 should be url_safe...I think I might be able to get that, too 18:52:04 gyee: i don't think so 18:52:05 gyee: i don't think we should add more headers 18:52:18 dolphm, why not? 18:52:24 jamielennox, we should move everything to headers. 18:52:28 jamielennox, EVERYTHING 18:52:30 jamielennox, ;) 18:52:33 gyee: because the token should be opaque 18:52:33 jamielennox, I think I can check the length, and, if it is greater then len of a uuid, perform a base64decode on it 18:52:56 CMS tokens are not opaque, they contain stuff 18:52:59 ayoung: if it's a CMS token though we should be able to still check for MII 18:53:01 OK..I have some work to do...I think I can drop all of the PEM stuff 18:53:07 jamielennox, nope 18:53:13 MII won't work if it is compressed 18:53:17 they are essentially signed documents 18:53:24 it MII is length of the token dependent 18:53:25 * dolphm going to cut the meeting 5 minutes short as i have to run soon 18:53:31 I think I'm good 18:53:40 ayoung: it won't work if we gzipped a token - if we have compresseddata in cms it probably will 18:53:50 jamielennox, ++ that is what I meant 18:53:50 jamielennox: if openssl supports compressed CMS, then ++ 18:53:53 gyee: clients don't know that yet 18:53:58 s/yet/though 18:54:00 . 18:54:01 * ayoung going to test 18:54:04 jamelennox, they will right? :) 18:54:09 gyee: no 18:54:15 ayoung: thanks for looking into this. I think it will make a lot of users happy to have smaller tokens. 18:54:16 gyee: they're not opaque to keystone tools, they're opaque to clients and need to remain so 18:54:19 gyee: for no reason i can think of 18:54:29 it should be transparent. When the check the signature, library should see it is compressed 18:54:55 try: zlib.decompress(token) except: pass 18:54:59 I might abandon the old review, if the code is radically simpler 18:55:29 cool 18:55:36 cutting it short... thanks everyone! 18:55:38 #endmeeting