18:01:55 #startmeeting keystone 18:01:56 Meeting started Tue Sep 10 18:01:55 2013 UTC and is due to finish in 60 minutes. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:57 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:02:00 The meeting name has been set to 'keystone' 18:02:03 YAY HAVANA 18:02:21 #topic Havana release blockers 18:02:26 #link https://launchpad.net/keystone/+milestone/havana-rc1 18:02:32 hi 18:02:53 speaking of packstacking all day... ayoung, you've got three RC blockers assigned to you -- are you working all of those? 18:03:17 so #link https://bugs.launchpad.net/keystone/+bug/1201487 is available fore review 18:03:18 Launchpad bug 1201487 in keystone "listing projects for a user omits those that only have group related roles" [High,In progress] 18:03:22 i just want to make sure that everything we have targeted towards RC1 is making progress 18:03:58 #link https://bugs.launchpad.net/keystone/+bug/1221889 working on it 18:04:00 Launchpad bug 1221889 in keystone "Invalid X-Subject-Token results in HTTP 401 rather than 404" [High,In progress] 18:04:01 (cc henrynash) i also just updated the meeting agenda with code reviews to havana release blockers https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting#Agenda_for_next_meeting 18:04:09 dolphm, when is the submission freeze for RC1? 18:04:23 ayoung: ASAP? it's done when it's done 18:04:31 ayoung: hence 'blockers' 18:04:35 is there a reason keystone returns 401 and not 404 (security?) 18:04:42 bknudson: context? 18:04:47 dolphm: excellent 18:04:54 https://bugs.launchpad.net/keystone/+bug/1221889 18:05:03 bknudson: i think that's just a bug 18:05:14 bknudson: disagrees with spec, and with auth_token's expectations 18:05:27 dolphm ++ on that being a bug 18:05:32 ok. I hadn't looked at it, just looked fishy. 18:05:35 atiwari: thanks! 18:05:56 yrw 18:06:12 dolphm: as an aside, I will also post a review for a solution to #link https://bugs.launchpad.net/keystone/+bug/1153645 tonight as well 18:06:13 Launchpad bug 1153645 in keystone "Deleting a role should revoke any tokens associated with it" [High,Confirmed] 18:06:32 dolphm: not in the rc1 list, but it is rated High 18:06:48 henrynash: should that be a release blocker? 18:07:20 henrynash: "need to refactor the code" means i'm scared to land in havana :) 18:08:16 dolphm: turned out to be much easier than I thought..I forgot I had written the list_role_assignments() method, I can just use that to get all the uses that have the roel we are deleting 18:08:41 dolphm: so very relatively small change 18:08:53 henrynash: cool 18:09:00 henrynash: targeted 18:09:05 dolphm: ok 18:09:26 if anyone has any other bugs in progress that *should* be release blockers, feel free to point them out :) i've been trying to identify as many as possible 18:09:44 i also have this one targeted which just needs to be investigated, at least https://bugs.launchpad.net/keystone/+bug/1131590 18:09:45 Launchpad bug 1131590 in keystone "the folsom keystone database schema can't upgrade to latest version" [High,Confirmed] 18:09:54 How's morganfainberg 18:10:06 's work on domains coming? 18:10:12 ayoung, review is posted 18:10:20 morganfainberg: passing jenkins? 18:10:24 dolphm, yes 18:10:41 link? 18:10:43 sec 18:10:48 #link https://review.openstack.org/45649 18:10:51 https://review.openstack.org/#/c/45649/ 18:10:51 #link https://review.openstack.org/#/c/45649/ 18:11:02 #winner dolphm 18:11:09 heh, anyone else? 18:11:14 lol 18:11:34 morgafainberg: so I noticed you kind of pulled back from removing the user/group validation checks form the grant calls 18:11:37 morganfainberg, anything in that that you would consider "scope creep" besides URL encoding the IDs? 18:12:01 morganfainberg: was that just too minimise change/risk? 18:12:02 ayoung, i didn't address full DNs as user_ids. 18:12:13 ayoung, that could be a large amount of change risk 18:12:19 morganfainberg: ++ 18:12:27 save that for icehouse, and do it project-wide :) 18:12:47 wow, user_id is not globally unique? 18:13:02 henrynash, i was working towards minimizing risk, but i pulled a some of "check user_id/project_id" out of the drivers and pushed to the controllers where i could 18:13:09 user_id won't be globally unique if use DNs 18:13:10 gyee: it's now "complicated" in the case of users and identity-backend-per-domain 18:13:12 or .. brought it in line with the other implementations 18:13:23 dolphm, that will be messy 18:13:28 gyee: +++++++++ 18:13:31 gyee: yes, it still is…but you might not know which domain to look in 18:13:39 bknudson, yes they will 18:13:55 DNs will be per LDAP server....if you use the same DN twice, you are wrong wrong wrong 18:14:17 if you have a complete DN being duplicated, i'm going to raise an abiguous ldap server error :P 18:14:30 bknudson: shouldn't they vary per DC? 18:14:39 dolphm, they should. 18:14:49 morganfainberg: i assume that's how you're selecting the LDAP driver to use? 18:14:49 DN, by definition, is globally unique 18:14:53 morganfainberg, I still think we want to look at the ID and say "ah, that is a DN, let us parse it, and figure out which LDAP server to talk to" in icehouse 18:14:54 i though user_id was unique username was supposed to be unique per domain 18:15:00 DNs could vary, but then someone might just store by cn=Users. 18:15:01 ayoung, that was my thought 18:15:28 bknudson, and then only one LDAP server will be allowed to do that in a deployment...and everyone else is SOL 18:15:31 jamielennox: ++ 18:15:32 jamielennox, user_id should be globally unique. 18:15:32 also, our LDAP can also use cn for the userid attribute 18:15:50 jamielennox, and you're right, username is domain unique 18:15:52 morganfainberg: ++ 18:16:08 ok, must of misread the conversation 18:16:20 bknudson: user.name=cn, user.id=dn ? 18:16:37 bknudson, but does the fully qualified DN ever change? 18:16:39 jamielennox, no, you did not, he is explicitly putting a domain check on the lookup by user_id 18:16:42 dolphm, that's usually the case 18:16:52 dolphm: that would be ok, but we currently have the user id is the cn, I think? 18:17:01 bknudson: i think so too 18:17:03 oh, wait...not, that is not what you are doing, is it 18:17:13 bknudson, we do. that is why i didn't want to tackle this change in havana 18:17:20 you are just confirm after lookup..that is much better 18:17:32 ayoung ++ 18:17:49 if we're going to change it, it'll take a release of deprecated. 18:18:02 bknudson, and migration paths. 18:18:05 #topic open discussion 18:18:29 I'm getting asked about https://review.openstack.org/#/c/43041 18:18:39 "Support client generate literal ipv6 auth_uri base on auth_host" 18:18:56 my opinion is that it's supported already if you can put [] in auth_host 18:19:30 but others might have different opinions so would be nice to get wider input 18:20:02 bknudson, yeah, host is just a string 18:20:12 also, https://review.openstack.org/#/c/45488/ -- it's a doc change 18:20:35 you can also tunnel IPv6 over IPv4 right? 18:20:36 bknudson all the bug fix does is add []. you are saying that is an uneeeded convenience function? 18:20:58 topol: yes, an unnecessary complication of the code. 18:21:36 bknudson, yeah, seem unnecessary 18:21:40 note that I haven't actually tried putting [] in auth_host myself yet... haven't had time. 18:21:48 gyee, I don't thien IPv6 over IPv4 is what people are looking for 18:22:11 ayoung, point is auth_token itself doesn't know 18:22:17 and probably don't care 18:22:54 gyee: bknudson: ayoung: i suspect this is poor interplay between token providers, business logic distributed between controller/manager/driver, driver separation etc, but there's a lot of redundant queries to LDAP in a single POST /tokens call http://ix.io/7Xg 18:23:24 bknudson, if they are doing IPv6, then the problem is, I think, DNS. THe hostname should only resolve to a AAAA record, and the client should deal with that directly 18:23:34 dolphm: the logging when using LDAP is pretty crazy 18:23:42 ayoung: yes, I'd also say use DNS. 18:23:42 we should not be advocating oputting IP address, V4 or V6, into URLs 18:23:48 morganfainberg: caching reduces the load on LDAP in the above request from like ~22 queries to ~5 18:23:54 bknudson, rip out the logging! 18:24:02 dolphm, nice. 18:24:19 dolphm, it'll only get better as we implement more caching 18:24:24 ayoung: bknudson: it's not just logging. it's the issuing tons of redundant queries to LDAP for the same data over and over 18:24:24 Shoulda been gone, long ago, far away! 18:24:27 ayoung: but the logging is also an indication we're doing something wrong (e.g. new connection per query) 18:24:34 yeah, that logging's a lot of noise 18:25:16 bknudson, it seems like a pool like sqlalchemy session pooling would make sense for LDAP. though… that has other pitfalls 18:25:16 ayoung: bknudson: blueprint and make go! https://pypi.python.org/pypi/ldappool/ 18:25:27 dolphm, +++++ 18:25:43 No 18:25:52 that is the wrong approach for LDAP 18:26:01 is connection pool reliable in Python land? 18:26:11 What we are doingright now is using a manager account 18:26:27 we should, instead, be using the same ldap connection that we used to authenticate the user, and do things as that user 18:26:44 That wauy, LDAP ACLs are enforced 18:26:48 ayoung: how does that make connection pooling 'the wrong approach'? 18:27:04 dolphm, a connection pool only makes sense for the manager case 18:27:23 dolphm, we create a connection for authentication right now, using the simple bind 18:27:45 there are other things we can do that have the same capabilities 18:27:55 ayoung: pretty sure ldappool covers that use case, check out the docs 18:28:23 i also think we can restructure the ldap driver to not make as many redundant queries. but it'll be a lot of shuffling code around. 18:28:23 looks like the pool can provide differently-auth'd conns: 18:28:26 with cm.connection('uid=adminuser,ou=logins,dc=mozilla', 'password') as conn: 18:28:31 bknudson: ++ 18:28:45 morganfainberg: ++ 18:28:48 we need to stop providing user-list 18:29:13 huh 18:29:14 ayoung: that's what filtering was going to do 18:29:17 dolphm, can we deprecate user list 18:29:42 ayoung, why? 18:29:57 gyee, because it is thinking like a small business, and we need to think cloud scale 18:30:06 you wouldn't do a user list for all citizens of the US 18:30:11 or all BofA customers 18:30:27 ayoung, force filtering on any user-list requests? e.g. no filter (no sane filter that is) no result? 18:30:42 there is a case for getting users like 18:30:47 morganfainberg, yes and no 18:30:51 even in the case of bofa customers. 18:30:57 think federation, and you realize there is no user list accessable 18:31:04 ayoung: i'd like a list of all BofA customers so I can send them letters politely suggesting that they find a better bank 18:31:09 assume that you don't have enumeration access to the backing store 18:31:23 dolphm, don't hate (i've just been too lazy to switch...) 18:31:24 dolphm, heh 18:31:24 :P 18:31:42 dolphm, I bet even BofA can't generate that list....maybe the NSA can 18:31:58 ayoung: I'd have thought that filtering + policy rules that , for instance, you must at least specify a domain_id would get us a long way 18:32:02 ayoung: you're saying we need to think NSA scale? 18:32:02 ayoung, good idea, lets ask the NSA for our user-list! 18:32:12 dolphm, absolutely 18:32:29 morganfainberg: i think we need message bus for that 18:32:34 morganfainberg, I have no desire to go through the background security clearance check to be able to see NSA data 18:32:52 * topol just got a weird phone call that said tell your buddies to stop making fun of the nsa 18:33:32 ayoung, but even with a federated login, you need some kind of breadcrumb to match up right? so a filtered userlist could still work, just not in the enumerate the canonical list way 18:33:38 ayoung: i think the immediate answer is that user list can be denied for everyone via RBAC, a custom driver could not implement it, etc 18:33:41 topol, sigint is trolling this channel right now 18:33:46 given that it is there is there any restriction to user-list that makes sense? 18:34:00 ayoung: long term we can pursue filtering, and then perhaps *require* filtering 18:34:15 dolphm, no, long term we say "we don't have a user list to give you" 18:34:24 when it comes to LDAP, is all about the filter :) 18:34:28 dolphm, we can only provide a list scoped to something in the assignments backend 18:34:29 to my mind any backend that supports user-add can probably handle user-list, but that probably indicates it should be an extension rather than core 18:34:38 so we can provide a list of all users in a project, or something like that 18:34:45 but not all users in a domain 18:34:45 ayoung: but i totally understand the "getting a list of all users doesn't make sense" objection 18:34:50 jamielennox: that soundslike a good idea 18:35:12 dolphm, thinkg oauth. You don't have any way to query a liast of users in an oauth setup...saml, ABFAB, etc...nonte of them will provide that 18:35:17 ayoung, all current users of all current projects in a domain wouldn't work? 18:35:32 user-list could just be a "known user list" via assignment 18:35:33 morganfainberg, yse, but that is based on assignments, not identity 18:35:55 ayoung, that's because in federation, user don't exist in a single place 18:35:56 morganfainberg, except that it makes no sense to do that, but yes, in theory it is possible 18:36:18 gyee, and that is exactly what Keystone is becoming: a Federated identity service 18:36:34 i can see a case (audit) for getting all "currently active" users in a domain (e.g. in assignment) 18:36:37 gyee even with federation you still can get user info from the assignments. Is that not good enough? 18:36:53 I think it would be nice to get keystone out of the identities game. 18:36:54 keystone is a restful federated centralized auth gateway service api 18:36:56 topol, with federation, forget assignment 18:37:06 will will be all mappings 18:37:18 bknudson, we can split identity off of the rest of Keystone, and make it an optional component 18:37:25 jamielennox, my concern with supporting the concept of "if we can add, we can list" is that it is very inconsistent 18:37:40 gyee, what do you think assignments are? They are all mappings 18:37:46 ayoung, make identity an extension? 18:37:55 morganfainberg, make it a separate service 18:38:05 deployed on a separate machine 18:38:16 and optional 18:38:23 keystone-id, keystone-assignment, keystone-quota (sounding like nova's services in my mind) 18:38:25 doesnt assigments = mappings? 18:38:27 ayoung: sounds better all the time. if you want keystone users, use that, if you want ldap, use that 18:38:33 or kerberos or whatever. 18:38:34 that is the idea 18:38:35 topol: ++ i think that's what he meant 18:38:37 windows 18:39:03 ayoung: as in the current sql identity be held on a different machine, or LDAP etc as well? 18:39:09 i'm seeing a summit session here. 18:39:16 jamielennox, no need to puyt LDAP on a separate machine 18:39:28 just need a way to specify what queries to run for a given domain 18:39:30 ayoung: that was my thought 18:39:49 ayoung: with a whole new paste file, i think you might be able to deploy like that today? minus the cross driver calls between identity and assignment 18:40:15 i like the idea of making the current user operations against a sql backend a new service 18:40:21 dolphm, well, we need a way to apss the authentication data from one to the other... 18:40:30 SAML is, I think a likely candidate for that 18:40:31 you could develop middleware for REMOTE_USER 18:41:21 dolphm, I've been arguing with davidchadwick that his Federaion API should not be a separate API at all, but rahter should be the coding standard for new plugins. 18:41:33 jamielennox why the need for a new service? Lots of new queries to be expected in the future? 18:41:43 The ABFAB proposal would be the first Federated plugin, but it would be method: abfab 18:41:47 wondering if some body can review https://review.openstack.org/#/c/37141/ 18:42:48 topol, to split the identity that keystone manages into a separate container, just like ldap, etc would be separate. 18:42:51 topol: removes this whole question about user-list, if you have a service where you can add to and manage via an API user-list moves over to query that directly rather than via keystone 18:43:00 atiwari, needs a bug report to explain what problem you are solving 18:43:15 jamielennox, +1 18:43:24 it was a new BP 18:43:34 for an extension 18:43:35 atiwari, then link it in the commit message 18:43:41 ok 18:43:50 sorry for that 18:44:16 this change: https://review.openstack.org/#/c/45581/ makes it so that you can do ./run_tests.sh test_backend_sql again 18:44:21 (don't need keystone.tests.) 18:44:22 atiwari, a better way to get people's attention is to explicitly add them as reviews, too. 18:44:42 bknudson, neat 18:44:46 sure 18:44:48 bknudson, mind rebasing the chain? 18:45:07 bknudson, i was going through slowly and doing the rebases as i was reviewing them 18:45:08 morganfainberg: I can rebase it, I just don't think the rebase is necessary? 18:45:16 bknudson, um, no 18:45:19 no nononononono 18:45:19 the only change was the commit message 18:45:28 di you just put all those test back into the global namespace? 18:45:36 bknudson, click the rebase button. the parent is outdated 18:45:55 ayoung: the tests have been in the global namespace for some time. 18:46:31 bknudson, I thought we moved them out of the global namespace when we put them in keystone/tests? 18:46:32 ayoung: no, he hasn't - or at least not in that review 18:46:34 that was the intention 18:46:34 the only change this makes is to not use relative imports for packages in tests. 18:47:13 bknudson, so that change would eliminate the need to specify keystone.tests? 18:47:39 gyee: yes. I think I tried it once... let me try it again. 18:47:45 gyee, I care naught about that...I do care if the tests are out of the global namespace 18:47:56 bknudson, i thought that was an artifact that was resolved by going to testr 18:48:05 we need the tests in the keystone namespace so that we can start integrating our tests with Tempest 18:48:05 ayoung: how does one get the tests out of the global namespace? 18:48:07 not relative imports. 18:48:10 tests has an __init__.py 18:48:13 bknudson, __init__.py 18:49:05 OK...maybe I panicked... 18:49:43 for example, now ./run_tests.sh test_cache fails, NameError: name '_' is not defined 18:49:57 because keystone.tests. __init__.py wasn't run at the right time 18:50:10 NO...I think I misread what you are doing...I think they are still in the keystone.tests namespace 18:50:21 ayoung: did you look at the change? 18:50:25 ayoung, they are still in the keystone.tests namespace 18:50:42 its just how he pulls them in 18:51:00 hmm, well test_cache still doesn't work. not sure the problem. 18:51:11 bknudson, i can take a look at that 18:51:17 dolphm, I did...but I misread what I was looking at...I thought he was doing the relative imports out of the global namespace... 18:51:22 dumb mistake... 18:51:57 bknudson, but i don't see why test_cache should be different than anything else. 18:52:16 ayoung: hmm, i guess i'm not understanding your concern :( 18:52:19 morganfainberg: it's not... others have the same time. 18:52:29 others do the same time. 18:52:32 bknudson, ah ok 18:52:32 do the same thing 18:52:45 bknudson, i think infra said testr would solve this (as i recall) 18:53:03 and moving the tests under keystone's namespace was a prereq to that? 18:53:18 morganfainberg: did you just volunteer to move us to testr? 18:53:27 morganfainberg: I didn't make the fix to make it so I could run_tests. I just thought it was really ugly to have these imports not follow the convention 18:53:32 dolphm, oh crap. i might have :( 18:53:48 dolphm, he's doing the exact opposite of what I panicked about...I basically read the patch backwards, and thought he was putting the imports into the global namespace when he is doing the exact opposite. We want the tests to be keyste.tests.tes_backend_ldap for example, so that, if we decide to call something from tempest, we have deconflicted with, say, the novate_test_backend_ldap (if there was one) 18:53:52 bknudson, and i like that it fixes a hacking issue (more flake8 checks!) 18:53:55 somebody hit #action fast for morgan 18:54:12 topol, if i leave the channel you can't find me! 18:54:20 * morganfainberg runs and hides 18:54:22 yeah, this is good stuff.... 18:54:24 #action morganfainberg to switch keystone to testr and then we can all buy him beer 18:54:27 ok, 6 minutes left 18:55:13 dolphm, OK if I opent the packaging thing as a bug, and I attempt to fix in the Havana timeframe? If it is too invasive when the patch comes out, I will gladly accept the -2 18:55:23 the activation of extensions.... 18:55:29 morganfainberg: as i recall, there's some nasty problem with our test architecture that prevents us from easily moving to testr... so no one's going to make the switch for us 18:55:42 dolphm, i'll look at it once RC1 is cut 18:55:53 ayoung: definitely open a bug 18:55:58 will do 18:55:59 dolphm: i think the way we do git checkouts of old keystoneclients will bite us with testr 18:56:00 morganfainberg dolphm: did rebase on https://review.openstack.org/#/c/45581/ 18:56:22 bknudson, thanks. 18:56:37 ayoung: did you poke at creating a factory to instantiate oauth1.Manager() ? 18:57:02 jamielennox, ++ 18:57:08 jamielennox: ooh, that too 18:57:18 morganfainberg: any excuse to get out of it 18:57:32 jamielennox, nah, i'll still do the work 18:57:38 we need to move that into tempest 18:57:38 morganfainberg: so you're going to move all our integration tests to tempest, then? 18:57:45 dolphm, uh. 18:57:55 dolphm, i'll see what i can do. 18:57:56 that's a trap 18:58:02 but i don't think that is in scope 18:58:04 :P 18:58:11 it could be fixed relatively easily for testr, just have each client checkout to a different folder 18:58:13 morganfainberg: lol 18:58:34 and do it up front 18:58:41 jamielennox, likely a good plan 18:58:54 we do need to "fix" that test mechanism in either case 18:59:07 just wondering if anyone else has been looking at the docs? 18:59:31 whilst we're talking fixing test cases another plug for: https://review.openstack.org/#/c/44014/ 18:59:38 remember the last time we fixed test and then half us could test and the other half were dead in the water. lets not repeat that 18:59:41 jamielennox, haha, i was going to plug that for you. 18:59:43 I'd consider our docs to be in rather poor shape. I'll try to work on it. 18:59:43 bknudson: which docs? /answer in -dev 18:59:47 #endmeeting