18:02:20 #startmeeting keystone 18:02:21 Meeting started Tue Aug 13 18:02:20 2013 UTC and is due to finish in 60 minutes. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:02:22 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:02:24 The meeting name has been set to 'keystone' 18:02:36 #topic FeatureProposalFreeze 18:02:37 I sent him email, but he is in the Ukraine, so it is pretty late in the day for him 18:02:48 #link http://lists.openstack.org/pipermail/openstack-dev/2013-August/013278.html 18:02:53 #link https://wiki.openstack.org/wiki/FeatureProposalFreeze 18:03:12 hopefully this isn't a surprise because i already spammed everyone about this, but read the above if you haven't ^ 18:03:29 Looks good here. 18:03:36 hi 18:03:37 makes sense to me 18:03:46 dolphm, format date is...? 18:03:51 hi 18:03:51 formal 18:03:52 i don't expect any surprises, but wanted to bring everyone's attention to it again anyway :) 18:03:57 dolphm: i think the people who need to know about it, know about it 18:04:11 we're already drowned in code reviews 18:04:15 bknudson: ++ 18:04:18 #topic morganfainberg 18:04:25 thankfully we have a new core reviewer to help with that :) 18:04:29 yay! 18:04:31 awesome! 18:04:32 +1 18:04:33 official welcome to morganfainberg :) 18:04:33 congrats! 18:04:34 i've been trying. :) 18:04:35 +1 18:04:37 thanks 18:04:41 congrats! 18:04:47 dolphm, is there some other launchpad setting he needs in order to assign bugs to people? 18:05:01 congrats!!! Very wel deserved 18:05:15 morganfainberg, you are buying beer on the next summit? 18:05:21 morganfainberg: ayoung: not sure, poke me after the meeting if something is wrong with lp 18:05:23 gyee… hrmm… we shall see. 18:05:29 yay 18:05:36 #action morganfainberg to buy everyone beer 18:05:40 ha 18:05:43 #topic critical issues 18:05:54 #link https://bugs.launchpad.net/keystone/+bug/1210590 18:05:57 Launchpad bug 1210590 in keystone "Split backend crashes with AttributeError" [Critical,Confirmed] 18:06:01 #link https://bugs.launchpad.net/keystone/+bug/1211445 18:06:03 Launchpad bug 1211445 in keystone "deleting an unassigned role causes 500" [Critical,Confirmed] 18:06:10 these are two nasty ones on v3 that i've seen reported a couple times 18:06:25 dolphmL I can't seem to reproduce…tried what you di and it worked 18:07:19 dolphm: at least could not reproduce #link https://bugs.launchpad.net/keystone/+bug/1210590 18:07:19 henrynash: hmm... i'll try to reproduce once more 18:07:21 Launchpad bug 1210590 in keystone "Split backend crashes with AttributeError" [Critical,Confirmed] 18:07:33 dolphm: is you db migrated or new? 18:07:56 henrynash: brand new 18:08:03 dolphm: hmmm 18:08:10 henrynash, let me know if you want a hand with either 18:08:21 henrynash: i have the steps to repro mostly scripted for a different bug 18:08:26 dollphm: and we unit test exactly the url you tried 18:08:50 dolphm: ok…if you can shed any more light…I'll get in and debug 18:08:55 henrynash: will do 18:09:08 the role exists but it's not assigned? 18:09:17 dolphm, suspect that the differenc might be the LDAP backend. THe unicode thing leads me to think it might be a Directory server issue 18:09:37 bknudson: yeah, new user, new project, new role ... and create a role assignment of the three -> 500 18:09:42 dolphm ayoung: wasn't there a recent unicode relate ldap thing? 18:09:53 morganfainberg: i'm not aware of one? 18:10:02 dolphm: i might be thinking internal code 18:10:04 bknduson: so I made a change where the list of roles for a user/probject pair became a list of dicts instead 18:10:08 dolphm: i'll check 18:10:11 spzala had a work in progress for unicode 18:10:21 bknudson: ah that might be what i saw. 18:10:29 my guess is that I missed something somewhere and something is still returning an old style list 18:10:33 morganfainberg, yes, there is a well known issue. 18:10:50 https://bugs.launchpad.net/keystone/+bug/1172106 18:10:53 Launchpad bug 1172106 in keystone "Live LDAP tests fail on unicode names" [Medium,In progress] 18:10:54 could use some LDAP-expert feedback on bug 1211643 18:10:56 Launchpad bug 1211643 in keystone "Update user name failed with LDAP back end by CLI" [Undecided,In progress] https://launchpad.net/bugs/1211643 18:11:07 maybe that needs to be configurable 18:11:09 yep, it's specific to the LDAP though 18:11:32 it's strange we don't allow change name in ldap backend. 18:11:35 ayoung: thanks for the bug link 18:11:50 since it's not like you can't change the name attribute in an entry. 18:12:28 dolphm, assign any ldap bugs to me 18:13:06 you should be able to update user name 18:13:20 sounds like a bug in the code 18:13:52 unless your LDAP ACL is configure to have it read only 18:13:54 let them try to change the name and if the ldap server doesn't like it it can reject the request. 18:14:03 gyee: it was expected behavior when it was implemented 18:14:09 gyee: i'd be careful about suddenly allowing it 18:14:48 bknudson: hmm... that's probably a safe approach 18:15:02 sql could do the same thing 18:15:14 dolphm, you mean we don't allow updating user name? 18:15:19 maybe ldap doesn't check for duplicates? 18:15:20 username is typically modifiable, so long as the userid is immutable. Should be enforced by ACLs. 18:15:22 gyee: historically, no 18:15:26 so, yeah, lets allow it. 18:15:30 bknudson: depends on schema 18:15:45 bknudson: and acls 18:16:17 dolphm, is that a general rule across all backends? If so, then the bug can be closed notabug 18:16:26 continue this discussion in the bug / review? 18:16:29 ok then, we need to distinguished what we can do versus what LDAP can do 18:16:37 dolphm, agreed 18:16:47 this isn't high priority, just wanted to bring some attention to it 18:16:51 #link https://review.openstack.org/#/c/41603/ 18:17:04 #topic pagination 18:17:20 :) 18:17:30 #link http://lists.openstack.org/pipermail/openstack-dev/2013-August/013493.html 18:17:32 ok, so quite lively ML discussion going on 18:17:42 So I think we need pagination short term, but we should be wary of depending on it over the long. LDAP and pagination is a bad mix 18:17:57 listing all users is also a bad practice. 18:17:59 ayoung: +1 18:18:09 lets pass the query parameters into the drivers and let the drivers optimize 18:18:45 so it's worth reading the ML trail 18:18:54 dolphm, I opened a handful of related wishlist items yesterdat. 18:19:04 would be hard to standardize if drivers don't speak the same language 18:19:10 ayoung: use google as an example :) you can't go to google.com and see "all search results" with a blank query string 18:19:17 Jay certainly advocating we support the same thing that other projects do, i.e. limit/marker 18:19:22 dolphm, good point 18:19:47 I think it would be best if we followed what other OS projects are doing. 18:19:58 Is there a reason we WOULDN'T do what the other projects have done? 18:20:06 bknudson: at the very least it makes it easier for developers to interact with keystone then 18:20:19 http://bit.ly/168qd2d 18:20:27 should be able to use shared code to do paging. 18:20:29 those are new and wishlist bugs 18:20:33 sort key and sort order are particularly important to have consistent, even if we don't expose that to the api yet 18:20:43 bknudson, we don't follow, we lead :) 18:20:50 Here's my most recent ML post: 18:20:51 https://bugs.launchpad.net/keystone/+bug/1211582 18:20:51 1) Support 'limit' and 'marker' (as opposed to 'page', 'page_szie', or anything else). These would be standard, independent of what backing store keystone was using. If neither are included in the url, then we return the first N entires, where N is defined by the cloud provider. This ensures that for at least smaller deployments, non-pagination aware clients still work. If either 'limit' or 'marker' are specified, then we pagi 18:20:51 down into the driver layer wherever possible to ensure efficiency (some drivers may not be able to support pagination, hence we will do this, inefficiently, at a higher layer) 18:20:52 2) If we are paginating at the driver level, we must, by definition, be doing all the filtering down there as well (otherwise it all gets mucked) 18:20:53 3) We should look at supporting the other standard options (sort order etc.), but irrespective of that, by definition, we must ensure that we any driver that is paginating must be getting is entries back in a consistent order (otherwise, again, pagination doesn't work reliably) 18:20:54 Launchpad bug 1211582 in keystone "Filter user list by partial attributes" [Wishlist,New] 18:21:27 btw - the identity API spec doesn't have page / page_size anymore... I submitted a change to remove them since they weren't implemented. 18:21:52 can always add them back in again. 18:21:59 bknudson…has that gone in…I checked earlier today and they were still there... 18:22:04 yeah, better to add them when we have support. 18:22:04 let me, once again, reiterate the LDAP specific concerns, 1) limit the number of entries returnsed. 2) LDAP does not guarantee order, which means paging requires a cursor. 3) Cursors don't scale 18:22:15 https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v3/src/markdown/identity-api-v3.md doesn't have them 18:22:20 #link https://review.openstack.org/#/c/39828/ 18:22:37 ldap servers can have their own limit on results anyways 18:22:56 ayoung: so that's fine, we just return N items as I describe 18:23:02 I think Active Directory has paging of member attribute. 18:23:28 bknudson, yeah, but we should also allow Keystone to specify the limit. See the problem with the HP ED taking an hour+ to return 18:24:14 ayoung: agreed, we should have a keystone limit 18:24:16 ayoung, that's because HP ED is miss configured :) 18:24:24 misconfigured 18:24:27 --sizeLimit 200 18:24:41 option value for all LDAP queries, I think? 18:24:43 lol 18:24:48 gyee: right, but regardless, we should limit (or optionally be able to provide a limit) to solve that. 18:25:12 sure, I am fine with having limit on the client-side 18:25:15 s/solve/make keystone better regardless of misconfiguration 18:25:18 of the server 18:25:25 LDAP servers usually don't take hours and return you thousands of entries 18:25:35 if configured correctly 18:25:44 so are there any objections to my most recent proposal…it does seem to cover all the above issues? 18:25:50 on paging, there was some question about the implementation. Are we just punting on it? 18:25:59 LDAP, by design, is for *fast lookup* 18:26:21 should there be a knob for ldap search time limit though? 18:26:41 rcrit, it can be optional, too 18:26:45 at least with 389-ds there is one, even if the user doesn't set it themself 18:26:52 rcrit, file that as a wishlist bug 18:27:03 I don't wish it, just asking :-) 18:27:08 or tag it on to the limit 18:27:26 rcrit, I know better than that.... 18:27:32 why just ldap search? can't sql query take a long time, too? 18:27:49 bknudson, I think because timeout is a standard part of an ldap query 18:28:00 bknudson, if an LDAP query takes a long time, something is misconfigured 18:28:06 pretty sure SQL has not such standard 18:28:08 it's a server-side thing. 18:28:17 I think, anyway 18:28:27 ldapsearch --hostname localhost -p 1389 --baseDn 'uid=user.0,ou=people,dc=example,dc=com' \ 18:28:27 --searchScope base --sizeLimit 1 --timeLimit 1 '(&)' @inetOrgPerson 18:28:46 keystone doesn't have any control over the ldap server settings. 18:29:05 bknudson, no, but it can chose to use server side controls if they are available 18:29:06 bknudson: but we should totally change that 18:29:15 bknudson: keystone-manage configure_ldap_correctly 18:29:25 proposal: I'll create an etherpad that contains my most recent proposal from the ML (that is essentially to do things the way other projects do), link it to the bp and let others comment? 18:29:27 damn straight 18:29:45 henrynash +1 18:29:51 bknudson, there is already something like that for AD, where it can make use of a control if the server supports it. I'd have to dig up the commit. Was probably done by CERN 18:30:02 henrynash: the way other projects do it is a great approach for SQL 18:30:07 henrynash, blueprint, link to the wishlist bugs 18:30:36 dolphm: I believe that my proposal at least ensures that we don't get long delays from LDAP 18:31:05 ayoung: this isn't a wishlst, we are implementing a solution for H3 18:31:08 ldap.search_ext_s has a timeout= parameter. 18:31:20 henrynash, then up the priority of the bug 18:31:38 and can pass server and client controls if that's an option. 18:32:43 OK, I think we have an approach.Just please add me as a review on any LDAP changes. I'll try to keep an eye out for them. make sure LDAP is in the patch description 18:32:53 meee 2 18:33:02 ayoung, gyee: ok! 18:33:23 dolphm: probably time for a new topic 18:33:26 what's the marker parameter? a value from the next entry? 18:33:53 marker is suppose to be the last entry from the previous batch 18:33:54 henrynash: ++ 18:34:01 henrynash I will be happy to review as well 18:34:03 so like uid? 18:34:09 bknudson: yep 18:34:10 ayoung: skip common client auth? 18:34:15 dolphm, no 18:34:25 #topic common client auth 18:34:25 just want people to know: 18:34:30 #link https://review.openstack.org/#/c/28043/ 18:34:38 (ok, I have to duck out….sorry folks…., be back on later) 18:34:43 I submite a revert review for the osl change to auth client 18:34:44 giant patch 18:34:55 https://review.openstack.org/#/c/41578/ 18:35:12 who reviewed the original patch in oslo-incubator? 18:35:14 and I talked with aababilov 18:35:24 we are going to work to get this integrated directly into the keystone client 18:35:41 jamielennox is aware and has responded to him as well. This should be a good approach 18:36:18 I guess we can move it oslo-incubator after it's been used in keystoneclient if that's necessary 18:36:27 ayoung: i'm in favor of the notion... if absolutely nothing else, it's more likely we'll have more security-sensitive eyes on the code that way 18:36:27 bknudson, doesn't really matter. I don't think they were aware that this was supposed to be a Keystone thing, too 18:36:28 but other clients should be able to import keystoneclient? 18:36:33 bknudson, no 18:36:44 bknudson, other clients should pull in keystone client as a library 18:36:56 won't go into oslo 18:37:13 bknudson, "but other clients should be able to import keystoneclient?" yes 18:37:53 much easier to keep it "correct" if keystoneclient owns it and it's not in oslo 18:38:00 ++ 18:38:07 it means we wont run into projects lagging in sync 18:38:19 and causing auth issues. 18:38:21 morganfainberg: +1 18:38:25 we may decide we want to break up keystone client over time. I could easily see it as: command line, keystone common library, keystone client library, and middleware. 18:38:29 morganfainberg: we need to make it stupid easy for other projects to consume us, which is NOT the case today :( 18:38:39 dolphm: yes, that is related to the topic. 18:38:40 different installs might have different levels of the client, and now we have to be very careful of backwards compatibility. 18:38:46 dolphm: and ++ 18:39:08 everything from auth options -> client side token management -> authenticating requests to other services 18:39:12 bknudson: i think we are already having to watch closely on that wrt auth_token. 18:39:19 it would be nice if we could build that out of a single git repo. 18:39:39 dolphm, like secured by default? which does nothing :) 18:39:42 Does shared common client auth also include the service catalog stuff? 18:39:51 cody-somerville, yes 18:40:11 cody-somerville, it is implicit that when you get a token you get the service catalog with it. 18:41:05 because it's broken in keystoneclient trunk currently - region_name get passed to AccessInfo (which would pass it on to ServiceCatalog if it was) and thus region_name is ignored. 18:41:28 and everyone seems to do things like url_for differently. 18:41:36 cody-somerville, file it in launchpad, or if it is filed, please link the bug 18:41:48 I haven't filed it yet but will be doing so today. 18:41:52 service catalog needs a bit more work 18:42:03 the way we expose the service catalog is sad 18:42:07 cody-somerville, there was an expired review for region work, which jaypipes is planing on resubmitting as an extension. 18:42:13 oops, I typoed: region_name *does not* get passed to AccessInfo 18:42:15 cody-somerville, thanks 18:42:25 +1 18:42:27 dolphm, yeah, we need to go back to the drawing board on this one 18:42:45 like how to facilitate API versioned urls 18:43:47 I'll note that it would probably be much appreciated if you guys kept the heat people appraised of any changes here 18:43:55 cody-somerville, will do 18:44:22 eventually we want all clients to use the common auth stuff 18:44:24 #topic open discussion 18:44:45 which means someone has to go through all the other clients and update them to use common auth 18:44:46 there's some high priority code reviews on the agenda -- 18:44:56 #link https://review.openstack.org/#/c/39530/ 18:45:02 #link https://review.openstack.org/#/c/38308/ 18:45:07 #link https://review.openstack.org/#/c/40692/5 18:45:36 Ahh good, KDS has a new patchset. 18:45:39 also, is there anything happening regarding external authentication methods? 18:45:54 #link https://review.openstack.org/#/c/29130/ dolphm, this one too :) 18:46:03 morganfainberg, the API does, yeah. simo will sync the code to API once it is clear there is not too much churn 18:46:12 ayoung: yep, that was what i meant 18:46:57 code reviews from here on out 18:47:04 stevemar, I think oauth is close 18:47:16 ayoung, yay 18:47:22 stevemar: yeah, it's looking really good. 18:47:32 JoeHazzers, what do you have in mind? 18:47:33 stevemar +1 18:47:53 I'll give it another look. I assume no major changes since last I looked. I withdraw the requests for making access tokensi nto Keystone tokens....although it might be worth revisint that in the future. 18:48:07 revisiting 18:48:08 oauth ftw 18:48:19 ayoung: that reminds me... https://gist.github.com/dolph/6198529 18:48:26 gyee: i know someone who wants to integrate kerberos, x509 and other authentication methods with keystone, such that the client and server (if running under say apache) can negotiate and authenticate via other methods than a simple username and password 18:48:44 dolphm, neat 18:48:46 ayoung: my first pass was with PKI-based oauth access_keys ... i switched to AES in the current gist, but will be switching back 18:48:49 dolphm: thats cool. 18:48:58 JoeHazzers, already done in Apache HTTPD 18:49:12 JoeHazzers, would not recommend trying to do it in Eventlet 18:49:14 basically the oauth secret is encrypted into the access key along with basic authz attributes 18:49:29 yes, but how does client discovery or knowledge of these methods work? 18:50:02 project_id, role_names, secret = verify_access_key(access_key) 18:50:03 JoeHazzers, we are working on an extension for that. THe kent federation review is going to split that off into its own extension 18:50:16 okay! 18:51:37 JoeHazzers, the review is here: https://review.openstack.org/#/c/39499/ see the first two items on the list: listing IdPs and listing protocols supported. 18:52:09 dolphm, should that piece end up part of keystone client? 18:52:35 One other review: https://review.openstack.org/#/c/41471/ 18:53:00 ayoung: i am checking on that FK constraint right now 18:53:20 I don't see it in the DB, but I'm going to 2x check to see what is going on. 18:54:08 trying it from a clean slate. 18:54:18 ayoung: yeah, parts of it 18:54:56 ayoung: keystoneclient doesn't have any business creating access tokens, but keystoneclient should be able to verify them (a la keystoneclient.common.cms.verify_token()) 18:55:11 dolphm, yeah, that is what I was thinking 18:55:43 ayoung: keystoneclient.contrib.oauth.verify_access_token() or something? 18:55:48 oauth1* 18:56:52 4 minutes. Any last burning topics? 18:58:03 who's going to IceHouse summit? 18:58:12 I'll be there 18:58:22 I plan on being there 18:58:24 I'll be there. 18:58:29 same here 18:58:33 gyee: hopefully! 18:58:43 jamielennox and simo from RH as well representing IdM 18:58:49 gyee: o/ 18:58:55 ayoung, a much better answer than last time 18:59:07 topol, I even have a book on Cantonese 18:59:21 ayoung, wow 18:59:21 ayoung, I can teach the good stuff :) 18:59:23 ayoung: hehe 18:59:35 我迷路了,請幫我 18:59:39 Im just gonna follow gyee everywhere 18:59:42 hehe 18:59:47 nice one ayoung 18:59:48 topol: that is a good idea 18:59:56 gyee, where we going to dinner. gyee what is this on the menu 19:00:16 topol, I'll let you know what I ordered, after dinner :) 19:00:32 我的氣墊船​​鰻魚 19:00:38 gyee, I've fallen for that before 19:00:55 #endmeeting