18:00:22 #startmeeting keystone 18:00:23 Meeting started Tue Sep 24 18:00:22 2013 UTC and is due to finish in 60 minutes. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:25 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:00:27 The meeting name has been set to 'keystone' 18:00:31 \0 18:00:34 hi 18:00:40 #topic havana-rc1 18:00:42 Oyez 18:00:44 #announcement SO CLOSE 18:00:53 YAY 18:01:08 fyi, you can see every project's progress on http://old-wiki.openstack.org/rc/ 18:01:09 #link https://launchpad.net/keystone/+milestone/havana-rc1 18:01:13 hi 18:01:14 woot. 18:01:14 just 3 bugs left? 18:01:17 just 3 18:01:20 hi 18:01:20 yay 18:01:27 4 with the one that dolphm just triaged 18:01:33 and isn't one gating now? 18:01:35 nooo 18:01:37 or ready to 18:01:54 ayoung: well, that's already in progress, but we're waiting on upstream changes to openstack/requirements 18:02:03 nice 18:02:04 so, i guess let's run through every bug for status 18:02:10 bug 1182861 18:02:12 Launchpad bug 1182861 in trove "Switch to oslo.config 1.2.0 final" [High,Triaged] https://launchpad.net/bugs/1182861 18:02:15 morganfainberg: iep, mine is gatting 18:02:20 (yep) 18:02:22 in progress in keystone, requires an upstream changes to openstack/requirements 18:02:27 i think that'll be trivial 18:02:33 bug 1153645 18:02:36 Launchpad bug 1153645 in keystone "Deleting a role should revoke any tokens associated with it" [High,In progress] https://launchpad.net/bugs/1153645 18:02:50 gating now! 18:02:55 yay! 18:03:10 bug 1210515 18:03:12 Launchpad bug 1210515 in keystone "keystone chokes on empty "description" field in active directory" [Medium,In progress] https://launchpad.net/bugs/1210515 18:03:19 chmouel, can you hit the changes to ^^ 18:03:23 should be trivial 18:03:29 looks like brant had a couple nits, but as far as i can tell, this is really close 18:03:42 dolphm, the order of checks should be fixed 18:03:54 we shouldn't be allowing adding an immutable attribute 18:03:59 ayoung: ahh, i glossed over that comment 18:04:10 I had made the same comment on previous patch set 18:04:28 oh yeah i messed the ordering 18:04:33 tks 18:04:52 I wouldn't -1 for missing ' in didn't 18:04:58 chmouel: thank you! 18:05:00 chmouel, ping when done 18:05:04 yep will do 18:05:10 bknudson: yes you would ;) 18:05:20 I'd -2 18:05:21 i was writting some additional tests for that but i think someone did 18:05:23 haha 18:05:25 hehh 18:05:43 chmouel: dstanek wrote a few in a dependent patch, but i haven't reviewed yet 18:05:51 aaand... 18:05:52 bug 1221889 18:05:53 Launchpad bug 1221889 in keystone "Invalid X-Subject-Token results in HTTP 401 rather than 404" [High,In progress] https://launchpad.net/bugs/1221889 18:06:09 i have two patches with tests and a third that i'm trying to get to pass 18:06:13 morganfainberg, can you explain your comment https://bugs.launchpad.net/keystone/+bug/1221889/comments/7 18:06:14 Launchpad bug 1221889 in keystone "Invalid X-Subject-Token results in HTTP 401 rather than 404" [High,In progress] 18:06:57 ayoung, the tempest change to allow the test to continue has been merged. once we get this bug in, there is another tempest change needed to un-skip and change expected http status to 404 18:07:02 the fix for this one has been slow because it's dependent on changes to oslo and tempest first 18:07:10 now we need to sync up keystone policy with oslo policy 18:07:20 considering we extend oslo policy, i'm paranoid we'll see breakage there 18:07:28 dolphm: yeah.. 18:07:32 oslo changes is in 18:07:36 gyee: ++ 18:07:37 I think they changed stuff to be oo 18:07:37 lets get lots of eyes on that review 18:07:41 but not synced over to keystone 18:07:43 ayoung: ++ 18:07:43 ayoung, i was responding to dolph's comment. 18:07:51 dolphm, I'll take a look at that one 18:07:55 but the oslo's version of policy is way different than Keystone 18:08:06 gyee: thanks 18:08:21 for anyone that has free time, this is definitely the most complex patch of what we have left 18:08:22 we probably haven't merged oslo policy for a while... 18:08:26 bknudson: we haven't 18:08:29 bknudson: in a looong while 18:08:41 time to roll the dice :) 18:08:44 dolphm: well, not since the end of Grizzly 18:08:49 eek 18:09:00 stevemar, oh comon, live a little 18:09:04 henrynash: i was about to say, i think you were the last to do so 18:09:22 hehe 18:09:24 I hit that on the sync for notifier on accident, a lot of the new stuff in oslo's policy is object oriented, its gonna take a bit a refactoring in Keystone 18:09:28 why isn't it synced at the same time we do the rest of an oslo sync? 18:09:34 dolphm: guilty, as charged….although not sure it was actually in oslo at that point 18:10:04 is this going to conflict with henrynash 's changes for entity level policy rules? 18:10:50 ayoung: I don't see why it should…as long as they haven't reduced the capability of the engine 18:10:58 so...question is do we really want to sync now...or can we delay until Icehouse 1. What is in Oslo that we need? 18:11:15 ayoung, have faith in our tests :) 18:11:33 ayoung, honestly, if we can avoid syncing till Icehouse1, i'd vote that. 18:11:33 gyee, jk? 18:11:38 ayoung: I never changed the engine…. 18:11:41 * topol my scaredy cat anture says wait till Icehouse 1 18:11:42 the tests for oslo-incubator are in oslo... do we have our own tests for oslo stuff? 18:11:52 I think sooner is better otherwise two will deviate a bit 18:11:55 we don't have a review for the sync yet, do we? 18:12:02 ayoung, not that i see 18:12:03 so for reference, a full sync to oslo right now would look like this- http://pasteraw.com/h7q6rihq5hhtsz3q478ogtjxtveyeyj 18:12:21 oh shit 18:12:22 and that causes new test failures 18:12:32 dolphm, can you WIP that review? 18:12:35 ayoung: sure 18:12:41 that looks harmless :-) 18:13:00 that actually doesn't look as bad as i'd expect 18:13:06 so.. lots of locale and language stuff... 18:13:11 full sync to oslo: https://review.openstack.org/#/c/48111/ 18:13:38 most of that is fairly simple 18:13:52 policy is +274, -196 18:14:09 and imports fileutils, jsonutils and common logging 18:14:23 isnt policy tricky enough that there maybe uintended consequences??? 18:14:24 and oslo.config, of course 18:14:27 those ain't that bad 18:14:43 topol: yep 18:15:09 policy has new config options, so would need to update sample config too 18:15:36 policy looks relativly minimal. We are one change ahead of their GenericCHeck, the rest looks like they have made an object that gets replaced for refreshing rules 18:16:27 whats the sense of urgency to do this now as opposed to when there is more runway to fix stuff? 18:16:32 on the upside, the entire test suite takes like 2 minutes now 18:16:35 mostly because they all fail 18:16:41 hehe 18:16:56 it introduces a new exception that would need to be plumbed 18:17:05 DuplicateOptError: duplicate option: policy_file http://pasteraw.com/sqlxfrvg4mh7gf9wkzy797v8kgw4ypq 18:17:17 so..the argument for fixing this now is that if thereis a CVE in icehouse, we can replace the coder in all projects at once 18:17:26 a CVE in oslo code, that is 18:18:00 replace the code...the coder will likely have already been replaced.... 18:18:10 topol: this change is currently a release blocker (that could be changed, as i see this as a *very* compelling nice-to-have) https://review.openstack.org/#/c/46123/ 18:18:47 topol: which is currently dependent on a recent change to oslo policy due to the change made in etc/policy.v3cloudsample.json 18:18:55 and how that should be handled in keystone to support this use case 18:19:06 dolphm, topol, we'll have to fix it one way or the other as API spec says 404 should be returned 18:19:13 gyee, ++ 18:19:16 so, we either need an alternative solution to the policy.json change, or we need to sync oslo to make the proposed change work 18:19:34 so its really the cost of backporting we are considering here 18:20:03 or make whatever changes are required to fix to our policy.py and then get full policy in icehouse 18:20:36 yes, we'll have to sync policy, eventually 18:20:49 bknudson: you mean carry some fork of common policy? :( 18:20:55 well if you have folks who feel they can contain it and its impact sounds cool to have... 18:21:01 again, cost of backporting versus the cost of delaying the RC1 release 18:21:01 dolphm: yes :( 18:21:06 i'd like to see a new & old client ran against that 18:21:12 dolphm, maybe sync policy icehouse 1, tag for backport? 18:21:24 morganfainberg, +1 18:21:33 doesn't delay RC, and gives us a longer runway to hit bugs. 18:21:54 morganfainberg +1. thats brilliant 18:22:04 morganfainberg, +1 18:22:16 morganfainberg, thinking things through :P 18:22:18 morganfainberg: you don't see it as backporting features via oslo? 18:22:23 typically don't backport config changes. 18:22:41 bknudson, fair enough. but this might be worthwhile doing in this case. 18:23:09 dolphm, We would probably want to get our change to GenericCHeck merged into Oslo 1st anyway 18:23:16 otherwise we are not really in sync 18:23:24 ayoung: ? i thought that merged 18:23:34 dolphm, don't see it in your review 18:23:41 we've already got a fork of poicy? 18:23:55 ayoung: ? is that not the try catch on line 848 https://review.openstack.org/#/c/48111/1/keystone/openstack/common/policy.py 18:24:03 bknudson: in review, not in master 18:24:43 dolphm, duh, yes it is...I was reading it backwards. I was thinking we had applied that change to our repo. 18:25:37 ayoung: *whew* i thought i was crazy for a minute 18:25:41 * dolphm but thankfully it's just you 18:25:42 :) 18:26:13 ayoung: it's proposed here (and the reason for my -1) but not merged https://review.openstack.org/#/c/46123/11/keystone/openstack/common/policy.py 18:26:26 dolphm, if you are taking about https://review.openstack.org/#/c/46589/, I think it is in master now. 18:26:27 dolphm, just cuz we know I'm crazy doesn't mean that you aren't. You are just less crazy than me: you really need a higher bar than that. 18:26:50 ayoung: now you're being too logical 18:27:00 heh 18:27:15 Im sure in Hong Kong we can find interesting food options and we can really see who is crazy... 18:27:21 atiwari: in oslo master, yes... but not keystone.common 18:27:27 topol: ++++ 18:27:28 i'm in 18:27:35 topol, should be fun. 18:27:37 ok 18:27:41 chicken feet was new for me not that long ago 18:28:00 topol, but gyee has a serious advantage in that he knows what the waiters will be saying. 18:28:02 atiwari, we just need to leave that one out as oslo sync is in a separate review 18:28:24 ayoung, I'll make sure I translate "properly" 18:28:30 yes. yes he does. 18:28:33 gyee, I'm sure you will 18:29:03 I'll be worries when gyee's translation is "there is no word for it in english, but… it's good, just eat it" 18:29:14 ha! 18:29:21 :) 18:29:36 atiwari, so just make your review a dependent of dolphm's 18:29:37 right, or when he will say, it is like chicken nuggets :-) 18:30:01 sounds like sync for RC it is then, gyee ? 18:30:08 gyee, OK 18:30:31 what is the official Keystone stance on etc/policy.v3cloudsample.json ? Is that going to be the default approach to policy moving forward? 18:30:41 gyee: except the full oslo sync is failing tests 18:30:55 I think we should fix it as it impacts API spec 18:31:15 cuz I like it a lot better than the default policy.json. I would like to see the tests run against that. 18:31:30 ayoung, + 18:31:31 + 18:31:42 henrynash, nice work on that, BTW 18:31:48 ayoung: thx 18:31:54 dolphm, lets fix the tests first 18:32:31 gyee, I'm going to try and splitting the sync, to see what happens if we sync only the policy.py file first 18:32:36 ayoung: fyi, there are a set of tests already that run against it….but only as a specific unit test 18:32:50 ayoung, lets do this 18:33:00 henrynash, yeah...understood. And we can't just blindly swap over to it, as that will break all the current deployments out there 18:33:04 #topic open discussion 18:33:11 So.. policy 18:33:31 should we absorbe the policy code into Keystone again 18:33:37 and if so, does it go into keystone client 18:33:52 or should be make it a standalone library 18:33:53 there was a comment on the mailing list that they wanted middleware out of keystoneclient 18:34:12 bknudson, or should we make a keystonemiddleware project 18:34:21 ayoung, we probably should do that 18:34:32 how does the standalone library work? it would be treated more like keystoneclient? 18:34:33 middleware should stay with keystone client 18:34:38 or is it released like keystone? 18:34:41 too much of a dependency 18:34:42 standalone library that is. 18:34:50 (policy) 18:34:56 what's the dependency between keystoneclient and middleware? 18:34:56 I would not mind seeing something like python-openstackmiddleware python-openstackpolicy and python-libkeystone 18:35:15 ayoung, ++ (not oslo.policy?) 18:35:16 auth_token will be using keystoneclient's requests part 18:35:46 bind token check 18:35:49 cms 18:36:01 pluggable auth for admin token request, etc, etc 18:36:04 we'd have to duplicate cms again. 18:36:14 that would be bad 18:36:23 what's the convincing argument for split it out? 18:36:38 short of moving cms into a (similarly) common library, it would be silly to duplicate it again 18:36:40 gyee actually there is no dependency on client from auth_token 18:36:55 morganfainberg, the policy rules are tightly coupled with the token implementation. The generic policy engine is more general purpose. I don't care about the naming, but I would like to have the policy available as a reusable component 18:36:58 there should be... 18:37:11 jamielennox, thought you are going to make to fix for using requests lib for everything? 18:37:14 libkeystone should be for code common to server and client 18:37:20 gyee, he is... 18:37:21 what's the argument in favor of splitting middleware out of keystoneclient? 18:37:35 yea, requests has happened, but it doesn't use the client's requests path it's got its own 18:38:20 ayoung, I think policy probably does not belong in keystoneclient. but i think it should absolutely be moved into it's own library (and i don't think it would be improper to be handled by keystone to maintain it, but that is a separate argument) 18:38:20 jamielennox, thought you are going to do that next as we are currently duplicating code 18:38:22 bknudson: there should be a stronger dependency there, i know it's on jamielennox's wishlist as well as mine to rewrite auth_token to use the regular old keystoneclient ... http client 18:38:36 dolphm: the best one i've heard is being able to release auth_token without having to do a full release of the client code 18:38:57 jamielennox, that's not a convincing argument :) 18:39:04 there are reasons that auth_token should not be pegged to the current minimum required version of keystoneclient which i think is still 0.2.1 18:39:05 jamielennox: that a weak argument, without an example of a specific reason why you'd want to do one without the other 18:39:25 jamielennox: 0.2.1 according to who? 18:39:30 whoa, this isn't my push... 18:39:37 dolphm, I suspect it is just whinging from people that have pegged their client dependencies to a low version... 18:39:56 we had some tempest/gate breakage due to that and incompatbile client versions 18:39:58 ayoung, one line chef change :) 18:39:59 ayoung, and that isn't solved with spliting it. they'd just pin the middleware too :P 18:40:06 heh 18:40:17 apologies it's been updated to 0.3.2 - i ddidn't know that 18:40:20 ayoung: that's why we mandate minimum client versions, but not to pin them to a max 18:41:01 gyee, and it is definitely on my todo list 18:41:20 morganfainberg, so...I don't really havea burning reason to split them, rather it is more from a cleanliness perspective. If I am doing CLI operations, I don't need middleware, and if I am runnign a service, I don't need CLI. There is no real reson to split them, other than they pull in other package dependencies that may not be appropriate in all situations 18:41:36 ayoung, agreed. 18:41:43 auth_token middleware is differnt from the client in that it is priamrily a token consumer 18:41:53 CLI is p[rimarily a token requestor 18:41:53 speaking of which, can somebody take a look at https://review.openstack.org/#/c/47661/ 18:42:03 CLI in keystoneclient is deprecated. 18:42:07 I sorta leave the door open for pluggable auth for admin token request 18:42:11 now, auth_token does request tokens for some usages, but that almost seems gratuitous 18:42:17 or, to be deprecated 18:42:20 ayoung, if we had say, libkeystone i could see auth_token being split logically. 18:42:29 you should not have to ask Keystone for a token in order to to a keystone based operation 18:42:35 but right now, we're still cli+libkeystone 18:42:58 this might be worth reconsidering when openstackclient is ready to go? 18:43:00 use the admin token 18:43:13 gyee, any chance that we can just do that base on admin_password == None for now rather than a new config option? 18:43:14 I don't see any real problem with splitting a growing project into more granular packages. 18:43:50 jamielennox, but that's a hack, I rather do this properly 18:43:53 ayoung, don't get me wrong, i'm infavor of more granular development. 18:44:22 i'm thinking the issue will go away if we can get the APIClient stuff in and then move that into auth_token 18:44:25 ayoung, but we might have bigger fish to fry until a bit later on. 18:44:28 morganfainberg, so the question is: what should the layout be? 18:44:36 ayoung, that is a valid question 18:44:53 p.s. i thought of an interesting use case the other day for not deprecating keystoneclient's CLI that no one here probably cares about 18:45:13 gyee, also the admin_token is required for fetching a revocation list - which is required for PKI operations as well, so we do kind of still need it around 18:45:17 If we assume that the typical usage is client on one machine, server on a second, and auth_token on a third...and we only want to put code essential to those operations on each machine, I think the layout becomes clear 18:45:35 dolphm, do tell 18:45:36 dolphm, ok i'll bite, share :) 18:45:54 "I want to deploy keystone for use with not-openstack, so I don't want to use python-openstackclient to interact with keystone on the CLI." 18:46:02 jamielennox, I basically make it optional 18:46:15 libkeystone gets common functionality, but is only usable when called from python. auth_token gets a reduced role to call into the library for its operations. CLI can be replaced with the common CLI without disrupting or dual deploying 18:46:29 hmmm. sounds like heresy 18:46:40 I hope openstackclient would be pluggable, so you could only plug-in keystone function 18:46:58 we'll need to update keystone CLI for V3 18:47:09 dolphm, actually, good point. my friend is looking ot use keystone for a non-openstack project. 18:47:13 bknudson, ++ 18:47:15 every now and then someone will be interested with deploying keystone as a standalone authn/z service for something totally unrelated to openstack 18:47:21 i've never heard of anyone actually doing it 18:47:23 dolphm, actually, I've had a request for that. And then I pointed out that they were a Java based project and Keystone was python. I politely suggested they look into Jython and never heard back from them 18:47:24 oh no its spreading 18:47:43 dolphm, it's likely going to happen in a month or two for sure (was talking about this last night) 18:47:53 ayoung: lol 18:48:02 morganfainberg: let me know when it happens :) 18:48:07 dolphm, for sure. 18:48:54 this is just a theoretical, and it's not a strong argument for NOT using openstackclient, but i've heard similar "but i don't want to use completely-useful-thing X just because it's called X" 18:49:04 Looks like change for bug 1201251 is ready. 18:49:05 Launchpad bug 1201251 in keystone "issues of updating user via keystone rest api" [Medium,In progress] https://launchpad.net/bugs/1201251 18:49:12 dolphm, so...keystone is really two things now. One is a very limited IdP. The other is that it is an authorization mapping service. I think it is this second role that people want to tie in with. As such, all of the discussion around Federation becomes much more interesting 18:49:42 I've started a Q&A around Federation on the old federation etherpad, and dchadwick has answered some questions on it today 18:49:52 https://review.openstack.org/#/c/42826/ Add user to project if project ID is changed 18:49:52 ayoung, link? 18:49:52 ayoung: i'd argue that we've always been that, and both sides of our authn + authz abilities have expanded quite a bit 18:50:03 https://etherpad.openstack.org/keystone-federation 18:50:06 gyee ^ 18:50:09 we have stuff for the new federation etherpad when it becomes avail 18:50:13 still can't require a password to change every 6 months with sql 18:50:21 #link https://etherpad.openstack.org/keystone-federation 18:50:22 bknudson, i'll look at that review as soon as we're done here. (1201251) 18:50:27 topol, i'm going to add to it now 18:50:38 still can't require certain chars in password, can't lock out users after bad attempts 18:50:43 Thanks stevemar 18:50:50 meetbot working? 18:51:00 lets try to keep that page somewhat oragnized, so we can use it as the basis for making decisions around Federation based features 18:51:02 #link https://etherpad.openstack.org/keystone-federation 18:51:02 hmm #link https://etherpad.openstack.org/keystone-federation 18:51:09 wtf? 18:51:11 gyee, meetbot has been odd lately 18:51:23 gyee, probably og picked up, 18:51:31 it doesn't tell you on all successful operations 18:51:35 k 18:52:07 oh, is this going in: #link https://review.openstack.org/#/c/46363/ 18:52:44 did we want to get the optional dep stuff in for Havana? 18:52:56 i thouht we did? 18:53:01 thought 18:53:09 so, it sounds like the three main approaches to Identity are going to be LDAP for corporate, SQL for a self contained deployment, and SAML for communicating with a remote IdP. OAuth is very SAML like. The old mapping blueprint/backend is proably best thought of as an extension to the identity bakcend: I got it in form X, here it is as a userid or as a group. 18:53:09 dolphm ? ^ 18:53:26 the oauth dependency was a problem for some packagers. 18:53:42 bknudson, ok, i'll look at that one as well when we are done here 18:53:50 sorry, was looking back to check on meetbot 18:54:04 stevemar, thanks for brining that up. I'd like it to go in. Even if it is a short lived feature, 18:54:29 why shortlived? 18:54:29 yeah, i think it went by the way side for a little 18:54:32 I'd like to Keep keystone from pulling in the Oauth dependency at the package level 18:54:44 ayoung: stevemar: that's a good breakdown 18:54:45 topol, because, as the comment suggests, it is backwards 18:54:58 didnt see that. thanks 18:55:32 topol, I've been thinking that the token provider should be a pipeline for some time now...suggested it back when gyee did the origianl -plugability stuff. 18:55:48 We want to be able to inject or extract something like OAuth in to the larger workflow 18:55:54 ayoung ++ 18:55:58 something built out of tree, say 18:56:03 that sounds cool 18:56:09 without having to change core Keystone to support it 18:57:10 ayoung: thankfully we have the WSGI pipeline to do exactly that :) 18:57:11 topol, and the paste code supports that kind of pipeline, but we don't have time to do it completely for Havana. So I would say, do option dependencies for Havana, and use that to insule the oauth changes. Then, we can pipeline up token creation later. 18:57:18 ayoung: we just have to take better advantage of it 18:57:19 dolphm, ayoung, if that stuff is going in, we should give the bug an rc1 tag 18:57:28 dolphm, ayoung , henrynash: last question while we have people here. the domain_scope cleanup. should I even try to get that ready for RC timeline? or did we just want to mothball it until I-1? i think it's got a lot of pitfalls we can't address even in cleanup until Icehouse 1 18:57:32 dolphm, yes, exactly 18:57:41 dolphm, I want to make better use of that across the board. 18:57:43 ayoung ++ 18:57:49 * dolphm hugs ayoung 18:57:57 lets do this 18:58:02 i'm not fond of the optional dependencies as implemented. it feels like a bandaid 18:58:10 dolphm, for instance, we should not be explicitly starting each of the managers in service.py. They should be activated by their inclusing in the pipeline, like the extensions are. 18:58:14 dstanek: it very much is a bandaid 18:58:24 dstanek: post comments on the review! 18:58:25 #endmeeting