18:02:07 #startmeeting Keystone 18:02:07 Meeting started Tue Nov 18 18:02:07 2014 UTC and is due to finish in 60 minutes. The chair is morganfainberg. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:02:08 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:02:10 The meeting name has been set to 'keystone' 18:02:12 hi 18:02:17 I was guilty of that last year... 18:02:26 poor jamielennox probably will get an alarm clock wake up in 1 hour 18:02:27 Welcome back! Hope people had a good summit and a nice week since the summit 18:02:28 I feel accomplished that I remember 18:02:49 dolphm reminded of that yesterday 18:03:01 #topic Stable Maintenance Liaison 18:03:07 #link https://wiki.openstack.org/wiki/CrossProjectLiaisons#Stable_Branch 18:03:23 I'd be willing to sign up for this since we kind of need it 18:03:27 While I can do this, I would prefer to have someone else to also rely on. 18:03:58 #startvote Nominate bknudson 18:03:58 Only the meeting chair may start a vote. 18:03:59 i'm on board as well 18:04:05 bknudson: is the liaison of liaisons 18:04:13 dolphm's got 2 already 18:04:20 haha 18:04:23 #winning 18:04:32 i'll let you guys figure out which one will do it. 18:04:38 * ayoung winning by not having to be stable liason 18:04:58 #action dolphm or bknudson as stable liason. They'll battle it out - mad max style to see who gets it. 18:05:35 bknudson: fisticuffs in #openstack-keystone later? 18:05:51 "Two man enter, one man get extra work!" 18:05:53 just update the wiki and let me know who it ends up being. 18:06:02 morganfainberg: ++ 18:06:07 dolphm: you can take it. I'm busy enough. 18:06:18 bknudson: lol alrighty 18:06:18 I'll try to do more reviews in stable 18:06:21 #topic Mapping engine - do we need some enhancements? 18:06:26 marekd o/ 18:06:32 hello 18:06:42 plus I can never figure out the test failures. 18:06:49 morganfainberg, I think we need, at least the ability to split the REMOTE_USER value into userid and domain 18:06:53 bknudson: that's the hard part 18:06:58 * morganfainberg needs to also do more stable reviews. 18:07:04 jamielennox, good morning! 18:07:06 sorry 18:07:12 emergency 18:07:15 no worries 18:07:19 ayoung: mmm 18:07:27 i think marekd is running away? 18:07:31 i can answer the question - we do need enhancements. 18:07:32 so, during the summit i heard some plans and opinions about 18:07:34 mapping engine 18:07:47 stevemar: i am not running away 18:07:53 here comes that singularity 18:08:01 e.g. https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting there was a thread here 18:08:18 i remember nkinder was mentioning something about dynamic group creation 18:08:35 marekd, ++ 18:08:40 i see everybody have some plans and opinions and i'd like to gather them and try to work something out. 18:08:48 marekd, more specifically group-passthrough, so it was possible to just pass group info across not need a keystone group every time 18:08:53 etherpad? 18:08:55 ++ 18:08:57 that too 18:09:01 etherpad would be good. 18:09:32 18:09:40 https://etherpad.openstack.org/p/mapping-engine-enhancements 18:09:53 #link https://etherpad.openstack.org/p/mapping-engine-enhancements 18:10:00 morganfainberg: thanks. 18:10:27 we want to help with that as well 18:10:32 I just worry about going too far here. like the dev thread suggested 18:10:33 cc vsilva_ 18:10:44 rodrigods: fine, thanks, but we still don't have a clear roadmap :-) 18:11:01 marekd, we can also help with the roadmap =p 18:11:02 #action please contribute information to the etherpad, we can distill out the requirements and what is really needed and what would be too much. 18:11:08 #link https://wiki.openstack.org/wiki/Summit/Kilo/Etherpads#Keystone 18:11:19 concrete use cases would be nice 18:11:39 topol: ++ 18:11:43 marekd, i have some use-cases for enhancements that "make sense" afaict, but they should be easy sells compared to some of the other things. 18:11:51 topol, See the complexit y from gyee's X509 spec he pulled: bascially, the subject of a cert 18:11:54 but yes, include the use-case in the etherpad 18:12:01 morganfainberg: ok, thanks. 18:12:11 topol, also, REMOTE_USER as set by mod_auth_kerb 18:12:16 and the REMOTE_UISER value from SAML 18:12:32 ayoung, I vote we make the variable REMOTE_UISER in all cases now 18:12:32 ayoung that should work now.. with https://review.openstack.org/#/c/133037/ merged 18:12:44 morganfainberg, can't 18:12:51 ayoung, darn 18:12:52 anyway 18:12:54 morganfainberg: and to be honest i don't know much about nkinder's proposition 18:12:57 for example, REMOTE_USER might not be the right value from mod_ssl 18:12:57 regarding groups 18:13:08 marekd, we'll get details outlined in the etherpad 18:13:11 i was hoping he would explain it here pro publico bono :-) 18:13:16 #link http://www.freeipa.org/page/Environment_Variables 18:13:21 morganfainberg: allrighty. 18:13:27 And SAML is pretty much An LDAP query 18:13:34 does it all deserve a spec for the kilo release? 18:13:35 marekd, it should be straight forward once it's laid out like that. 18:13:50 marekd, likely we will want a spec to encompass the changes. 18:14:02 but lets see what comes from the ehterpad first 18:14:08 yeah, especially if it's new functionality/feature 18:14:09 might be that we say "nope not going to happen" 18:14:12 and not just a bug fix 18:14:20 for $REASONS$ 18:14:22 probably the most common case is split an env var into two pieces, one is username or userid, the other is domain info 18:14:26 engine is not broken as is... 18:14:32 it simply may lack some features 18:14:53 ayoung: i'd still concatenate it 18:14:55 marekd, correct. thats why i think we need to see the details and then write a spec to encompass the enhancements - clearly detailing the SoW/scope 18:15:05 morganfainberg: great 18:15:15 i can take care of the specs if we need it. 18:15:19 #link https://jdennis.fedorapeople.org/doc/mapping.pdf John Dennis' proposal 18:15:22 it will help us prevent things sneaking in that are "bad ideas"™ 18:15:48 ayoung: they made more powerful engine 18:16:07 ok, if anybody else has some usecases please add it to the etherpad 18:16:19 marekd, "They?" he's on my team. He did this all by himself. jdennis kindof been through this once or twice before 18:16:29 I am going to say right now we will for this cycle at least *not* be making the mapping engine configurable - we will maintain 1 mapping engine, even if it means swapping out what we have whole-sale. I don't want to fight with pluggable mapping engines at this point. too many ways for things to be exposed in bad ways 18:16:53 morganfainberg +++ 18:16:59 ayoung: ok, great, jdennis, *himself*, from ayoung's team has made a much powerful engine. 18:17:02 I think that is fine 18:17:09 morganfainberg: ++ seems dangerous if done wrong 18:17:33 he was doing it for opendaylight. So one benefit to his approach is it would work with another open source project 18:17:37 I'm on board with that 18:17:51 ayoung: i know. I really appreciated his e-mails 18:18:04 and it was *really* meritorical stuff. 18:18:15 seems like if there's an existing mapping tool we should use it rather than try to develop our own 18:18:19 that's why i want to drag your attention 18:18:22 I would say, though, that he lacks the APIs we need for management. Those are really part of the Federation spec today, and assume the existing mapping language 18:18:34 ayoung, and it might make sense to be compatible - my only hard line is i don't want pluggable engines for many many reasons, mostly around security and making the features solid before we open it up to massive headaches. 18:18:40 bknudson, technically, ours existed first 18:18:41 and answer the questions: do we need something new? some small fixups, or maybe we should take John's engine? 18:18:56 also, his iirc is java and would need to be ported to python if we used it. 18:19:02 he wrote his this year as part of his work on Open Daylight. 18:19:13 marekd, ++ those are exactly the questions we should answer. 18:19:17 if nothing else that doc is awesome 18:19:18 java ugghh 18:19:28 git clone git://fedorapeople.org/~jdennis/federated-mapping.git 18:19:34 and would john's engine "fix" the other needs or would we need to expand it as well. 18:19:47 s/"fix"/"meet" 18:19:49 looking to see if he did the python yet 18:19:57 maybe we can make the python version an openstack project 18:20:13 bknudson, I'd support that. 18:20:15 bknudson: really? 18:20:22 if we go down that path 18:20:22 we're just dealing with arrays and dicts, it shouldn't be hard to port it over 18:20:28 bknudson: morganfainberg like a standalone project? 18:20:32 marekd, makes sense, since it's goiing to be pluggable 18:20:33 it's pretty common... I think this was done with some other python projs that openstack uses. 18:20:45 marekd, it would be a lib like policy 18:20:49 sqlalchemy-migrate for example 18:20:51 morganfainberg: ok 18:21:00 i could probably give a helping hand with that. 18:21:02 marekd, not a "stand alone service" 18:21:07 let's just start simple on the etherpad with requirements and gaps - then this discussion will be more valuable 18:21:15 +1 dstanek 18:21:15 dstanek: ++ 18:21:16 but yes, lets get requirements in the etherpad 18:21:18 there is python code there 18:21:30 or a spec to detail why this approach is needed and what problems it solves 18:21:55 ayoung, if you could poke john to look at the etherpad as well and comment re his mapping engine once we have the use-cases 18:22:05 will do 18:22:07 see where we land. 18:22:38 #action ayoung to talk to john dennis about mapping engine and use-cases from etherpad: https://etherpad.openstack.org/p/mapping-engine-enhancements once populated with requested enhancements 18:23:00 morganfainberg, me and vsilva_ will be on top of that as well 18:23:03 joesavak, thats the point of the etherpad. 18:23:10 woot 18:23:14 joesavak, post meeting need to bug you fyi. 18:23:18 ok moving on. 18:23:23 #topic K2K federation - status 18:23:27 marekd, all you again! :) 18:23:35 allrighty, was hoping to see gyee here 18:23:40 here 18:23:46 gyee: oh, sorry! 18:23:47 sorry I got stuck in traffic 18:23:48 magic. 18:23:57 * morganfainberg waves hands "magic" 18:24:12 so, K2K 18:24:25 hoping to see megan fox here 18:24:30 joesavak, shh 18:24:30 here! 18:24:39 ++ :) 18:25:19 so, K2K I wanted to ask if any of you had some experience or tried it out. It was marked as experimental in Juno and I feel there should be some fixes. 18:25:29 i know rodrigods make a setup without security 18:25:34 marekd, we did and it works! 18:25:36 marekd, i am planning on setting up a multi-way federated cloud test env this week. 18:25:42 i know gyee and Sam were playing with that 18:25:49 gyee: did you turn off crypto? 18:25:50 so i expect i'll also have feedback on it. 18:25:55 Is K2K still marked as experimental? 18:25:59 no 18:26:02 bknudson, yes. 18:26:09 gyee, marekd, AFAIK, Sam disabled security as well 18:26:12 bknudson, i hope it will move to "stable" this cycle. 18:26:20 at least was the last update he sent to me 18:26:23 gyee: so it worked with a proper assertion signature validation? 18:26:23 I do have a question on federation though, with the way it works, we have to use different URL for different providers 18:26:27 bknudson, but we're just starting to get drive-time on it now. 18:26:38 rodrigods: that's what he told me . 18:26:39 we need tempest tests! 18:26:42 yes 18:26:49 bknudson, ++ that is part of why i'm setting up the test env 18:26:49 rodrigods, marekd, yes, that's right, I have the signature validation disabled 18:26:56 leonchio_: :/ 18:27:00 =( 18:27:07 because of the way redirect works 18:27:08 morganfainberg: which test env specifically? 18:27:23 leeantho, did you try using the same issuer for both IdP and SP? 18:27:24 also 18:27:34 lbragstad, i'm going to be standing up a multi-node / cloud env with k2k + other federation. specifically so we can outline how we're testing it 18:27:37 leonchio_, * 18:27:40 oh, it was disabled? 18:27:44 AFAIR there were some plans of different regions list depending on who is having a local token 18:27:47 morganfainberg: ah, that's right... 18:27:49 stevemar: recall anything like that? 18:27:54 lbragstad, infra can do multi-node environments now. (but it might be expirimental still) 18:27:58 we are missing the region in the catalog as well 18:28:07 rodrigods: hmmmmm, are we? 18:28:10 yep 18:28:14 last time i tried it I think i had it 18:28:30 not for me, here 18:28:41 joesavak: we don't want *any user* to see all the clouds that one can burst into, do we? 18:28:44 stevemar: ^^ 18:28:56 this should also be fixed, right? 18:29:05 marekd, i think that can be fixed with the filtering? 18:29:19 leverage endpoint filtering and expand it to region filtering? 18:29:25 marekd - they should be able to pull a list of trusted service providers so they know which provider to reference when getting the saml assertion from the local cloud to pass to other (service provider) clouds 18:29:28 marekd, i don't think we put any regions in the catalog right now 18:29:45 stevemar, ++ 18:30:13 joesavak: but all the SPs ? 18:30:15 but we can group endpoints based on regions 18:30:18 stevemar: i thought we do 18:30:36 stevemar: rodrigods i will check it, maybe i messed sth up. 18:30:36 its an attribute of endpoint isn't it? 18:30:48 marekd - perhaps rbac, or relating which domains can access which SPs is needed... 18:31:02 oh i didn't think about filtering at all 18:31:08 i can see that some restriction would be good - but I think it's supposed to be a list-all right now 18:31:14 also regarding having SPs still feel that it's a bad idea to put anything in the catalog that can't be accessed with the current token 18:31:17 stevemar: morganfainberg but isn't filtering a client-initiated action? 18:31:26 marekd, no endpoint filtering is server side 18:31:36 https://github.com/openstack/keystone/blob/master/keystone/catalog/schema.py#L73 18:31:38 morganfainberg: on what basis? 18:31:39 it doesn't prevent use of a target 18:31:49 joesavak: aha, ok 18:31:55 joesavak: i thought you would be concerned. 18:31:58 marekd, but it's a cloud provider can filter endpoints per project etc 18:32:08 for now i think filtering out targets is an enhancement 18:32:16 not a design of the current system 18:32:20 aha, joesavak ++ 18:32:40 one last thing. 18:32:51 currently we add only one url to a region 18:33:05 which is a SP URL where the assertion should be send. 18:33:07 a little concerned - but should be manageable - via RBAC or even endpoint assignment to identity 18:33:41 joesavak, security is a manageable condition :D 18:33:48 gyee, hah 18:33:54 implementors choice. ; ) 18:34:00 but this means, a client needs to know another URL apriori, an endpoint where he should go once authenticated. 18:34:08 marekd, ++ 18:34:17 and it put a lot of concern on people. 18:34:21 yeah, that's a tricky point 18:34:25 yes 18:34:40 so I suggest we add two url to the bursting regions 18:34:40 since we do not have a real relay_state =P 18:34:51 marekd, fallback url? 18:35:01 you mean? 18:35:12 one being the foreign keystone URL and the other being what? 18:35:23 joesavak, the auth URL 18:35:29 joesavak: let's say the protected url is sp-keystone.com/secure 18:35:38 rodrigods, but that should be discoverable? 18:35:48 joesavak: but the assertion is gonna be sent to sp-keystone.com/shibboleth.sso/ecp 18:36:03 morganfainberg, I need to know how the SP called my IdP 18:36:13 which should be a out-of-band operation 18:36:14 and today we configure the latter only 18:36:14 I think 18:36:43 response from assertion call is a scoped token, right? So can I get the auth URL from that scope? 18:36:56 GET v3/endpoints -H X-Auth-Token: $scopedToken 18:37:17 or unscoped even 18:37:17 joesavak, that's exactly the URL we need to go to get the scoped token 18:37:34 joesavak: is this GET to IdP or SP ? 18:37:42 GET to SP 18:37:48 v3/identity_providers//protocols//auth 18:37:54 how do you know this url 18:38:12 no, wait. 18:38:18 marekd, isn't that the URL in the region? 18:38:39 morganfainberg, the one in the region is the SP url 18:38:43 morganfainberg: no, the one in the region is for POST /Shibboleth.sso/ADFS (e.g) where you send your assertion 18:38:44 like Shibboleth.sso/ECP 18:39:24 so marekd wants another field to store v3/identity_providers//protocols//auth 18:39:26 right? 18:39:28 and then, usually SP responds with HTTP 302 and location set to keystone.example.com instead of keystone.example.com/v3/OS-FEDERATION/identity_providers/protoc/.../auth 18:39:33 rodrigods: yes 18:39:51 it's all because the workflow is so called IdP initiated authentication 18:39:55 Gotcha - because we made the protocols extensible - the URL may differ outside of the known keysotne contract for assertion processing - so 2 URLs are needed 18:40:00 ah 18:40:05 joesavak: exacly! 18:40:07 joesavak, ++ 18:40:09 20 minutes left 18:40:10 jamielennox, how's that going to work with keystoneclient? 18:40:12 that seems reasonable 18:40:23 but it will need client side work i think 18:40:24 I don't think that auth url is discoverable 18:40:25 Ok , I'm ok with that - federationURL and keystoneURL (optional) 18:40:54 joesavak: yeah. 18:41:01 joesavak, would easy a lot to automate the clients 18:41:13 is this for the Horizon use case? 18:41:36 gyee: i think it's specific to the federation auth plugin 18:42:05 rod - didn't understand your comment 18:42:06 jamielennox, I mean it doesn't seem to follow the usual client workflow 18:42:21 gyee: having all of this stuck in a region doesn't follow the regular workflow 18:42:27 joesavak: he meant that clients would need to figure out one extra utl 18:42:28 we could put the auth urls in the json-home document somehow 18:42:30 url 18:42:34 yup 18:42:36 joesavak: they could work fully automated. 18:42:37 marekd, joearnold exactly 18:42:44 i tried to break the region == SP think prior to kilo but didn't get there in time 18:42:45 joesavak, * 18:43:13 jamielennox, why is it wrong to assume a region == SP ? 18:43:21 in this case? 18:44:07 1. makes it confusing with how people currently use regions 2. breaks the concept that everything in the service catalog can be accessed with the current token 18:44:29 (from memory #2 still applies - maybe changed since then) 18:44:36 morganfainberg, the client discovery code won't work as is 18:44:38 except the region *only* has a url. 18:44:45 not endpoints in this case. 18:44:56 morganfainberg: joesavak one thing, just remembered. 18:45:09 shouldn't we also add some *another* parameterer to the region? 18:45:17 like the federated protocol? 18:45:18 ok this needs more discussion - we can enhance/change this but we have other topics to get to in this meeting. 18:45:21 morganfainberg: which seems like we shoehorned something into a current resource with edge cases rather than just make a new resource 18:45:42 we can make this better since k2k is expirimental if it's really warranted 18:45:43 today we have only saml2, but tommorow regions/SP A,B,C will be saml2 only, but M,N OIDC 18:45:47 this is the point of experimental 18:45:53 morganfainberg: we could have made an SP resource and put that into the catalog 18:45:53 but the case needs to be really strong. 18:45:54 how am i going to be able to distinguish? 18:46:33 marekd, so i think we should evaluate shifting to an SP resource or similar 18:46:39 jamielennox: that would be not a bad idea 18:46:40 marekd: at this point you're taking over the region concept and i would vote to making a new SP resource in which all of this makes sense 18:46:41 morganfainberg: ++ 18:46:42 marekd, if SP is a resource in SC, we could filter it 18:46:44 or at least a clear way to identify a "Region" is a resource 18:46:50 yeah - we should have the protocol the SP uses to be discoverable. 18:46:50 or whatever 18:46:54 but yes. 18:47:06 lets evaluating making that shift before we stabilize k2k 18:47:13 morganfainberg: so, whole new api for adding service providers, just like we have IdP now? 18:47:21 stevemar: joesavak ^^ ? 18:47:24 #action Evaluate Service Provider as part of the Catalog 18:47:27 +1 18:47:35 Heh...that was inthe origianl Kent proposal 18:47:46 ayoung, funny how things come full circle 18:47:51 ok #topic HM releases planning 18:47:53 erm 18:47:56 #topic HM releases planning 18:48:00 \o 18:48:03 raildo, rodrigods o/ 18:48:06 o/ 18:48:12 So, this is our planning for HM in Kilo: 18:48:13 release early release often 18:48:23 Kilo-1 the base implementation about HM merged and the new specs about HM improvements https://review.openstack.org/#/c/135309/ and the Reseller Use case (I'm writing it) merged too. 18:48:29 ayoung ++ 18:48:43 raildo, thats a good target 18:48:49 Kilo-2 the entire code about HM and Reseller use case committed. 18:48:56 ++ 18:48:58 so, for Kilo-1 18:49:04 please review https://review.openstack.org/#/c/117786/ and the following patches 18:49:05 =) 18:49:12 rodrigods, ++ 18:49:13 raildo, once the base code is merged we will need to resolve the rebase against master to topic branch 18:49:22 then we do a simple topic branch -> master merge 18:49:42 morganfainberg, yes, i believe that we can do that in kilo-1 too 18:49:44 should be straight forward. 18:49:45 yes 18:49:51 really wanted it to land before henrynash refactoring 18:49:51 hehe 18:49:59 rodigods: eek 18:50:01 rodrigods, race to rebase. 18:50:07 hah 18:50:21 and for Kilo-3 just reviews and resolve some bugs related to HM (if it appears :) ) 18:50:38 makes sense 18:50:43 So, We have this Wiki about HM Meeting: 18:50:47 rodigods: we should just talk about which way makes the most sense 18:50:47 #link https://wiki.openstack.org/wiki/Meetings/HierarchicalMultitenancyMeeting 18:50:56 (just follow the Keystone pattern) 18:51:00 the idea is that Kilo-2 code won't be a feature branch anymore 18:51:08 rodrigods, that is my hope 18:51:20 the HM meetings on Friday at 16:00 UTC here in #openstack-meeting I invite everyone to participate 18:51:26 :) 18:51:43 raildo, do what we do at the start of this meeting: 18:51:47 rodigods: my go 18:52:03 paste in the names of the keystone contributors that you want there 18:52:08 ok so we can quickly get to the last couple topics we have. 18:52:09 ayoung, ++ 18:52:11 we need to keep moving. 18:52:12 ayoung, sure 18:52:25 morganfainberg, ok 18:52:27 #topic Oslo-Incubator Policy Graduation to Keystone Program 18:52:36 We are adopting the policy lib 18:52:44 w00t! 18:52:48 w00t 18:52:55 it'll be run like pycadf (separate core team), if you're not interested as keystoen-core you wont need to be on core 18:52:59 and we are waiting on fileutils to move to oslo.utils, 18:53:01 though i hope everyone will want to be core on it 18:53:07 ayoung, we decided to wait? 18:53:15