18:00:40 #startmeeting keystone 18:00:41 hi-hoooooo 18:00:41 Meeting started Tue Aug 11 18:00:40 2015 UTC and is due to finish in 60 minutes. The chair is morgan_503. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:41 samueldmq, vs 18:00:42 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:00:45 The meeting name has been set to 'keystone' 18:00:49 #topic Agenda 18:00:54 #link https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting 18:00:55 gyee: I was about -20 then 18:01:03 gyee, ++ hehe 18:01:10 #topic Centralized Policies Distribution 18:01:22 david8hu, just kidding :) it has been fun to run with it 18:01:28 and there we go 18:01:28 o/ 18:01:32 o/ 18:01:38 we have lots to cover 18:01:40 o/ 18:01:50 so... going to timebox some. 18:01:51 so, operators feedback is what we missed on this point 18:01:59 you have ~15m 18:02:01 max 18:02:11 people know what we want to this cycle (distribute centralized policies) 18:02:18 we had the specs, etc, and we got operators feedback 18:02:20 samueldmq, we'll bring it in up in the ops midcycle next week 18:02:21 morgan_503, got it, thanks 18:02:30 under "burning issues" 18:02:34 samueldmq, will talk to the operators 18:02:40 \o 18:02:57 so, operators feedback 18:03:03 #link https://www.mail-archive.com/openstack-operators@lists.openstack.org/msg02805.html 18:03:32 about customizing and distributing we got things like: 18:03:35 "Not the easiest task to either get right, or make sure that the files are distributed around in an HA setting. But absolutely necessary." - Kris G. Lindgren, Senior Linux Systems Engineer at Go Daddy 18:03:57 and others, so I think the lack of stackholders shouldn't be an issue anymore 18:04:42 and I'd like to request an official decision on the SFE request 18:04:46 (since we still don't need a FFE) 18:04:55 #link https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg57416.html 18:05:22 any objection? comment? :) 18:05:24 morgan_503, should we vote? or anything else? 18:05:44 sure. can start a vote unless someone has something else to add 18:05:53 how ready is it? 18:06:16 gyee, yes, bringing it up in the ops midcycle is great, however independent from having it approved internally, given the good feedbacks, good progress in the code/specs 18:06:18 regardless of state it will be classified as expirimental and optional in liberty at best 18:06:20 :) 18:06:37 do we have a sponsor? 18:06:37 breton, all the code/specs is under https://review.openstack.org/#/q/status:open+branch:master+topic:bp/dynamic-policies-delivery,n,z 18:06:38 samuldmq, I am all for it 18:06:47 pending adoption and/or solving outstanding critical issues in mitaka it could be considered for stable 18:06:55 morgan_503: let;s calrify thta…since when I asked that before, ayoung said it would not be experimental 18:07:04 breton, specs have 2x +2, code is 90%+ submitted and reviewable 18:07:07 it will be expirimental or i -2 18:07:19 it is the same as all new code in keystone 18:07:22 samueldmq: I've been pinging our core operations team to give a feedback, but we have a use case for centralizing the policy file. It is more nice to have, but not critical. 18:07:27 a cycle for sussing out before contract is set in stone 18:07:39 can we approve the spec for backlog? 18:07:39 * morgan_503 plays PTL card on this one 18:07:43 I agree it is experimental 18:07:44 morgan_503”: ++ 18:07:50 you build it, we'll kick the tire 18:07:58 ayoung: I already +2’d if fo rthat 18:08:05 so i'm happy to vote on SPFE, which would in turn allow the specs to be approved for liberty 18:08:13 lhcheng, nice, one more good feedback in the list, thanks 18:08:20 no need to backlog shuffle for the ones targeted at this cycle 18:08:30 morgan_503, yes I agree with you and henrynash on the experimental flag 18:08:41 so any concerns before vote? 18:08:48 ayoung: any spec can be approved for backlog 18:08:54 ayoung: no SPFE etc needed ever. 18:09:03 morgan_503, I know...It was more a call to action than a request for approval 18:09:10 ok so. 18:09:20 #vote yes 18:09:25 opening vote in... 1min unless someone says otherwise 18:09:37 *do it* 18:09:41 ayoung, we are voting on making it possible to merge them to L 18:09:47 ayoung, o/ 18:10:16 #startvote Approve SPFE for policy distribution and scaffolding for dynamic policy? yes,no,i_dislike_concrete_options_but_still_vote_yes 18:10:16 Begin voting on: Approve SPFE for policy distribution and scaffolding for dynamic policy? Valid vote options are yes, no, i_dislike_concrete_options_but_still_vote_yes. 18:10:17 Vote using '#vote OPTION'. Only your last vote counts. 18:10:27 #vote yes 18:10:29 #vote yes 18:10:31 #vote yes 18:10:31 #vote yes 18:10:34 #vote yes 18:10:36 if it's experimental then do whatever you want 18:10:40 #vote yes 18:10:42 #vote yes 18:10:46 #vote i_dislike_concrete_options_but_still_vote_yes 18:10:46 #vote yes 18:10:54 #vote yes 18:10:57 #i_dislike_concrete_options_but_still_vote_yes 18:11:03 #vote i_dislike_concrete_options_but_still_vote_yes 18:11:05 1m left on vote 18:11:13 #vote yes 18:11:14 I'm not signing up to review the changes. If others don't review it then it's not going to get in. 18:11:16 * lbragstad had to try out the new option 18:11:20 i think there are still a few details i dont' like, but we can work them out 18:11:23 don't care 18:11:25 bknudson: *nod* 18:11:41 that's why I asked if there was going to be sponsor 18:11:42 bknudson: i've been reviewing a bit and working with samueldmq 18:11:54 dstanek, thanks 18:11:56 #endvote 18:11:57 lbragstad, on my todo list 18:11:57 Voted on "Approve SPFE for policy distribution and scaffolding for dynamic policy?" Results are 18:11:58 yes (9): gyee, dstanek, ayoung, lhcheng, bknudson, david8hu, samueldmq, henrynash, breton 18:11:59 i_dislike_concrete_options_but_still_vote_yes (2): morgan_503, lbragstad 18:12:05 ok so i think that is a yes. 18:12:22 ayoung: you win! 18:12:25 morgan_503, looks right :) 18:12:27 Whaddaya know...miracles do happen 18:12:52 dstanek, he's like Charlie Sheen, he always wins 18:12:52 I also think that we must be careful in terms of priority, in Kilo, reviews of Fernet tokens (whcih was an SFE) took precedence over other things that where dubmitted before the deadline…some of which didn’t get in….that can’t happen again 18:12:58 ayoung, samueldmq: please respond to the ML thread reference this vote as approval, lets merge the specs - if there are no critical -1s on them, we can merge them today, no 2x+2 needed on the specs (refer to this vote when approving) 18:13:18 be sure to clearly indicate that this will be exprimental like all other code. 18:13:18 we still need to focus on reviews for fernet tokens to fix the bugs 18:13:20 morgan_503, ++ will do my best. 18:13:23 morgan_503, great 18:13:25 gyee: i only have nsfw responses for that 18:13:28 bknudson: ++ 18:13:29 #link https://review.openstack.org/#/c/134655/ 18:13:29 #link https://review.openstack.org/#/c/197980/ 18:13:34 samueldmq, theres one with a jenkins issue...you are getting that, right? 18:13:38 the specs are those two ^ 18:13:43 ok moving on 18:13:44 ayoung, sure 18:13:48 ++ 18:13:48 #topic Raising an exception if no domain specified on user/group/project create 18:13:52 henrynash: 5m 18:13:54 ok 18:13:58 yes so please let's get specs merged and review code 18:14:00 thanks all 18:14:01 :) 18:14:26 so as the bug indicates, this is where we try teh default domain in cases when you don’t supply a domain in a create entity call 18:14:55 henrynash: #link the bug? 18:15:00 this is not in the spec, nor do we really want this….it was in there due to tempest failing (a long time ago) 18:15:09 bug #1482330 18:15:09 bug 1482330 in Keystone "Creating a user/group/project without a domain should raise an exception" [Medium,In progress] https://launchpad.net/bugs/1482330 - Assigned to Henry Nash (henry-nash) 18:15:09 #link https://bugs.launchpad.net/keystone/+bug/1482330 18:15:09 henrynash: this sounds like it should be a simple validation error 18:15:09 Launchpad bug 1482330 in keystone "Creating a user/group/project without a domain should raise an exception" [Medium,In progress] https://launchpad.net/bugs/1482330 18:15:11 Launchpad bug 1482330 in keystone "Creating a user/group/project without a domain should raise an exception" [Medium,In progress] 18:15:13 thx 18:15:23 stevemar: and so it should 18:15:39 won't this break things? 18:15:47 I agree with stevemar this seems like a validation error 18:15:47 henrynash, did we talked about this sometime back, like defaulting it to the caller's domain? 18:15:50 however.... 18:15:52 the question is - is it safe to cut this off after it being out ther for a while? 18:16:05 changing the behavior is API incompatible 18:16:08 you'd have to deprecate the old behavior 18:16:12 bknudson: ++ 18:16:13 morgan_503: ++ 18:16:16 henrynash, we should also consider that in reseller, it won't be necessary 18:16:21 have a config option for it 18:16:26 gyee: we do default to teh domain token…but if they, for insatnce, used a proejct token tehn we take this odd path 18:16:28 we'd infer it from the parent 18:16:29 V3 ... domain should be specified. V2, default only. We all agree that is the proper semantics, right? 18:16:34 bknudson: i'd like to avoid that kind of option 18:16:52 morgan503: we don’t specifiy this behaviour in the spec 18:17:01 bknudson: but that is a fine work around for *today* 18:17:05 another option is something on the request (a header or something) like microversioning 18:17:08 we shouldn't be defaulting to the default domain 18:17:25 ayoung, yes 18:17:28 i'm ok with an option to turn off the behavior today 18:17:31 and we shouldn't default 18:17:38 ayoung's assessment is correct 18:17:56 so lets just say make it an option, deprecate old behavior - lets not get into microversions for now 18:18:01 morgan_503L so turn it off (by default), but allow a config switch to truen it back on 18:18:19 when we validate a token, we should validate V3 only, so as to make the domain information explicit, too 18:18:22 morgan_503: so turn it off (by default), but allow a config switch to truen it back on 18:18:30 bknudson: ^ i think this can't be done that way because this isn't security driven 18:18:35 thinking in terms of enforcing policy.... 18:18:44 I am curious if tempest or something well break if we turn it off 18:18:51 ayoung: understand, I’ll check this 18:18:55 we have to default it to crrent behavior and in 2 cycles we can look at turning off by default 18:18:56 gyee: no, it all passes now 18:19:03 henrynash, oh good 18:19:37 morgan_503: even thow we don’t specify it in the spec? 18:19:39 i think it needs to maintain current behavior..but bknudson any input before moving on? 18:19:40 microresponse code 400.301 18:20:05 "Bad request, but we used to accept it." 18:20:09 henrynash: sadly even if the spec doesn't say it... behavior is the contract [unless massive security hole] 18:20:13 hah 18:20:16 morgan_503: I agree the default for this release should be to be the same as old behavior 18:20:21 ayoung: 400.201 18:20:30 ++ 18:20:36 "Bad request but we caaaept it anyway?" 18:20:39 and log a message when it's used 18:20:40 morgan_503: ok, got it….and that’s why I riased it here. Ok, I got what i neded 18:20:45 cool 18:20:49 moving on.. next topic 18:20:52 then change default to the new behavior next release 18:21:11 maybe change devstack to the new behavior 18:21:12 #topic HTTP caching support 18:21:15 dstanek: 5m 18:21:26 this is 2 parts... and ties back a little to dynamic policy 18:21:30 Did we ever find out what caching support we get from the requests library? 18:21:33 i put together an experimental client change 18:21:42 #link https://review.openstack.org/#/c/211396/ 18:21:52 this will allow you to enable HTTP caching when creating a session 18:21:59 my next step is to get the new deps in g-r if there are no objections to my change 18:22:07 jamielennox also wanted to see if we could make those optional deps 18:22:11 thoughts? 18:22:19 dstanek: make sure to replicate this in keystoneauth if no complaints 18:22:36 * morgan_503 has no real complaints with this 18:22:43 so my only thing was i always considered this something that like OSC would do, but i like the idea 18:23:05 mainly because CacheControl lets you cache to a file so to cache things like discovery between cli invocations 18:23:06 jamielennox: i think this is putting the features into session so osc can do it 18:23:19 unless we want it a layer above session 18:23:26 (if even possible) 18:23:27 jamielennox: in theory we could add it there, but we still need support in ksc so that ksm can take advantage of it 18:23:28 so there's a few options there i'd like to see if we can somehow expose 18:24:03 sounds to me like dstanek and jamielennox have some conversations pending on this, but generally no opposition 18:24:06 dstanek: yep, i'm good with the idea in theory i just want to see if we can make it a bit more generic thatn on/off 18:24:07 dstanek, is cache shared globally? 18:24:22 or is it per sesssion 18:24:22 gyee: just with the user running the code 18:24:24 we should be able to specify a secure directory to store the cache, no? 18:24:33 so that is the client side 18:24:46 yes....i have a script that i used 18:24:55 #link http://paste.openstack.org/raw/412596/ 18:25:02 to test my experimental patch 18:25:08 i also want to make a spec for M to add in real caching support 18:25:15 a real small sliver of the work needs to be done to support dynamic policy, but there is lots more to do 18:25:21 i wanted to get some feedback on the approach 18:25:28 #link https://review.openstack.org/#/c/211693/ 18:25:31 ayoung read my mind, just don't want to into shared cookie situation 18:25:38 generally i want to issue cache-control everywhere and see clients be smart 18:25:47 and cache where they are allowed to 18:25:49 sess = session.Session(auth=auth, http_cache=caches.FileCache('.os_cache')) ... could we make that a secure temporary by default? 18:25:53 this is moving down that path 18:25:56 so i like it 18:26:01 ayoung: gyee: look at the sample in paste - you control that 18:26:08 from cachecontrol.cache import DictCache -- We're limiting to specific cachecontrol implementations? 18:26:11 nice! 18:26:16 dstanek, if no param is passed, what do you get? 18:26:20 ayoung: i don't want to cache to file by default from a library 18:26:27 bknudson: no, those are just shortcuts - you can pass in any object 18:26:33 ayoung: no caching 18:26:34 we could just tell them to use cachecontrol directly 18:26:51 jamielennox, I mean if called like this sess = session.Session(auth=auth, http_cache=caches.FileCache()) 18:27:02 like jamielennox eluded to there are some adapter options that i'm not passing though that may be useful 18:27:41 bknudson: they'd have to implement a crazy subclass like i do to get around us using a TCP adapter 18:27:41 ayoung: i'm sure we can come up with something, it's not as easy as i thought because we have the TCPAdapter with like socket timeout on it 18:27:51 why do we have the TCP adapter? 18:28:07 bknudson: it fixes a TCP keep alive bug 18:28:16 we're probably going to need to be able to share the cache between all threads in an apache instance, and all workers in an eventlet server, too 18:28:25 I don't have a problem telling them they have to implement a crazy subclass. 18:28:28 ayoung: that already works 18:28:58 "them" is us in many cases 18:29:05 dstanek, this is something we've needed for a long time. Nice. 18:29:15 dstanek: how do you tie them together? i would have thought it was per session object 18:29:29 jamielennox: the cache? 18:29:32 right 18:29:36 memcache backend or something? 18:29:45 the filecache is a shared directory that uses file locking 18:29:55 ok, still filecache 18:30:04 lets not memcache if we can avoid it. This should not go off system 18:30:08 there are only 3 upstream backends a memory cache, a file cache and a redis cache 18:30:22 yay no "memcache" 18:30:40 will it play nicely with eventlet? 18:30:44 and i'd love to see this eventually being automatic for users so things just get cached 18:30:51 ayoung: it should yes 18:30:56 ayoung: most things do... by accident, but they still do 18:31:04 gevent is smart-ish 18:31:08 memcache did not 18:31:15 hopefully I can cache my tokens in there 18:31:22 ayoung: that is because of explicit thread.local and a horrrrrrrible library 18:31:29 bknudson, yes, its very safe 18:31:37 you have to work hard to make it that bad 18:31:40 dstanek: i definetly want people to opt-in to a cache, particularly one that is stored on a filesystem and will be reused between runs 18:31:42 bknudson: if we are crazy enough to set the cache headers then sure! 18:31:54 it makes sense for the CLI and applications but not as a library 18:32:22 i'll poke around osc some to see how much damage i can so there 18:32:26 ayoung: and it used all FDs for the process, so an effective DOS but the cache was still stable/secure 18:32:54 dstanek: pretty neat stuff dude 18:32:55 so everything using the session gets this? 18:32:55 #action dstanek to work with jamielennox and continue exploring this path. more work to be done before it's really ready to go 18:32:57 not just keystone? 18:32:59 i'm over time...closing thoughts? 18:33:08 bknudson: correct anything using session can leverage this 18:33:10 bknudson: yes 18:33:13 so...what happens if we put a cache header on, say a user record, then we delete the record. Will the client library be smart enough to invalidate the cache? Of will we need a "ignore cache" switch for those case? 18:33:16 seems like this should go to keystoneauth first 18:33:21 dstanek: let me know if you need help on the osc side :) 18:33:40 ayoung: cache-control says you are allowed to keep that cache until expiry - regardless of server response 18:33:44 so. 18:33:58 don't put permissive cache control where you don't want it 18:34:17 ayoung: a client would not ask for that until the local copy times out 18:34:21 no way to flush it 18:34:44 dstanek, so the workflow I'm doing is that I need to check if something is deleted... 18:34:44 ok we're at time for this topic 18:34:46 this is eventual consistency at it's best 18:34:51 and there is no way except to busy wait. 18:35:02 ayoung: why would you need to do that? 18:35:06 ayoung: this just means we need to be smarter about what we send in our cache-control headers 18:35:08 ayoung: don't put a cache-control header on those resources that allow it. 18:35:10 So: delete router-interface...then check if it exists...sleep. check again 18:35:11 then 18:35:20 I think we should document how to do it yourself and also support that. 18:35:23 you also can ignore cache-control as a client 18:35:23 i think most things won't be cacheable 18:35:34 usually, caching is what I want, but in those cases, its a user override...like hitting ctlr r in the browser 18:35:34 you don't *have* to use it. 18:35:48 clients can ignore caching at any point 18:35:55 ayoung: delete your local cache 18:36:00 morgan_503, that is my question "how?" 18:36:24 either delete the local cache or don't use a caching session 18:36:33 or disable caching in the session for those requests 18:36:35 ayoung: don't configure it on your sesson or delete local cache 18:36:37 do I need to delete the whole cache just to force a requery on a single resource? 18:36:56 ayoung: you'll have a hard time finding a single resource 18:37:13 better to create an uncached session 18:37:21 this still needs work 18:37:24 did you look at other libraries for caching with requests? 18:37:26 lets take this convo and impl details offline 18:37:36 next topic. 18:37:59 bknudson: a few..that was a recommendation from sigmavirus24 18:38:02 #action dstanek to work with ayoung to ensure caching use-cases and cache avoidance for specific requests can occur 18:38:14 again, lets take the rest of this offline 18:38:23 we have a few more big ticket topics to hit 18:38:37 #topic X.509 Tokenless Authz needs a verdict 18:38:43 gyee: 12m max 18:39:02 so the patch's been independently tested by folks from Yahoo and Mirantis 18:39:04 VOTE! 18:39:18 are there anything we can do at this point? 18:39:24 what verdict is needed? isn't it just reviews? 18:39:24 i'm fairly happy with the changes in general 18:39:27 gyee, its experiemental and opt in, right? 18:39:33 ayoung, yes 18:39:36 I have questions 18:39:39 looks like it needs reviews.. and it needs ot be clearly marked expirimental 18:39:40 lets do it 18:39:43 related and unrelated to client side 18:40:05 breton, client side change will come, after this patch is landed on the server side 18:40:12 1. Is getting a catalog for the user? 18:40:25 #link https://review.openstack.org/#/c/156870/ 18:40:25 ah. 18:40:34 *is getting a catalog for the user planned? 18:40:37 breton: if you're acting on keystone you don't need a catalog 18:40:42 /v3/auth/catalog ? 18:40:44 gyee: every figure out service tokens? 18:40:45 breton: if you get a token, you have a catalog 18:40:48 X-Service-token 18:40:50 also /v3/auth/catalog ? 18:41:10 jamielennox, we'll need to support mapping on the client side 18:41:17 gyee: mapping? 18:41:20 client side? 18:41:32 map a cert to auth context 18:42:08 #action Cores: Please review code for tokenless auth (service users) 18:42:09 jamielennox: the x509 tokenless auth work leverages the mapping engine from the federation code 18:42:38 gyee, stevemar: what does that have to do with client side/ 18:42:40 I saw that Session already supports `cert` option 18:42:53 gyee: what do you mean support mapping on the client side? the x509 specific mapping? or the ability to create a mapping from the client side? 18:43:01 jamielennox: i dunno, that's what i'm wondering ;) 18:43:08 breton: yes it does but you'll need to handle the catalog so you're better off doing it as a auth plugin 18:43:15 but they don't work with auth=None (keystoneclient/session.py, line 577, in _auth_required) 18:43:20 jamielennox, we'll need to be able to fetch a mapping from server to map the cert into auth context 18:43:33 gyee: those words do no make sense 18:43:33 instead of token to auth context 18:43:37 jamielennox: so, a kind of no-op plugin then? 18:43:56 breton: you'll need to pass through the url for keystone, but mostly it'll be no-op 18:44:00 gyee: i'm still confused 18:44:03 :) 18:44:03 service users are used for more than just auth_token. 18:44:14 gyee: I don't quite understand too 18:44:16 gyee: if the service user needs a token, get a token via the cert 18:44:17 breton: there is also like get_connection_parameters from which you can set the cert from the plugin which is better 18:44:31 bknudson: for bootstrapping! 18:44:33 morgan_503, sure, we can do that 18:44:34 gyee: I almost make it work with minimal changes 18:44:39 if the service user only talks to keystone don't need a token 18:44:40 simple 18:44:55 service users are used by neutron to send notifications to nova. 18:44:57 this is opening doors for more options and moving away from tokens elsewhere as needed. 18:44:58 to get a token via cert we need an auth plugin on ks side, no? 18:44:58 morgan_503, cert for token is supported without code changes 18:45:11 so for now, don't try and grab mappings and such 18:45:15 that is way way way way too complex 18:45:17 morgan_503: the X-Service-Token problem is for swift and others that expect you to pass the service that is calling this as well 18:45:19 just get a token 18:45:30 morgan_503: in which case there is no token to pass on 18:45:33 morgan_503, sounds good 18:45:35 moving away from tokens everywhere: http://adam.younglogic.com/2015/08/tokenless-keystone/ 18:45:37 jamielennox: so get a token for now :) 18:45:46 this is iterative 18:45:54 morgan_503: making x509 not useful for at least a couple of services 18:46:06 it is still useful 18:46:08 and a growing list 18:46:11 but sure 18:46:14 we're saying it's experimental now so if it doesn't work then we can stop supporting it 18:46:28 it is also expirimental so changes can occur in mitaka 18:46:29 bknudson, its both experimental and optional 18:46:45 in short: review code 18:46:46 X Service token can be handled via a separate middleware and an X59 as well. 18:47:02 doesn't invalidate this patch 18:47:07 don't try and make the client handle the mapping 18:47:25 morgan_503: ++ 18:47:32 provide a way for x509 to grab the token for x-service-token when needed *for now* 18:47:42 gyee: ^ work for you? 18:47:53 morgan_503, yes 18:47:55 (as in when sending x-service-token) 18:47:57 ok 18:47:58 cool 18:48:03 #topic Bug 1482701 - Federation: user's name in rules not respected 18:48:03