Wednesday, 2015-06-24

*** richm has joined #openstack-keystone00:04
*** arunkant__ has joined #openstack-keystone00:11
*** arunkant_ has quit IRC00:14
*** spandhe has quit IRC00:20
*** dims has joined #openstack-keystone00:23
*** spandhe has joined #openstack-keystone00:31
*** bradjones has quit IRC00:34
*** bradjones has joined #openstack-keystone00:35
*** bradjones has quit IRC00:35
*** bradjones has joined #openstack-keystone00:35
*** jasondotstar has quit IRC00:43
openstackgerritBrant Knudson proposed openstack/python-keystoneclient: Prevent attempts to "filter" list() calls by globally unique IDs
openstackgerritBrant Knudson proposed openstack/python-keystoneclient: Stop using .keys() on dicts where not needed
*** dramakri has quit IRC00:51
morganfainbergbknudson: nice ^^01:04
*** spandhe has quit IRC01:10
*** charlesw has joined #openstack-keystone01:12
*** _cjones_ has quit IRC01:15
*** davechen has joined #openstack-keystone01:22
davechenayoung: sorry, I miss our meeting and just read your email.01:24
ayoungdavechen, we actually hadn't met yet01:25
davechenayoung: I am lucky actually. :-)01:26
ayoungdavechen, so, I'll ask you the same thing I floated by samueldmq earlier01:26
davechenayoung: just the time is too bad for me, I am unable to catch up the meeting I am afraid.01:26
davechenayoung: yes.01:27
*** darrenc is now known as darrenc_afk01:27
ayounglets take the absolute basic:  we enable fetch by URL.  Let's work through just that use case01:27
ayoungassumption is we have a stock policy in place01:27
davechenayoung: I saw the spec is ready.01:27
ayoung now we fetch one from Keystone.  My thought is that we ignore the stock and only use the dynamic.01:27
ayoungTHoughts on that.01:27
ayoungmorganfainberg, ^^ you should probably read that, too, as it is the core of the discussion01:28
davechenayoung: stock policy, you mean nova do that?01:28
ayoungdavechen, yes, the policy.json out of the nova repo01:28
morganfainbergayoung: so how does keystone know what the baseline policy should be?01:29
davechenayoung: if we ignore stock policy, will it broke someting from nova?01:29
ayoungmorganfainberg, it is uploaded by the operator.  Either manually, or via puppet01:29
ayoungmorganfainberg, and, yes, you cut to the chase01:29
ayoungif we were to automate this, based on updating the code, nova itself could be responsble for updating the policy file01:30
morganfainbergayoung: and then my next question: how do you control when new policy goes out - so as not to end up with situations where you have partial policy01:30
morganfainbergright now with CMS you have control on the windows01:30
morganfainbergcentralized, you may not.01:30
morganfainbergand could end up with either a group of endpoints out of sync (different policies) or partial changes01:31
ayoungmorganfainberg, I assume by partial policy you mean:  we have a new microversion and a new API, and no policy covers that?01:31
morganfainbergno, i made ½ of the change i intended to01:31
morganfainbergand there is an update01:31
morganfainbergand now we have lots of broken things01:31
ayoungmorganfainberg, lets simplify this a bit...I'm not talking about database driven, so lets assume policy changes are, to start with, all or nothing01:32
ayoungand byt htat I mean that a user will upload a full policy file...01:32
*** timsim has joined #openstack-keystone01:32
ayoungwe can discuss whether that means: for thewhole openstack versuse per service in a bit...lots of weeds to distract here01:32
morganfainbergok great lets ignore the partial issue then01:33
ayounglets leave those off for the moment, and just say that it is a full policy file to start, and we are using the endpoint-policy means of assigning it to an endpoint when it requests it01:33
morganfainbergthe issue is the different policy for different endpoints now [think ha-proxy], it's a cache coherency probelm01:33
morganfainbergs/different endpoints/different processes in the same "endpoint"01:34
ayoungmorganfainberg, I don't think we support that today, do we?01:34
ayoungI think I misunderstood01:34
ayoungyou are just talking about reuse of the thread01:35 policy is, today, enforced by reading a full file out of the file system01:35
ayoungI don;'t think oslo has the concept of caching built in.01:35
ayounglet me check...01:35
ayoungah, I guess it depends on when init is called01:36
morganfainbergand also if you have say multiple nova apis (physically different systems) behind a load balancer01:36
morganfainbergor docker containers01:36
morganfainbergor similar isolation01:36
morganfainbergnot a totally uncommon thing to do.01:36
morganfainbergHA reasons.01:36 we need a way to signal "refresh your policy"01:37
ayoungtotally legitimate concern01:37
davechenayoung: do you have someting in mind when it's get out of sync?01:37
morganfainbergand oslo.messaging isn't sufficient since it's only guaranteed one worker (even multi eventlet workers) will see it01:38
ayoungmorganfainberg, right.  So, one option is to always read policy from the cache directory01:38
ayoungand, slightly less bad is to check the directory to see if there is a newr version, and read it there is01:38
openstackgerritBrant Knudson proposed openstack/keystone: Remove "tenants" from user_attribute_ignore default
ayounglets see what Nova does...01:39
*** vg_ has joined #openstack-keystone01:39
morganfainbergi'm not sure what happens when you have multiple physical nodes acting as a single API [HA], i think only one would see the notify01:39
morganfainbergif it's not fanout?01:40
ayoungit looks like they call init every time01:40
ayoungmorganfainberg, so, lets assume the fetch is done by either ATM or by a listener daemon, Nova will pick up the new policy file next time it enforces01:41
ayoungmorganfainberg, so in ATM, we fetch policy from Keystone and write to a temp dir on the same particion as the policy dir, and then do a rename01:44
ayoungnext time nova calls enforce, it will reread the policy file01:44
morganfainbergayoung: standard fare for posix01:44
*** fangzhou has quit IRC01:45
ayoungmorganfainberg, down the road, if we want something more effecient, I could see using a notification from Keystone to a listener on the nova host, but the first iteration would be fetch on demand01:45
morganfainbergwait every time?01:45
ayoungmorganfainberg, the fetch from keystone?  Not, that is on a timeout, based on the HTTP headers from Keystone01:46
morganfainbergayoung: that breaks the multiple physical node mode01:46
ayoungor do you mean read from the directory and parse the rules?01:46
ayoungyes, that is currently done every time01:46
morganfainbergnode 1 will fetch on a different timeout than node 201:46
ayoungmorganfainberg, is that a problem?01:46
morganfainbergbecause now in a HA [master/master] model01:47
morganfainbergwhihc isn01:47
morganfainbergt that uncommon01:47
morganfainbergyou end up with different enforcement based on where your request goes01:47 the only way to keep them in sync would be to do a puppet type thing?01:47
morganfainbergand it could result in some requests failing/working where they shouldnt01:47
morganfainbergi think - or a notify to the group of nodes somehow01:47
ayoungeven that, though, is not atomic01:47
ayoungthere will always be some lag01:47
morganfainbergbut you have a window that is known01:47
ayoungmorganfainberg, how about this...01:47
ayoungwe put a "no earlier than" time on the policy01:48
morganfainbergi am running puppet now, we might have 5 minutes of bad requests01:48
*** arunkant_ has joined #openstack-keystone01:48
ayoungATM fetches it, but will not enforce on that file until then.01:48
*** Raildo_ has joined #openstack-keystone01:48
morganfainbergbut with timeout/on demand pulls you don't have that level of control.01:48
ayoungmorganfainberg, sure we do.  We always pull a new file01:48
*** Raildo_ has quit IRC01:48
ayoungand the file has an expiry01:49
ayoungpull like, once a minute01:49
morganfainbergayoung: unless we do math that says we pull on the same request timeout [that isn't based on _init_ time]01:49
*** Raildo_ has joined #openstack-keystone01:49
morganfainbergwhich we might be able to do01:49
morganfainbergso something like take the $sha of the endpoint id (URL)01:49
morganfainberguse that to seed the RNG01:50
morganfainbergpull an RNG out, and offset from <top of minute> by that RNG01:50
davechen"no eariler than" is good idea I think.01:50
morganfainbergso you know at <time> these nodes will all fetch policy01:50
morganfainbergand it will be in sync [aka next request after the timeout]01:50
openstackgerritBrant Knudson proposed openstack/keystone: Update developer docs for discretion
ayounglets separate the fetch time from the active time, though01:51
morganfainbergayoung: thats fine01:51
morganfainbergi'm not disagreeing on no-earlier-than01:51
morganfainbergthat is a fine model01:51
morganfainbergbut i think we also want to ensure all nodes in a HA group fetch at the same indicator01:51
*** arunkant__ has quit IRC01:51
morganfainbergall TTL at the same time(ish) vs. one being even 5 mins offset or whatever01:51
ayoungmorganfainberg, lets take that as accepted...we work hard to make sure that all things that need the smae policy get it in sync01:52
ayoungwhat is the relationship between the stock policy on the node at start up and the dynamic?01:52
morganfainbergthat is the next step i wanted to jump to01:52
ayoungI think we say the dynamic overwrites it01:52
*** arunkant has joined #openstack-keystone01:52
ayounghowever,  microversions01:52
morganfainbergi think we need to do an overlay.01:52
*** darrenc_afk is now known as darrenc01:53
morganfainbergdynamic is meant to replace elements01:53
morganfainbergit can replace everythign01:53
ayoungmorganfainberg, is where I think splitting the policy file would make sense01:53
ayounglets say that we can identify two halves of the rule01:53
ayoungone half is RBAC, one half is scope01:53
ayoungthe scope is something that tells us "here is the scope on the requested resource"01:53
ayoungand that we overlay01:54
ayoungthe RBAC, we accept the defaults from stock if and only if it has not been overwritten in the central server01:54
bknudson suggests some review policy updates.01:54
morganfainbergayoung: sec, jumping into another convo brb01:55
ayoungmorganfainberg, that is fine.01:55
*** arunkant_ has quit IRC01:55
ayoungvg_, did you try the policy check tool I pointed you at?01:56
*** arunkant_ has joined #openstack-keystone01:56
vg_yes that works..01:56
vg_I also switched my API from 2.0 to 3 yesterday01:57
davechenayoung: do you what's the "policy management api" means? which is marked as todo in the wiki page.01:57
ayoungvg_, ah, very good01:57
ayoungdavechen, hold on...01:57
davechenayoung: ye01:57
*** stevemar has joined #openstack-keystone01:57
*** ChanServ sets mode: +v stevemar01:57
ayoungvg_, make sure you can do the operation from the command line.01:58
*** Ctina___ has quit IRC01:58
ayounguse the openstack common CLI, you still cannoot create users?01:58
vg_+ayoung I used this link
vg_for some reason my UI got unresponsive01:58
*** Ctina___ has joined #openstack-keystone01:58
vg_yes using command line01:58
*** Ctina___ has quit IRC01:59
vg_any other doc you can point for switching the API for keystone01:59
*** Ctina___ has joined #openstack-keystone01:59
*** arunkant has quit IRC01:59
ayoungvg_, cool,  so, to cbe clear, a user with only the project_admin role on a project is capable of adding a role to another user for that same project?02:00
openstackgerritBrant Knudson proposed openstack/keystone: Update developer docs for discretion
ayoungvg_, and the only problem is that you cannot do that from Horizon?02:00
ayoungdavechen, let me see.02:00
ayoungdavechen, which wiki do you mean?02:00
vg_user with project_admin role can also add new users in that tenant02:00
morganfainbergayoung: so i think i still like the idea of /policy on the endpoints -- *not* for strictly the reason sdague was proposing.02:00
ayoungmorganfainberg, what would /policy have?02:01
morganfainbergayoung: just as a way of knowing at least what the endpoint thinks it's policy is supposed to be02:01
morganfainbergayoung: it lest nova ask glance "hey can i do X" without making keystone in the middle02:01
davechenayoung:, this page02:01
morganfainbergglance already knows02:01
ayoungmorganfainberg, you really think it is useful?  And pracitcal to implement?02:01
morganfainbergayoung: this is a side thought02:01
morganfainbergayoung: nova knows it's calling glance for X02:01
morganfainbergtake this in isolation of what we just discussed02:02
morganfainbergthis really is just a drizzle that occured ot me02:02
ayoungI mean,  I could see an automatic post from the endpoint to Keystone with that data, scripted at startup but exposing it via na API is...going to be a lot of work02:02
morganfainbergayoung: i'm thinking whatever long term way we query for horizon -> keystone or whatever02:02
morganfainbergcan be the same on /policy02:02
morganfainbergayoung: not about ownership just a the base RBAC + what was sourced from keystone02:03
morganfainbergor whatever the central thing is02:03
morganfainbergmight be pointless though.02:03
ayoungmorganfainberg, so...let me just state that I don;'t think we can prcially do it.  If we wait for that to be implemented in every project, we will be waiting for several releases.  And, we would need a way to find it in the service is a nice thought, but I can;'t really connect the critical pieces of it02:03
*** Ctina___ has quit IRC02:04
ayoungmorganfainberg, OTOH, if ATM wants to read the stock policy out of a file and post it to Keystone...that would be practical to do02:04
morganfainbergayoung: i don't think the stock policy can be a file02:05
ayoungI mean, ugly, sure...but we could make it happen02:05
*** arunkant__ has joined #openstack-keystone02:05
morganfainbergi think it's going to have to be code02:05
ayoungmorganfainberg, its a file right now02:05
morganfainbergto support transition from <bad default> to <good default>02:05
ayoungmorganfainberg, so...again, with the split02:05
ayounglets say that we make the code portion the "find the scope" part of policy02:05
morganfainbergso why do we need it to be a file on disk?02:05
morganfainbergit could just be strictly code02:05
ayoungthen the RBAC portion is just "this role for this API"02:06
morganfainbergthis is just for the base-line policy in a service02:06
morganfainbergthe "from disk" can be the overlay from keystone02:06
openstackgerritMerged openstack/keystone: Python 3: Use range instead of xrange for py3 compatibility.
*** tobe has joined #openstack-keystone02:06
*** bknudson has quit IRC02:06
ayoungmorganfainberg, how about we state that, if we can find a good way to autogenerate the JSON from code in the future, we will entertain it then02:06
ayoungI don;'t want to wait on that02:06
morganfainbergayoung: i think that is a really easy solve though02:07
morganfainbergserializing data to json is *easy*02:07
ayoungmorganfainberg, yes, but gathering the data from every single API is not02:07
morganfainbergyou've already said you need to upload overrides to keystone manually02:07
ayoungmorganfainberg, just enumerating all the APIs is tricky02:08
*** arunkant_ has quit IRC02:08
morganfainbergnot seeing the benefit to providing a stock policy on disk02:08
morganfainbergif it's already massively manual02:08
ayoungmorganfainberg, so...lets table that for a moment02:08
ayoungI promise to get back to it in a sec02:08 you understand what I am saying about the split?02:08
morganfainbergayoung: only partially - i'm not sure what you're looking to gain with it02:09
morganfainbergthe scope bit is effectively already solved - it's passed down to the context from middleware02:10
morganfainbergand is already enforced on02:10
morganfainbergsure we do it a bit weird from the URL today in some cases02:11
morganfainbergbut scope can be sourced directly already and is enforced on for *most* projects [ignore the new "compute_admin" idea]02:11
morganfainbergit's just part of the auth context02:11
morganfainbergso lets back up02:12
morganfainbergwhat happens if there is no RBAC override in keystone02:12
morganfainbergwe use the default policy02:12
morganfainbergwho defines the default policy02:13
morganfainbergthe service team?02:13
ayoungmorganfainberg, so you are cool with saying "we are really only dealing with the RBAC side here?02:13
morganfainbergwe already only really deal with RBAC here.02:13
ayoungheh...not if you look at the current policy files, but I thin we are in agreement02:14
ayoungok....if that is the case, then we have two choices02:14
morganfainbergthe other stuff is outside what we can call our purview, we have no way to reach into nova and say scope allows me to do X on project Y02:14
ayoung1.  overlay.  2.  Default from dynamic02:14
morganfainbergsorry on project Y's resource02:14
morganfainbergthat is a seprate check02:14
morganfainbergand usually below the Can I call "boot me a vm" check02:14
morganfainbergor "delete my vm, am i allowed to even try?"02:15
ayoungmorganfainberg, ok,  so the problem with Overlay is they might define a default policuy RBAC that is based on a role that the operator has deleted from Keystone02:15
ayoungWe really only have two roles right now, Admin and Memmber,  and nothing right now enforces on Member02:15
ayoungso really, we have Admin or not admin02:15
morganfainbergi'd say we have 3 states02:15
morganfainberg"admin, not admin, and GTFO"02:15
ayoungso...lets default to "if nothing is specified, you have to have Admin"02:16
morganfainbergthat would be not-member [unscoped]02:16
morganfainbergor bad-scope02:16
morganfainbergbut same argument02:16
ayoungthe operator can change the default (and I think we should have per service defaults)02:16
ayoungbut I think "admin until set otherwise" is the safest"02:17
morganfainbergayoung: don't disagree, in fact i think we should push towards that being baseline [hence sdague's desire to make it code, so he can do fallthrough to old-admin as needed for transition]02:17
ayoungso...if we do an overlay...we could say "it will work so long as you use the default set of roles"02:17
ayoungso..if there is a stock policy, and they use Member there, and we have member in the main one, and we limit policy to will work02:17
ayoungkeep just the RBAC portion in the policy.json file.  The rest can get enforce based on a different policy.json, or based on code.  We don't really care02:18
*** nkinder has joined #openstack-keystone02:18
ayoungmorganfainberg, now want the cool part?02:18
*** jasondotstar has joined #openstack-keystone02:18
ayounglets say we do this...split the policy file.  Now...we can enforce the RBAC portion, from Middleware, based on the URL, not on the  internal name of the target02:19
*** fifieldt_ has quit IRC02:23
morganfainbergayoung: i don't think you need the split to do that.02:24
morganfainbergayoung: since the REQ URL is still passed down.02:24
morganfainbergi am not convinced i'd want middleware doing that enforcement02:24
ayoungmorganfainberg, you do, for at least some calls, as they need to fetch the object out of the database.  Keystone, at least does that02:24
morganfainbergfwiw, i'm not convinced middleware shouldn't do the enforcement02:24
morganfainberglets pretend keysotone is special here02:25
morganfainbergbecause it is02:25
ayoungwell, Nova does it too02:25
morganfainbergyes and we want to get out of the substitution bits in the URL02:25
morganfainbergso i don't think middleware needs to be doing the enforcement02:25
morganfainbergi think the controller should be [or something hooked into the router]02:25
morganfainbergi'm worried middleware is too detached from this enforcement02:27
ayoungmorganfainberg, wouldn;'t it be nice to be able to match the URL to the RBAC policy that will be enforced for it?  It would make Horizon happy02:28
morganfainbergayoung: oh i'm saying we should still move to URL based02:28
morganfainbergi'm just not convinced that enforcement needs to be in middleware02:28
morganfainbergit could be done below the middleware02:28
morganfainbergbecause what happens is you end up with a new API, middleware is not aware of it [or sub-apis? enforced differently?] and you end up with gaps02:29
ayoungmorganfainberg, *could* shmould.02:29
morganfainbergif this is at the controller level it's easier to say "this is enforced"02:29
ayoungWe should do is part of the keystone team's job02:29
samueldmqI am still reading up .. interesting conversation though :)02:29
ayoungmorganfainberg, if middleware is not aware of it,  default policy02:30
ayoungmorganfainberg, but...I'll concede.  We don;t need it now02:30
ayoungit can be done later02:30
morganfainbergayoung: i'd rather keep all enforcement at one layer02:30
morganfainbergbut thats just for simplicity sake02:30
*** lhcheng has joined #openstack-keystone02:30
*** ChanServ sets mode: +v lhcheng02:30
morganfainbergit means nova can define the default for new apis/slice up apis sanely02:30
morganfainbergit also allows them to tie certain things together more easily. "if you can't do X you definitely can't do Y"02:31
morganfainbergwhich middleware could never really do easily02:31
ayoungmorganfainberg, so...even if we do the split, we could do it by just making two policy files, one that we allow an update for, one we don't.  We check policy against both files. If either fails, policy fails02:31
morganfainbergayoung: lets revisit middleware enforcement02:33
morganfainbergdown the line02:33
morganfainbergi want more eyes on that conversation02:33
morganfainbergthat just you, me and samueldmq02:33
ayoungmorganfainberg, deal.02:33
ayoungmorganfainberg, the big thing is figuring out how to do the overlay02:34
morganfainbergayoung: that is all oslo.policy work02:34
morganfainbergand i think we do it in code (post load) regardless of if it was originally a .json or not02:34
ayoungmorganfainberg, except for the actual fetch.  That is keystone specific02:34
morganfainbergayoung: that is outside the overlay bit02:34
morganfainbergayoung: fetch is orthognal to how we merge / overly the policy02:35
ayoungmorganfainberg, I don't follow you on those last statement02:35
morganfainbergfetch happens.02:35
morganfainbergwe load policy02:35
morganfainbergwe merge / overlay02:35
morganfainbergfetch is orthogonal, it has lots of traps around it02:35
morganfainbergbut oslo.policy is where the merge/overlay goes02:35
ayoungby "enforec in code" you just mean call policy from inside the controller, or something else?02:35
morganfainbergno, i mean however we load the policy - in memory/code structure02:36
morganfainbergwe do the merge, not in the json pre-load02:36
ayoungmorganfainberg, let me see if I understand what you are saying...02:37
ayoungwe have stock policy...lets say it is in some python form02:37
ayoungthat will generate a set of Rules.  Then we fetch and parse those, it makes sescond set of rules.  Then we merge those based on our own logic?02:37
morganfainbergyes. because we can know what the rules boil down to then02:38
ayoungOK...I think that will work fine02:38
morganfainbergso if an override exists we can pop it in02:38
morganfainbergmerging it at the .json layer could get icky02:38
morganfainbergbut the rules are a little easier to compare since they've been compiled [especially if you have an is_admin rule, that is combined with an OR, and an AND, for one api - and you then override that api]02:39
ayoungand...we can start today with the stock policuy in JSON.  If Nova ever rewrites theirs in python, it will be a level 0 replacemente for the json stock.02:39
ayoungpretty sure that the merge happens at the rule level already02:39
*** nkinder has quit IRC02:39
morganfainbergwe probably will get help putting code structures that make sense in oslo.policy02:39
ayoungmorganfainberg, there was the "dirs" thing02:39
morganfainbergayoung: yeah that thing scares me02:39
ayoungmorganfainberg, I think that is a pipe dream02:40
morganfainberga lot02:40
morganfainbergayoung: i think we're going to see it happen honestly02:40 long as we dont push the /policy url I think I can work with what we have here02:40
ayoungmorganfainberg, Nova may do it...02:40
morganfainbergthe /policy url is useful for different things02:40
morganfainbergayoung: if nova does it, other projects are likely to do02:40
*** nkinder has joined #openstack-keystone02:40
ayoungagain, so long as it doesn't get in the critical path, I do not care02:40
morganfainbergpolicy.json is awful and almost no one understands it as is02:41
morganfainbergmy guess is people will understand a code represenation better02:41
ayounggo look at XACML.  You'll come back loving policy,json02:41
morganfainbergayoung: i looked at XACML02:41
morganfainbergit makes more sense than policy.json02:41
morganfainbergnot a lot more i admit02:41
morganfainbergbut it does make more sense.02:41
ayoungIt does all the stuff we do with mapping and assignments02:42
morganfainbergthat is why.02:42
morganfainbergbut lets not get too buried in that conversation02:42
morganfainbergthat has enough political overhead i'm not wiling to spearhead it02:42 think I can extract a todo list from tonights conversation02:42
* ayoung thankful for evesdrop02:42
morganfainberggood. and lets hit the core targets for L02:43
ayoungand I think we have a path forward02:43
morganfainbergseriously, having clear markers will help get the buy in02:43
morganfainbergespecially from nova and sdague02:43
morganfainbergand the other projects will follow02:43
ayoungso...for L, lets work on:  enable fetch by URL, and the fetch.  Using the merge srtategy you laid out here02:43
morganfainbergmaybe enforce by URL *if* it makes sense/possible to do02:44
ayoungwe need the sync stuff...that is a new spec02:44
ayoungcan we punt on that to start, or is it an essential upfront?02:44
morganfainberglets put it out on the table02:44
morganfainbergbut it's our stretch goal for L02:44
morganfainbergit's the first thing we drop if there is any question02:44
morganfainbergmostly i think it's making oslo.policy and the enforcer smart enough to handle that case02:45
morganfainbergor the old-style02:45
morganfainbergwe might get sdague or someone from nova helping on that front02:45
ayounglet's hold off on enforce by is cool, and we can drop it into the mix at the M summit02:45
morganfainbergayoung: like i said i'd just put it out as a "if we have someone making those smarts available we wont say no"02:46
ayoungI think that even the split can be tabled for the moment02:46
ayoungits just a better way to organize02:46
morganfainbergfocus on fetch, url, and merge02:46
morganfainbergthat is our baseline02:46
ayoungwe do have the potential on conflict  based on internal rules like "context_admin"02:46
morganfainberg"Basic Policy local to project", Fetch by url, and merge02:47
*** csoukup has joined #openstack-keystone02:47
ayoungwe should try to phase those out02:47
morganfainbergwe can stab that stuff once we have a way to phase it out, and i think we're going to see the merge bit help there long ter02:47
morganfainbergjust seeing where it leads02:48
samueldmqfinally read all the conversation so far :-)02:49
samueldmqmorganfainberg: ayoung so looks like we have a plan/scope02:49
ayoungsamueldmq, yes, I think we do02:49
samueldmqayoung: CMS uploads stock policy to keystone?02:50
morganfainbergayoung: and to be clear we can rewrite the current policy API in keystone to whatever (we may need to call it something else) but if it's bad / bad ux we can02:50
samueldmqmorganfainberg: ++02:50
morganfainbergayoung, samueldmq: the only impl detail i'd like to add in is CMS upload should be able to upload to the URL-ID02:50
morganfainbergnot require upload then associate02:50
ayoungsamueldmq, I am going to exercise dilligent laziness here:  can you summarize in the Wiki page?  I'll correct it if I think anything is out of whack (as will, I am sure, morganfainberg ) but that gives you the onus of geting into all the details02:50
morganfainbergCMS folks hate *hate* the "send something in, get an ID, then do something with ID"02:50
samueldmqmorganfainberg: so policy would be associated when uploaded, directly?02:51
morganfainbergsamueldmq: it should be possible to do so.02:51
ayoungmorganfainberg, I think we could all agree on that02:51
morganfainbergayoung: great02:51
ayoungand..that solves a big problem, too02:51
*** woodster_ has quit IRC02:51
morganfainbergayoung: yes.02:51
*** dims has quit IRC02:51
ayoungmorganfainberg, if the one URL maps to 3 endpoints...all three endpoints get set the same way02:51
ayoungwin win win02:51
morganfainbergURL is the important peice here02:51
ayoungmorganfainberg, you are, once again, my favorite PTL02:52
morganfainbergayoung: don't say that too quickly02:52
samueldmqayoung: those were not your words from earlier :)02:52
morganfainbergyou'll regret it in the morning02:52
samueldmqjust kidding :p02:52
ayoungmorganfainberg, let us now both quickly go off line02:52
*** nkinder has quit IRC02:52
morganfainbergi have to do expense reports and such02:52
morganfainbergso.. no rest for the wicked...or something02:53
samueldmqI have a question regarding the 'no earlier than' in the policy02:53
samueldmqhow does that solve the cache coherence issue with HAproxy and multiple nova processes ?02:55
samueldmqayoung: ^02:55
ayoungsamueldmq, yeah, it is kindof hard to do in the JSON format we have, isn;t it02:55
morganfainbergsamueldmq: we use the magic math02:56
ayoungsamueldmq, ok,  so  lets say that there are 30 machines...10 clusters of 302:56
morganfainbergif we set a fixed window for refresh, use a known seed for RNG [calculated] we cna guarantee the RNG gives us the same offset02:56
ayoungand they fetch at random internvales...lets say weveryt 5 minutes, so there can be a 4:59 delay between one and another02:56
morganfainberge.g. if we seed RNG with the sha of the url02:56
ayoungwe say "this policy file is active at 10:00.02:56
samueldmqmorganfainberg: RNG,02:56
morganfainbergthen we can do a not-before logic as well to give us a buffer02:56
morganfainbergrandome number generator02:57
ayoungso, even though they fetch at random times, they don't actually use that policy file until the first fetch after 10:0002:57
samueldmqhm, my brain is processing this, wait02:57
morganfainbergayoung: we might want to add an IMS support for the fetch02:57
samueldmqit's on bash mode for a bit02:57
morganfainbergayoung: side thought02:57
*** nkinder has joined #openstack-keystone02:57
morganfainbergif modified since02:57
ayoungmorganfainberg, absolutely02:57
morganfainberguse HTTP spec to our advantage02:58
ayoungso.. I think we want to put a header outside the rules?02:58
morganfainbergwe could *also* use Cache-control to ensure we cache for the same time everywhere02:58
ayoungor can we do all this in HTTP headers?02:58
morganfainbergif we used cache-control headers we don't need a RNG we can say this is valid until X02:58
morganfainbergand IMS check it02:58
morganfainbergwe might be able to do this all in HTTP headers02:59
morganfainbergwith a little metadata storage along side the policy on disk02:59
ayoungmorganfainberg, lets make that a design goal, shall we?02:59
morganfainbergayoung: yep02:59
morganfainberghonestly token validate (HEAD call) should have been an IMS instead03:00
ayoungFreshness is, I think our primary tool here03:00
morganfainbergand we can IMS which *should* [i think] be able to refresh cache-control03:00
morganfainbergand if keystone controls cache-control strictly on a clock, the remote ends know when to try a refresh and stay in sync03:00
morganfainbergthat is waaaaay less work03:01
morganfainbergthan needing to be all sneaky about things03:01
samueldmqis the cache the same for all endpoints in a cloud ?03:01
morganfainbergand the not-valid-until can be controlled from kesytone via the cache-control and IMS checks03:02
samueldmqsince we have no control where a token is going to be used03:02
morganfainbergsamueldmq: all endpoints of a fixed url03:02
morganfainbergsamueldmq: aka any nova that is http://nova-endpoint-1/03:02
morganfainbergthat would have a cache control for it03:02
samueldmqmorganfainberg: how do we define the cache timetou, based on a url?03:02
openstackgerritDave Chen proposed openstack/keystone: Upgrade Foreign key in Endpoint with ondelete='CASCADE'
samueldmqmorganfainberg: still didnt get thsi though03:02
ayoungsamueldmq, its is based on the last file fetched from that URL03:03
ayoungwe'll need to record the headers, and check on each validation03:03
morganfainbergsamueldmq: so keystone knows when a file is updated, it knows when it tells the services to refresh the cache03:03
morganfainbergsamueldmq: when a cache is expired, the endpoint does an "if-modified-since" request03:03
morganfainbergkeystone either gives it the new file (and cache-control) or says not-modified and hands back cache-control03:04
morganfainbergcache-control tells the endpoint when to fetch again03:04
morganfainbergit is keytone's responsibility to figure out the window for refresh [internally managed]03:04
*** richm has quit IRC03:04
morganfainbergand communicate that with headers to the endpoints03:04
samueldmqmorganfainberg: so cache timeout is +infinite, and when policy changes, it says 0 (update right now)?03:05
morganfainbergcache_control will say "try again in X number of seconds"03:05
morganfainbergthat X will result in all endpoints for <URL> to try at a fixed time03:05
*** nkinder has quit IRC03:06
morganfainbergsamueldmq: keystone knows it wants a URL to refresh on a 5 minute window03:06
samueldmqsince a token may hit rocess X 10 seconds later than process Y03:06
*** nkinder has joined #openstack-keystone03:06
morganfainbergit also knows the policy was updated at 120003:06
morganfainbergso it calculates 5-minute windows from 12:0003:06
morganfainbergand passes a correct cache_control header so that no matter when the policy is requested fro <url> - the endpoint will ask on the 5minute window03:07
morganfainbergso you'll see a refresh at 12:05, 12:10, 12:15, etc03:07
lifelessisn't that risky03:08
morganfainbergbecause keystone can calculate time to next window03:08
lifelessthundering herd03:08
morganfainberglifeless: the idea would be keystone would do an internal offset03:08
morganfainbergthis was strictly an example03:08
morganfainberglifeless: and the offset would be per-url03:08
morganfainberglifeless: oh i've done this enough to know not to let the thundering herd happen ;)03:08
lifelessmorganfainberg: one never knows03:08
morganfainberglifeless: but adding that into the description makes the description much harder to communicate03:08
morganfainbergwhen just conveying the concept03:09
morganfainbergwhat i like about this is it is abusing HTTP in the right ways03:09
morganfainbergsince we're already using HTTP03:09
ayoungsamueldmq, you get all that?  Take notes, there will be a short quiz at the end.03:11
samueldmqayoung: hehe I am still re-reading the caching strategy thing03:11
samueldmqI am sorry about still not getting this, I think I am too tired :/03:12
samueldmqit's been a long day :)03:12
morganfainbergsamueldmq: this falls into the category of "my past life was running origin servers for $large_social_media_site and doing all the CDN integration with Akamai"03:12
ayoungsamueldmq, No problem.  Its all recorded in evesdrop03:12
morganfainbergsamueldmq: so i can try and describe it again otmorrow in different ters03:12
Raildo_ayoung, if samueldmq don't take A+ in the test he will be kick out from keystone? =P03:12
ayoungRaildo_, he just needs a passing grade03:13
samueldmqRaildo_: hey you there o/03:13
Raildo_samueldmq, we don't sleep :)03:13
morganfainbergayoung: depends on what you define on passing :P03:13
ayoungAs a Junior at WP once told me:  if the minimum standard wasn't good enough, it wouldn't be the minimum standard.03:13
ayoungmorganfainberg, we need a 2.0 to graduate03:14
ayoungthe lowest ranked guy in the class was called the goat03:14
morganfainberg2.0. wow03:14
ayoungour Goat graduated with a 1.99603:14
morganfainbergi think i needed a 3.003:14
samueldmqmorganfainberg: sure, I will re-read it tomorrow, and we talk about it again :)03:14
* morganfainberg opted out of number grades03:14
ayoungmorganfainberg, I assure you, as 2.0 at WP was well earned03:14
* morganfainberg has a transcript that is about 100pages of evaluations03:15
morganfainbergand essays03:15
morganfainbergit equates to about a 3.503:15
morganfainbergbut i had to write contracts for all my classes03:15
morganfainbergand negotiate them with the teachers03:15
* morganfainberg never took a final exam either03:15
samueldmqmorganfainberg: ayoung actually ..... what we've been discussing so far is not very distant form what is in the spec already03:15
ayoungI had a 3.1 Academic...slighly lower when military was factored in03:15
samueldmqthat only add some (before) hidden details03:16
ayoung3.0 somethings...graduated 309 out of 100303:16
morganfainbergi think i graduated at like 8 of 3003:16
morganfainbergin equivalency03:16
ayoungand I assure you, we diud not "negotiate" with teachers03:16
*** bradjones has quit IRC03:16
* morganfainberg went to a hippy liberal arts college03:16
samueldmqayoung: in the case of updaitng a nova endpoint ? CMS uploads the new stock policy to keystone, right ?03:18
ayoung19 years later...but it looked just the same03:18
ayoungsamueldmq,'ll spoil the moment03:18
*** bradjones has joined #openstack-keystone03:18
*** bradjones has quit IRC03:18
*** bradjones has joined #openstack-keystone03:18
ayoungno, I don't think we want to automate that03:19
*** iamjarvo has joined #openstack-keystone03:19
ayoungsince we are going to an overlay, we can  make the update of the policy file a manual task to start03:19
samueldmqand what if nova doesnt find the rule for the new API03:20
samueldmquses the default rule ?03:20
ayounguses the rule from stock03:20
samueldmqayoung: yeah that's what I am talking about03:20
morganfainbergwe're going to make the operator upload a new policy to start with03:20
samueldmqand when an update occurs?03:20
morganfainbergit is manual because *oh god lets not solve that problem*03:20
ayoungmorganfainberg right03:21
* Raildo_ thinking that we need a dynamic police mid-cycle03:21
samueldmqthe operator re-upload the stock policy03:21
morganfainbergsamueldmq: upload a new file. with new overrides if needed03:21
morganfainbergor delete the file to go back to default stock03:21
morganfainbergdelete stored in keystone file03:21
ayoungsamueldmq, that is one choice, but I think it would be more like:  fetch the dynacmig policuy, add the new rules, upload new dynamic policy03:21
morganfainbergsamueldmq: if it is not defined in the overrides in keystone it falls back on the default policy03:21
ayounglater on, we'll get the database stuff03:22
samueldmqmorganfainberg: so update teh custom policy manually to introduce the new added API03:22
samueldmqmorganfainberg: or update teh stock policy could be a chocie from the deployer03:23
samueldmqayoung: ^?03:23
*** boris-42 has joined #openstack-keystone03:23
vg_when u get time..03:23
morganfainbergsamueldmq: assume the deployer cannot / should not ever update stock policy03:24
ayoungvg_, did you answer my last quetsion here?03:24
*** vg_ has quit IRC03:24
morganfainbergsamueldmq: once everything is truely cut over03:24
*** vg_ has joined #openstack-keystone03:25
samueldmqopenstackgerrit: ok so I was assuming stock represents a version of the api03:25
samueldmqopenstackgerrit: ahhahaha03:25
samueldmqmorganfainberg: ^03:25
* samueldmq might be tired03:25
morganfainbergsamueldmq: the stock policy is owned by say nova03:25
morganfainbergoverrides are *not* owned by nova03:26
samueldmqmorganfainberg: yes and tht changes in different versions03:26
morganfainbergif you update nova and don't provide an override for a new api03:26
morganfainbergyou get the stock policy for it03:26
samueldmqmorganfainberg: and why do we need to upload stock policy to keystone at start?03:27
morganfainbergso when you update, if you want an override for a newly introduced API, you need to upload a new file to keystone03:27
*** Raildo_ has quit IRC03:27
morganfainbergsamueldmq: because keystone only cares about overrides right now03:27
ayoungmorganfainberg, so, I think we are going to need ot  make "overlay" wv "replace and default" into a confiug option03:27
ayoungIf we do overlay, and the rule from stock goes back to a global admin,  it will break the security rules of the deployment03:27
morganfainbergayoung: maybe. maybe we just also still support just specifying a policy file on disk [location]03:27
morganfainbergand if someone doesn't do a fetch they can still CMS deploy03:28
ayoungmorganfainberg, yes, that is base line03:28
ayoungdynamic needs to be enabled03:28
ayoungbut if you do enable dynamic, use only dynamic needs to be an option03:28
morganfainberg"do you really want this?!"03:28
morganfainberguhm. why do we need that?03:28
ayoungmorganfainberg, because right now the stock policies suck03:28
ayoungand if a microversion means a new API, that has critical info in it, uses the current approach]03:29
morganfainbergi'm not seeing why we need another option03:29
ayounga deployer will be sad03:29
ayounglets say that...cinder adds a "fetch private key" API03:29
ayoungabout as ugly as it can get03:29
ayoungand the policy for this is defaulted to "context_is_admin"03:30
ayoungwhich is pretty common03:30 dynamic, they've gone and done everything scoped03:30
ayoungbut the "context_is_admin" rule is defined in stock, and not in dynamic03:30
ayoungso, it just sayd "user must have the admin role" but does not check scope"03:30
morganfainbergyou're still going to have a fallthrough issue03:31
ayoungmeanwhile in the deployment, they are saying "ok, if you want to be able to assign users to a role in the project, you get admin on that project"03:31
morganfainbergsince you have no idea what that rule should be03:31
morganfainbergor.. you override context_is_admin03:31
ayoungcontext_is_admin is defined differenlyy in each file...but our theoretical org has gotten rid ofi ti...or so they thought03:32
ayoungbut, since it is not defined in the dyanimc policy, it sneaks back in in the static03:32
morganfainberglets circle back on that03:32
morganfainbergi'm not convinced we need *another* option for this atm.03:32
ayounglets make it a config option, and we can avoid the CVS by saying "set the config option"03:33
morganfainberglets not worry about that yet03:33
morganfainbergits easy for us to add an option in03:33
samueldmqmorganfainberg: so when the CMS uploadds the initial stock policy, admin is able to read the policy and modify a single existing api03:33
morganfainbergbut honestly, i think we're going to run into issues with "use dynamic only"03:33
samueldmqmorganfainberg: reading the stock is the way he knows what apis are available, and eventually modify some of them03:33
*** jasondotstar has quit IRC03:33
morganfainbergsamueldmq: you don't upload the whole policy file in this iteration03:33
morganfainbergsamueldmq: just the overrides03:33
morganfainbergwhen we get to the more DB managed stuff we're moving to possibly keeping state03:34
ayoungmorganfainberg, ++03:34
morganfainbergi know it doesn't solve horizon's immidiate need03:34
ayoungok...tuning out for the night03:34
morganfainbergbut it's because we need clear targets we can build on03:34
samueldmqmorganfainberg: the CMS uploads only the overrides? a policy which already represents the overrides?03:34
*** ayoung is now known as ayoung_ZZZZzzz--03:34
morganfainbergsamueldmq: just a policy that represents the overrides03:34
samueldmqmorganfainberg: oh I thought it was the stock one initially03:35
morganfainbergwe merge the overrides with the stock one *at the endpoint*03:35
samueldmqmorganfainberg: and how do the admin customize the nova policy? where does he lookup to see available apis?03:35
morganfainbergsamueldmq: you'll need to have the URLs documented03:35
morganfainbergand what that customize looks like03:36
morganfainbergwe can probably autogenerate a doc on it from the enforcement03:36
morganfainbergeither it's a policy.json file or it's code that can output something to show options for overrides in a doc03:36
morganfainberglikely it is going to be code (long term)03:37
samueldmqmorganfainberg: ok I got the main workflow, which doesn't change too much from what we have on that wiki03:37
morganfainbergsamueldmq: but the idea is only the override is uploaded to keystone03:37
samueldmqmorganfainberg: what changes is that we have more details now03:37
morganfainbergyou can override *everything*03:38
samueldmqmorganfainberg: sure I got it, you can even override something that doesn't exist03:38
samueldmqthis way ..03:38
samueldmqmorganfainberg: you can use it for notes..03:38
morganfainbergor if you have your own extension03:38
morganfainbergwith it's own policy03:38
morganfainbergyou could override in the same manner03:38
morganfainberguh, no don't use the override for notes :P03:39
morganfainbergreally a bad idea03:39
morganfainbergwrong tool for the job03:39
samueldmqjust kidding though :)03:39
samueldmqok so .. let's talk 2 minutes about timing ..03:39
samueldmqmorganfainberg: I can write all this up in the wiki until  this weeken03:40
morganfainbergeither that or ayoung_ZZZZzzz-- can help write it up tomorrow03:40
samueldmqmorganfainberg: and needed specs could be in a good state by the end of next weekend03:40
samueldmqso 1 week to work hard on them03:40
morganfainbergwe can do spec exception emails03:41
morganfainbergit's not hard to do03:41
morganfainbergjust a "hey we are here and here is our target"03:41
morganfainbergthe api-impacting freeze is friday this week03:41
morganfainbergfor specs03:41
samueldmqsure, but running with specs is also part of the plan, time is running :)03:41
morganfainbergso we'll do a couple exception emails here03:41
morganfainbergjust to clearly outline the targets03:41
samueldmqmorganfainberg: will we try to synchronize all this with otehr folks now? sdague and others ..03:42
morganfainbergyes we will chat with them03:42
morganfainbergprobably tomorrow03:42
samueldmqmorganfainberg: great! I was expecting next week :)03:43
samueldmqmorganfainberg: have to go sleep now ... almost 1 am here where I am :(03:46
samueldmqmorganfainberg: thanks, talk to you tomorrow (or later today in my case)03:46
*** dramakri has joined #openstack-keystone03:50
*** vg_ has quit IRC03:58
*** kiran-r has joined #openstack-keystone04:05
*** ayoung_ZZZZzzz-- has quit IRC04:06
*** kiranr has joined #openstack-keystone04:14
*** kiranr has quit IRC04:15
*** arunkant_ has joined #openstack-keystone04:20
*** iamjarvo has quit IRC04:22
*** charlesw has quit IRC04:23
*** arunkant__ has quit IRC04:24
*** kiran-r has quit IRC04:30
*** csoukup has quit IRC04:36
*** tobe has quit IRC05:29
*** tobe has joined #openstack-keystone05:29
*** mgarza_ has joined #openstack-keystone05:41
*** mabrams has joined #openstack-keystone05:55
*** e0ne has joined #openstack-keystone06:07
*** mabrams has quit IRC06:07
*** mabrams has joined #openstack-keystone06:07
*** Kennan has quit IRC06:12
*** Kennan has joined #openstack-keystone06:13
*** lhcheng has quit IRC06:14
*** kiran-r has joined #openstack-keystone06:16
*** lhcheng has joined #openstack-keystone06:17
*** ChanServ sets mode: +v lhcheng06:17
*** belmoreira has joined #openstack-keystone06:18
*** kiranr has joined #openstack-keystone06:18
*** kiranr has quit IRC06:18
*** _kiran_ has joined #openstack-keystone06:18
*** e0ne has quit IRC06:20
*** bradjones has quit IRC06:20
*** kiran-r has quit IRC06:21
*** bradjones has joined #openstack-keystone06:22
*** bradjones has quit IRC06:22
*** bradjones has joined #openstack-keystone06:22
*** _kiran_ has quit IRC06:23
*** e0ne has joined #openstack-keystone06:24
*** yottatsa has joined #openstack-keystone06:30
*** e0ne is now known as e0ne_06:36
*** bradjones has quit IRC06:36
*** stevemar has quit IRC06:38
*** bradjones has joined #openstack-keystone06:39
*** bradjones has quit IRC06:39
*** bradjones has joined #openstack-keystone06:39
*** e0ne_ has quit IRC06:42
*** e0ne has joined #openstack-keystone06:45
*** e0ne is now known as e0ne_06:46
marekdrodrigods: i'd say yes.06:48
*** spandhe has joined #openstack-keystone06:52
*** henrynash has joined #openstack-keystone06:53
*** ChanServ sets mode: +v henrynash06:53
*** e0ne_ has quit IRC06:55
*** browne has quit IRC06:55
*** dramakri has quit IRC07:00
*** _cjones_ has joined #openstack-keystone07:03
*** dguerri` is now known as dguerri07:12
*** kiran-r has joined #openstack-keystone07:13
*** kiran-r has quit IRC07:14
*** rlt_ has joined #openstack-keystone07:16
*** dramakri has joined #openstack-keystone07:16
*** kiran-r has joined #openstack-keystone07:17
*** dramakri has quit IRC07:18
*** yottatsa has quit IRC07:24
*** dguerri is now known as dguerri`07:27
*** lhcheng_ has joined #openstack-keystone07:30
*** lhcheng has quit IRC07:33
*** mgarza_ has quit IRC07:33
*** dguerri` is now known as dguerri07:42
*** _cjones_ has quit IRC07:43
*** dguerri is now known as dguerri`07:44
*** spandhe has quit IRC07:53
*** henrynash has quit IRC08:13
*** yottatsa has joined #openstack-keystone08:13
*** openstack has quit IRC08:25
*** openstack has joined #openstack-keystone08:25
*** e0ne is now known as e0ne_08:34
*** rushiagr_away is now known as rushiagr08:38
*** e0ne_ has quit IRC08:39
*** e0ne has joined #openstack-keystone08:40
*** kiran-r has quit IRC08:41
*** e0ne has quit IRC08:45
*** bradjones has quit IRC08:49
*** bradjones has joined #openstack-keystone08:51
*** bradjones has quit IRC08:51
*** bradjones has joined #openstack-keystone08:51
*** lhcheng_ has quit IRC08:55
*** lhcheng has joined #openstack-keystone08:57
*** ChanServ sets mode: +v lhcheng08:57
openstackgerritDave Chen proposed openstack/keystone: WIP - closes bug: Ambiguous error
*** yottatsa has quit IRC09:08
*** bradjones has quit IRC09:09
*** bradjones has joined #openstack-keystone09:11
*** bradjones has quit IRC09:11
*** bradjones has joined #openstack-keystone09:11
*** lhcheng has quit IRC09:12
*** dguerri` is now known as dguerri09:13
*** boris-42 has quit IRC09:22
*** afazekas has joined #openstack-keystone09:42
*** e0ne has joined #openstack-keystone09:46
*** vg_ has joined #openstack-keystone09:47
*** jasondotstar has joined #openstack-keystone09:49
*** dims has joined #openstack-keystone09:50
*** davechen has left #openstack-keystone09:55
*** henrynash has joined #openstack-keystone10:24
*** ChanServ sets mode: +v henrynash10:24
*** kiranr has joined #openstack-keystone10:36
*** kiranr has quit IRC10:40
*** vg__ has joined #openstack-keystone10:44
*** arunkant__ has joined #openstack-keystone10:44
*** yottatsa has joined #openstack-keystone10:45
*** vg_ has quit IRC10:45
*** tobe has quit IRC10:45
*** yottatsa has quit IRC10:46
*** arunkant_ has quit IRC10:48
*** vg__ has quit IRC10:50
*** vg_ has joined #openstack-keystone10:51
*** yottatsa has joined #openstack-keystone10:52
*** yottatsa has quit IRC10:53
*** yottatsa has joined #openstack-keystone10:53
*** arunkant_ has joined #openstack-keystone10:58
vg_anyone to answer this please
*** arunkant__ has quit IRC11:02
rushiagrmorganfainberg: gyee: Sorry, I was unable to attend the keystone meeting yesterday. It was evening here in India, and I had some prior commitments.11:02
rushiagrThis is regarding the stable driver interfaces work. Do we have better clarity now?11:03
rushiagrThe stable driver interfaces work is important to us at Reliance, and we can dedicate resources to get this work done if things are going a bit slow right now..11:05
rushiagrI'll wait for some hours now :)11:07
*** jaosorior has joined #openstack-keystone11:15
*** fhubik has joined #openstack-keystone11:16
*** jasondotstar has quit IRC11:21
*** vg_ has quit IRC11:22
*** vg_ has joined #openstack-keystone11:22
*** henrynash has quit IRC11:24
*** bradjones has quit IRC11:29
*** bradjones has joined #openstack-keystone11:30
*** bradjones has quit IRC11:30
*** bradjones has joined #openstack-keystone11:30
openstackgerritSean Dague proposed openstack/keystone: WIP: make keystone-wsgi-public a console entry point
*** aix has quit IRC11:37
*** kiranr has joined #openstack-keystone11:48
*** kiranr has quit IRC11:52
*** amakarov_away is now known as amakarov11:58
*** ajayaa has joined #openstack-keystone12:09
*** jasondotstar has joined #openstack-keystone12:09
*** jasondotstar has quit IRC12:11
*** rushiagr is now known as rushiagr_away12:14
*** bknudson has joined #openstack-keystone12:26
*** ChanServ sets mode: +v bknudson12:26
*** packet has joined #openstack-keystone12:34
*** edmondsw has joined #openstack-keystone12:39
*** yottatsa has quit IRC12:39
*** yottatsa has joined #openstack-keystone12:43
*** yottatsa has quit IRC12:43
*** richm has joined #openstack-keystone12:50
*** EmilienM is now known as EmilienM|off12:50
*** aix has joined #openstack-keystone12:56
*** vg_ has quit IRC12:59
*** ajayaa has quit IRC13:02
*** rushiagr_away has quit IRC13:04
*** jasondotstar has joined #openstack-keystone13:06
*** nkinder has quit IRC13:06
*** Ctina___ has joined #openstack-keystone13:19
*** zzzeek has joined #openstack-keystone13:28
*** e0ne is now known as e0ne_13:33
*** iamjarvo has joined #openstack-keystone13:34
*** iamjarvo has quit IRC13:34
*** rushiagr_away has joined #openstack-keystone13:35
*** vilobhmm has joined #openstack-keystone13:35
*** ayoung has joined #openstack-keystone13:35
*** ChanServ sets mode: +v ayoung13:35
*** yottatsa has joined #openstack-keystone13:35
*** kiranr has joined #openstack-keystone13:36
*** yottatsa has quit IRC13:36
*** e0ne_ is now known as e0ne13:37
*** janonymous_ has joined #openstack-keystone13:38
*** yottatsa has joined #openstack-keystone13:39
*** kiranr has quit IRC13:41
*** charlesw has joined #openstack-keystone13:49
*** vilobhmm has quit IRC13:49
*** zzzeek has quit IRC13:49
*** vilobhmm has joined #openstack-keystone13:49
*** Raildo_ has joined #openstack-keystone13:52
*** yottatsa has quit IRC13:53
*** yottatsa has joined #openstack-keystone13:55
*** vhoward has left #openstack-keystone13:56
*** yottatsa has quit IRC13:56
*** vhoward has joined #openstack-keystone13:56
*** jasondot_ has joined #openstack-keystone14:00
*** dims has quit IRC14:02
janonymous_``/info``   -->  ?14:02
*** bknudson has quit IRC14:02
*** marzif has joined #openstack-keystone14:02
*** dims has joined #openstack-keystone14:03
*** timsim has left #openstack-keystone14:04
*** ajayaa has joined #openstack-keystone14:10
*** mabrams has left #openstack-keystone14:11
*** iamjarvo has joined #openstack-keystone14:12
*** sigmavirus24_awa is now known as sigmavirus2414:14
*** fhubik is now known as fhubik_afk14:18
*** davechen_away has quit IRC14:24
*** davechen_away has joined #openstack-keystone14:25
*** Ctina___ has quit IRC14:27
*** yottatsa has joined #openstack-keystone14:33
*** stevemar has joined #openstack-keystone14:34
*** ChanServ sets mode: +v stevemar14:34
*** fhubik_afk is now known as fhubik14:34
openstackgerritDave Chen proposed openstack/keystone: Show friendly message when request body is empty
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Updated from global requirements
*** janonymous_ has quit IRC14:44
*** jasondotstar has quit IRC14:45
*** fhubik is now known as fhubik_afk14:46
*** jasondot_ has quit IRC14:47
*** e0ne is now known as e0ne_14:47
*** e0ne_ is now known as e0ne14:48
*** jasondotstar has joined #openstack-keystone14:54
*** mestery has joined #openstack-keystone14:57
openstackgerritAlexander Makarov proposed openstack/keystone: Group role revocation invalidates all user tokens
amakarovayoung, hi! ^^14:57
marekdstevemar: hi, would you care taking a look at this: and up ?14:58
amakarovayoung, please look at that revocation change - I hope I understand revocation by scope correctly14:58
*** stevemar has quit IRC14:59
*** charlesw_ has joined #openstack-keystone15:00
ayoungamakarov, looking15:01
openstackgerritMarek Denis proposed openstack/keystone: Accept both formats of federation mapping rules
*** charlesw has quit IRC15:01
*** charlesw_ is now known as charlesw15:02
ayoungamakarov, do you really still plan on listing all users in the group?15:03
amakarovayoung, it's for TRL15:03
ayoungah...ok, so old code only15:03
*** rwsu has joined #openstack-keystone15:04
ayoungthat really should be wrapped by an if block15:04
*** thedodd has joined #openstack-keystone15:04
*** r-daneel has joined #openstack-keystone15:05
amakarovayoung, if block with witch condition?15:06
ayoungamakarov, one sec, I'll post the review15:06
ayoungamakarov, it seems pretty close15:07
*** jasondotstar has quit IRC15:08
*** nkinder has joined #openstack-keystone15:10
amakarovayoung, thank you, I'll proceed15:11
*** Lactem has joined #openstack-keystone15:12
*** jasondotstar has joined #openstack-keystone15:12
*** fhubik_afk is now known as fhubik15:12
LactemMy IP was blocked yesterday for some reason.15:13
*** rwsu has quit IRC15:17
*** rwsu has joined #openstack-keystone15:17
*** Ctina___ has joined #openstack-keystone15:18
*** browne has joined #openstack-keystone15:19
*** geoffarnold has quit IRC15:21
*** vilobhmm has quit IRC15:22
*** ajayaa has quit IRC15:27
*** nkinder has quit IRC15:30
*** bknudson has joined #openstack-keystone15:34
*** ChanServ sets mode: +v bknudson15:34
*** fhubik is now known as fhubik_afk15:38
*** e0ne is now known as e0ne_15:40
*** dramakri has joined #openstack-keystone15:43
*** diazjf has joined #openstack-keystone15:44
*** e0ne_ is now known as e0ne15:45
*** pballand has joined #openstack-keystone15:47
*** jasondotstar has quit IRC15:50
*** jasondotstar has joined #openstack-keystone15:50
openstackgerritFernando Diaz proposed openstack/keystone: Adding Documentation for Mapping Combinations
*** charlesw_ has joined #openstack-keystone15:53
*** jasondot_ has joined #openstack-keystone15:54
*** charlesw has quit IRC15:54
*** charlesw_ is now known as charlesw15:54
*** jasondot_ has quit IRC15:55
rodrigodsbknudson, ping... are you ok with the API spec being uploaded in a different review?
*** Ephur has joined #openstack-keystone15:59
*** fhubik_afk is now known as fhubik16:00
*** spandhe has joined #openstack-keystone16:03
*** iamjarvo has quit IRC16:04
*** jasondotstar has quit IRC16:05
*** aix has quit IRC16:06
*** jasondotstar has joined #openstack-keystone16:07
*** geoffarnold has joined #openstack-keystone16:10
morganfainbergLactem: weird.16:11
LactemIt's cool now.16:12
*** edmondsw has quit IRC16:12
LactemI'm just waiting to talk to Dolph about the bug.16:12
*** belmoreira has quit IRC16:12
*** spandhe has quit IRC16:18
openstackgerritVictor Morales proposed openstack/keystone: Integrate OSprofiler in Keystone
*** afazekas has quit IRC16:23
*** _cjones_ has joined #openstack-keystone16:24
*** henrynash has joined #openstack-keystone16:27
*** ChanServ sets mode: +v henrynash16:27
*** tqtran has joined #openstack-keystone16:27
bknudsonrodrigods: sure.16:29
rodrigodsbknudson, great! thx16:31
bknudsonrodrigods: where the different review?16:31
rodrigodsbknudson, we are waiting for this spec to be approved in order to update it16:32
*** jasondotstar has quit IRC16:36
openstackgerritAlexander Makarov proposed openstack/keystone: Group role revocation invalidates all user tokens
*** jasondotstar has joined #openstack-keystone16:37
*** rushiagr_away is now known as rushiagr16:38
openstackgerritRodrigo Duarte proposed openstack/keystone-specs: WIP: API changes for Reseller
*** Jason10258 has joined #openstack-keystone16:39
*** Lactem has quit IRC16:39
rodrigodsbknudson, rebased ^ made the API spec follow up patch from henrynash's change16:39
*** Lactem has joined #openstack-keystone16:39
henrynashrodigods: thx16:39
henrynashrodigods, samueldmq: who was it in Horizon who was working on a patch for domain tokens….if we are going to do this project token idea in L, we probably don’t want them to land a patch to start using domain tokens16:40
*** vg_ has joined #openstack-keystone16:40
henrynashbetter to get peope to start using project tokens with is_domain=True….that way domain tokens can die an early death16:41
morganfainbergbknudson: we should be looking to back port the simplified wsgi scripts to kilo as well. If at all possible.16:42
rodrigodshenrynash, totally agree. Maybe we can ask lhcheng to help with this16:42
morganfainberghenrynash: yay!16:42
*** fangzhou has joined #openstack-keystone16:44
Raildo_henrynash, ++16:45
samueldmqhenrynash: hi16:48
henrynashsamueldmq: hi16:49
samueldmqhenrynash: I think there is a patch from david-lyle for this16:49
openstackgerritAlexander Makarov proposed openstack/keystone: Group role revocation invalidates all user tokens
samueldmqhenrynash: let me see if I can find it, I remember to give a link to someone (maybe you) a few days ago16:49
henrynashsamueldmq: yes, I think you posted it here….but I can’t find it right now16:50
*** iamjarvo has joined #openstack-keystone16:50
samueldmqhenrynash: easy one16:51
samueldmqhenrynash: and
*** stevemar has joined #openstack-keystone16:51
*** ChanServ sets mode: +v stevemar16:51
bknudsonmorganfainberg: I've been trying to figure out how to install the scripts16:52
bknudsonpbr / setuptools isn't working as expected16:52
bknudsonthe docs say that with pbr you can tell it to install a file wherever you want but I've tried and no worky16:52
*** dontalton has joined #openstack-keystone16:52
morganfainbergYeah we shouldn't really be installing *anywhere*16:53
morganfainbergit would break on different distros16:53
*** zzzeek has joined #openstack-keystone16:53
bknudsonif that's the case then installing somewhere isn't going to work16:53
morganfainbergIf anything the bin dir where other console scripts go.. But don't try and put it someplace else.16:53
*** lsmola has quit IRC16:53
bknudsonas far as I can tell pbr / setuptools only supports console scripts16:54
morganfainbergThen either it's a console script or it's not.16:54
bknudsonbut a wsgi script isn't a console script16:54
bknudsonit has to export the application symbol which pbr / setuptools console scripts don't do16:54
*** gyee_ has joined #openstack-keystone16:55
morganfainbergThen let's just do it as we do today and back port the fixes to make it simple / same on kilo and master (and potentially Juno)16:55
morganfainbergI don't think we can do anything else because it becomes too distro specific16:55
*** jasondot_ has joined #openstack-keystone16:56
morganfainbergAnd the packagers will do something else as well.16:56
morganfainbergThis is not something pip really should be handling.16:56
bknudsonwe'll just not know where the script is going to be and have to document it16:56
morganfainbergAnd that is what I've been saying from the start.16:56
*** fhubik is now known as fhubik_afk16:56
bknudsonsdague won't be happy.16:56
morganfainbergWell he can be unhappy. I'm going to block trying to make this a pip thing for now.16:57
morganfainbergSo we can fix the issue at hand. If there is a way in the future to do it we can look at it, but let's not spin our wheels too much. This is a separate concern from what pip usually does.16:58
morganfainbergBlock = -1 and say not worth spending too much time on it right now.16:58
bknudson is the change to keystone16:58
bknudsonand is the change to devstack16:59
bknudsonif you want to comment there16:59
*** spandhe has joined #openstack-keystone16:59
openstackgerritSteve Martinelli proposed openstack/keystone: Relax the formats of accepted mapping rules for keystone-manage
samueldmqmorganfainberg: ayoung hi, I have a question regarding our conversation on dynamic policies yesterday17:04
morganfainbergbknudson: commented on your patch17:05
morganfainbergbknudson: i'm not a fan of the devstack change... but that is acceptible17:06
*** stevemar has quit IRC17:06
samueldmqmorganfainberg: ayoung so keystone doesn't own anything besides the overrides, my only concern is about UX17:06
ayoungsamueldmq, go on17:06
morganfainbergsamueldmq: for the first step, keystone only owns overrides17:06
samueldmqsince admins need to lookup to another place to see what operations are available17:06
*** stevemar has joined #openstack-keystone17:07
*** ChanServ sets mode: +v stevemar17:07
samueldmqmorganfainberg: ayoung ah ok, just making sure this is not the final solution in keystone side, as you saidin this first step17:07
morganfainbergsamueldmq: i would prefer it to be documentation not an example policy file that they upload to keystone17:07
morganfainbergsamueldmq: for now.17:07
*** jasondot_ has quit IRC17:07
morganfainbergif it is policy-in-code, we generate docs from it (likely we should be generating docs based upon the enforcement with sphinx) so it can be clear what things are doing what17:08
morganfainbergin either case, we should have it documented not be "here is a blob, good luck knowhing what these really do"17:08
*** fhubik_afk is now known as fhubik17:09
samueldmqmorganfainberg: I mean, I am customizing nova policy is keystone side17:10
samueldmqmorganfainberg: I need to be looking to other docs to see what I can put there17:10
samueldmqmorganfainberg: actually that's not bad17:10
morganfainbergsamueldmq: so, stop thinking of customising policy being a keystone thing17:10
samueldmqmorganfainberg: just different from what we have today, where you can see what is available at the same time you customize17:10
morganfainbergsamueldmq: the centralization of policy will be 100% optional17:11
samueldmqmorganfainberg: when you look at policy.json ..17:11
morganfainbergif the deployer wants to put the overrides on disk on the nova node(s), and never fetch from keystone, that will also work17:11
*** iamjarvo has quit IRC17:11
samueldmqmorganfainberg: sure, that's like an extension17:11
samueldmqwith a config switch17:11
morganfainbergno not an extension17:11
morganfainbergnot a config switch17:11
samueldmqoh, looks like I know nothing then17:12
morganfainbergwell maybe a config switch to not fetch17:12
*** jasondot_ has joined #openstack-keystone17:12
morganfainbergyou will still source the policy overrides from a known location (path like today)17:12
morganfainbergthe fetch policy code will just collect that override from keystone17:12
morganfainbergand place it on disk17:12
samueldmqgot it17:13
morganfainbergcustomizing policy is still customizing policy - we are changing it so only overrides are needed instead of a complete replacement17:13
samueldmqanother thing ..17:13
samueldmqyes, we control customization on keystone side, if one decides to use it17:13
*** stevemar has quit IRC17:14
samueldmqmorganfainberg: one more thing, endpoint_url should be a property of a policy entity, right?17:14
samueldmqcan we have an override doesn't belong to an endpoint_url?17:14
samueldmqso yes, every override (policy) must have an endpoint_url with it17:15
morganfainbergit doesn't need to be a FK on the endpoint in the catalog17:15
morganfainbergit just needs to be specifically tied to an endpoint_url17:15
samueldmqgoing to update wiki17:16
samueldmqmorganfainberg: looks like this solution is very simple17:16
*** yottatsa has quit IRC17:16
*** e0ne has quit IRC17:16
samueldmqmorganfainberg: and very interesting17:16
morganfainbergsamueldmq: we have ~4-5 total things to write here this cycle17:16
samueldmqmorganfainberg: and I am sure we'll have all them :)17:17
ayoungGAH!.  OK, the Ansible module to create a router (  HA!)  needs to convert a tenant name to id, and does a project list...but the user is not allowed to do a project list by policy17:17
morganfainbergoslo.policy overlay / merge [overrides and baseline], updates to local policy files (where appropriate), fetch policy overrides from keystone, store policy overrides in keystone, (stretch goal): enforce on url instead of internal-name17:17
ayoungour naming sucks17:17
samueldmqmorganfainberg: let's just keep moving, and working ahrd, we'll get there17:17
*** iamjarvo has joined #openstack-keystone17:18
ayoungmorganfainberg, customizing policy is totally going to be keystone thing.17:18
samueldmqmorganfainberg: I dont undertand the last one17:18
samueldmq(stretch goal): enforce on url instead of internal-name17:19
ayoungnot a stretch goal samueldmq17:19
morganfainbergayoung: nope. it has to allow CMS to load overrides from disk and never fetch - it does not have to be a keystone thing17:19
morganfainbergayoung: yes it is.17:19
morganfainbergayoung: i told you i'm putting it on the table. if someone wants to do it17:19
morganfainbergi wont say no17:19
ayoungmorganfainberg, I would say it is a core goal,. mnot stretch17:19
morganfainbergayoung: core goal overall, stretch for liberty17:19
morganfainbergayoung: i can agree with that17:19
morganfainbergayoung: yeah :)17:19
ayoungI was thinking of the fetch by URL17:20
morganfainbergno no the enforce by url :)17:20
morganfainbergfetch by url is def. part of liberty goals17:20
samueldmqI enforce by url?17:20
ayoungok..we are in agreement...with the customize, It is also outside of what we are trying for in liberty17:20
morganfainbergsamueldmq: enforce by url - if we have someone who wants to work on it - we want it, but unless someone provides resources we can't commit to it for liberty17:20
ayoungline level modifications of the policy files...needs the DB backend, I think17:21
samueldmqmorganfainberg: sure, I just don't remember what this thing is17:21
*** sigmavirus24 is now known as sigmavirus24_awa17:21
morganfainbergsamueldmq: enforce by url is instead of saying "nova_create" it's "/path/to/boot/instance"17:21
morganfainbergin the policy file17:21
*** Ctina___ has quit IRC17:21
morganfainbergayoung: yep, need the DB stuff before we can do line-by-line stuff.17:21
samueldmqmorganfainberg: ohmm, got it thanks17:21
*** roxanaghe has joined #openstack-keystone17:21
morganfainbergayoung: and i don't think we can commit to that in liberty.17:21
morganfainbergsince that logically is built behind/on top of the other stuff.17:22
*** Ctina___ has joined #openstack-keystone17:22
ayoungmorganfainberg, so...I don;t think we actually need upload by URL17:22
*** pnavarro|off has joined #openstack-keystone17:22
ayoungI think it will more work like this:17:23
ayoungafter a base install, upload the policy to Keystone for the service, not the endpoint17:23
ayoungall endpoints for the same service should use the same policy file17:23
samueldmqayoung: the policy file ? overrides?17:23
ayoungsamueldmq, I'll link top the API, but it is in the endpoint-policy extension right now17:24
morganfainbergayoung: ok we will be delayed on unified catalog work then i think.17:24
morganfainbergayoung: erm, cleaned up catalog17:24
samueldmqayoung: oh for the same service? I think we should go in the URL17:24
ayoungmorganfainberg, why?17:24
morganfainbergayoung: and actually some endpoints will have different policy (you might have a beta endpoint that requires different roles) - this is done today17:24
morganfainbergi think we need to stick to endpoint for now.17:25
samueldmqayoung: when morganfainberg said we should be thinking in a new policy api, I was thinking like a CRUD on policy overrides, where the ID is the URL, or simething like that17:25
ayoungmorganfainberg, that will still work, but they are not the norm.  Those are the one offs...but, yeah, those could be done by URL17:25
david8humorganfainberg, store policy override per endpoint?17:25
morganfainbergayoung: the other issue is right now service is not unique17:25
morganfainbergayoung: service type17:25
morganfainbergayoung: so you're back to by id17:25
* morganfainberg sighs17:25
morganfainbergyou can have 3 different services today called compute17:26
ayoungmorganfainberg, but, in general, won;t they all share the same policy?17:26
ayoungI mean, normale case?17:26
morganfainbergayoung: not nessicarily17:26
morganfainbergayoung: you might have distinct things running on those services that are wildly different17:26
ayoungmorganfainberg, I'm talking expected deployments, not customizations17:26
*** jasondot_ has quit IRC17:26
morganfainbergayoung: so, expected deployment is "don't make assumptions about what people do - our catalog is bad and people do bad things in deployments"17:27
david8humorganfainberg, ayoung, I likt it to be flexible enough, so we can achieve policy per project or domain at some point in the future.17:27
ayoungand...does endpoint link to service by ID or type...17:27
morganfainbergwe've already broken people by accident making a similar assumption17:27
morganfainbergendpoint links to service by id17:27
morganfainbergas a FK17:27
ayoungmorganfainberg, we haven't broken any ability to do things here, though.  Actually, the issue might be higher up...17:28
ayoung1 sec...17:28
morganfainbergwe broke people in a similar way with the domain SQL and ldap change17:28
morganfainbergwe assumed no one would ever be using the LDAP driver as the default and override with SQL on one domain17:28
morganfainbergwell, we have people doing that17:28
morganfainbergand we need to revert [i need to check on that] the limitation out17:28
morganfainbergi am saying we can't assume a deployment is expected if we have the flexibility in place that lets things be sloppy17:30
morganfainbergdavid8hu: i can say i expect policy per project is, if ever, a looooooong way out17:30
morganfainbergdavid8hu: policy per domain i also think is a long way out17:30
*** rlt_ has quit IRC17:31
*** jasondot_ has joined #openstack-keystone17:31
david8huLet's aim for Z, then :)17:31
morganfainbergdavid8hu: eh, maybe the next A release :P17:31
ayoungmorganfainberg, nah, I am just trying to cover the expected case cleanly.  I know people don;'t like the "get back and ID and work with it" approach, but for policy, which was designed with ID as the only idenitifier, ID is the only way to share17:32
morganfainbergcan we just dump that old api and stop referencing it17:32
ayoungmorganfainberg, It would be better if we used the SHA256 of the policy file as the ID17:32
morganfainbergbecause no one uses it17:32
ayoungthat way, two different uploads of an identical file get the same thing.17:32
ayoungAnd, it should make it easier to update things in sync when the stock policies change17:32
morganfainbergi'm fine with a behind the scenes using a sha for the id17:33
*** Jason10258 has quit IRC17:33
ayoungmorganfainberg, what I don't want to have become the norm is "I installed a new endpoint, I need to explicitly set up the policy for it"17:34
morganfainbergayoung: we are already heading that way17:35
david8humorganfainberg, ayoung, Once a new policy is updated for an endpoint, it should persist in keystone and become the new default for that endpoint.17:35
morganfainbergayoung: because you also can't be too generic, services bridge regions17:35
morganfainbergand policy may be different per region17:35
ayoungdavid8hu, I'm puishing for a global policy you see the difference in emphasis17:35
ayoungmorganfainberg, *may be* is great17:36
morganfainbergi have a meeting i need to go to17:36
ayoungbut say I've customized the policy for my region, and I add another endpoint,  I should not fall back to stock17:36
openstackgerritBrant Knudson proposed openstack/keystone: admin and public httpd files
david8huayoung, a global policy, then customizable per endpoint needs?17:36
ayoungdavid8hu, that is my goal, yes17:36
ayoungso we have sane, and cooperating defaults17:36
morganfainbergayoung: we may need to map this out at the midcycle instead of on irc17:36
ayoungmorganfainberg, ++  but I think we are still making progress17:37
*** lhcheng has joined #openstack-keystone17:37
*** ChanServ sets mode: +v lhcheng17:37
ayoungjust the fact that we are having this discussion gives me some small hope17:37
*** dguerri is now known as dguerri`17:37
david8huayoung, good idea, so there is always a default policy, not a NULL.17:37
morganfainbergayoung: lets get the basic support stuff [the biggest peiece we're going to need is the overlay/merge bit]17:37
morganfainbergayoung: and where we stick the fetcher.17:37
samueldmqmorganfainberg: yes as we discussed yesterday17:37
*** openstackgerrit has quit IRC17:38
morganfainbergwe can go back and forth on the way policy is stored in keystone a bit more easily17:38
*** jasondot_ has quit IRC17:38
morganfainbergbut those two (first) peices we need in either case17:38
samueldmqmorganfainberg: ayoung and we will have time to discuss and improve it based on this base things17:38
ayoungmorganfainberg, what I would like to see the workflow of a new install in Liberty default to is to upload the policy upon service registration.  Later on, I would like there tio be a unified policy file, and the workflow should be only upload once...and then customize deliberately17:38
david8huayoung, need to keep the ball rolling...still...:)17:38
*** openstackgerrit has joined #openstack-keystone17:38
morganfainbergdavid8hu: the thought is the services own the default policy. keystone should not own it17:38
morganfainbergdavid8hu: ever17:39
*** henrynash has quit IRC17:39
*** jasondotstar has quit IRC17:39
morganfainbergbecause i don't want services coming to me and saying "hey can we update our policy"17:39
morganfainbergthey know the APIs and how things link together, we don't17:39
samueldmqmorganfainberg: ++17:39
* morganfainberg goes off to meeting.17:39
morganfainbergbe back a bit later17:39
*** yottatsa has joined #openstack-keystone17:40
david8humorganfainberg, agreed.  nova does not need keystone core to +2 its default policy changes.17:40
ayoungdavid8hu, I was thinking it should be a cross project repo...maybe Oslo, with a section for each project17:41
ayoungsomething like this:17:41
*** henrynash has joined #openstack-keystone17:41
*** ChanServ sets mode: +v henrynash17:41
ayoungyou update the nova file then regenerate17:41
david8huayoung, deployer does the update?17:42
*** diazjf has quit IRC17:42
ayoungdavid8hu, that is the default, shipped by Openstack17:43
ayoungdeployer will fetch that, and can even modify before deploying17:44
ayoungdavid8hu, the review proces is just to make sure that the policy changes are sane across the projects.17:44
david8huayoung, the files in the repo got to have tighter controll in terms of who gets +2 default or stock policy file changes.17:44
ayoungdavid8hu, one rep per project or something like that17:45
david8huayoung, for example, only keystone cores should +2 keystone-policy.json17:45
ayoungjust to have a second set of eyeballs on it.  Changes should be very infrequest, or require a lot of corss project talk17:45
ayoungdavid8hu, nah17:45
ayounglet Nova read Keystone policy17:45
ayoungand vice versa17:45
*** mgarza_ has joined #openstack-keystone17:45
david8humore visibility to each others's policy is not a bad thing17:46
*** Raildo_ has quit IRC17:47
*** stevemar has joined #openstack-keystone17:48
*** ChanServ sets mode: +v stevemar17:48
david8huayoung, to solve 6689874657, perhaps have sub dir structure under openstack-core-policy.  openstack-core-policy/base openstack-core-policy/service_admin17:48
david8huwhatever that number is17:50
david8huthe global admin bug17:50
david8huyes, 96869617:50
david8huayoung, we can attempt to get it right this time.17:51
ayoungdavid8hu, it needs a unified policy...all the dyanmic stuff is leading to that17:51
ayoungand I need to try out that unified policy file on a deployemnt17:52
david8huayoung, thoughts on what the chances are to have unified policy for liberty?  We can implment openstack-core-policy/base openstack-core-policy/service_admin for liberty if unified should come after L.17:55
morganfainbergdavid8hu: i'd -2 an attempt to split policy into a separate project like that.17:57
ayoungdavid8hu, I think the effort can go in parallel.  My understanding of how this works has changed over time.  There really is no reason the unified has to start off as a blessed version;  it could be devstack, or even just a community resource until the policy is shaken out17:57
ayoungits not the stock policy, it is *a* unified policy17:57
morganfainbergdavid8hu: plain and simple. the projects should own the base policy17:57
ayoungmorganfainberg, not17:57
ayoungthey should own a section of it17:57
ayoungbut not the roles part17:57
morganfainbergayoung: you and i will continue to disagree on this17:57
morganfainbergi am against centralizing the base policy17:58
ayoungits more complicated than blanket statements like that will support17:58
morganfainbergvery much against it17:58
ayoungYou run wuith SELinux disabled, too, I bet17:58
*** thedodd has quit IRC17:59
ayoungoh, wait...App Armor...Debian based17:59
morganfainbergayoung: SELinux is disabled in most places because no one can be bothered with the awful syntax. it's why ubuntu uses apparmor and mostly only RH based distros (or specific distributions) use SELinux17:59
morganfainbergayoung: and with apparmor, the individual packages own the policy/install the policy. i think even SELinux works like that17:59
david8humorganfainberg, for keystone, we currently have policy.json and policy.v3cloudsample.json.  We need a better way to structure that.17:59
ayoungSecurity is hard, lets go shopping17:59
bknudsonfedora provides SELinux configs for openstack18:00
morganfainbergit's not some other repo that everyone has to commit to updating18:00
ayoungmorganfainberg, nope.  Default policy is in a single repo18:00
morganfainbergayoung: yeah i'm still against it for OpenStack.18:00
ayoungthere are a few projects that have their own, but that is short lived18:00
morganfainbergi much prefer the apparmor model18:00
ayoungand app armor is on the \Dentry, not the Inode, which is just...18:00
*** harlowja has quit IRC18:00
ayoungmisunderstanding the problem....18:01
morganfainbergbecause what will happen is no one will keep the central store up to date or it'll be impossible to get anything landed18:01
morganfainbergthe services should own their base policy.18:01
ayoungbut, regardless, I don't even really care if the unified becomes an official thing, so long as we can support one18:01
*** e0ne has joined #openstack-keystone18:01
ayoungmorganfainberg, so each project should then specify the roles they require18:02
ayoungcuz Policy is enforced on roles18:03
ayoungand then we need hierarchical roles18:03
ayoungor soemthing18:03
ayoungbecause people are not going to assign the individual you see how the design progresses....18:03
*** spandhe has quit IRC18:03
david8humorganfainberg, ayoung, This is what I like to achieve.  For customers that wants segregate their service, I can tell them to grab a set of policy files and move on.  We don't have that today.18:03
*** harlowja has joined #openstack-keystone18:04
ayoungmorganfainberg, so, what would work is something like this:18:04
ayoungwe put a base policy file under oslo.  It has rules that are shared by the other proejcts18:04
*** spandhe has joined #openstack-keystone18:04
ayoungthe other proejcts sync down, uincubator style18:04
ayoungand that becomes the base of the project specific policy files18:05
*** jaosorior has quit IRC18:05
*** yottatsa has quit IRC18:08
*** e0ne has quit IRC18:08
*** stevemar has quit IRC18:08
*** stevemar has joined #openstack-keystone18:09
*** ChanServ sets mode: +v stevemar18:09
*** stevemar has quit IRC18:10
samueldmqayoung: I think oslo.policy should know the role hierarchy, through middleware18:11
samueldmqayoung: and then it exapneds the policy, whne it is overriding it as well18:11
*** stevemar has joined #openstack-keystone18:11
*** ChanServ sets mode: +v stevemar18:11
samueldmqayoung: that old idea where role:member is replaced by (role:member or role:admin), in the case admin inherits member18:11
david8huayoung, I don't think is going to work, since "is_admin:True" is all over the place, and there is no context_is_admin defined.18:13
ayoungdavid8hu, that one is broken18:13
ayoungdavid8hu, I just was testing the ability to merge them based on the rule names, not even if the policy itself made sense18:13
david8huLOL... ok18:13
*** diazjf has joined #openstack-keystone18:14
ayoungI had started by merging in the changes on at a time...but the turn around was way too slow.  It really needs something like a temptest run to see if it will work18:14
rushiagrmorganfainberg: did you hear back from folks who are working on stable driver interfaces stuff?18:14
samueldmqayoung: I forgot what is tht tool we've been using to define those diagrams in the wiki18:15
*** shaleh has joined #openstack-keystone18:15
ayoungdavid8hu, if you go back a few revisions, I did have a working one for nova, cinder, and neutron (I think)  maybe glance?18:15
samueldmqayoung: I can look in the logs, but asking you would be quicker :)18:15
ayoungsamueldmq, seqdiag...I have it installed locally18:15
*** jaosorior has joined #openstack-keystone18:16
samueldmqayoung: thanks18:16
ayoungpython-seqdiag-0.9.0-4.fc21.noarch samueldmq18:16
rushiagrmorganfainberg: sorry for being a pest of sorts :)18:16
*** jasondotstar has joined #openstack-keystone18:17
*** csoukup has joined #openstack-keystone18:17
*** boris-42 has joined #openstack-keystone18:18
*** yottatsa has joined #openstack-keystone18:18
*** stevemar has quit IRC18:20
*** fhubik is now known as fhubik_afk18:21
*** jasondot_ has joined #openstack-keystone18:21
ayoungrushiagr, so, the current driver contract sucks18:22
ayoungits all dictionaries and I don't really know if it is a contract we should try to support18:22
ayoungso...I'm not that excited about it18:22
rushiagrayoung: I assume by 'it' you mean dictionaries?18:24
*** Rockyg has joined #openstack-keystone18:24
ayoungrushiagr, yeah18:24
gyee_ayoung, I don't think we should do params and return type enforcement, something Python is not natural to do18:24
rushiagrayoung: and not the stable driver interface work :)18:24
ayoungrushiagr, so,  If I thought we could actually do things in a timely manner, thuis is what I would suggest18:24
ayounggyee_, python is totally capable of doing it18:25
gyee_enforcing return type?18:25
ayoungrushiagr, here is my starting point:
*** stevemar has joined #openstack-keystone18:25
*** ChanServ sets mode: +v stevemar18:25
rushiagrgyee_: I didn't like that idea either. BUT what we can totally do is write tests which try to see if the return value matches expected values18:26
ayoungrushiagr, I would have Keystone token construction work as a pipeline, calling each of the Managers in turn to return an object added to the token-WIP18:26
gyee_we should think about openssl fips-140 certification approach18:26
gyee_come up with a set of test vectors18:26
gyee_if the lib pass the tests, digitally signed the code18:27
ayounggyee_, we should start by defining a reasonable contract18:27
gyee_enhance stevedor to validate signatures on lib loading18:27
ayounggyee_, stop.  I know you are only half joking18:27
gyee_no I am serious18:27
ayoungand it is the other half that scares me18:27
ayoungthen I am even more scared18:27
gyee_loading digitally signed lib?18:27
ayoungthat is toally not what we need here18:27
gyee_what's scary about that18:28
rushiagrayoung: so are you implying we move _every_ database backend driver to use objects first, and then solve stable driver interfaces? I think stable driver interface work was to avoid people headaches who already maintain out-of-tree drivers :)18:28
*** amakarov is now known as amakarov_away18:28
ayounggyee_, people are loking for stable interfaces adn you are going full blown crypto enforcement.18:28
ayoungShill dude18:28
gyee_we for to enforce compliance18:28
*** fhubik_afk is now known as fhubik18:28
gyee_we want18:28
*** fhubik is now known as fhubik_afk18:28
ayoungrushiagr, meh...I am loosing more and more motivation for doing things in a sustainable way18:28
*** diazjf has quit IRC18:28
ayoungI'm not going to get in the way of other people doing what they think best...just not going to actively contribute to things I think are suboptimal18:29
rushiagrayoung: that is what technical debt is for :)18:29
*** diazjf has joined #openstack-keystone18:29
david8hugyee_,  Requiring driver developer to pay a fee, go through certication process, then get their driver signed?  I thought only Microsoft does that.18:29
ayoungdavid8hu, fips-140 is a government process18:30
gyee_david8hu, that's how they certify openssl I think18:30
*** dguerri` is now known as dguerri18:30
rushiagrayoung: I agree with your spirit of doing work in a way which is beneficial for the long term benefit of the project18:30
gyee_I don't know any python magic out there that can enforce return type on lib loading18:31
*** dguerri is now known as dguerri`18:32
rushiagrayoung: but how about this: we implement stable driver interfaces first. Yes, with dictionaries, which is kind-of suboptimal, so as to get other people who are already struggling to keep up with the pace at which we move. And then take on the problem of passing objects instead of dictionaries?18:32
*** fhubik_afk is now known as fhubik18:33
*** thedodd has joined #openstack-keystone18:33
*** fhubik is now known as fhubik_afk18:33
rushiagrayoung: it will help the motivation of people who maintain out-of-tree drivers a little bit18:33
gyee_rushiagr, what are we *enforcing* then?18:34
*** Rockyg has quit IRC18:34
rushiagrayoung: and just to reiterate, I completely agree with you that object passing _should_ be the long term way forward18:34
rushiagrgyee_: what can we enforce with objects that we can't with dictionaries?18:36
gyee_ayoung, btw, I got sssd working on my devstack somewhat, but there are issues18:36
gyee_rushiagr, how do we enforce return type at load time?18:37
morganfainberggyee_: you don't get to do that in python :P18:37
gyee_morganfainberg, exactly, so test vector is one way to do it18:38
gyee_and make sure the code stay the same as tested18:38
morganfainberggyee_: i'm not advocating driver signing - lets not go down that path yet18:38
morganfainberglets just get stable defined contracted interfaces first ;)18:38
gyee_morganfainberg, yes, they are two different issues18:38
morganfainbergwe can build on top of that18:38
gyee_but yes, stable interface first is a must18:38
morganfainberggyee_: ++18:39
*** sigmavirus24_awa is now known as sigmavirus2418:39
morganfainbergrushiagr: ^18:39
gyee_ayoung, sssd and mapping choke on multiple groups18:39
ayounggyee_, is a different separator character18:39
gyee_and it does not work with UserGroupsIter18:40
ayounggyee_, I had it at one point...lets see18:40
gyee_ayoung, don't think our mapping is capable of handling separators in one string18:40
rushiagrgyee_: yes, test vector is what I was thinking when I was talking about SDI18:40
gyee_we'll need a patch to fix that18:40
*** fhubik_afk is now known as fhubik18:40
ayounggyee_, you don't have to,  it is a mod_lookup_id config18:40
gyee_ayoung, I have both LookupUserGroups and LookupUserGroupsIter set18:41
ayoungLookupUserGroups REMOTE_USER_GROUPS ";"18:41
morganfainbergbknudson: fixed with test-requirements.txt missing oslosphinx change18:42
gyee_env['REMOTE_USER_GROUPS'] = group1;group2;group318:42
ayoungthat worked for me...18:42
gyee_you stuff the entire string in group name?18:42
gyee_group name is a string, not a set18:42
ayounggyee_, what do you have for your mapping?18:43
ayoung  was what I got working18:43
ayounggyee_, I think it was the difference between "group"  and "groups" in the mapping18:44
gyee_local [group {name: {0}}], remote [REMOTE_USER_GROUPS]18:44
ayoungmake that18:44
ayounglocal [groups {name: {0}}], remote [REMOTE_USER_GROUPS]18:44
gyee_change group to groups18:44
ayoungyeah...subtle.  messed me up too.  sorry I didn't not that in the blog.18:45
gyee_ayoung, thanks, let me give it a try18:45
ayounggyee_, if it works, please post a comment to the blog...lets record it this time18:45
gyee_ayoung, sure, will do18:46
gyee_also, for sssd group lookup to work, I had to add memberUid to the posixGroup18:46
*** jasondotstar has quit IRC18:48
rushiagrgyee_: morganfainberg: so does that mean we're agreeing on doing the stable driver interface work first, without moving to python-objects in place of dictionary passing? Sorry, this part wasn't clear..18:49
rushiagror did I miss something?18:49
morganfainbergrushiagr: pretty much. a contract that the drivers can rely on for multiple releases is the important part18:50
morganfainbergif we move to passing objects, it has to be another version of the interface [easy to handle really]18:50
morganfainbergeither we convert dict -> object -> dict where needed [use the new version of the interface] - or vice versa18:51
bknudsonmarshmallow does object marshalling18:51
gyee_rushiagr, morganfainberg, yeah I agree18:51
*** jasondotstar has joined #openstack-keystone18:51
gyee_bknudson, mmm, smores18:52
rushiagrmorganfainberg: oh. This thought never occurred to me before. Yes, moving from dict -> object can be a version bump...18:52
gyee_bknudson, you have something that lookup keystone with scim?18:53
*** jasondotstar has quit IRC18:53
gyee_just curious18:53
*** stevemar has quit IRC18:54
*** stevemar has joined #openstack-keystone18:55
*** ChanServ sets mode: +v stevemar18:55
*** stevemar has quit IRC18:56
*** stevemar has joined #openstack-keystone18:57
*** ChanServ sets mode: +v stevemar18:57
rushiagrmorganfainberg: gyee_: Okay. I'm also assuming we're going to have a test matrix kind of a thing to check drivers are sticking to the contract. Coz there really is no other way I guess. At least with the method return types part..18:57
ayoungI want to move to Rust18:57
morganfainbergayoung: sure do it.18:57
morganfainbergayoung: it's a cool language.18:57
ayoungmorganfainberg, I need some more time to learn it.18:58
morganfainbergrushiagr: gate/check jobs - it's what they're for18:58
ayoungI like what I've seen so far, though18:58
morganfainbergayoung: it really is hitting the mark for me as i'm learning it18:58
morganfainbergit's fun.18:58
gyee_rushiagr, yes, phase 218:58
ayoungI want to write a PXE server in Rust18:58
morganfainbergrushiagr: return values would fail gate/check type jobs18:58
morganfainbergayoung: i know someone is writing a gearman server in rust18:58
ayoungmaybe iuntegrated with DHCP..although I'd rather move to something IPv6 ish instead18:59
morganfainbergayoung: PXE, etc that is great stuff for rust18:59
ayoungmorganfainberg, yeah, and TFTP is not a bad way for cutting teeth on a new network stack18:59
ayoungone of the easier protocols to hack19:00
rushiagrmorganfainberg: 'return values would fail gate/check type jobs' means?19:00
*** nkinder has joined #openstack-keystone19:01
*** jasondotstar has joined #openstack-keystone19:01
*** jasondotstar has quit IRC19:02
samueldmqmorganfainberg: ayoung middleware fetches the override policy and uses oslo.policy to do the overlay19:04
ayoungsamueldmq, yes19:04
samueldmqayoung: k19:04
morganfainbergrushiagr: if you don't pass correct values back our testing would show the driver fails19:04
morganfainbergrushiagr: or your testing [if using tempest] would19:05
*** jasondot_ has quit IRC19:05
morganfainbergrushiagr: so not worried about return types/values atm19:05
rushiagrmorganfainberg: ah. I thought you were saying gate/check type jobs are unable to catch return values mismatch type of problems :)19:06
rushiagrmorganfainberg: got it now. Clearer19:06
morganfainbergrushiagr: yep19:06
morganfainbergrushiagr: :)19:06
rushiagrmorganfainberg: I think I'm pretty much in sync with the agreed approach. Now about implementation..19:07
rushiagrhas anyone started it? If yes, I'd like to have a look, and contribute..19:07
morganfainbergrushiagr: i think we're needing to merge the spec update, cc gyee_ ?19:08
gyee_rushiagr, morganfainberg, I haven't started on it yet19:08
morganfainbergrushiagr: gyee_ is the goto on this one.19:08
gyee_rushiagr, if you have some cycles, please feel free19:08
morganfainbergrushiagr: so we need to merge the spec... gyee_ ^^ please review the move the spec to liberty19:08
gyee_morganfainberg, I think the spec's already there19:08
gyee_moved to liberty19:09
morganfainberggyee_: i think it isn't targeted to liberty19:09
*** stevemar has quit IRC19:09
rushiagrgyee_: sure19:09
*** dramakri has quit IRC19:10
*** stevemar has joined #openstack-keystone19:10
*** ChanServ sets mode: +v stevemar19:10
gyee_rushiagr, you want to add your name to it?19:10
gyee_or I can add your name to the assignee list19:11
*** fhubik is now known as fhubik_afk19:11
rushiagrgyee_: I'd love to contribute19:11
rushiagrgyee_: please add my name, if you're pushing up a newer version of spec. Else I'll do the same, but tomorrow19:12
rushiagrit's past midnight here right now :)19:12
gyee_rushiagr, thank, I'll push a new patch with your name added19:12
*** fhubik_afk is now known as fhubik19:13
*** fhubik is now known as fhubik_afk19:13
openstackgerritguang-yee proposed openstack/keystone-specs: Moved driver interface from backlog to liberty
*** harlowja has quit IRC19:15
*** harlowja has joined #openstack-keystone19:15
morganfainbergrushiagr: strictABC needs to be pulled out19:16
morganfainbergrushiagr: and made into a library or some such19:16
*** yottatsa has quit IRC19:16
morganfainbergrushiagr: i'll take a look at it today19:16
morganfainbergit's been on my todo for a while now19:16
rushiagrmorganfainberg: okay, thanks for clarifying..19:17
morganfainbergrushiagr: but it wont be part of keystone, it shouldn't be.19:17
*** mgarza_ has quit IRC19:17
morganfainbergit should be it's own lib19:17
rushiagrmorganfainberg: agreed19:18
*** vg_ has quit IRC19:20
*** stevemar has quit IRC19:22
*** stevemar has joined #openstack-keystone19:22
*** ChanServ sets mode: +v stevemar19:22
*** mgarza_ has joined #openstack-keystone19:24
*** nkinder has quit IRC19:24
*** stevemar has quit IRC19:32
*** stevemar has joined #openstack-keystone19:33
*** ChanServ sets mode: +v stevemar19:33
*** e0ne has joined #openstack-keystone19:33
*** _cjones_ has quit IRC19:35
*** jasondotstar has joined #openstack-keystone19:36
*** diazjf has quit IRC19:36
*** _cjones_ has joined #openstack-keystone19:39
*** dramakri has joined #openstack-keystone19:40
*** dguerri` is now known as dguerri19:41
*** stevemar has quit IRC19:42
*** stevemar has joined #openstack-keystone19:43
*** ChanServ sets mode: +v stevemar19:43
*** dguerri is now known as dguerri`19:44
*** diazjf has joined #openstack-keystone19:50
*** dguerri` is now known as dguerri19:52
*** packet has quit IRC19:53
*** gordc_ has joined #openstack-keystone19:56
*** dguerri is now known as dguerri`19:59
*** stevemar has quit IRC19:59
*** stevemar has joined #openstack-keystone20:01
*** ChanServ sets mode: +v stevemar20:01
samueldmqmorganfainberg: ayoung I just updated the sequence diagrams based on our discussions20:01
samueldmqmorganfainberg: ayoung please check and let me know what you think about it20:02
samueldmqmorganfainberg: ayoung at a glance, it looks much simpler than what we had before, looks a very consistent and realistic scope for L20:02
ayoungsamueldmq, let me add some verbage, first20:03
ayoungI think explaining the scope of the changes we are going to do in Liberty, and then we can link to the actual specs, might bea better introduction to other people20:03
samueldmqayoung: how those diagrams are defined in  terms of specs ?20:05
*** e0ne has quit IRC20:06
samueldmqayoung: most of those specs could go in a kind of backlog, so we keep only the needed specs for that scope in hands20:06
*** stevemar_ has joined #openstack-keystone20:06
ayoungsamueldmq, all specs are submitted to backlog20:06
samueldmqayoung: I mean in that wiki, we could have a separation between what is in the backlog and what is scoped to L20:08
*** afazekas has joined #openstack-keystone20:08
samueldmqayoung: matching what is defined in those diagrams20:08
stevemar_dolphm: help me!!!20:08
stevemar_i have no idea what i'm doing on a mac20:09
marekdgyee_: you should like this patch:
stevemar_lbragstad: help mojo20:09
ayoungstevemar_, what you should be doing on a Mac is installing Fedora20:09
* marekd cc/ stevemar_20:09
stevemar_marekd: pretty accurate representation20:10
stevemar_expect my reviewing to take a nose dive for a week or so20:10
samueldmqayoung: looks good, thanks20:10
ayoungsamueldmq, those should probably be updated to be the links, or contain the links, to the specs, but leave them for now20:11
bknudsonstevemar_: just set up kvm on it so you can run ubuntu in a vm20:11
bknudsonor install ubuntu on it20:11
samueldmqayoung: will do20:11
marekdTorvalds uses MacBook Air as a hardware20:11
*** afazekas has quit IRC20:12
marekdi wonly wish OSX had some tiling WMs20:13
marekdmorganfainberg: re: i was not seeint this in terms of bug, because it's like an enhancement but i can file a bug.20:15
*** harlowja has quit IRC20:18
*** harlowja has joined #openstack-keystone20:18
dolphmmarekd: stevemar_: i use
dolphmstevemar: welcome to the dark side?20:20
stevemar_dolphm: can i install it via not-the-app-store?20:20
*** Lactem has quit IRC20:21
dolphmstevemar: no, it's a paid app to upgrade OS X to be useable20:21
*** Lactem has joined #openstack-keystone20:21
stevemar_screw that noise!20:21
*** iamjarvo has quit IRC20:22
marekddolphm: ++ looks like i am ready for Mac now.20:23
dolphmstevemar: my OS X config
Lactemdolphm: Hi. I think my internet should actually work today.20:24
dolphmthe most important change:
dolphmLactem: lol welcome back to the internet then20:24
LactemUmm okay so about that bug...20:25
LactemYou added the test-improvement tag. What were you saying I need to do to finish it?20:25
dolphmLactem: add a test to, i think, keystone/tests/unit/ which does exactly what you did to show the bug could not be reproduced20:27
LactemOkay so a .sh script with the commands I did?20:27
LactemCool so then where would I put that script? Just in a private GitHub repo and put the link in the bug page or...?20:28
*** jasondotstar has quit IRC20:29
openstackgerritMarek Denis proposed openstack/keystone: Relax the formats of accepted mapping rules for keystone-manage
marekdmorganfainberg: stevemar_ rodrigods dstanek dolphm lbragstad ^^20:30
*** rushiagr is now known as rushiagr_away20:31
marekdstevemar_: thanks for cleaning the commit msg.20:32
stevemar_marekd: np sir20:32
dolphmLactem: p.s. if you prefix your message with my nick then i'll get a notification when you're talking to me!20:32
dolphmLactem: no, not a .sh -- python tests!20:32
gyee_marekd, k2k, yay!20:32
LactemYeah I probably should've done that prefix thing.20:33
dolphmLactem: we have a bunch of tests can you read through to find where yours would best fit in
dolphmbunch of related* tests20:34
*** mestery has quit IRC20:35
*** stevemar_ has quit IRC20:35
*** Lactem has quit IRC20:35
*** jasondotstar has joined #openstack-keystone20:36
marekdgyee_: if you have time/resources - go ahead and test it (I did). I feel it's been there for too long.20:37
marekdgyee_: and we need people who are familiar with that kind of stuff :-)20:37
*** Lactem has joined #openstack-keystone20:37
gyee_marekd, yeah, I'll need to test it locally20:38
marekdgyee_: please do!20:38
*** gordc_ has quit IRC20:38
gyee_I have an environment setup for sssd right now, might as well test them both20:38
Lactemdolphm: So I have to change all of those cmd-line commands into python code?20:39
*** stevemar has quit IRC20:39
*** stevemar has joined #openstack-keystone20:39
dolphmLactem: yep!20:39
*** stevemar2 has joined #openstack-keystone20:39
*** ChanServ sets mode: +v stevemar220:39
dolphmLactem: all of our tests are written in python20:39
*** diazjf has quit IRC20:40
*** stevemar2 has quit IRC20:43
*** stevemar2 has joined #openstack-keystone20:44
*** ChanServ sets mode: +v stevemar220:44
*** stevemar has quit IRC20:46
*** mestery has joined #openstack-keystone20:49
*** stevemar has joined #openstack-keystone20:50
*** jasondotstar has quit IRC20:54
ayoungseen in the keystone client code:   # TODO(heckj): supporting backwards compatibility with environment20:54
ayoung        # variables. To be removed after DEVSTACK is updated, ideally in20:54
ayoung        # the Grizzly release cycle.20:54
ayoungI miss heckj20:54
david8husamueldmq, for, there is really no need for step 10, because when you call a rule, that rule is already a osla policy rule(forgot the exact name) object..20:56
lbragstadstevemar: who is mojo?20:57
stevemarlbragstad: you fail at the simspons quiz20:57
*** belmoreira has joined #openstack-keystone20:58
samueldmqdavid8hu: step 10 loads the policy.json file20:58
samueldmqdavid8hu: from the cache, isn't that needed?20:58
lbragstadstevemar: I don't think I ever completed an episode of the simpsons...20:59
samueldmqdavid8hu: oslo policy reads from the policy.json file and then intantiate the rules objects20:59
david8husamueldmq, In that case, which component calls something like policy.Rules.load_json(policy_data, "default") ?21:00
*** stevemar has quit IRC21:01
Lactemdolphm: For testing all of those methods in, what do I use as the self parameter? For example: stack@Ubuntu64:~/keystone/keystone/tests/unit$ python -c 'from test_v3_catalog import *; print test_create_endpoint_enabled_false()'21:02
Lactemstack@Ubuntu64:~/keystone/keystone/tests/unit$ python -c 'from test_v3_catalog import *; print test_create_endpoint_enabled_false()'21:02
Lactemstack@Ubuntu64:~/keystone/keystone/tests/unit$ python -c 'from test_v3_catalog import *; print test_create_endpoint_enabled_false()'21:02
Lactemstack@Ubuntu64:~/keystone/keystone/tests/unit$ python -c 'from test_v3_catalog import *; print test_create_endpoint_enabled_false()'21:02
*** stevemar has joined #openstack-keystone21:02
Lactemdolphm: Those weren't showing up. Okay so for that method/function/whatever it's called in Python, what would the self parameter be?21:02
*** stevemar has quit IRC21:03
samueldmqdavid8hu: I think it is the oslo.policy itself, since policy_file and policy_dirs is a config defined there21:04
samueldmqdavid8hu: it gets teh policy.json, instantiates the rules, and check the APIa agianst the instantiated rules21:04
samueldmqdavid8hu: if that makes sense21:04
Lactemstack@Ubuntu64:~/keystone/keystone/tests/unit$ python -c 'from import *; test_create_endpoint_enabled_false()'21:04
LactemMaybe it's supposed to be like that, but either way there's an error.21:04
david8husamueldmq,  It felt like backard, because the enforcer has to be instantiated before doing rule enforcement in  step 9.21:09
*** csoukup has quit IRC21:10
*** diazjf has joined #openstack-keystone21:10
david8husamueldmq, when it gets to step 11, all keys and rules are loaded, so it doesn't need to go to the cache again.21:11
openstackgerritBrant Knudson proposed openstack/keystone: Tests for correct key removed
openstackgerritBrant Knudson proposed openstack/keystone: Simplify fernet rotation code
*** stevemar2 has quit IRC21:12
ayoungHey bknudson , If I do a keystone tenant-list (CLI, v2.0) as a non-prived user, I get a list of tenants that I am a member of.  If the Ansible module trys to do the same thing, it errors out, because it is trying to use the admin endopoint.  Why does the CLI work?  It should not have an unsocoped token.21:13
ayoungI know it is a jamielennox|away question21:13
bknudsonayoung: ansible doesn't use the CLI?21:13
ayoungbknudson, in this case, ansible makes a keystoneclient v2 call21:14
ayoungbknudson, I hacked my version to always do the fall back, and now ansible works...but I don't understand what the CLI is doing21:14
*** dramakri has quit IRC21:14
ayoungif there is a SQL backend, and OS_TENANT_* ios set, we should get a scoped token.  So why does the CLI know to use something that has no admin endpoint?21:15
bknudsonthe code says it does admin query first and if that doesn't work use auth_url21:17
bknudsonwhich is weird.21:17
bknudsonthe APIs are totally different so you really should have to pick one or the other21:17
ayoungbknudson, right, and that is, I think, to help enumerate projects for an unscoped token21:17
bknudsonis the auth_url being set differently in the cli case?21:18
ayoungbknudson, if a user logs in to Horizon, and there is no default project set, the user gets an un scoped token. That call is used to then list the tenants to select the first one for the webui21:18
ayoungno,  auth url is passed in like this21:19
ayoungansible localhost -m quantum_router  -a "name=ayoung-test-router  login_username=$OS_USERNAME auth_url=$OS_AUTH_URL  login_password=$OS_PASSWORD login_tenant_name=$OS_PROJECT_NAME"21:19
*** dramakri has joined #openstack-keystone21:19
ayoungmaybe it is the fact that login_tenant_name=$OS_PROJECT_NAME  is set,  where as maybe the CLI ignores it?21:19
bknudsonit's weird that a non-priv user works at all since this comment says that it uses the admin endpoint21:20
ayoungheh, I bet the CLI is looking for TENANT and I jhave PROJECT set21:20
ayounglets see...if I set tenant does it break....21:20
ayoungnope, still works21:21
ayoungagh, but I have my hack in still21:22
ayoungyeah, now it fails21:22
ayoung keystone --debug --os-tenant-name $OS_PROJECT_NAME   tenant-list21:22
ayoungYou are not authorized to perform the requested action: admin_required (HTTP 403)21:23
*** ankita_wagh has joined #openstack-keystone21:23
bknudsonwhat's the action, is it trying to get the tenant-id ?21:23
ayoungbknudson, yeah,  from the tenant-name21:23
ayoungI don't thin the module will accept tenant-id as a param...lets see21:23
morganfainbergayoung: i think we need to support a get-project-by-name (and domain) at the REST API level21:24
bknudsonthat's kind of strange you can't look up the tenant id for name without admin auth21:24
ayoungmorganfainberg, if you ayq take ayspryn21:24
*** morganfainberg is now known as ayspryn21:24
aysprynayoung: >.>21:24
*** ayspryn is now known as morganfainberg21:24
ayoungmorganfainberg, yeah.  But I think from an ansible standpoint, we need to make the whole thing work with V321:25
morganfainbergayoung: it's silly we need to do crazy things to filter/look that info up21:25
morganfainbergayoung: yes that much for sure21:25
*** pballand has quit IRC21:25
ayoungmorganfainberg, I just did a git blame on the file that does the core auth, it is all mordred21:25
bknudsonI'd say switch ansible to v3.21:25
*** jaosorior has quit IRC21:25
morganfainbergayoung: yes he owns upstream ansible stuff21:26
morganfainbergayoung: shade does a lot of the v2/v3 maic21:26
morganfainbergayoung: depending on what you're trying to accomplish, shade might do what you need (it's ansible + smart hacks for openstack smoothing out)21:26
mordredmust support both21:26
*** pballand has joined #openstack-keystone21:26
ayoungmorganfainberg, I was able to do everything I needed with V321:26
ayoungmordred, yeah, I understand that21:26
ayoungmordred, what would be really nice is supporting the whole range of auth plugins, to include SAML for authorization21:27
morganfainbergayoung: is nkinder around? wanted to ask if we were getting some resources to do get-domain-by-name and get-project-by-name REST APIs?21:27
mordredI agree - in theory we shoudl support them21:27
morganfainbergayoung: he was interested in that.21:27
mordredbut in practice it has seen no testing21:27
ayoungmorganfainberg, he's at the RH summit, and there is basically me and jamielennox|away21:28
mordredjust saw the scrollback21:28
mordredif you see ANYTHING that says login_$blah21:28
mordredit is old and deprecated21:28
morganfainbergmordred: i need to get KeystonAuth going again21:28
* morganfainberg goes and stares at the outstanding reviews21:28
morganfainbergmordred: I *think* we are relatively close once the reviews land21:28
mordredayoung: quantum_router is broken and old and will be destroyed for all of the reasons you stated21:28
mordredmorganfainberg: ++21:28
ayoungmordred, yeah, and I'm using the version of ansible that ships with F22.  I've seen that there are some minor changes moving forward21:28
*** jasondotstar has joined #openstack-keystone21:29
mordredayoung: there are MAJOR COMPLETE REWRITE NOT COMPATIBLE changes21:29
mordredin the openstack modules21:29
ayoungmordred,  where does this rewrite live?21:29
mordredbecaue the existing ones are completely unworkable21:29
morganfainbergbknudson: last of the stevedore BPs just got +A21:29
ayoungis it in upstream openstack devel branch yet?21:29
morganfainbergbknudson: for auth plugins in keystone server21:29
morganfainbergbknudson: except sample config (for obvious reasons)21:29
mordredayoung: about half has landed in ansible devel branch21:29
mordredayoung: the other half are in these PRs:
ayoungmordred, so...timing on that?  Can we still affect change?21:30
bknudsonmorganfainberg: neat!21:30
mordredayoung: yes and no - depending on what you want to change21:30
ayoungmordred, Auth....21:30
ayoungas I just said21:30
mordredyah. the interface we have for that SHOULD handle everything21:30
mordredwe designed it from scratch with auth plugins in mind21:31
ayoungmordred, I saw that the core modules code was pulling code in from anible repo.  Is that split going away?21:31
mordredfor now the core modules repo will still be a submodule in the ansible/ansible repo21:31
mordredwe might rejoin them in the future21:31
morganfainbergmarekd: are you happy with the k2k plugins in keystoneauth?21:31
morganfainbergmarekd: i'm about to press go on them.21:31
ayoungmorganfainberg, I'm not talking about that21:32
ayoungI mean that is uses the copenst helper out of anisble..21:32
mordredyes. the helper should be in released ansible at the moment21:32
*** Ctina___ is now known as Ctina21:32
mordredyou're talking about
morganfainbergCtina: /wave cause /wave is possible21:32
ayoungfrom ansible.module_utils.openstack import *21:33
mordredayoung: and openstack_full_argument_spec is the one that matters - yah21:33
mordredayoung: openstack_full_argument_spec from 1.9 should work with the openstack modules in devel21:33
Ctina@morganfainberg hi!21:33
morganfainbergCtina: just saw you un-idle and had to wave. thats all. :)21:33
mordredayoung: the README here: has the best succint discussion of what the auth dict stuff looks like - that's carried over now into shade and ansible modules and friends - so if there is a deficiency in it, we should definitely sort that out21:34
*** csoukup has joined #openstack-keystone21:34
ayoungmordred, very cool.  I'm just learning ansible.21:35
mordredayoung: yay! welcome to the party21:35
openstackgerritBrant Knudson proposed openstack/keystone: Update
mordredayoung: I actually have a pile of shade changes up adding keystone api support and several ansible PRs - so if you get bored, reviews/patches are more than welcome21:36
ayoungmordred, so, one thing I need to do, that is shade-like is this:21:36
ayoungcreate a router, a network, a subnet, attach the router to the network...blah21:37
ayoungmordred, so I had coded it up using the python APIs directly21:37
mordredayoung: so - first step of that is here:
mordredayoung: but I want to have a "just give me a working end to end thing please"21:37
mordredayoung: similar to what we do with create_server21:38
mordredwhcih will do all the things you need it to21:38
ayoungmordred, so...I have some ideas along those lines.21:38
ayoungmordred, what I wrote was designed to do 3 things21:38
ayoungcreate , destroy, display state21:38
ayoungand it had to do it for chains of tasks...21:38
ayounglike the one you have there...21:38
ayoungPOC code is :
ayoungmordred, one thing it allows is sharing of a session across all the calls, which should speed up processing.  Better than each thing needing to go to Keystone to get another token21:39
ayoungto build a network setup
ayoungit runs through the create function on each of those classes21:40
mordredyah - totally agree on shared session - we do that in shade too - the collection of tasks is interesting21:40
ayoungmordred, yeah, that was the idea I wanted to share,  because this way, they are also reversable21:41
mordredand is totally potentially either a thing we could layer on top - or just expose as api calls21:41
ayoungtear down a netwokr is painful21:41
ayoungso...reverse the Array and call teardown on each WorkItem21:41
mordredyah - that's not terrible at all21:41
ayoungmordred, its a technique from Network marshalling code21:42
mordredayoung: you know what would be great? if it wasn't so painful in teh first place ...21:42
ayoungmordred, well, I think they are probably being careful.  If you allowed a delete while something was still connected, you'd orphan the VM.  Since that is kindof a deal breaker...make it deliberate.  I don;'t fault the decision21:43
ayoungbut having a script to tear down makes a lot of sense21:44
mordredoh sure - I mean, having the functions broken out is great21:44
mordredbut also having a "please give me a vm with a workign IP" or "please give me a network that can talk to the internet"21:44
*** dguerri` is now known as dguerri21:44
mordredare some basic rollups of that sort of thing that are super useful21:44
mordredbecause at first, I do not want a network a subnet and a router - I want a place form which I can get an IP21:44
ayoungmordred, what I need is a recipe I can hand over to QA saying:  here is a Federated setup21:46
ayoungmordred, so nkinder has been working on a bunch of shell scripts for that, but it all assumes that you are working on local VMs, using libvirt.  We want to start by splitting out what  he uses into a "secutp the VMS" versus "configure the oopenstack stuff"21:48
ayoungand...the second half can probably use the OSAD approach,21:48
ayoungmordred, what is the relationship between shade and ansible?  None?21:49
morganfainbergayoung: shade makes working between clouds where things are ... "different", not terriblew21:51
mordredayoung: all of the new ansible openstack modules use shade21:51
mordredayoung: so, infra owns the library, but we basically put any logic into shade rather than the ansible modules21:51
mordredso that it can be reused for not-ansible21:51
ayoungmordred, right, but that should actually make ansible better21:51
mordredand we keep the ansible modules about data marshalling and parameter stuff21:52
ayoungbecause not one process per21:52
mordredwe're also moving nodepool to using shade21:52
mordredso - it'll be very battle tested :)21:52
ayoungmordred, OK, let me give shade a test run21:53
mordredsweet! bugs more than welcome21:54
*** pballand has quit IRC21:54
ayoungmordred, I think you mean bug reports.  If you are accpeting my code, you will get plenty new bugs...21:54
* ayoung has a realistic view of his own potential for damage21:54
*** diazjf has left #openstack-keystone21:56
mordredheh. we welcome both21:57
*** pballand has joined #openstack-keystone21:57
openstackgerritBrant Knudson proposed openstack/keystone: Document use of wip up to developer
openstackgerritBrant Knudson proposed openstack/keystone: Document update sample config up to developer
*** dguerri is now known as dguerri`21:58
morganfainbergbknudson: will be doing the strip out CLI next on that branch22:00
bknudsonmorganfainberg: cool!22:00
bknudsonmorganfainberg: I'm a little concerned about the branch now... I tried a dry-run of the merge with master and there was a conflict22:00
bknudsonso at this point I'd prefer it if nothing more was merged there until we're synced22:00
morganfainbergbknudson: i expect there to be a conflict22:00
morganfainbergbknudson: because some ksa specific things have been landed22:01
morganfainbergbknudson: i'll propose the strip out cli patch and then resolve the conflict(s)22:01
bknudsonthe conflict was pretty minor22:01
morganfainbergbknudson: we can rebase things from there22:01
morganfainbergbknudson: yeah there should be a conflict at this point22:01
bknudsonit was in exceptions22:01
morganfainbergsince iirc jamie had some ksa porting stuff merged over22:01
*** Ctina has quit IRC22:03
bknudsonneed 1 more +2 and we'll have the power to push merges.22:03
bknudsonhe he he22:03
bknudsonI don't know if that makes a review or if it's ninja.22:03
morganfainbergbknudson: it makes a merge commit22:04
morganfainbergbknudson: and we need to approve it22:04
bknudsonsince I'm not going to run tempest on it I hope there's a review22:05
*** bknudson has quit IRC22:09
*** jasondotstar has quit IRC22:09
ayoung$ ./.tox/py34/bin/shade-inventory --list | wc -l22:13
ayoung147022:13 works22:13
*** Lactem has quit IRC22:13
*** fhubik_afk is now known as fhubik22:16
*** fhubik has quit IRC22:16
*** henrynash has quit IRC22:18
*** pballand has quit IRC22:18
*** belmoreira has quit IRC22:19
*** thedodd has quit IRC22:21
openstackgerritMerged openstack/keystoneauth: Add Keystone2KeystoneAuthPlugin for K2K federation
openstackgerritMerged openstack/keystone: Short names for auth plugins
*** pballand has joined #openstack-keystone22:27
openstackgerritMerged openstack/keystoneauth: Keystone2KeystoneAuthPlugin scoping capabilities
openstackgerritMerged openstack/keystoneauth: Support discovery on the AUTH_INTERFACE
openstackgerritMerged openstack/keystone: Document entrypoint namespaces
*** belmoreira has joined #openstack-keystone22:29
openstackgerritMerged openstack/keystoneauth: Add get_communication_params interface to plugins
*** belmoreira has quit IRC22:35
openstackgerritMerged openstack/python-keystoneclient: Updated from global requirements
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Updated from global requirements
*** pballand has quit IRC22:47
*** arunkant_ has quit IRC22:48
*** charlesw has quit IRC22:48
openstackgerritMorgan Fainberg proposed openstack/keystone-specs: Moved driver interface from backlog to liberty
openstackgerritMorgan Fainberg proposed openstack/keystone-specs: Cleanup and removal of StrictABC requirement
*** pballand has joined #openstack-keystone22:50
*** belmoreira has joined #openstack-keystone22:53
*** ankita_w_ has joined #openstack-keystone22:55
*** arunkant has joined #openstack-keystone22:56
morganfainbergdolphm, jamielennox|away:
*** mgarza_ has quit IRC22:57
ayoungmorganfainberg, I can very happy +2 that22:57
morganfainbergayoung: :)22:57
ayoungmorganfainberg, will it pass gate?22:57
morganfainbergayoung: no idea22:57
morganfainbergayoung: it passes unit tests22:58
ayoung+2 anyway22:58
morganfainbergayoung: we'll see if anything is using the keystone CLI now wont we ;)22:58
ayoungmorganfainberg, yes we will22:58
*** ankita_wagh has quit IRC22:58
morganfainbergno more middleware in keystoneclient22:58
ayoung+3, -440823:00
ayoungmorganfainberg, you snuck bandit in there?23:00
ayoung,cm  morganfainberg ?  that a mistake?23:01
morganfainbergayoung: oh must have snuck in elsewhere23:01
*** zzzeek has quit IRC23:01
morganfainbergwell it should be in there but it probably was a merge oddity23:01
morganfainbergi'll get resolved when we merge back from master23:01
ayoung2015 oddity 223:02
morganfainbergremoving ~6k lines of code from keystonelcient = nice23:02
ayoungmorganfainberg, you don;t want to resubmit it?23:02
morganfainbergayoung: i need to do a rebase once we merge back from master23:02
morganfainbergso i'll wait till then23:02
morganfainbergwaiting on to land23:03
ayoungmorganfainberg, so that is due to the fact that this branched prior to bandit going in, but bandit is now in master23:03
morganfainbergand it snuck in when i did the cherry-pick from master23:03
morganfainbergso once i rebase from master we that should go away23:03
morganfainbergbut ^^ that infra change needs to land so we can merge to the feature branch23:04
ayoungmorganfainberg, so I'm going to abandon the initialize/sample code for client23:04
ayoungI'm guessing that should go somewhere, but not sure where23:04
ayoungmaybe shade23:04
ayoungalthough, it is more of an install thing.23:05
morganfainbergdevstack maybe?23:05
morganfainbergis it really a dev-only tool23:05
morganfainbergor is it something genrrally useful23:05
ayoungnay, this is using the python API.  devstack uses the CLI23:05
* morganfainberg isn't sure where it should land23:05
ayoungmorganfainberg, there was a repo I found that was hugely helpful with  examples.  I would loveto have something like that...I'll find it23:06
*** Ephur has quit IRC23:06
ayoung  morganfainberg something like that23:07
morganfainbergyeh not sure where that really belongs23:07
ayoungit showed python-*client code for all the things I wanted to do23:07
morganfainbergthis almost feels like something that belongs under docs somewhere23:07
ayoung  is a little dated, but the neutron ones were good23:08
ayounghe hasn't touched it in a year.23:10
morganfainbergok back in a bit23:13
morganfainbergneeed to finish running errands23:13
*** pballand has quit IRC23:13
*** belmoreira has quit IRC23:15
*** darrenc is now known as darrenc_afk23:16
openstackgerritMerged openstack/python-keystoneclient-saml2: Updated from global requirements
openstackgerritVictor Morales proposed openstack/keystone: Integrate OSprofiler in Keystone
*** darrenc_afk is now known as darrenc23:23
*** bradjones has quit IRC23:26
*** bradjones has joined #openstack-keystone23:26
*** bradjones has quit IRC23:26
*** bradjones has joined #openstack-keystone23:26
*** geoffarnold has quit IRC23:29
*** roxanaghe has quit IRC23:35
*** mestery has quit IRC23:36
openstackgerritMerged openstack/pycadf: Updated from global requirements

Generated by 2.14.0 by Marius Gedminas - find it at!