Wednesday, 2014-12-10

*** jorge_munoz has quit IRC00:00
*** shakamunyi has joined #openstack-keystone00:02
*** shakamunyi has quit IRC00:03
morganfainbergdstanek: I want to circle back on the osprofiler issues. Will ping you tomorrow on it.00:03
jamielennoxcan someone kick off https://review.openstack.org/#/c/139512/  - it's an additional test and in queue needed for a middleware release00:05
jamielennoxalso: https://review.openstack.org/#/c/132652/00:08
jamielennoxekarlso-: we need that review of yours - i just don't really like the interface and want to know if there is a better way to do it00:09
*** samuelms_ has quit IRC00:10
*** david-lyle is now known as david-lyle_afk00:11
*** gokrokve has quit IRC00:14
*** packet has quit IRC00:15
*** gokrokve has joined #openstack-keystone00:15
*** dims has quit IRC00:17
*** dims has joined #openstack-keystone00:19
*** gokrokve has quit IRC00:20
*** philloooo has quit IRC00:23
*** lhcheng_ has joined #openstack-keystone00:23
*** lhcheng has quit IRC00:24
*** _cjones_ has quit IRC00:27
*** dims has quit IRC00:30
*** nkinder has joined #openstack-keystone00:43
*** dims has joined #openstack-keystone00:43
*** dyasny has joined #openstack-keystone00:45
*** jamielennox is now known as jamielennox|away00:48
*** gyee has quit IRC00:50
dyasnyhey all, can anyone help me with a v3 endpoint? I have one working, but it's 2.0, the codebase is supposed to support v3 as well, but I've no idea how to get a v3 auth url to work00:51
*** avozza is now known as zz_avozza00:56
*** mikedillion has quit IRC01:00
*** _cjones_ has joined #openstack-keystone01:01
*** mikedillion has joined #openstack-keystone01:01
*** tellesnobrega_ has joined #openstack-keystone01:05
raildo_hi dyasny, do you need to enable the v3 configuration to use the API v3. Here there a quickly documentation (this is not the best way but works) explains how you can do this: http://www.symantec.com/connect/blogs/how-switch-keystone-v20-v301:08
raildo_dyasny, so, after that, you can use the  Endpints API v3: http://developer.openstack.org/api-ref-identity-v3.html#endpoints-v301:09
dyasnyraildo_, cool, I have seen that link, but it requires a token, what I have is admin user and password instead01:10
*** dims has quit IRC01:11
*** mikedillion has quit IRC01:14
*** xianghui has joined #openstack-keystone01:15
*** zzzeek has quit IRC01:16
*** gokrokve has joined #openstack-keystone01:18
raildo_the other way is setting the global variable: =OS_IDENTITY_API_VERSION=v3 (or just 3 i don't remember now) and the OS_AUTH_URL=http://127.0.0.1:35357/v301:18
raildo_dyasny, I think this changes are enough to enable keystone v3.01:19
dyasnyraildo_, and I don't need to run keystone endpoint-create for the v3 URL?01:20
raildo_after that you'll need. This changes is to enable the keystone v3.01:20
dyasnyraildo_, ah, ok, so first I enable, then I create01:21
raildo_dyasny, yes01:21
*** afaranha_ has joined #openstack-keystone01:21
dyasnymakes sense :)01:21
raildo_dyasny, you can use the POST /v3/endpoints with the body http://paste.openstack.org/raw/148284/01:22
dyasnyraildo_, I keep seeing this notation, where do I apply it?01:22
* dyasny is a newb at this stuff, can you tell?01:23
raildo_hahaha no problem01:23
raildo_you can use a rest client like postman for the API calls http://www.getpostman.com/ or you can use the keystone client to create a endpoint01:24
raildo_$ pip install python-keystoneclient01:25
raildo_:)01:25
*** marcoemorais has quit IRC01:26
raildo_so you just need to make a source openrc with your user and password, and after that, you can use the keystone client01:26
dstanek@morganfainberg: ok, i'm around now and i'll be around all day tomorrow01:28
morganfainbergi'm avout to head to dinner01:28
morganfainbergi think01:28
morganfainbergso. don't want to need to run middle of the convo01:28
morganfainbergi also might have a fix to help SQL token backend not suck as much until we can ditch it 100%01:28
raildo_dyasny, you can see more about the keystone client here: http://docs.openstack.org/cli-reference/content/keystoneclient_commands.html01:29
dyasnyraildo_ I have keystone client, I think01:30
dyasnyat least I can run keystone commands01:30
dyasnyproblem is, most of them return 40401:30
dyasnyraildo_ the worst part is I get this for admin, but not a simple user01:31
*** xianghui has quit IRC01:32
*** xianghui_ has joined #openstack-keystone01:32
raildo_dyasny, hum.. that is a little problem here, you need the permission to do this action and in the policy.json, you need to be a admin to create a endpoint: https://github.com/openstack/keystone/blob/master/etc/policy.json#L2401:33
*** xianghui_ is now known as xianghui01:33
dyasnyraildo_, the 404 happens as admin01:33
dyasnyI wonder how I can check where the request is going, maybe there's more than one service and one of them is down01:34
*** lhcheng_ has quit IRC01:34
raildo_but you can get a token for the admin user?01:34
*** lhcheng has joined #openstack-keystone01:35
dyasnyraildo_, my rc file only contains username and pass, no tokens in there01:35
*** mitz- has quit IRC01:39
*** lhcheng has quit IRC01:39
*** mitz_ has joined #openstack-keystone01:41
raildo_dyasny, right, so, you can use this file to get a token to use in keystone. you can run a commend like $ source openrc <user> <password>01:41
raildo_and after that you can use the keystone in command line01:41
raildo_or you can export a token like OS_SERVICE_TOKEN=admin01:42
raildo_:)01:42
dyasnyraildo_, http://fpaste.org/158197/18175764/ this is what my openrc file looks like01:43
dyasnyfor another user I simply change the user, tenant and password lines01:43
dyasnyraildo_ could a service-create command pointing to a v3 URL before enabling v3 cause this?01:44
raildo_no. because the keystone client service-create use the v2.0 =/01:45
raildo_I always use the API calls to do this =/01:46
raildo_dyasny, you can see how use the keystone v3 API here: http://adam.younglogic.com/2013/09/keystone-v3-api-examples/01:46
raildo_that is very simple :)01:46
dyasnyraildo_ seems to be quite straightforward01:49
raildo_dyasny, ++01:50
raildo_dyasny, Any questions, you can talk here :)01:52
dyasnyraildo_, thanks, let me get into the thingie first :)01:52
*** shakamunyi has joined #openstack-keystone01:53
*** oomichi_ has joined #openstack-keystone01:57
*** htruta_ has joined #openstack-keystone02:02
dyasnyraildo_, {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}02:04
dyasnyafter I create the token file and run the curl command02:04
raildo_dyasny, ok so run the command echo $TOKEN02:05
raildo_what is the return?02:05
dyasnyempty02:06
dyasnytried to port 5000 and 3535702:06
dyasnyraildo_ ^^02:07
raildo_so, you don't export the token =/02:07
dyasnyno, only username and password02:08
raildo_you can try this examples too http://adam.younglogic.com/2013/11/more-keystone-v3-api-examples/02:08
*** harlowja has joined #openstack-keystone02:10
dyasnyraildo_, ok, I got the token, had a typo in the json file initially02:13
raildo_yeah \o02:13
*** erkules_ has joined #openstack-keystone02:14
raildo_so, you can use this token, to create a endpoint02:14
raildo_you just need to create a .json file, with the body http://paste.openstack.org/raw/148284/02:15
*** r-daneel has quit IRC02:15
raildo_curl -si -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" http://localhost:35357/v3/endpoints -d @<file_json_name>.json02:16
*** erkules has quit IRC02:16
dyasnyraildo_, it looks, when I enable --debug (good thing I googled that one up, strace is insane) I can see that the initial request goes to the v2.0 portal, but that one returns v3 somewhere down the road, which brings me to a 404. I can pastebin it02:22
*** raildo_ has quit IRC02:24
*** henrique has joined #openstack-keystone02:26
*** htruta_ has quit IRC02:29
*** afaranha_ has quit IRC02:40
*** oomichi_ has quit IRC02:44
openstackgerritwanghong proposed openstack/keystonemiddleware: support micro version if sent  https://review.openstack.org/13091602:45
*** henrique has quit IRC02:45
*** _cjones_ has quit IRC02:49
*** _cjones_ has joined #openstack-keystone02:50
*** drjones has joined #openstack-keystone02:51
*** htruta_ has joined #openstack-keystone02:51
*** tellesnobrega_ has quit IRC02:51
*** wanghong has quit IRC02:52
*** tellesnobrega_ has joined #openstack-keystone02:53
*** _cjones_ has quit IRC02:54
*** r-daneel has joined #openstack-keystone02:56
*** r-daneel has quit IRC03:00
*** sluo_wfh has joined #openstack-keystone03:01
*** drjones has quit IRC03:05
*** _cjones_ has joined #openstack-keystone03:05
*** chrisshattuck has joined #openstack-keystone03:06
*** wanghong has joined #openstack-keystone03:09
*** harlowja is now known as harlowja_away03:12
*** r-daneel has joined #openstack-keystone03:15
*** chrisshattuck has quit IRC03:25
*** jamielennox|away is now known as jamielennox03:26
*** sluo_wfh is now known as sluo_laptop03:26
*** richm has quit IRC03:34
*** oomichi_ has joined #openstack-keystone03:35
*** htruta_ has quit IRC03:42
*** Shohei has quit IRC03:49
*** Shohei has joined #openstack-keystone03:50
*** r-daneel has quit IRC03:50
*** Shohei_ has joined #openstack-keystone03:52
*** Shohei has quit IRC03:54
*** diegows has quit IRC03:59
dyasnyraildo, still here?04:03
*** Shohei_ has quit IRC04:33
*** Shohei has joined #openstack-keystone04:34
*** Shohei has quit IRC04:36
*** Shohei has joined #openstack-keystone04:36
*** Shohei has quit IRC04:44
*** Shohei has joined #openstack-keystone04:45
*** _cjones_ has quit IRC04:49
*** _cjones_ has joined #openstack-keystone05:00
*** _cjones_ has quit IRC05:08
*** tellesnobrega_ has quit IRC05:12
*** _cjones_ has joined #openstack-keystone05:13
*** _cjones_ has quit IRC05:14
*** radez is now known as radez_g0n305:15
*** _cjones_ has joined #openstack-keystone05:15
*** lhcheng has joined #openstack-keystone05:19
*** harlowja_away is now known as harlowja05:28
*** harlowja is now known as harlowja_away05:29
*** davechen_ has joined #openstack-keystone05:38
*** KanagarajM has joined #openstack-keystone05:46
*** gokrokve_ has joined #openstack-keystone05:53
*** gokrokve has quit IRC05:57
*** gokrokve_ has quit IRC05:58
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Imported Translations from Transifex  https://review.openstack.org/13624306:05
*** gyee has joined #openstack-keystone06:08
*** ChanServ sets mode: +v gyee06:08
openstackgerritJamie Lennox proposed openstack/python-keystoneclient: Expose version matching functions to the public  https://review.openstack.org/12993506:15
*** gokrokve has joined #openstack-keystone06:23
*** gokrokve has quit IRC06:25
*** gokrokve has joined #openstack-keystone06:25
*** Shohei has quit IRC06:28
openstackgerritMerged openstack/keystonemiddleware: Auth token tests create temp cert directory  https://review.openstack.org/12228006:28
*** Shohei has joined #openstack-keystone06:29
*** gokrokve has quit IRC06:30
*** gokrokve has joined #openstack-keystone06:32
*** Shohei has quit IRC06:33
*** gokrokve_ has joined #openstack-keystone06:34
*** Shohei has joined #openstack-keystone06:35
*** gokrokve has quit IRC06:36
*** gokrokve_ has quit IRC06:39
*** oomichi_ has quit IRC06:46
*** lhcheng has quit IRC06:47
*** lhcheng has joined #openstack-keystone06:48
*** henrynash has joined #openstack-keystone06:48
*** ChanServ sets mode: +v henrynash06:48
*** lhcheng has quit IRC06:53
*** ajayaa has joined #openstack-keystone06:55
*** Shohei has quit IRC07:07
*** _cjones_ has quit IRC07:13
*** henrynash has quit IRC07:18
*** henrynash has joined #openstack-keystone07:19
*** ChanServ sets mode: +v henrynash07:19
*** zz_avozza is now known as avozza07:24
*** k4n0 has joined #openstack-keystone07:25
ekarlso-jamielennox: uhm, ok, so who'll figure out that ?07:39
*** nellysmitt has joined #openstack-keystone07:55
*** gyee has quit IRC07:58
*** Shohei has joined #openstack-keystone08:00
*** mzbik has joined #openstack-keystone08:02
*** bdossant_ has joined #openstack-keystone08:05
*** bdossant has quit IRC08:09
*** _cjones_ has joined #openstack-keystone08:15
*** avozza is now known as zz_avozza08:16
*** jistr has joined #openstack-keystone08:17
*** henrynash has quit IRC08:17
*** _cjones_ has quit IRC08:20
*** stevemar has quit IRC08:25
jamielennoxekarlso-: i'm not sure - this is why it's been left for a while08:26
jamielennoxi was talking to gyee about it a bit and we were talking about like allow=(DEPRECATED, UNKNOWN) or something08:26
*** oomichi has joined #openstack-keystone08:26
jamielennoxhe was suggesting just allowing setting a base endpoint_filiter as a dictionary on the adapter08:27
jamielennoxi just don't want to have to keep adding these booleans that represent one thing08:28
*** gokrokve has joined #openstack-keystone08:32
*** Shohei has quit IRC08:37
*** gokrokve has quit IRC08:37
*** Shohei has joined #openstack-keystone08:37
ekarlso-jamielennox: well it'd be awesome if u could look at it08:38
*** Shohei has quit IRC08:38
*** Shohei has joined #openstack-keystone08:38
*** zz_avozza is now known as avozza08:39
*** oomichi has quit IRC08:40
*** nellysmitt has quit IRC08:41
*** andreaf has quit IRC08:47
*** andreaf has joined #openstack-keystone08:48
*** marekd|away is now known as marekd08:55
*** yasu_ has joined #openstack-keystone09:17
*** lsmola has quit IRC09:18
*** i159 has joined #openstack-keystone09:20
*** lsmola has joined #openstack-keystone09:22
*** nellysmitt has joined #openstack-keystone09:29
*** gokrokve has joined #openstack-keystone09:32
*** gokrokve has quit IRC09:37
*** tellesnobrega_ has joined #openstack-keystone10:10
*** bdossant_ has quit IRC10:12
*** bdossant has joined #openstack-keystone10:12
*** aix has joined #openstack-keystone10:14
*** tellesnobrega_ has quit IRC10:23
*** tellesnobrega_ has joined #openstack-keystone10:31
*** gokrokve has joined #openstack-keystone10:32
*** gokrokve has quit IRC10:36
*** lhcheng has joined #openstack-keystone10:42
*** tellesnobrega_ has quit IRC10:43
*** ajayaa has quit IRC10:54
*** ajayaa has joined #openstack-keystone10:56
*** ajayaa has quit IRC11:01
*** NellyK has joined #openstack-keystone11:01
*** NellyK is now known as nellysmi_11:02
*** nellysmitt has quit IRC11:04
*** ajayaa has joined #openstack-keystone11:19
*** lhcheng has quit IRC11:22
*** lhcheng has joined #openstack-keystone11:23
*** nellysmi_ has quit IRC11:25
*** esco-franc has joined #openstack-keystone11:27
esco-franchi people11:27
*** lhcheng has quit IRC11:27
esco-franci would like to talk about the OTP implementation in keystone...11:29
esco-francAre there volotaries?11:29
*** gokrokve has joined #openstack-keystone11:32
*** gokrokve has quit IRC11:37
*** jamielennox is now known as jamielennox|away11:38
*** nellysmitt has joined #openstack-keystone12:04
*** david-lyle_afk is now known as david-lyle12:09
*** aix has quit IRC12:09
*** ajayaa has quit IRC12:10
*** esco-franc_ has joined #openstack-keystone12:12
*** esco-franc has quit IRC12:13
*** david-lyle is now known as david-lyle_afk12:17
*** david-lyle_afk is now known as david-lyle12:17
*** jistr has quit IRC12:20
*** david-lyle is now known as david-lyle_afk12:23
*** david-lyle_afk is now known as david-lyle12:23
*** jaosorior has joined #openstack-keystone12:27
*** ajayaa has joined #openstack-keystone12:28
*** david-lyle is now known as david-lyle_afk12:31
*** david-lyle_afk is now known as david-lyle12:31
*** gokrokve has joined #openstack-keystone12:32
*** diegows has joined #openstack-keystone12:36
openstackgerritMarek Denis proposed openstack/keystone: Identify groups by name/domain in mapping rules.  https://review.openstack.org/13901312:36
*** gokrokve has quit IRC12:37
marekdrodrigods: ^^ you were instersted in it as well12:38
*** jistr has joined #openstack-keystone12:40
rodrigodsmarekd, nice!12:40
marekdfeel free to review :-)12:40
openstackgerritRodrigo Duarte proposed openstack/keystone-specs: Policy enforcement library  https://review.openstack.org/13348012:43
rodrigodsayoung, ^ we need to discuss the middleware bits, so we can update this spec12:44
openstackgerritMarek Denis proposed openstack/keystone: Scope federated token with 'token' identity method  https://review.openstack.org/13059312:52
marekdmorganfainberg: coul you take a look at this patch ^^ ?12:53
*** dyasny has quit IRC12:53
marekdayoung: websso + ipsilon. you wanted to talk yesterday12:54
samuelmsdolphm, morning .. just saw my patch for refactoring list role assignments got out of the list of blocking reviews for the release12:57
samuelmsdolphm, I'll work hard to have everything ready to review again until tomorrow (now considering the inherited roles on projects)12:57
samuelmsdolphm, I hope it will still be able to be back on that list and then we have it in k1 (-:12:59
*** david-lyle is now known as david-lyle_afk13:00
*** aix has joined #openstack-keystone13:04
openstackgerritRodrigo Duarte proposed openstack/keystone-specs: Policy enforcement library  https://review.openstack.org/13348013:13
dhellmannrodrigods, ayoung : oslo.concurrency installs a command line app called 'lockutils-wrapper' for use in tox.ini13:18
*** amakarov_away is now known as amakarov13:18
rodrigodsdhellmann, hmm, thanks! Will update the code base, we also need to figure out if lockutils is *really* needed13:22
*** yasu_ has quit IRC13:31
*** gokrokve has joined #openstack-keystone13:32
ayoungdhellmann, thanks.  What's it do?13:33
ayoungmarekd, OK, lets start with that.13:34
ayoungrodrigods, stay tuned!13:34
*** richm has joined #openstack-keystone13:34
*** diegows has quit IRC13:36
ayoungmarekd, so here is the link to Ipsilon  https://git.fedorahosted.org/cgit/ipsilon.git/tree/README13:36
*** gokrokve has quit IRC13:37
ayoungmarekd, think of Ipsilon as an analogue to ADFS for *nix Systems13:37
dhellmannayoung: it takes the place of the old "python -m" hack to set OSLO_LOCK_PATH for you13:41
openstackgerritAlexander Makarov proposed openstack/keystone: Memcache connection pool excess check  https://review.openstack.org/14068113:42
openstackgerritAlexander Makarov proposed openstack/keystone: Memcache connection pool excess check  https://review.openstack.org/14068113:44
openstackgerritSamuel de Medeiros Queiroz proposed openstack/keystone: Improve List Role Assignment Tests  https://review.openstack.org/13702113:47
*** esco-franc_ has quit IRC13:48
ajayaaHi guys. Would keystone devs be okay with a NOSQL backend for keystone? We have started with the easiest one, token backend. Hopefully we would be able to migrate other backends as well. I just wanted to know people's opinion on this thing before we have invested too much in this. :)13:51
ajayaaayoung, dstanek, dolphm ^^13:52
*** rushiagr has joined #openstack-keystone13:53
*** stevemar has joined #openstack-keystone13:54
*** ChanServ sets mode: +v stevemar13:54
stevemarmarekd, ping13:54
marekdhey14:01
marekdwhat's up?14:01
ayoungajayaa, don't do it!14:06
ayoungWe are trying to drop the Token backend14:07
ayoungand the other backends need transactions14:07
ayoungmarekd, hey, I got dibs..I not only contacted you first, I didn't send a naked ping like stevemar did.14:07
marekdayoung: hey. I think we can talk in parallel, just no more than 5 topic at the same time in a group of three....14:08
marekdayoung: actually..sorry...i didn't read up and didn't notice your msgs ;/14:09
marekdayoung: ok, so let's suppose Ipsilon is like ADFS. To me it's like a standarlone service that can act as a legitimate SAML2 IdP.14:09
ayoung'Salright14:09
marekdayoung: you said ipsion will have discovery page14:10
marekdgreat14:10
ayoungmarekd, so I want the discovery page to live in Ipsilon.  Question then is how to sync the list of IdpS14:10
marekdayoung: i'd handle this in ipsilon.14:10
marekdand again, stay away with that from Keystone.14:11
marekdleave discovery to services like ADFS or ipsilon.14:11
*** rushiagr is now known as rushiagr_away14:11
marekdkeystone is only a service provider here, nothing else.14:11
*** nkinder has quit IRC14:13
amakarovajayaa, there is an effort to make policies manageable, I'd suggest you look there. You may propose your storage for policies since there is none now :)14:15
amakarovajayaa, btw, this is not mongodb, i hope?14:16
ajayaaamakarov, nope.14:17
*** KanagarajM has quit IRC14:17
ajayaaamakarov, cassandra or a a generic api layer on top of it.14:17
ajayaaThe transactional requirements can be avoided by schema design, I suppose. I can't say with 100% surety unless I take each backend case by case and analyse.14:18
ajayaaayoung ^14:19
ajayaabtw, the token backend being removed thing, is it a part of making tokens ephemeral?14:20
amakarovajayaa, hmm... Isn't it an overkill? And Cassandra, AFAIK, tends to be an SQL backend :) It has it's API, but as I recall, they shifted to recommend using SQL interface14:21
*** joesavak has joined #openstack-keystone14:21
amakarovajayaa, do you have actual performance problem with existing backends to introduce a new one?14:22
ajayaaamakarov, for small deployments it seems an overkill. But failure scenarios and things like geo-replication are better handled by cassandra.14:22
*** bknudson has joined #openstack-keystone14:23
*** ChanServ sets mode: +v bknudson14:23
amakarovajayaa, can you give us a link to BP?14:23
ajayaaamakarov, No bp yet. Just floating the idea and measuring the reaction to it. :)14:24
amakarovajayaa, I think it needs discussion. Anyway, bp has it's advantages - at list your idea become somewhat more persistent :)14:25
ajayaaamakarov, Yes. I would go ahead and try to put as much detail I can think of. :)14:26
amakarovajayaa, for example, I wand to use Redis: https://blueprints.launchpad.net/keystone/+spec/redis-storage-backend14:26
amakarovajayaa, I don't have enough time for now to implement, but it is something I may show my colleagues14:27
*** mzbik has quit IRC14:27
ajayaaamakarov, that's cool.14:28
*** gokrokve has joined #openstack-keystone14:32
openstackgerritMarek Denis proposed openstack/keystone: Identify groups by name/domain in mapping rules.  https://review.openstack.org/13901314:34
*** gokrokve has quit IRC14:37
marekdrodrigods: thanks for review :-)14:38
ayoungajayaa, We are currently splitting up the assignment backend.  I'd probably recommend starting there.14:39
ayoungProjects and domain  probably make sense for NosQL...so long as the revocation events are transactional14:39
*** dims has joined #openstack-keystone14:39
*** darren-wang has quit IRC14:41
ajayaaayoung, Thanks for the suggestion. BTW, regarding the token backend, I thought UUID tokens are not going away very soon and if UUID token is used then we would need a table to keep track of user and project and role associated with that token.14:41
ajayaaand PKI tokens are too large to fit into http headers where you have large catalogs.14:43
*** radez_g0n3 is now known as radez14:44
ayoungajayaa, well, there is already Dogpile support for multiple Key Value store backends.  A Cassandra one would not be too hard to add.14:46
ayoungJust, I'd rather people spent effort on strategic changes as opposed to bandaids for things we are looking to fix anyway14:46
ayoungthere really is no good reason to persist tokens14:47
ajayaaayoung, correct. So the tokens backend is going to use dogpile apis which in turn could use any backend?14:47
ayoungrodrigods, looking at the policy file now14:47
ayoungOne issue is that domain scoped tokens are strange14:48
*** rushiagr_away is now known as rushiagr14:48
openstackgerritMehdi Abaakouk proposed openstack/keystone-specs: tokens swift persistent backend  https://review.openstack.org/13151514:48
ayoungideally, everything would be the same scope.  For example, if I were to say "role:admin and domain_id:%(domain_id)s"  that is actually an error14:48
ayoungbecause a project scoped token will match that14:49
ayoungthus, we need a test that a token is domain scoped14:49
openstackgerritRodrigo Duarte proposed openstack/keystone: Inherited role assignments to projects  https://review.openstack.org/13855214:53
openstackgerritRodrigo Duarte proposed openstack/keystone: Fix inherited user role test docstring  https://review.openstack.org/14070514:53
rodrigodsayoung, which one?14:53
ayoungrodrigods, for example, all of the user operations that require domain scoped token should have:14:53
ayoung:"scope:domain" in their rules (I think)  testing now14:54
rodrigodsayoung, that's true14:55
rodrigodsnever noticed that problem14:55
rodrigodssamuelms, afaranha ^14:55
samuelmsayoung, nice catch... check for scope is really the right thing to do14:58
ayoungsamuelms, for now...I'm actually trying to do away with the distinction between projects and domains14:58
samuelmsayoung, please gimme a ping (not naked btw) to say if it worked :p14:58
ayoungand move to a domain Is-A project, just a special onw14:59
ayoungone14:59
*** nkinder has joined #openstack-keystone15:00
samuelmsayoung, gotta have lunch.. back soon15:02
*** samuelms is now known as samuelms-away15:02
openstackgerritAlexander Makarov proposed openstack/keystone: Memcache connection pool excess check  https://review.openstack.org/14068115:04
*** dims has quit IRC15:05
*** dims has joined #openstack-keystone15:05
*** topol has joined #openstack-keystone15:09
*** ChanServ sets mode: +v topol15:09
*** dims has quit IRC15:10
*** ajayaa has quit IRC15:11
marekdstevemar: do we need urls in regions for something else than K2K ?15:12
marekdstevemar: oh, and wantem me to ping you when it's ready: https://review.openstack.org/#/c/13901315:13
*** r-daneel has joined #openstack-keystone15:24
vsilvahey marekd, how can we help you with the mapping enhancements?15:27
vsilvaor better put, do you want any help?15:27
*** gokrokve has joined #openstack-keystone15:30
*** gordc has joined #openstack-keystone15:35
*** philloooo has joined #openstack-keystone15:43
phillooooHey I am not sure if it's a good place to ask problems about federated keystone...  I setup keystone with federation in devstack, and was able to generate scoped token for federated user. But It seems the federated token can not be used as a normal scoped token?15:43
*** dyasny has joined #openstack-keystone15:44
phillooooThe token can be validated via keystone api, but I can't use the token to construct a session object in keystoneclient, report "user not found error". If it's not a good place to ask, please point me to the right place...15:46
rodrigodsone line fix (noticed while writing inherited roles to project tests): https://review.openstack.org/#/c/140705/1/keystone/tests/test_backend.py15:47
rodrigodsayoung, morganfainberg ^15:47
*** dyasny has quit IRC15:50
morganfainbergmorning15:50
rodrigodsmorganfainberg, morning :) need your feedback in the oslo.policy codebase, whenever you have a moment15:52
morganfainbergyeah15:52
ayoungrodrigods, if you need a comment to explain what a test is about, you really should rename the test15:54
rodrigodsayoung, I think the intention of the docstring is to give a testplan, and since we have a pattern to be followed, that's why this first line is there15:56
ayoungrodrigods, yeah...the name 'grants' is what caught my attention15:56
ayoungI think we should not be calling things grants anymore15:56
ayoungthey are role assignments.15:56
rodrigodsayoung, true...15:58
marekdvsilva: hey, so first of all first part is implemented here https://review.openstack.org/#/c/13901315:58
marekdplease review, and put lots of +1s so cores have easier job :-)15:58
openstackgerritayoung proposed openstack/keystone: Identify groups by name/domain in mapping rules.  https://review.openstack.org/13901315:58
vsilvayeah, I saw that marekd15:59
vsilvawhat then? :)15:59
marekdvsilva: so, please review :-)16:00
marekdvsilva: there is one last point not yet touched16:00
marekdah, no.16:01
marekdsorry16:01
marekdtwo work items16:01
marekdimplementation of whitelisting/blacklisting16:01
marekdand 'groups' keywork16:01
marekdand 'groups' keyword16:01
ayoungmarekd, I gave you aplaceholder BP for that one:  https://blueprints.launchpad.net/keystone/+spec/mapping-enhancements  but I can't find the actual spec16:01
ayounglink?16:01
marekdhttps://review.openstack.org/#/c/138035/14/specs/kilo/mapping-enhancements.rst16:02
rodrigodsayoung, since I got your attention here, can you elaborate about the middleware part in https://review.openstack.org/#/c/133480 ?16:02
*** dyasny has joined #openstack-keystone16:03
ayoungrodrigods,yeah.  OK, look at the code that is in Keystone and in Nova beyond just the straight policy.py file.  There is stuff for pulling the contexts together before passing to Policy. That is the stuff I was origianlly thinking about.16:03
ayoungin keytstone, rules.py16:03
afaranhaayoung, the difference between projects and domain is that a project admin cannot create/list/delete projects, neither CRUD of users16:03
vsilvacool marekd. By 'what then' I meant what else other than reviewing the first part? It looks like it's already going pretty well16:04
ayoungrodrigods, http://git.openstack.org/cgit/openstack/keystone/tree/keystone/policy/backends/rules.py16:05
marekdvsilva: so i mentioned that there are two work items not touched yet.16:05
ayoungafaranha, one sec...I'll address that after helping rodrigods16:05
rodrigodsayoung, thought this part was in the controller16:05
marekdvsilva: you can implement one and i may take another.16:05
rodrigodsthe decorator controller16:05
ayoungrodrigods, also http://git.openstack.org/cgit/openstack/keystone/tree/keystone/common/authorization.py16:05
ayoungrodrigods, and16:05
ayounghttp://git.openstack.org/cgit/openstack/keystone/tree/keystone/common/controller.py#n5416:06
ayoungrodrigods, then, if you look at what nova does:16:06
marekdvsilva: there is whitelist/blacklist impl and later 'groups' keyword.16:06
vsilvasweet marekd, toss me whichever one you don't pick16:06
ayounghttp://git.openstack.org/cgit/openstack/nova/tree/nova/policy.py16:06
ayoungwe need to drop "is_admin"16:07
marekdvsilva: how about you make white/blacklisting ? and I will toss anoher patch on top of yours?16:08
marekdvsilva: fine for you?16:08
vsilvathat sounds good marekd16:09
rodrigodsayoung, so nova doesn't build a auth_context like Keystone, just uses the credentials dict16:09
marekdgreat16:09
ayoungrodrigods, and that credentials dict is built in middleware16:09
rodrigodsayoung, hmm16:10
rodrigodsayoung, so for the first version (without caches) we'd remove this logic from middleware?16:10
*** jistr has quit IRC16:10
ayoungrodrigods, no...I think we also need the caches up front16:11
ayoungmiddleware can still build the context16:11
rodrigodsayoung, the idea is to make a really generic context?16:11
ayoungmeaning?16:11
rodrigodsayoung, a context that could be used directly by Keystone/Nova/Glance, without the need of http://git.openstack.org/cgit/openstack/keystone/tree/keystone/common/authorization.py16:12
rodrigodscould be used directly by Keystoneclient, to pass it to the policy lib, I mean16:12
rodrigodsand could work for everyone16:12
ayoungrodrigods, so, yeah,  that is what I was targetting.  I also want the same context to be usable for revocation events.16:13
openstackgerritAndre Aranha proposed openstack/keystone: Modify the policy v3 sample  https://review.openstack.org/12350916:13
ayoungThat is what I have in the WIP patch:  https://review.openstack.org/#/c/138519/16:13
openstackgerritayoung proposed openstack/python-keystoneclient: Access Info  https://review.openstack.org/13851916:14
morganfainbergrodrigods, did you follow the steps in the graduation document?16:14
morganfainbergrodrigods, also, what is the git repo?16:14
morganfainberggit/github16:14
ayounglets see if a rebase makes it through the neutron tests16:14
rodrigodsmorganfainberg, yes... https://github.com/rodrigods/oslo.policy me and ayoung were wondering if could remove the dependency from lockutils16:15
svashekahi, is there a way to create trust using python keystone client?16:15
morganfainberghm.16:15
rodrigodsmorganfainberg, it's used here: https://github.com/rodrigods/oslo.policy/blob/master/oslo_policy/tests/test_policy.py#L12116:16
ayoungrodrigods, BTW, did you see dhellmann 's post earlier about the cli for concurrency?16:16
rodrigodsayoung, yes :)16:16
ayoung"rodrigods, ayoung : oslo.concurrency installs a command line app called 'lockutils-wrapper' for use in tox.ini"16:16
morganfainbergrodrigods, then no. since it's in the test.16:16
ayoungsvasheka, yes16:16
morganfainbergrodrigods, removing that dependency has to be done post import into gerrit16:16
morganfainbergrodrigods, so it goes under normal review16:16
rodrigodsmorganfainberg, hmm ok16:17
rodrigodsmorganfainberg, but I can use the libs instead of incubated code, right?16:17
openstackgerritDavid Stanek proposed openstack/keystone-specs: Adds a spec for fixing Keystone's DI  https://review.openstack.org/13593116:17
morganfainbergrodrigods, i would graduate it as-is then convert to the lib under gerrit16:17
morganfainbergrodrigods, *or* convert to the lib before graduation16:17
morganfainbergthe change needs to go through gerrit imo16:17
morganfainbergdstanek, nice ^!16:18
dstanekmorganfainberg: if people would stop finding things wrong with it, it would be done :-)16:18
morganfainberghwhw16:19
morganfainberghehe*16:19
rodrigodsmorganfainberg, great, will have another look in it and I think it is free for you and ayoung perform the other steps16:19
morganfainbergrodrigods, great.16:19
ayoungsvasheka, I'm working on some client examples.  By Client, I mean python code, mind you, not CLI16:19
rodrigodsayoung, thanks for the middleware explanation, will try to address dstanek comments in the spec and ping you if I need help16:20
*** samuelms-away is now known as samuelms16:20
ayoungrodrigods, ++16:20
*** timcline has joined #openstack-keystone16:21
*** ajayaa has joined #openstack-keystone16:22
*** gokrokve has quit IRC16:23
rodrigodsmorganfainberg, so I think it's ready for the oslo folks take a look :)16:24
dstanekrodrigods: is that for the policy spec?16:25
rodrigodsdstanek, the github repo?16:26
dstanekrodrigods: ah, the comment you linked to yesterday16:26
dstanekrodrigods: are you thinking about removing the lock?16:27
rodrigodsdstanek, yes... we couldn't figure out if it is really needed16:27
rodrigodsbut I guess we will propose via regular gerrit review once the lib graduates16:27
dstanekrodrigods: i can look right now and see if i can tell - do you have that link handy?16:27
rodrigodsdstanek, https://github.com/rodrigods/oslo.policy/blob/master/oslo_policy/tests/test_policy.py#L12116:28
rodrigodsdstanek, but morganfainberg suggested we proceed like I described above16:28
*** mikedillion has joined #openstack-keystone16:29
morganfainbergdstanek, we either need to make change prior to move to it's own repo or post, we probably shouldn't do it as part of the "move to it's own repo" if it works as is16:29
*** baker_ has joined #openstack-keystone16:30
*** gokrokve has joined #openstack-keystone16:30
dstanekmorganfainberg: yes, i totally agree - i would probably wait until after the move to make it cleaner/quicker16:31
*** dims has joined #openstack-keystone16:32
*** jaosorior has quit IRC16:33
*** baker_ has quit IRC16:33
*** baker__ has joined #openstack-keystone16:33
*** zzzeek has joined #openstack-keystone16:35
*** dims has quit IRC16:36
*** dims has joined #openstack-keystone16:36
openstackgerritBrant Knudson proposed openstack/keystonemiddleware: Refactor auth_token revocation list members to new class  https://review.openstack.org/10240316:37
openstackgerritBrant Knudson proposed openstack/keystonemiddleware: Refactor identity version handling to strategy pattern  https://review.openstack.org/14076516:37
samuelmsmorganfainberg, did you see the spec for use Swift as kvs storage for tokens in Keystone ?16:38
morganfainbergsamuelms, i don't know what the advantage to that would be16:38
morganfainbergthat sounds like a lot of overhead to store tokens16:38
morganfainbergi don't know if i saw an actual spec for it though16:38
samuelmsmorganfainberg, yep .. in addition, I'm concerned about the increase of network traffic16:38
morganfainbergsamuelms, yeah.16:39
samuelmsmorganfainberg, https://review.openstack.org/#/c/131515/5/specs/kilo/tokens-swift-backend.rst16:39
morganfainbergbecause you also need a token to get a ... token if swift is configured poorly16:39
samuelmsmorganfainberg, please see my comment at line 6016:39
samuelmsmorganfainberg, well, take your time .. you may be busy right now16:40
morganfainbergsamuelms, i'm actually working on a fix *right now* to handle token bloat with some advanced sql work16:40
morganfainbergsomething we should have done before - as a stopgap and if we move to revocation events (we need to get that done *hint ayoung*) we can fix an issue that makes the token table really hard to work with16:41
dstaneksamuelms: you are close on the lazy loading idea16:42
morganfainbergin-fact if we didn't have to update the rows for deletion [easier with rev. events] we *could* move that table back to MyISAM and eliminate overhead/locking problems (no gap-lock) for deletes, since append to MyISAM doesn't readlock.16:42
dstaneksamuelms: in what i am proposing all for deps would need to be provided to the object's __init__. they could be real instances or possibly fakes16:42
morganfainbergdoesn't solve the Postgres case(s), but we can't solve this issue db-agnostic16:42
samuelmsmorganfainberg, nice, do you have already submitted any patch?16:44
samuelmsmorganfainberg, would like to take a look at =B16:44
*** mikedillion has quit IRC16:44
samuelmsdstanek, cool16:45
*** shakamunyi has quit IRC16:45
samuelmsdstanek, for tests, we reload api's, etc for each test?16:45
samuelmsdstanek, or we do that for each test class?16:46
*** ctracey has quit IRC16:46
*** zhiyan has quit IRC16:46
dstaneksamuelms: we would just be creating instances in the setUp - just like we do now16:46
*** serverascode___ has quit IRC16:47
dstaneksamuelms: but you'll see Object(Dep1(), Dep2()) instead of clearing out the DI cache and getting new instance automagically16:47
openstackgerritBrant Knudson proposed openstack/keystonemiddleware: Refactor auth_token revocation list members to new class  https://review.openstack.org/10240316:47
openstackgerritBrant Knudson proposed openstack/keystonemiddleware: Refactor identity version handling to strategy pattern  https://review.openstack.org/14076516:47
*** jraim has quit IRC16:48
*** _cjones_ has joined #openstack-keystone16:50
*** _cjones_ has quit IRC16:50
samuelmsdstanek, got that. but we pass Object(Dep1(), Dep2()) or something like Object('dep1', 'dep') and then Object class would call a kind of 'object dependency manager' to instantiate them:16:50
dstanekrodrigods: i'm not sure why that would be needed; if we run tests in parallel they are using different objects anyway16:50
*** _cjones_ has joined #openstack-keystone16:50
samuelmsdstanek, ?16:50
samuelmsdstanek, that's a question :p16:50
morganfainbergsamuelms, no code yet, i'm testing this out16:51
morganfainbergsamuelms, making sure it will work16:51
dstaneksamuelms: no, Object would just get the instances; the pattern of DI separates the construction of an object graph from the object graph itself16:51
rodrigodsdstanek, yeah, and they pass (greenbar)16:51
samuelmsmorganfainberg, ok :)16:51
samuelmsmorganfainberg, feel free to ping me once you post any code16:52
morganfainbergsamuelms, absolutely, it is (initially) going to be mysql specific16:52
morganfainbergand sqlite will *never* see this solved.16:52
morganfainbergcause i just don't care enough to try and do this for sqlite16:52
samuelmsmorganfainberg, haha ++16:52
dstaneksamuelms: the new objects are essentially like this: http://paste.openstack.org/show/129171/16:53
dstaneksamuelms: they don't know how their dependencies are created16:53
openstackgerritZhiQiang Fan proposed openstack/python-keystoneclient: Enable hacking rule F821  https://review.openstack.org/13409616:53
samuelmsdstanek, yeah, I got that bit ... what I was thinking was if we had instances created for each test, we could have:  def __init__(self, credential_api, identity_api, revoke_api=None, ***LAZY=False***):16:54
dstaneksamuelms: the creation is handled in what i was calling an uber factory http://paste.openstack.org/show/129179/16:54
*** remix_tj has joined #openstack-keystone16:54
samuelmsdstanek, and if Object called 'object dependency manager' to instantiate, we just need to lazy evaluate, and then just load  credential_api if the test uses it16:54
samuelmsdstanek, but I got your idea, looks great16:55
samuelmsdstanek, great, so that you can even define at the 'uber factory' if the api's will be singletons or not, etc.16:56
samuelmsdstanek, and anyone who needs an api, request it to that factory16:56
samuelmsdstanek, right?16:56
dstaneksamuelms: exactly - all the objects know is that they are constructed with the right things16:57
samuelmsdstanek, perfect ++16:57
samuelmsdstanek, looking forward to see the patch16:57
dstaneksamuelms: our tests would essentially acts as factories creating object, fakes and mocks; and then glue them together to do interesting things16:57
samuelmsdstanek, ++16:57
openstackgerritOpenStack Proposal Bot proposed openstack/python-keystoneclient: Updated from global requirements  https://review.openstack.org/13479417:03
*** jsavak has joined #openstack-keystone17:04
openstackgerritgordon chung proposed openstack/pycadf: sync oslo  https://review.openstack.org/13838117:05
*** i159 has quit IRC17:05
david-lyle_afkayoung, morganfainberg, if you get a chance will you take a look at this https://review.openstack.org/#/c/136980/ It's a proposal by cinder toward an API to provide list of actions the user can perform17:05
*** joesavak has quit IRC17:05
*** david-lyle_afk is now known as david-lyle17:06
* david-lyle guess I'm here now17:06
openstackgerritBrant Knudson proposed openstack/keystonemiddleware: Change tenant to project  https://review.openstack.org/12706617:06
openstackgerritBrant Knudson proposed openstack/keystonemiddleware: Change admin user to service user.  https://review.openstack.org/12707517:06
*** baker__ has quit IRC17:08
openstackgerritBrant Knudson proposed openstack/keystonemiddleware: Change admin user to service user.  https://review.openstack.org/12707517:09
remix_tjHello, i'm planning a multi region setup of openstack swift with keystone auth. I'm looking for infos about the setup of keystone: can i have a redundant keystone setup on every region or i can have only one active server?17:11
*** k4n0 has quit IRC17:12
openstackgerritgordon chung proposed openstack/pycadf: deprecate audit middleware  https://review.openstack.org/13838617:13
ayoungdavid-lyle, so...sort of17:13
ayoungdavid-lyle, https://adam.younglogic.com/2014/11/dynamic-policy-in-keystone/  is my attempt to lay out the steps to get there.  It should not be cinder specific17:14
*** joesavak has joined #openstack-keystone17:15
*** browne has joined #openstack-keystone17:15
david-lyleayoung: so a grander vision17:16
david-lyle:)17:16
ayoungdavid-lyle, driven by a conversation long ago with a Horizon dev....17:16
ayoungWho was asking exactly the same question.17:16
ayoungI think that conversation was in Portland17:17
*** jsavak has quit IRC17:18
david-lyleI think we've had this conversation off and on for a couple years now :)17:18
david-lyleI'd love to see this happen17:18
david-lyleand I really don't want to see each project do it individually, we'll arrive at chaos17:19
david-lylethis is a TC or at least larger cross-project related question/decision17:19
david-lylemore likely the cross-project, the TC doesn't seem to take on these types of discussions17:21
raildomorganfainberg, do you have some free time to review the HMT specs? :)17:24
*** zzzeek has quit IRC17:25
*** nellysmitt has quit IRC17:25
*** zzzeek has joined #openstack-keystone17:25
ayoungsamuelms, OK,  so it looks like Keystone does not preserve the "scope" value when creating the context17:27
morganfainbergpopulating a table with 5MM rows is a slow process...17:27
morganfainberga test table that is.17:27
*** marcoemorais has joined #openstack-keystone17:28
rodrigodsayoung, question: I understood the context passed to the enforcer should be as generic as possible, but since Keystone Server doesn't use the middleware, this https://github.com/openstack/keystone/blob/master/keystone/common/authorization.py will remain the same?17:33
ayoungrodrigods, sort of17:33
ayoungrodrigods, I want to replace it with this:17:33
*** browne has quit IRC17:33
ayounghttps://review.openstack.org/#/c/138519/  rodrigods17:33
ayoungrodrigods, which is very much keystone specific, which is why I was origianlly thinking that policy enforcement needed to be in keystoneclient17:34
ayoungand this stuff is hard and lots of details which is why you seem to get a different answer from me every time you ask a question.  Sorry about that.17:35
rodrigodsayoung, heh17:36
*** avozza is now known as zz_avozza17:36
rodrigodsayoung, ok... makes sense to me17:36
rodrigodsbut I'm still in the direction of the enforcement being in Keystonecleint17:36
ayoungrodrigods, let me try to lay it out step by step17:36
ayoungone, a user gets a token and calls an API on some remote service endpoint.  Let's say Glance for example17:37
ayoungkeystonemiddleware.auth_token.Auth is responsible for  unpacking the token and putting it in the request object somehow17:37
ayoungthat middleware does not check policy17:38
ayoungbecause most of the APIs need to fetch and object from their datastore first, and see if the user has access to that object17:38
ayoungso, lets say that the API is to write a new image into a project.  In this case,  the Glance server would want to run the policy check to determine if the user has write access to that project17:39
ayoungactually, in that case, everything is specified in the API...bad example17:40
ayoungbut most glance APIs are that simple...I think that glance objects tend to be immutable.17:40
*** gokrokve has quit IRC17:40
rodrigodsayoung, ok, anything that needs "target..."17:40
ayoungLet's change the example a bit:  swift17:40
ayounga user wants to modify an object stored in a swift container17:40
ayoungso first swift needs to fetch the record for that object from its store and check policy against that record17:41
*** jraim has joined #openstack-keystone17:41
* ayoung acknowledges that it is hard to come up with a proper example here17:41
ayoungso, yeah, target17:41
*** philloooo has quit IRC17:42
ayoungwhat we want is for the services to be able to call:17:42
ayoungenforce_policy(api_name, auth_context_from_auth_token_middleware, target_from_persisted_store)17:42
ayoungOK,  so the enforce_policy call is going to be passed through to oslo.policy, but then there is caching, so...17:43
ayoungpolicy.enforce_policy  needs to get the right set of rules17:43
ayounglets assume that this is the first call on this server.17:44
ayoungso there is no cache object to start17:44
rodrigodsayoung, hmm this looks like exactly how I was thinking... but I think we need to know about the future plans to have a cache, but not implement it right now17:44
ayoungpolicy.enforce_policy then needs to create a cache object17:44
ayoungwell., we have an ENFORCER object right now17:44
ayoungI think that can morph into the cache17:45
rodrigodsayoung, the enforcer can store the cache, all in Keystoneclient17:45
ayoungSo the enforcer needs a backing store, and the policy.py file assumes it is in the local filesystem, I think17:45
rodrigodsjust remembered about the cache need in middleware17:45
*** aix has quit IRC17:45
morganfainbergayoung, what is the cache object?17:45
morganfainbergayoung, as in what are you planning on caching?17:46
ayounghttp://git.openstack.org/cgit/openstack/keystone/tree/keystone/openstack/common/policy.py#n20017:46
*** gyee has joined #openstack-keystone17:46
*** ChanServ sets mode: +v gyee17:46
ayoungmorganfainberg, the cache object is to abstract away the remote call to fetch the policy file from keystone server17:46
* notmyname just logged in and sees you're talking about swift17:47
ayoungmorganfainberg, right now, policy.py assumes that it is getting its rules from a file in the file system, and that file is named by  CONF.policy_file17:47
morganfainbergnotmyname, yeah they're using it as an example17:47
*** afaranha has quit IRC17:47
ayoungor, an explictly passed in file17:47
ayoungnotmyname, were are talking policy enforcement.  I might be taking Swift's name in vain here.17:47
morganfainbergayoung, right - just making sure i wasn't missing something or seeing some other cache being created17:47
notmynameayoung: heh. let me know if I can help with details or use cases :-)17:48
ayoungmorganfainberg, right, so I want the cache to do two things:  hide how the file is actually fetched, and determine when it needs to fetch a new file17:48
rodrigodsmy question is: where is this cache being placed? middleware/kc/policy lib17:48
ayoungnotmyname, I need to look at the policy.json file deployed by swift.  I learn something new each time I look at that from a different service17:48
ayoungrodrigods, OK back to our example17:49
morganfainbergayoung, so, you *may* want to look at how dogpile.cache handles proxy objects - so the way you "add" the caching in is by a proxy object that caches the data - that layers on top of the "fetch" logic17:49
ayoungmorganfainberg, I suspect I do17:49
morganfainbergso the base functionality is fetch, and proxy does caching.17:49
morganfainbergor even proxy object could be smarter than that17:49
ayoungmorganfainberg, so you specify customer "fetchers" then?17:50
ayoungcustom17:50
*** stevemar has quit IRC17:50
morganfainbergayoung, so the way proxy in dogpile works is a lot like our managers work17:50
morganfainbergjust can layer them as deep as you want17:50
ayoungmorganfainberg, we have dogpile in auth_token, right?17:50
morganfainbergno17:50
morganfainbergwe need to get it there17:50
*** gokrokve has joined #openstack-keystone17:51
ayoungmorganfainberg, OK, so  lets talk WRT policy.  Where would each piece of the caching mechanism live?17:51
morganfainbergso ideally, i would like to see caching done via dogpile17:52
ayoungpolicy should not be reading from the file system17:52
morganfainbergwhether tha is an KVS-style or memoized, i'm not concerned17:52
ayoungso would we have a "file" backed dogpile impl?17:52
*** shakamunyi has joined #openstack-keystone17:52
ayoungpolicy should be able to stand alone.  So I would think it would be something like this:17:52
morganfainbergwhat you would have is a stevedore (or similar) plugin that says "I fetch files by XXX method", one of which would be file, one would be keystone etc17:52
*** afaranha has joined #openstack-keystone17:53
morganfainbergif you *wanted* to you could use memoize decorator to cache - alternative you use a proxy, an object that is called (works like our manager via getattr overload) but is a chain of objects not just a manager->driver17:53
morganfainbergthat could implement the dogpile proxy.17:54
ayoungthere is a RuleCache object.  Its an abstract base class that lives in  In oslo.policy.  In oslo.policy we have a default implementation that uses that cache.  Then, in middleware or in keystoneclient, we have a cache implementation that fetches its data from keystone based on endpoint id.17:54
amakarovmorganfainberg, hello! A case: we have a trust with depth 10, changed max_redelegation_count to 5, restarted keystone, and trying to redelegate. What's the result?17:54
morganfainbergamakarov, old trusts should maintain their max depth - it already exists, new trusts are constrained.17:54
ayoungamakarov, you can redelegate the trust to 5 only now17:54
morganfainbergamakarov, redelegation_count17:54
ayoungah, that is only enforced on trust creation?17:55
morganfainbergamakarov, i'd say so17:55
ayoungthen yeah,  delegate to 1017:55
morganfainbergayoung, ^17:55
ayoung++17:55
amakarovmorganfainberg, so redelegated trust will have depth 9 regardless of config constraint?17:55
morganfainbergamakarov, yeah the *max* is only enforced on initial trust creation17:55
*** ctracey has joined #openstack-keystone17:55
*** shakamunyi has quit IRC17:55
morganfainbergamakarov, the one that enables redelegation17:56
ayoungmorganfainberg, OK,  so the cache interface would need to get initialization data from somewhere.  I'm guessing that for keystone, that should be from CONF17:56
amakarovmorganfainberg, thanks17:56
ayoungactaully, for the default file based on, that would work, too17:56
morganfainbergamakarov, makes sense right? I'm open to be convinced otherwise, but i thnk the UX would be *very bad* otherwise, suddenly a trust breaks because someone changed a config in keystone17:56
rodrigodsayoung, your design up there... didn't get why you need a cache in oslo.policy if we have the cache in kc for the specific endpoint/service17:57
ayoungmorganfainberg, so the question is, should the keystone based cache implementation live in keystone client then?  I think it should.  Then keystonemiddleware really is out of the picture for enforcement17:57
*** timcline_ has joined #openstack-keystone17:57
*** timcline has quit IRC17:57
rodrigodsayoung, not totally out, since it builds the context :)17:57
ayoungrodrigods,  policy is a generic rules engine.  It should be able to function for use cases other than keystone17:57
morganfainbergayoung, so this might be a middleware thing.17:58
morganfainberggyee, ping - need to ask a question re HP stuff, but want to make sure you're here before typing it all out17:58
gyeek17:58
openstackgerritHaneef Ali proposed openstack/keystone: Fix wrong log message in token flush  https://review.openstack.org/14079017:58
gyeejust got back to my desk17:58
ayoungmorganfainberg, I know it feels like it should be, but I think it will work better as a straight function call.   It won't be a WSGI middleware17:59
rodrigodsayoung, the enforcement to be in the middleware, it should be (like) a decorator, right? (like to imagine how I'd implement designs :P)18:01
morganfainbergrodrigods, no.18:01
morganfainbergrodrigods, it can't be18:02
morganfainbergrodrigods, right now18:02
amakarovmorganfainberg, I agree about UX - pedigree tracking, for example, will be easier. I needed approval :)18:02
morganfainbergand i'm worried about too many PEPs if we do REST PEP, then object PEP, then other PEP18:02
morganfainbergpep = policy enforcement point18:02
morganfainbergso lets start with 1 pep, and break it up once we have all the mechanics working unless we have a good reason to invert it.18:03
morganfainbergright now we have 1 PEP and we can make that work, so i think we can probably keep with that for now.18:03
*** serverascode___ has joined #openstack-keystone18:03
morganfainbergand it wont *really* break anyone18:03
rodrigodsmorganfainberg, the enforcement logic being fully performed by oslo.policy?18:03
openstackgerritayoung proposed openstack/keystone-specs: Self Management of Domain Scoped Roles  https://review.openstack.org/13872818:04
morganfainbergayoung, i think you're right - the fetcher is a KSC thing - the file base fetcher is a plugin that likely is from oslo_policy18:04
morganfainbergayoung, so, oslo_policy has the "fetch" logic that uses a stevedore plugin?18:04
morganfainbergayoung, the default is file-based (compat for now with today) that is provided with oslo_policy. KSC (or even separate pip package, doesn't matter) implements a "get from keystone" one18:05
morganfainbergayoung, does that line up with your thoughts?18:05
ayoungmorganfainberg, I think so.18:05
ayoungmorganfainberg, the nice thing about this is then it is callable from middleware, but not tied to it.  I think that is the right abstraction18:05
*** lhcheng has joined #openstack-keystone18:05
*** jsavak has joined #openstack-keystone18:06
ayoungrodrigods, I could see adding the decorator to keystonemiddleware18:06
morganfainbergayoung, yeah eventually we *could* lift strictly rest level enforcement to it's own PEP in middleware18:06
morganfainbergbut to start we don't do that.18:06
*** shakamunyi has joined #openstack-keystone18:06
ayoungAHA!18:07
ayoungand then in Keystone server we have an alternate fetcher that pulls policy right out of the keystone.policy.Manager18:08
ayoungmorganfainberg, I think we move all of the common policy enforcement code from keystone.common.controll into the middleware18:08
ayoungto include the decorator, if at all possible18:09
ayoungwe could do the same thing with "fetchers" for the target data18:09
*** joesavak has quit IRC18:09
ayoungsame thing as we are talking about doing for the cahce18:09
ayoungcache18:09
*** zhiyan has joined #openstack-keystone18:09
ayoungrodrigods, we have this line: https://github.com/openstack/keystone/blob/master/keystone/common/controller.py#L12418:10
ayoungself.get_member_from_driver(kwargs[key])18:10
ayoungI think we would need to abstract that away somehow.  That is the thing that says "get the object"18:10
*** hugokuo has quit IRC18:10
*** charz has quit IRC18:10
ayoungsome sort of lambda object would be probably the right abstraction18:10
morganfainbergayoung, yep.18:11
morganfainbergayoung, the pulling out of the manager directly that is for keystone18:12
morganfainbergayoung, we need to ditch the decorator in ksytone i think18:12
ayoungmorganfainberg, I think we could leave it as a decorator if we specified the helper function in there, but the guts of the decorator function should get pulled out into something directly callable.18:13
*** gokrokve has quit IRC18:13
ayoungand then the decorator becomes relatively trivial:18:13
morganfainbergayoung, we already ahve that it makes it *very* hard to read18:14
ayoungmorganfainberg, I'm not wedded to it.18:14
morganfainbergayoung, the callback is in there for object fetching18:14
morganfainbergand enforcement18:14
ayoungthe callback is actually in the controller18:14
morganfainbergi think nova's model is much better18:14
ayoung self.get_member_from_driver(kwargs[key])18:14
morganfainbergno - the callback is passed to the decorator18:14
morganfainbergthere are cases we need to get *other* objects18:14
morganfainbergso we have even wackier decorator stuff18:15
morganfainbergi'd say we should use the nova model, explicitly call at the expected PEP not do all sorts of wierd stuff to get around the decorator18:15
rodrigodsthat's how I see it http://paste.openstack.org/show/148856/ :)18:15
rodrigodsmorganfainberg, so for that design, I can only see it working in keystoneclient18:16
*** esp has joined #openstack-keystone18:16
ayoungmorganfainberg, "the callback is passed to the decorator"  where?18:16
ayoungI think you are going from memory18:16
ayoungnot from the code:18:16
morganfainbergayoung, sec let me find it but i know we do it18:16
ayoungmorganfainberg, start here https://github.com/openstack/keystone/blob/master/keystone/common/controller.py#L12418:17
ayoungget_member_from_driver is implemented by each of the controllers18:17
morganfainbergayoung, https://github.com/openstack/keystone/blob/master/keystone/assignment/controllers.py#L54118:17
rodrigodskeystone/assignment/controllers.py:    @controller.protected(callback=_check_grant_protection)18:17
morganfainbergrodrigods, not sure what "client.enforce" is referencing in your example18:18
ayoungAh...so we have two ways of doing the same thing18:18
ayoungI missed that callback implementation18:18
rodrigodsmorganfainberg, a enforcer class in the client, like rules.py18:18
morganfainbergrodrigods, no the enforcer should *not* be implemented in keystoneclient18:18
morganfainbergthe enforcer comes from policy.py, in oslo_policy. oslo_policy has a plugin that knows how to fetch the policy file18:19
morganfainbergksc implements a plugin that can get it from keystone (or it could be a new packaged if needed)18:19
ayoungBut I+2ed that...18:19
ayounghttps://review.openstack.org/#/c/38308/18:19
*** nellysmitt has joined #openstack-keystone18:19
morganfainbergayoung, yeah we needed it - because the decorator is limited18:20
morganfainbergif the PEP could be in the controller function - we wouldn't need the callback18:20
*** charz has joined #openstack-keystone18:20
*** nellysmitt has quit IRC18:20
ayoungso we could start by removing the "get_member_from_driver" to the callback mechanism18:20
morganfainbergthe decorator wraps the function - so we needed a way to "do extra" work18:20
morganfainbergayoung, well - except the callback largely replaces (wholesale) the enforcement logic18:21
morganfainbergit's all hard to read.18:21
rodrigodsayoung, so this https://review.openstack.org/#/c/133480/ lost its meaning, right?18:21
ayoungrodrigods, not entirely18:21
openstackgerritAlexander Makarov proposed openstack/keystone: Trust redelegation  https://review.openstack.org/12689718:21
morganfainbergayoung, so i think we should remove the decorator - maybe have a decorator that ensures a PEP was actuated18:21
ayoungrodrigods, that is the part we are discussing now:  what to do about the code that is common to all things that need to enforce keystone policy18:21
ayoungmorganfainberg, you mean test on the way out?18:22
morganfainbergbasically the wrapper just causes a failure if enforcement wasn't called18:22
morganfainbergso there isn't an "oops" in the logic18:22
morganfainbergthis api requires enforcement - we make sure it's called [this is a wild thought don't take it as we "should do it"]18:22
morganfainbergmove away from using the decorator to do the real enforcement.18:23
ayoungmorganfainberg, what if we made the callback mandatory?18:23
morganfainbergayoung, i don't think it's going to make things more readable.18:23
*** hugokuo has joined #openstack-keystone18:24
morganfainbergayoung, or easier to understand18:24
morganfainbergagain i'm not set that we *need* to ditch the decorator, i just know it has limitations - and we have crazy/ugly workarounds for it18:24
ayoungmorganfainberg, yeah...Its just that we want to make it easy to audit that all methods have protection on them.18:24
ayoungI hear ya18:25
morganfainbergayoung, thats why i was thinking @enforce_required18:25
rodrigodsayoung, so about the spec, I don't think I can write all the changes being discussed here :)18:25
morganfainbergall that wrapper does is guarantee that enforcement was called in the function - or fails.18:25
morganfainbergayoung, s/does/would do18:25
dstanekwe don't do dynamic mapping in federation now right? we pick a single mapping based on IdP and use it?18:25
morganfainbergdstanek, i want to say that is correct18:25
ayoungmorganfainberg, we could even do that as a middleware component18:26
morganfainbergayoung, not sure how middleware does it - but that is because i haven't thought about it18:26
morganfainbergayoung, remember some APIs *wont* be enforced18:26
ayoungmorganfainberg, I just meant a middleware component that checkedthat policy is enforced18:26
dstanekmorganfainberg: well, two 'maybes' make a 'most likely' - i'll go with it18:26
morganfainbergdstanek, :)18:27
ayoungmorganfainberg, well,  a no-op policy should be OK if they stop enforce "TOKEN-REQUIERD_ON_ALL_APIS" at the auth_token level18:27
morganfainbergdstanek, gyee and marekd would be the ones to 100% confirm - only cause i don't see stevemar18:27
dstanekmorganfainberg: i see that the mapped auth plugin used federation_api.get_mapping_from_idp_and_protocol to it's a safe bet that mapping are based on IdP and protocol18:30
*** harlowja_away is now known as harlowja18:31
rodrigodsdstanek, yes, you tie mapping/protocol/idp18:31
gyeeyes, map to protocol is 1:1, protocol to idp is 1:*18:32
rodrigodsgyee, ++18:32
gyeesorry idp to protocol is 1 to many18:32
gyeerodrigods, what's your email? I am looking at k2k federation, your blog is very help btw18:33
dstanekthanks rodrigods & gyee18:33
rodrigodsgyee, rodrigods@lsd.ufcg.edu.br :)18:33
rodrigodsgyee, glad it helps18:33
dstanekmorganfainberg: i was frightened when i read https://review.openstack.org/#/c/138728/ - the pin idea :-( lots of holes to plug18:33
rodrigodsgyee, or rodrigodsousa@gmail.com . I might be more responsive in the former18:34
gyeerodrigods, thanks18:35
dtroyerKeystone-ites: with use_tls enabled for LDAP, is the server cert validated?  if so, does it use the system certs if tls_cacert* is not supplied?18:47
gyeedtroyer, should be18:50
gyeeit will fall back to system cert18:51
dtroyerok, cool.  thanks18:51
openstackgerritAndre Aranha proposed openstack/keystone: Modify the policy v3 sample  https://review.openstack.org/12350918:52
afaranhaayoung, rodrigods I just fixed yours reviews, could you review again, please? https://review.openstack.org/#/c/123509/18:55
ayoungafaranha, nope18:56
ayoungafaranha, I'm editing it as we speak :)18:56
ayoungafaranha, we don't have the cloud_admin role, although that is an interesting addition18:57
afaranha:P great!18:57
samuelmsayoung, :)18:57
ayoungafaranha, sorry about that.18:58
ayoungI'll post mine WIP18:58
ayoungbut I'm not done18:58
ayoungwas just running tox18:58
afaranhaayoung, no problem :), thanks!18:58
openstackgerritayoung proposed openstack/keystone: Modify the default policy  https://review.openstack.org/12350918:58
afaranhayou are editing the spec right?18:58
*** joesavak has joined #openstack-keystone18:59
*** jsavak has quit IRC18:59
afaranhaayoung Yes, our intention is to create a role to be clear to the admin who gets which role18:59
ayoungGAH!  ignore the changes to policy.json18:59
ayoungafaranha, so different forms of admin get different roles...maybe19:00
dstaneksamuelms: your comment on the self management review made me bust out laughing19:00
afaranhaayoung, policy.json is the next step :)19:00
ayoungI'm in the middle. Don't spend too much time onthat review19:00
afaranhaAs we discussed with Henry Nash19:00
*** timcline_ has quit IRC19:01
samuelmsdstanek, haha19:02
samuelmsdstanek, did I lie?19:02
afaranhaayoung I didn't try yet, does this scope:domain works? Didn't know we could have this key19:02
*** marcoemorais has quit IRC19:02
*** marcoemorais has joined #openstack-keystone19:03
*** marcoemorais has quit IRC19:03
dstaneksamuelms: no, i read it very quickly 'i think almost all comments were addressed' and did a double take when my brain processed 'none'19:03
afaranhaayoung, Ah, ok! gonna make the reviews after reading everything :P19:03
*** marcoemorais has joined #openstack-keystone19:03
ayoungafaranha,  yeah,  I want to get it such that each API is specified like this:19:03
ayoungrule:role_admin19:04
*** timcline has joined #openstack-keystone19:04
ayoungand then role_admin is defined by the base role and the scope19:04
samuelmsdstanek, haha :-)19:04
*** amakarov is now known as amakarov_away19:05
*** jistr has joined #openstack-keystone19:05
afaranhaayoung, Great! having a different way of checking for each set of rules is a pain19:05
ayoungafaranha, the target having the domain set on each object makes the rules really tricky19:06
ayounghence all of the19:06
ayoung "admin_and_matching_group_domain_id": "role:admin and domain_id:%(group.domain_id)s",19:06
*** gordc has quit IRC19:06
rodrigodsayoung, you didn't ack about updating https://review.openstack.org/#/c/133480/ :)19:07
*** ayoung is now known as shhh_im_busy19:07
*** chrisshattuck has joined #openstack-keystone19:07
afaranhaayoung this rule is deprecated, I just removed compatibility with the old admin role, as we agreed earlier19:08
*** shhh_im_busy is now known as ayoung19:10
ayoungafaranha, what I think we want is something that allows the service to fetch an object, and then to check attributes on that object without knowing the specific object type.  Those attributes we know of are;  domain, project,  and owner19:12
*** gordc has joined #openstack-keystone19:13
afaranhaayoung something like "target", but target need to know the object, an I correct?19:15
afaranhaI'll look at the policy code to understand better19:15
*** _cjones_ has quit IRC19:17
*** _cjones_ has joined #openstack-keystone19:18
vsilvaping dstanek; how's the infra going?19:20
dstanekvsilva: right now i'm stuck getting an IdP running and configured to talk to Keystone19:21
vsilvadstanek, is that within some special infra or any idp? you're using pysaml2, right?19:22
*** baker has joined #openstack-keystone19:22
dstanekvsilva: any IdP i haven't look at it again since Monday, but that's where it stopped19:23
dstanekvsilva: not sure yet how to tell pysaml2 to trust the keystone sp19:23
rodrigodsdstanek, you mean exchange metatada?19:24
samuelmsdstanek, rodrigods matadata* :p19:26
samuelmsdstanek, rodrigods, me metadata* :p19:26
*** erkules_ is now known as erkules19:27
*** jistr has quit IRC19:30
*** shakamunyi has quit IRC19:33
*** shakamunyi has joined #openstack-keystone19:34
*** timcline has quit IRC19:35
*** marcoemorais has quit IRC19:40
*** marcoemorais has joined #openstack-keystone19:41
*** marcoemorais has quit IRC19:42
*** marcoemorais has joined #openstack-keystone19:43
*** timcline has joined #openstack-keystone19:43
*** timcline has quit IRC19:44
bknudsonI changed /opt/stack/keystone/.tox/py27/local/lib/python2.7/site-packages/eventlet/wsgi.py so it raises an exception that's not caught so I could get a backtrace when the tests log...19:46
bknudsonhttp://paste.openstack.org/show/148908/19:46
openstackgerritayoung proposed openstack/keystone: Check and delete for  policy_association_for_region_and_service  https://review.openstack.org/14012219:46
bknudsonit looks like test_connection_pool_maximum_connection_get_timeout is starting a thread on a timer and when the timer goes off it gets an error.19:46
bknudsonmaybe something has been closed or something.19:46
*** timcline has joined #openstack-keystone19:56
ayoungafaranha, dagnabit...I don't like this.  I want to be able to say that whatever the fetched object is is the target19:56
ayoungand that user.project.id == target.object.project_id.19:57
ayoungwe should not have to specify a separate rule for each object type19:57
bknudsondoes the connection pool have to be closed or something?19:58
bknudsonI didn't even think the tests used eventlet.20:01
*** dims has quit IRC20:02
ayoungbknudson, unit tests have to specify which threading model to use20:06
ayoungand the default is eventlet.  Its set from the test setup using the environment module, IIRC20:06
bknudsonyep, environment.use_eventlet()20:07
ayoungbknudson, try setting STANDARD_THREADS20:08
bknudsondo we need that so we can run a server at the same time as the clients in a single thread?20:08
ayoungno idea20:09
bknudsonit might not even be that test that's the problem... could be another test that put the action on eventlet and it just notices it in this other test.20:09
ayoungafaranha, I missed you earlier post.  Yes 'something like "target", but target need to know the object, an I correct'20:09
*** timcline has quit IRC20:10
bknudsonmaybe there's a way to tell eventlet to run until all current actions are complete20:10
afaranhaayoung, agreed. But there's also some rules that we don't use target, other we don't use the object, etc. I didn't test if a rule that uses object.field_id works with TARGET.object.field_id20:11
afaranhamaybe not20:11
afaranhaor rules that only gets the field_id if work like: OBJECT.field_id....20:11
*** timcline has joined #openstack-keystone20:11
ayoungafaranha, Ok I think I can do this:20:12
ayounghttp://git.openstack.org/cgit/openstack/keystone/tree/keystone/common/controller.py#n12520:13
ayoung policy_dict['target'] = {self.member_name: ref, 'member':ref}20:13
ayoungthen collapse all of those rules into20:13
ayoung "admin_and_matching_target_group_domain_id": "rule:admin_required and domain_id:%(target.member.domain_id)s",20:14
ayounglets see if that works20:14
afaranhaayoung, "We deduce this by seeing if the class has specified a get_member() method and that kwargs contains the appropriate entity id."20:17
afaranhaSo, projects and domains need the method get_member in order to work?20:18
ayoungafaranha, nah, its on the controller right now20:18
ayoungafaranha, we'll work to make this smoother.  RIght now I'm trying to make it work at all for Keystone20:20
afaranhaayoung, I didn't get 100% how this method works....20:22
ayoungMAJIK!20:22
afaranhaIt assumes that exists a self.member_name, so this variable should be the the "domain_id" that we specify in the policy...20:22
ayoungAHHH but the grants code is not going to let that work20:23
afaranhaWhy? Is it something that we can change easily?20:24
ayoungyeah..I can beat it into shape20:25
ayoungthis code is getting uglier by the hour, though20:25
afaranhaah! Right, guess I understood that code now. At least what it does20:25
openstackgerritMerged openstack/keystone: Fix inherited user role test docstring  https://review.openstack.org/14070520:27
*** dyasny has quit IRC20:30
afaranhaayoung, Ok, grant will not work with this because it can be target to a project or a domain, right?20:30
ayoungafaranha, nah, I'll make it work20:33
*** jsavak has joined #openstack-keystone20:33
afaranhaayoung, Although I think it's not a problem because the rule will match with the id provided, project_id or domain_id.20:36
rodrigodsayoung, https://review.openstack.org/140161 cleaned it up.20:36
*** joesavak has quit IRC20:37
ayoungthanks rodrigods20:37
afaranha"project_admin_for_grants": "role:project_admin and project_id:%(target.member.id)s",20:37
ayoungafaranha, I got that to work20:37
afaranhaayoung, What did you modified?20:38
*** timcline has quit IRC20:39
ayoungpatience...let me work.20:40
ayoungI'll post it in time, but there is a little trial and error going on20:41
ayoungafaranha, OK...tests are running.  Here's what I did so far.20:43
ayoungI added a key "memeber" into the dictionary in a couple places20:43
ayoungfirst is in the commonn/controller.py code that adds the object to the context.  There other is in the callback method that is used for grants20:43
*** rushiagr is now known as rushiagr_away20:43
afaranhaNice!20:43
ayoungwith that, I can check the domain id on member in a bunch of rules, and collect them together20:44
ayoungI have a rule like this now:20:44
ayoung "domain_matches":"domain_id:%(domain_id)s or domain_id:%(target.member.domain_id)s "20:44
ayoungand then20:44
ayoung "domain_admin_for_grants"                    : "role:admin and rule:domain_matches",20:44
ayoungso I have a bunch that I will reduce to just "domain_admin"20:45
afaranhaayoung, great! Looking forward to see the code20:47
afaranhafeel free to ping me when it's comitted20:48
*** shakamunyi has quit IRC20:53
*** xxj has quit IRC20:55
*** shakamunyi has joined #openstack-keystone20:56
*** ajayaa has quit IRC20:58
openstackgerritBrant Knudson proposed openstack/keystone: Cleanup to stop server in test_wsgi  https://review.openstack.org/14083520:58
*** xxj has joined #openstack-keystone20:59
*** dims has joined #openstack-keystone21:03
*** xxj has quit IRC21:04
*** dims has quit IRC21:08
*** xxj has joined #openstack-keystone21:08
marekddstanek: so i looked at my scripts today and apparently openstack was at its best so i could confige everything via openstaclient, didn't need to write custom wrappers utilizing ksc.21:10
*** dims has joined #openstack-keystone21:17
openstackgerritBrant Knudson proposed openstack/keystone: Cleanup eventlet use in tests  https://review.openstack.org/14083521:21
ayoungbknudson, so I agree with the changes in keystone/tests/test_wsgi.py21:22
ayoungI think I was responsible for the changes in keystone/tests/test_auth.py  and that strikes an alarm chord21:23
bknudsonayoung: if the tests don't work without the call to "environment.use_eventlet(monkeypatch_thread=False)" then there's something wrong happening.21:24
ayoungdid you run  keystone/tests/test_auth.py alone?21:24
bknudsonayoung: Yes, since that was the comment... let me try it again.21:24
ayoungbknudson, I believe you.  I suspect that some other assumption has changed21:25
bknudsontox -e py27 keystone.tests.test_auth -- worked...21:25
bknudsonis there some other way that you run the tests?21:25
bknudsonmaybe I can do testr in the venv21:25
ayounglooks right21:25
ayoungyou can do21:25
ayoung/opt/stack/keystone/.tox/py27/bin/python    -m nose -s  keystone.tests.test_auth21:26
bknudson"$ .tox/py27/bin/testr run keystone.tests.test_auth" -- failed with an import error "keystone.tests.test_v3_federationNon-zero exit code (2) from test listing."21:26
ayoungbut that should be pretty much the same thing you ran21:26
ayoungI pip installed nose21:27
*** timcline has joined #openstack-keystone21:27
bknudsony, I needed to .tox/py27/bin/pip install nose21:27
*** timcline has quit IRC21:27
bknudson$ .tox/py27/bin/python -m nose -s keystone.tests.test_auth -- "Ran 70 tests in 11.987s"21:28
bknudsonworked for me.21:28
bknudsonthere's a warning about The IN-predicate on "assignment.actor_id" that I still haven't looked into21:28
ayoungyeah...that is a different issue though.  I think you are good21:28
ayoungbknudson, what was happening before is that the environement call was not part of the code path for this test, and was only being set if the server tests were run first21:29
ayoungif you ran this test alone, you got the errors you would expect if neither eventlet nor standard threads were set21:30
bknudsonayoung: I tried running an individual test and that worked, too... keystone.tests.test_auth:AuthWithToken.test_auth_token_project_group_role21:32
ayoungI think you are good21:32
*** timcline has joined #openstack-keystone21:32
*** timcline has quit IRC21:32
bknudsonnot much info in the commit... https://github.com/openstack/keystone/commit/811dfdb5dd9b21754a5bea76304ca62960e9474c ... maybe something was fixed, but it would be interesting to know why it didn't work before.21:34
bknudsonI'll try checking out that commit.21:34
ayoungbknudson, I'm guessing that the  environment.use_eventlet  call for the global tests was moved.21:35
*** afaranha_ has joined #openstack-keystone21:36
bknudsonmaybe there's a test runner that for some reason wasn't doing the test.core imports.21:36
ayounggit show aca10c1621:37
ayoungI changed that21:37
ayounglast January21:37
ayoungthe other change happened.21:37
*** timcline has joined #openstack-keystone21:38
ayoungTue Jan 21 14:45:06 2014 -050021:38
bknudsonrunning with the commit before 811dfd the tests still work.21:39
ayoung Mon Jan 6 21:13:04 2014 -050021:39
bknudsonprobably too many things have changed.21:39
ayoungbut that is when I made the change origianlly, not when it merged21:39
*** timcline has quit IRC21:39
ayoungbknudson, try it with STANDARD_THREADS set21:40
bknudsonayoung: like this? STANDARD_THREADS=1 .tox/py27/bin/python -m nose -s keystone.tests.test_auth21:41
ayoungmaybe it was something along those lines...I can't remember the order that this happened21:41
ayoung1?  True?  I think they are the same21:41
bknudsonmonkeypatch_thread = not os.getenv('STANDARD_THREADS') -- so it's just checking if it's a non-empty string21:44
bknudsontest_auth does "environment.use_eventlet(monkeypatch_thread=False)" whereas core.py does "environment.use_eventlet()" ... I wonder what difference that would make21:45
ayoungprobably an issue with memcached21:46
*** jsavak has quit IRC21:52
*** diegows has joined #openstack-keystone21:55
*** afaranha__ has joined #openstack-keystone21:56
*** afaranha_ has quit IRC21:56
bknudsonlbragstad: did you open a bug for the issue with the messages when running tests? I didn't find one... I think https://review.openstack.org/#/c/140835/ should fix it.22:00
*** jamielennox|away has quit IRC22:08
*** shakamunyi has quit IRC22:08
*** xxj has quit IRC22:09
*** charz has quit IRC22:09
*** serverascode___ has quit IRC22:09
*** jamielennox|away has joined #openstack-keystone22:10
*** jamielennox|away is now known as jamielennox22:10
*** ChanServ sets mode: +v jamielennox22:10
*** shakamunyi has joined #openstack-keystone22:10
*** zigo has quit IRC22:11
*** zigo has joined #openstack-keystone22:12
*** serverascode___ has joined #openstack-keystone22:12
*** charz has joined #openstack-keystone22:12
*** r-daneel has quit IRC22:16
*** andreaf has quit IRC22:17
*** andreaf has joined #openstack-keystone22:17
ayoungafaranha, OK, I have most of them, but I've run into another snag with the creation functions.  Those don't have a `member` so much as you need to check the container (proejct or domain) before creating something inside that container22:21
*** xxj has joined #openstack-keystone22:22
afaranha__ayoung, How is it working? in create project we have the rule checking target.project.domain_id22:30
ayoungafaranha__, those become target.member.domain_id22:30
ayoungits the ones in the request that are like this22:30
ayoung"rule:cloud_admin or (role:admin and project_id:%(scope.project.id)s)"22:30
*** mfisch has quit IRC22:31
ayoungor : "role:admin and domain_id:%(project.domain_id)s"22:31
*** mfisch has joined #openstack-keystone22:32
afaranha__ayoung, where does this scope comes from?22:32
ayoungI don't know.  I think it might be a typo22:32
ayoungbut the other is not22:32
*** mfisch is now known as Guest7648522:32
*** Guest76485 has joined #openstack-keystone22:32
*** Guest76485 is now known as mfisch22:33
*** topol has quit IRC22:34
afaranha__ayoung, Sorry, didn't get it all. If we have something like: project.domain_id we are not able to user target.member.domain_id or just member.domain_id, correct?22:34
afaranha__ayoung, I'll just review a few points, as you are working on that :)22:38
ayoungafaranha__, I'll post another WIP here so you can see where I'm at22:38
afaranha__great!22:41
*** mflobo has quit IRC22:42
ayoungafaranha__, OK,  I think I can go one step further22:44
openstackgerritayoung proposed openstack/keystone: Modify the default policy  https://review.openstack.org/12350922:44
ayoungafaranha__, I think I can get rid of those custom rules by expandind the meaning of domain_matches22:44
*** RichardRaseley has joined #openstack-keystone22:45
*** baker has quit IRC22:48
afaranha__ayoung, Sure, we can reduce even more the rules, there still duplications: admin_on_domain_filter and admin_on_project_filter22:49
ayoungyeah.22:49
afaranha__Can we reduce the rules that checks an id, like this one:22:50
afaranha__domain_id:%(target.member.domain_id)s22:51
afaranha__into:22:51
afaranha__domain_id:%(target.member.id)s22:51
afaranha__?22:51
afaranha__in this case the member is domain, we same also for project22:51
afaranha__ayoung, In the assignment controller we have: ref['member'] = ref['project']22:55
afaranha__don`t we need this for domain and group also?22:56
ayoungnon22:56
ayoungif the member is a project, we need the member.domain_id22:56
ayoungwe might also need to test22:56
ayoung member.project_id22:57
*** _cjones_ has quit IRC23:01
afaranha__ayoung, Right, we also need to check the domain_id on users23:04
*** _cjones_ has joined #openstack-keystone23:04
*** chrisshattuck has quit IRC23:04
*** zz_avozza is now known as avozza23:08
ayoungafaranha__, so we need to check these very closely.  I have a suspicion that it would be easy to fool the "or" checks into incorrectly passing by putting bogus data into a request23:12
ayoungIt might be OK if coupled with JSON home and strict parsing, so that putting, say a bogus group value into a create_project call would  cause it to fail.23:14
*** thedodd has joined #openstack-keystone23:14
ayoungafaranha__, I'm going to repost, but it is still very much WIP23:15
openstackgerritayoung proposed openstack/keystone: Modify the cloud policy  https://review.openstack.org/12350923:16
*** jdandrea has joined #openstack-keystone23:17
openstackgerritayoung proposed openstack/keystone: Modify the cloud policy  https://review.openstack.org/12350923:18
afaranha__ayoung, Let me check, I think I`ll understand better23:18
ayoungafaranha__, I think what I want to do is the same kind of thing that the "member" does.  Maybe called "request_member"  and make it an explicit set in the API.23:19
jdandreaQ: When requesting :5000/v2.0 I see "localhost" used for the rel="self" links, but I can't find where to change that in the keystone.conf. catalog shows the proper URLs though.23:19
ayoungso if you are creating a user you set request_member: user23:19
*** thedodd has quit IRC23:20
ayoungjdandrea, public_endpoitn23:20
ayoungendpoint23:21
jdandreaayoung: The public endpoints look good when I use keystone catalog and keystone endpoint-list. Just not when I go to the :5000/v2.0 URL.23:21
* jdandrea scratches head23:21
*** gordc has quit IRC23:23
afaranha__ayoung, don`t know the effect that it would have and if it`s necessary, let me think23:23
afaranha__ayoung, This is the same issue we have before, so that's the reason we declare rule for domain admin and project admin between each set of operations23:24
*** raildo_ has joined #openstack-keystone23:24
ayoungjdandrea, look at the code that generates it.  I don't think it is hard coded.  Its in keystone/service.py23:25
jdandreaayoung: sure ... checking.23:26
ayoungafaranha__, it might be necessary.  I'll lok tomorrow.  Got to go into family mode now23:26
*** ayoung is now known as ayoung_dad_mode23:26
jdandreaayoung_dad_mode: I don't see localhost in there.23:27
jdandreaayoung_dad_mode: I also bounced keystone just in case. No change.23:27
jdandreaWeird.23:27
jdandreaayoung_dad_mode: Ahhh, but the default public_endpoint DOES use localhost.23:28
jdandreaI bet it's not set in the conf.23:28
jdandreaIt's not, but it's not set on our other OS installs either and it is ok.23:29
jdandrea(on those)23:29
afaranha__ayoung_dad_mode, hehehe, gonna check the review, thanks for the help o/23:29
jdandreaayoung_dad_mode: Ok, I see a difference between common/config.py on one OS install and another. On the one that shows the proper self link, there is no public_endpoint or admin_endpoint URL. On the one that shows localhost, there's public_endpoint and admin_endpoint with the default using localhost.23:32
*** r-daneel has joined #openstack-keystone23:33
*** Tahmina has joined #openstack-keystone23:34
jdandreaayoung_dad_mode: Should I use %(public_port)s in the value or actually put in 5000?23:34
jdandreaAh, use the var. It works now!23:35
openstackgerritMerged openstack/keystonemiddleware: Add a test to ensure the version check error  https://review.openstack.org/13951223:37
jdandreaThx ayoung_dad_mode23:37
*** r-daneel has quit IRC23:38
*** RichardRaseley has quit IRC23:42
openstackgerritJamie Lennox proposed openstack/keystonemiddleware: Fix passing parameters to log message  https://review.openstack.org/14087123:46
*** r-daneel has joined #openstack-keystone23:50
*** tellesnobrega_ has joined #openstack-keystone23:55
*** tellesnobrega_ has quit IRC23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!