Monday, 2014-11-24

bknudsonshould have had the mid-cycle in australia00:03
bknudsonit wasn't on the poll as an option00:03
morganfainbergjamielennox, well crap. it's due to an ancient version of the LDAP lib in OS X00:15
morganfainbergjamielennox, 2.200:15
morganfainbergjamielennox, in fact is it the *same* issue as
morganfainbergwhich means -- we can't install python-ldap in venvs w/o massive amounts of special manual work.00:15
morganfainbergneeds custom include dirs added to setup.cfg00:16
morganfainbergso... i think this is where keystone no longer will run (at least w/ anything LDAP) on OS X00:16
jamielennoxbknudson: i have offered that in the past00:17
morganfainbergjamielennox, i'd be for it, but honestly i think you'd have like 3 or 4 people there total00:17
jamielennoxbknudson: everyone seemed to think it was too far to travel - which feels somewhat ironic :)00:17
morganfainbergdstanek, ^ but in short, that issue has resurfaced but worse with later versions of python-ldap00:18
jamielennoxmorganfainberg: that seems like a bug on there behalf, they should fix that right? if not can't you update the system libs?00:18
morganfainbergdstanek, i would expect LDAP-related tests to become increasingly hard to fix00:18
morganfainbergjamielennox, you can, but it gets harder and harder to maintain.00:19
jamielennoxthough i try not to mess with the system libraries myself, always breaks things unexpectedly later00:19
morganfainbergjamielennox, exactly00:19
jamielennoxyou can build them in home and LD_LIBRARY_PATH (i think - wow it's been a while), but yea it might not be worth it00:20
morganfainbergjamielennox, it's DYLD_LIBRARY_PATH on OS X and it gets really wonky00:20
morganfainberglibrary_dirs = /usr/local/lib00:21
morganfainbergis in the setup.cfg00:21
morganfainbergwell, the fixed one00:21
morganfainbergwhich means i am skeptical that DYLD_LIBRARY_PATH would even work right00:21
morganfainbergthe worst part is the OpenLDAP includes for OS X have been the same since like 2010. never updated - WHAT COULD BE WRONG WITH THAT?!00:22
jamielennoxyep, i guess file the bug with apple (no idea how or if they'll respond) but if they fixed it before then this is a regression, surely there are people on an enterprise login scheme that need a newer ldap librray than that00:22
morganfainbergjamielennox, i don't think they fixed it.00:23
jamielennoxmorganfainberg: right - the other side is to find a security problem in the old library and start exploiting it00:23
dstanekjust upgrade to Linux00:23
morganfainbergi think new versions of python-ldap require libs that are newer than ~2009.00:23
jamielennoxdstanek: i thought you were a mac user as well?00:23
* morganfainberg goes and works to setup vagrant again.00:24
morganfainbergor some such.00:24
dstanekjamielennox: i was, but Yosemite has tilted me towards hating apple00:24
morganfainbergdstanek, jamielennox - oh this is the *same* issue as OpenSSL w/ apple00:24
morganfainbergdstanek, jamielennox, they have their own OpenDirectory implementation00:24
morganfainberg"use that instead"00:24
* morganfainberg facepalms.00:24
jamielennoxapple did there own security libraries? that's hilarious - i never knew that00:25
morganfainbergjamielennox, yeah.00:25
morganfainbergjamielennox, GOTO FAIL - look that one up. the was the SSL bug in their x509 lib this year...00:25
morganfainbergor last year00:25
jamielennoxoh yea, i remember that00:26
morganfainbergjamielennox, part of it is because IOS doesn't have the same framework/libs, they want to make OS X and IOS development similar00:26
morganfainbergunfortunately, Objective-C is the answer there00:26
jamielennoxand openssl and nss are terrible to work with00:27
morganfainbergand OpenSSL doesn't exactly inspire confidence at this point either00:27
dstanekmorganfainberg: Objective-C is never the answer. you must be asking the wrong question :-)00:29
morganfainbergdstanek, or asking Apple.00:29
morganfainbergdstanek, i agree with you. Apple doesn't :P00:29
bknudsonLooks like keystoneclient / keystonemiddleware are broken because of some issue in icehouse tests00:32
*** dimsum__ has joined #openstack-keystone00:35
jamielennoxbknudson: yea, i thought they had fixed it00:35
jamielennoxbknudson: i put a recheck on the most recent failure - i hvaen't seen the result00:36
jamielennoxbknudson: would appreciate reviews on anyway - it's essentially the same as i had you reviewing before but i squashed the subsequent patch into it00:37
jamielennoxbknudson: some of the changes that were being asked for were going to be immediately changed again anyway00:37
bknudsonno real point to reviewing anything if it can't be merged.00:37
jamielennoxbknudson: these patches have been around for weeks - if that bug is the only thing preventing it being merged i'll be overjoyed00:38
boris-42morganfainberg: ping00:54
*** ncoghlan has joined #openstack-keystone01:14
*** lhcheng has quit IRC01:24
*** dimsum__ has quit IRC01:31
*** dimsum__ has joined #openstack-keystone01:31
*** NM has joined #openstack-keystone01:34
*** dimsum__ has quit IRC01:36
*** diegows has quit IRC01:36
openstackgerritMerged openstack/python-keystoneclient: Correct typos in man page
*** stevemar has joined #openstack-keystone01:44
*** ChanServ sets mode: +v stevemar01:44
openstackgerritMerged openstack/keystone: Remove duplicate setup logic in federation tests
*** dimsum__ has joined #openstack-keystone01:54
*** stevemar has quit IRC02:10
*** sigmavirus24_awa is now known as sigmavirus2402:12
*** tellesnobrega_ has joined #openstack-keystone02:20
*** lhcheng has joined #openstack-keystone02:24
*** lhcheng has quit IRC02:28
*** erkules_ has joined #openstack-keystone02:30
*** erkules has quit IRC02:33
*** NM has quit IRC02:42
*** KanagarajM has joined #openstack-keystone03:04
*** fifieldt has joined #openstack-keystone03:19
*** dimsum__ has quit IRC03:27
*** erkules_ is now known as erkules03:30
*** sigmavirus24 is now known as sigmavirus24_awa03:45
*** boris-42 has quit IRC03:57
morganfainbergdstanek, ping - some metaprogramming for you to eyeball03:57
morganfainbergdstanek, here would be some quick testing of it (interactive):03:58
dstanekmorganfainberg: trying to match signatures along with method names?03:59
morganfainbergdstanek, yeah03:59
dstanekdoes that break with decorators?04:00
morganfainbergdstanek, hm. good question04:00
morganfainbergdstanek, i think i need to dig into the decorator underpinnings in that case.04:00
morganfainbergdstanek, this is my first pass of how to handle "is this token provider actually going to work"04:01
morganfainbergthe other thing i was contemplating was using the .register for the ABC for the driver04:01
morganfainbergmake this instead of strict_abstract a special metaclass for the provider...04:01
morganfainbergah .register wont actually force __new__ to be called.04:03
morganfainbergdstanek, yep, fails with decorators atm.04:06
morganfainbergbut it shouldn't if functools.wraps is used.04:13
morganfainbergi think04:13
morganfainberg*this* might be the solution:
jamielennoxmorganfainberg: if you're doing down that route use wrapt04:26
*** dimsum__ has joined #openstack-keystone04:27
morganfainbergjamielennox, same concept, a little less friendly UX, a lot more functionality04:28
jamielennoxmorganfainberg: well i know that he didn't start off to write a decorator module, he wanted something that would exactly proxy from one object to another, so if you have any issues with wrapt and arg parsing or whatever then that's a wrapt bug04:29
*** dimsum__ has quit IRC04:32
jamielennoxalso i love the way wrapt handles optional arguments to decorators04:35
*** aix has quit IRC04:36
*** ajayaa has joined #openstack-keystone04:41
*** boris-42 has joined #openstack-keystone04:42
*** junhongl has quit IRC05:06
*** junhongl has joined #openstack-keystone05:08
*** ncoghlan is now known as ncoghlan_afk05:22
*** stevemar has joined #openstack-keystone05:42
*** ChanServ sets mode: +v stevemar05:42
*** ncoghlan_afk is now known as ncoghlan05:48
*** k4n0 has joined #openstack-keystone06:00
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Imported Translations from Transifex
*** ukalifon1 has joined #openstack-keystone06:33
*** tellesnobrega_ has quit IRC06:37
*** raildo has quit IRC06:46
*** boris-42 has quit IRC06:47
openstackgerritSridhar Gaddam proposed openstack/python-keystoneclient: Curl statements to include globoff for IPv6 URLs
*** afazekas has joined #openstack-keystone07:01
*** raildo has joined #openstack-keystone07:03
*** stevemar has quit IRC07:16
*** bjornar has quit IRC07:35
openstackgerritMarek Denis proposed openstack/keystone-specs: Yet another WebSSO spec
marekd|awaydolphm: and how was you moka?07:37
*** marekd|away is now known as marekd07:37
*** kobtea has joined #openstack-keystone07:43
*** ncoghlan has quit IRC07:59
*** KanagarajM has quit IRC08:19
*** bjornar has joined #openstack-keystone08:22
*** ukalifon1 has quit IRC08:24
*** jistr has joined #openstack-keystone08:55
*** ukalifon has joined #openstack-keystone09:04
*** viktors has joined #openstack-keystone09:06
*** lhcheng has joined #openstack-keystone09:34
*** kobtea has quit IRC09:38
*** lhcheng_ has joined #openstack-keystone09:54
*** dimsum__ has joined #openstack-keystone09:54
*** lhcheng has quit IRC09:56
*** dimsum__ has quit IRC09:59
*** boris-42 has joined #openstack-keystone10:01
*** aix has joined #openstack-keystone10:13
openstackgerritMarek Denis proposed openstack/keystone-specs: Service Provider for K2K
*** f13o has quit IRC10:26
*** f13o has joined #openstack-keystone10:31
openstackgerritMarcos Fermín Lobo proposed openstack/keystone: Split the assignments manager/driver.
openstackgerritMarcos Fermín Lobo proposed openstack/keystone: Implement group related methods for LDAP backend
*** bdossant has joined #openstack-keystone11:01
*** viktors is now known as viktors|afk11:09
*** amakarov_away is now known as amakarov11:13
openstackgerritLin Hua Cheng proposed openstack/keystone: Always return the service name in the catalog
*** NM has joined #openstack-keystone11:33
*** dimsum__ has joined #openstack-keystone12:03
afaranhaHello, I submitted a spec proposing to change the policy, could you read and give comments?
afaranhaHaneef, bknudson, morganfainberg could you review?12:13
openstackgerritBoris Pavlovic proposed openstack/keystone: Test authenticate (DO NOT MERGE)
*** kobtea has joined #openstack-keystone12:39
*** kobtea has quit IRC12:44
ekarlso-jamielennox: you around ? :)12:53
*** k4n0 has quit IRC13:00
openstackgerritAlexander Makarov proposed openstack/keystone: Speed up memcache lock
*** elynn has joined #openstack-keystone13:21
*** kobtea has joined #openstack-keystone13:23
*** jraim has quit IRC13:24
*** serverascode___ has quit IRC13:24
*** zhiyan has quit IRC13:24
*** jraim has joined #openstack-keystone13:25
*** boris-42 has quit IRC13:25
*** boris-42 has joined #openstack-keystone13:26
*** serverascode___ has joined #openstack-keystone13:27
*** zhiyan has joined #openstack-keystone13:27
openstackgerritBrant Knudson proposed openstack/keystone: Correct token flush logging
*** joesavak has joined #openstack-keystone13:35
*** sigmavirus24_awa is now known as sigmavirus2413:38
boris-42bknudson:  hi there13:41
*** thiagop has quit IRC13:41
*** ayoung has joined #openstack-keystone13:41
*** ChanServ sets mode: +v ayoung13:41
*** diegows has joined #openstack-keystone13:46
*** elynn has left #openstack-keystone13:49
*** pc-m has joined #openstack-keystone13:54
*** gordc has joined #openstack-keystone13:54
openstackgerritMarek Denis proposed openstack/keystone: Scope federated token with 'token' identity method
*** dimsum__ has quit IRC14:01
*** dimsum__ has joined #openstack-keystone14:02
*** ukalifon has quit IRC14:06
*** ukalifon2 has joined #openstack-keystone14:06
*** nkinder has quit IRC14:10
*** diegows has quit IRC14:11
*** richm has joined #openstack-keystone14:15
*** richm has quit IRC14:17
*** tellesnobrega has quit IRC14:19
*** tellesnobrega has joined #openstack-keystone14:19
*** lhcheng_ is now known as lhcheng14:24
openstackgerritMarcos Fermín Lobo proposed openstack/keystone: Implement group related methods for LDAP backend
boris-42ayoung: around?14:28
ayoungboris-42, nah, I'm more oblong14:28
ayoungFairly squarish14:28
boris-42ayoung: I am really worried about this stuff
boris-42ayoung: I mean you can't run 50 authentication per second14:29
ayoungboris-42, any idea what the bottleneck is?14:29
boris-42ayoung: as far as I know we are using sql backend in dsvm jobs?14:30
boris-42ayoung: I think that bottlneck is issue14:30
boris-42ayoung:  we were getting in our Mirantis scale lab issues with memchaced backend14:30
boris-42ayoung: without this patch
boris-42ayoung: I need to integrate osprofiler & rally finally, so I'll get traces under load14:31
ayoungboris-42, so the problem is storing tokens to the database?14:31
boris-42ayoung: or some lock* inside keystone backend14:32
ayoungboris-42, then the real issue is getting us off the SQL backend. Which means revocation events14:32
*** bdossant has quit IRC14:33
ayoungboris-42, Here's the deal.  Only the SQL backend survives a reboot.  Which means that only the SQL backend is viable right now14:35
ayoungwith UUID tokens, memcached would be fine14:35
openstackgerritLance Bragstad proposed openstack/keystone-specs: Authenticated Encryption Tokens
ayoungits only PKIZ tokens that require explicit revocation14:35
ayoungUUID tokens, if you dump the memcache backend,  are just all invalidated14:35
ayoungso, for now,  when UUID tokens are enabled, use memcached, and you are OK14:36
ayoungmeanwhile, we are trying to get tokens ephemeral,  PKIZ,  AE, whatever the approach.  In order to do that, we need a persisted store we can trust for revocation events14:36
boris-42ayoung: nope14:38
boris-42ayoung: you are not OK without
boris-42ayoung:  keystone is DDOSed by other projects14:38
boris-42ayoung: this is one of bottlencks14:38
ayoungboris-42, Ah, so the issue is memcached locking as well?14:39
ayoungboris-42, so instead of catastrophic failback, use random....14:39
boris-42ayoung: it's separated issue* But in FUEL we are deploying with memchaced. And testing at middle-scale (200 servers installation)14:39
*** htruta has quit IRC14:40
amakarovayoung, boris-42 hi! Let me explain this: in concurrent environment if memcache lock attempt hits 8 subsequent a collisions then overall waiting time ~108sec => 504 Time out14:40
boris-42ayoung: so we tested this fix in more or less real life situation. But sseems like SQL backend has much lower perf14:40
ayoungamakarov, I get it14:41
ayoungthe failures happen in lockstep14:41
ayoungthe random timeout makes more sense14:41
ayoungreminds me of flow control issues in TCP14:41
boris-42ayoung: but SQL backend is different story. I'll try during Kilo release cycle to get Rally & OSprofiler integrated so we will be able to analyze in more details14:41
ayoungboris-42, thanks.14:42
ayoungboris-42, I'd like to get rid of the SQL backend.  The more justification you can give me for that, the better14:42
boris-42ayoung: ya seems like it's quite useless for any non dev installtion14:42
amakarovayoung, :)14:43
amakarovA question: does it need a spec?14:43
ayounglbragstad, So what do you think of an implementation that merges AE tokens with PKI?  PKI-AE-Z.  Usese the compressed formate instead of JSON for the inner document?14:43
ayoungamakarov, I thought redis was already supported14:43
amakarovayoung, not everywhere14:44
ayoungamakarov, for the token backend it is.  I wouldn't use it for a caching mechanism, but we have other issues there14:44
amakarovayoung, as a cache - yes, as a storage - no14:44
ayoungamakarov, where do you want to use redis?  I thought tokens in redis was a done deal?14:45
amakarovayoung, actually, I haven't find Token persistence backend. Generic kvs only14:46
ayoungamakarov, nah, the Dogpile stuff works for multiple stores.  We inherited redis up front.  ask morganfainberg once he's awake and gotten coffee14:47
amakarovayoung, btw, why trusts don't use kvs?14:47
amakarovayoung, there are sql backend only14:48
ayoungamakarov, trusts need to be persisted.  I guess you would just have invalid trusts without....I didn't implement, but I wouldn't object if someone else did.14:48
ayoungJust time constraint14:48
amakarovayoung, I see, so it's ok if I make a redis backend for trusts too?14:49
*** htruta has joined #openstack-keystone14:49
ayoungreally didn't expect them to be a scalability constraint, but if they are starting to become one, a KVS impl should be trivial14:49
ayoungyes.  I think that makes sense, but make sure they don't time out14:49
ayoungtrusts are inherently long lived14:49
ayoungmemcached style "dump old records if you are full" will not cut it for trusts14:50
*** thedodd has joined #openstack-keystone14:50
amakarovayoung, thanks, and what about that extra LDAP attributes issue? AFAIK we want separate rw LDAP, right?14:51
lbragstadayoung: I'm not sure, I guess I'd have to think about that one for a bit14:51
lbragstadto work everything out14:52
lbragstaddstanek: around?14:52
*** ukalifon2 has quit IRC14:53
*** Ctina has joined #openstack-keystone14:53
amakarovayoung, - it was -1'ed long ago, can we reanimate it somehow? :)14:54
*** nkinder has joined #openstack-keystone14:54
marekddstanek: lbragstad any news on functional testing and federation testsuite in Jenkins ?14:59
lbragstadmarekd: I've been meaning to respond to you note14:59
marekdnote == e-mail?15:00
marekdlbragstad: please, do15:00
lbragstadI apologize that it's taking me a bit15:00
marekdlbragstad: no worries.15:01
*** ukalifon has joined #openstack-keystone15:01
ayoungamakarov, I don;t think bknudson 's concerns have been met.  It needs a rework to avoid breaking existing deployments15:01
marekdjoesavak: i need your opinion on 'alternatives' section in
ayounglbragstad, one of the things you are doing is coming up with a minimal, non-json format for the token data.  That is viable for any transportation mechanism, not just AE15:02
lbragstadayoung: would the entire service catalog be in that format then?15:03
ayounglbragstad, I don't think so.  I think we can find other ways to deal with service catalog, most notably fetching out of band and caching15:03
marekdayoung: +++15:03
marekddstanek: i need some help: this new class is 2 levels down from a class that is decorated with dependency.requires('federation_api') yet all the tests regarding this class complain about missing federation_api . Could you take a look and help me resolving this mystery?15:06
marekddstanek: the console output is here:
marekddstanek: the class 1-level down utilizes federation_api successfully15:07
zhiyanayoung: rodrigods hi folks, hope this one fit all we need on it: :) when/if you ok, pls let me know your thoughts, thanks!15:08
ayoungzhiyan, OK,  so, I was thinking this through over the weekend.  I think maybe we are missing a key factor.  Is it the policy that should vary *per-object* or attributes from the object that are needed to enforce policy?15:10
ayoungI think the changes alone are probably OK, but I am not certain we are actually solving the problem15:11
zhiyanayoung: i'm sure think fix could solving the problem which currently i'm handling for glance adoption.15:11
zhiyanayoung: any thing i missed?15:12
ayoungzhiyan, please answer my question.  I am concerned about multi-access cases, which you will probably not see in your testing15:12
zhiyanayoung: i'm not fully understand your question tbh15:13
ayoungzhiyan, I am not sure I can make it clearer....15:13
zhiyanayoung: actually i have some thougths on this as well during my weedend work15:13
ayoungzhiyan, you shouldn't need to reload rules15:13
ayoungI think that glance is doing it because it has per-object policy15:14
zhiyanayoung: why we don't need reload logic15:14
zhiyanper-object policy? you mean per-image? or something like that15:14
zhiyanayoung: under my understanding, under current design, if policy config files got changed, then next enforce() call should reload these new rules from these changed files.15:15
ayoungzhiyan, yes, per image15:15
zhiyanayoung: no, we haven't pre-image policy15:16
zhiyanwe have some in-fly policy which built on code, but when glance get started then rules are not changed15:17
ayoungzhiyan, If I understand what is going on, there are layers of policies here.  There is the policy.json file, and  something else that dynamically creates policy.  Where does that dynamic policy come from?15:17
zhiyanwhen an image get request in, these rules are applied on that image15:17
zhiyanimage are changing by each request, but rules are static (for each req)15:17
ayoungexplain to me how the " in-fly policy which built on code" get built15:17
zhiyanayoung: 1 sec, let me show you some code15:18
zhiyan & L15915:18
zhiyanayoung: ^15:18
openstackgerritAlexander Makarov proposed openstack/keystone: LDAP additional attribute mappings description
zhiyanayoung: that code build these dynamic policies15:19
marekdcan i ask for some reviews: ?15:19
ayoungzhiyan, I think it would make more sense to reapply those rules after a reload.15:19
ayoungIf you reload the file from disk,  dump all the old rules, then reapply the file based ones, and the code ones afterwards15:20
zhiyanayoung: we could applied these before or after, but i think from policy module's POV, it would be better to support both15:20
zhiyanayoung: and imo, currently glance domain design is good to me15:21
ayoungzhiyan, maybe.  I'd also have to think what kind of conflict resolution we should have if the policy.json and the auto-rules  don't agree.15:21
ayoungI'll give you review another look in a bit.  Got to step away for a moment.15:22
*** ayoung is now known as ayoung-afk15:22
*** r-daneel has joined #openstack-keystone15:22
*** thedodd has quit IRC15:22
*** aix has quit IRC15:22
zhiyanayoung-afk: ok, looking forward to see, thanks again15:22
*** stevemar has joined #openstack-keystone15:25
*** ChanServ sets mode: +v stevemar15:25
*** Ctina has quit IRC15:26
*** Ctina has joined #openstack-keystone15:27
*** joesavak has quit IRC15:30
*** dimsum__ is now known as dims15:32
*** NM has quit IRC15:37
*** NM has joined #openstack-keystone15:40
dstanekmarekd: still having that issue?15:41
dstanekmarekd: ah, you need an empty __init__ or the deps won't get injected15:42
marekddstanek: i will try15:43
marekddstanek: how was not happening in classes 1-level down ?15:43
bknudsonwhich review?15:43
marekdbknudson: 1 sec15:43
dstanekmarekd: what class does it work for?15:44
bknudsonI've got some fixes to the tests that might fix this...15:45
bknudsonmarekd: See ( dstanek )15:46
bknudsonI got rid of all the @dependency.requires in the tests.15:46
bknudsontests shouldn't be using dependency.requires.15:47
marekddstanek: FederatedTokenTests15:47
marekdline 77015:47
dstanekbknudson: ++, DI is runtine and not test time15:47
bknudsonmarekd: I bet if you rebase on it will work.15:47
marekdbknudson: i will do so15:47
*** ukalifon has quit IRC15:48
*** ayoung-afk is now known as ayoung15:53
*** zzzeek has joined #openstack-keystone15:56
*** kobtea has quit IRC16:10
*** kobtea has joined #openstack-keystone16:10
*** chrisshattuck has joined #openstack-keystone16:12
*** kobtea has quit IRC16:15
morganfainbergdstanek: so we would need to move to wrapt or decorator to use that sig validating metaclass16:21
morganfainbergOtherwise it does what I'm looking for. I have an alternative, that would error at runtime instead of at compile time as well.16:22
dstanekmorganfainberg: we could get rid of decorators!16:35
morganfainbergdstanek: where we can't I like the idea of using something like wrapt16:36
dstanekwrapt is interesting, but i've never used it16:37
bknudsonany other openstack projs using wrapt?16:38
bknudsonmust be, it's in global-requirements16:38
morganfainbergSo if we do something like this, that is enforce the method signature for certain plugin interfaces... Do we want at compile time or runtime?16:38
morganfainbergRuntime is a bit less spooky / metaclass Magic.16:39
morganfainbergRuntime = object instantistion.16:40
morganfainbergbknudson: I think we should probably look at wrapt in general for our decorators if it's already in global reqs.16:40
*** NM has quit IRC16:41
*** david-lyle_afk is now known as david-lyle16:41
*** _cjones_ has joined #openstack-keystone16:48
*** afazekas has quit IRC16:54
*** NM has joined #openstack-keystone16:55
afaranhaHello, I submitted a spec proposing to change the policy, could you read and give comments?
afaranhaHaneef, bknudson, morganfainberg, ayoung could you review?16:56
openstackgerritBrant Knudson proposed openstack/keystone: Fix tests using extension drivers
openstackgerritBrant Knudson proposed openstack/keystone: Avoid multiple instances for a provider
openstackgerritBrant Knudson proposed openstack/keystone: Add missing translation marker for depenency
bknudsonI'd never heard of wrapt before today and now it's everywhere: -- "recheck wrapt failure should be resolved"16:59
openstackgerritBrant Knudson proposed openstack/keystone: Add missing translation marker for dependency
openstackgerritBrant Knudson proposed openstack/keystone: Fix tests using extension drivers
openstackgerritBrant Knudson proposed openstack/keystone: Avoid multiple instances for a provider
*** kobtea has joined #openstack-keystone17:12
stevemarbknudson, oh took me a while to figure out that what you linked in wasn't what was merged17:12
*** spligak has quit IRC17:13
bknudsonstevemar: dolphm didn't like it17:13
*** radez_g0n3 is now known as radez17:13
bknudsonI thought it was super clever17:13
stevemardstanek, bknudson so lack of id's in the templated catalog - server's problem or clients/consumers should be smarter?17:13
bknudsonstevemar: what does the spec say?17:14
stevemarbknudson, making id the be region_name-service_name would address dstanek's concern17:14
dstanekstevemar: bknudson: i think it's a server problem;17:14
stevemarbknudson, doesn't say anything specifically about templated17:14
bknudsonthey <region_name>-<service_name> might not be unique either17:14
stevemarregion_id would be17:15
bknudsoncan I put the ID in the template?17:15
dstanekcan we force people to add IDs in the template?17:15
bknudsonget out of my brain.17:15
dstanekget out of mine!17:16
stevemarwe're all a part of the collective now anyway17:16
*** edmondsw has joined #openstack-keystone17:16
stevemarso ID's in templates works, that actually how are test data is17:16
*** kobtea has quit IRC17:16
stevemarbut the one we advertise does not contain that ...
bknudsonthat's not the endpoint id?17:17
bknudsonthat's the service id17:18
stevemarit's the service id17:18
ayoung"Dear Glance,  What are you doing.  Sincerly, Adam Young."   and
stevemarbknudson, i think the lack of a service id is still an issue17:18
ayoungzhiyan, why are the two patterns:  [^x_.*]    and [.*]17:19
bknudsonmaybe we could support , too?17:19
ayoungIt looks to me like you are doing regular expressions inside Oslo config headers.  "That might be the worst thing I've ever heard.  How marvelous."17:20
ayoungorder of processing matters, too17:20
ayoung[.*]   actually will match anything matched by [^x_.*]17:21
ayoungIf I read that correctly, Glance is using Python code to read in regular expresssions from an oslo-config managed file that has regular expressions as the section headers.  It then generates policy to protect individual properties on glance images.17:22
afaranhaayoung: As I understood if the rules match [^x_.*] it will stop without looking the [.*]17:23
afaranhaI still didn't understand this feature though17:23
ayoungafaranha, I think they got lucky17:23
ayoungafaranha, it only works because of ASCII ording17:24
ayoungquick,  which has a lower ascii values  .  or ^    ?17:24
afaranhaI think ^17:24
ayoungafaranha, I would not bet against you17:25
ayoungBut then, I don't take bets that I think I am not going to win17:25
afaranhaI don't know, I just searching now :P17:25
*** joesavak has joined #openstack-keystone17:25
afaranhaayoung: .17:25
ayoung. =4617:25
ayoung^  = 9417:26
ayoungafaranha, then maybe order from the file is maintained17:26
ayoungI thought it was put into a dictionary.   Which means it might be the hash value of the string that determines the ordering17:26
openstackgerritAlexander Makarov proposed openstack/keystone: LDAP additional attribute mappings description
afaranhaayoung: So, If I put [.*] first and [^x_.*] next, then this order will not be respected17:27
stevemarbknudson, i think that would be the simplest fix17:27
ayoungafaranha, I'll say "possibly"17:27
ayoungsince I don;t really know17:27
ayoungwhat they are looking for, IIUC, is for the .* rule to be the deafult17:27
stevemarbknudson, how would that affect existing deployments?17:27
afaranhaIf it's a hash, then it could be17:27
ayoungand  [^x_.*]  an explicit match17:28
afaranhaayoung: Let me check the implementation17:28
ayoungafaranha, and let me think about how this should actually work....17:28
ayoungafaranha, the default rule in glance is also defined outside that file17:28
ayoung"default": "",17:29
ayoung"context_is_admin": "role:admin",17:29
stevemarbknudson, dstanek if we doc it, and tell folks to add id's to their templatedcatalog, they can edit the file, then the next time they restart keystone it'll have the correct data?17:29
ayoungwhich means that for any policy points that are not defined, no RBAC or other attributes get checked17:29
afaranhabut the rules in the property-protections-policies.conf.sample doesn't cover this?17:30
ayoungafaranha, not that it references is_admin and defaul17:30
afaranhaayoung: If I have this file with the rule [.*] it will only apply to those rules not specified on the policy, or for all the rules?17:31
ayoungafaranha, I have no idea17:31
*** ajayaa has quit IRC17:31
ayoungafaranha, this is beyond what normal policy does.  I suspect it does not get applied17:31
afaranhaDo you have the spec, or blueprint for this? I didn't get the idea yet17:31
ayoungafaranha, Let me pull it up17:32
*** dims has quit IRC17:35
afaranhaSo, they want to define rules outside the policy. Say that I want a user to be able to read all information of the cloud without updating it17:35
*** dims has joined #openstack-keystone17:35
*** NM has quit IRC17:38
afaranhaayoung: Oh, it's from 2013. I don't think a good idea to have enforcement information in 2 different places, they might have a good reason for this17:38
*** NM has joined #openstack-keystone17:39
*** dims has quit IRC17:40
*** carlosmarin has joined #openstack-keystone17:42
*** mikedillion has joined #openstack-keystone17:44
amakarovayoung, about trust redelegation, can you please review the spec? I need it to get redelegation accepted and continue working on it17:44
*** DavidHu has quit IRC17:46
*** dims has joined #openstack-keystone17:48
*** ajayaa has joined #openstack-keystone17:50
*** jorge_munoz has joined #openstack-keystone17:51
*** harlowja_away is now known as harlowja17:52
*** mikedillion has quit IRC17:54
morganfainberglbragstad, yay! XML No-More!17:55
sigmavirus24xml-off, your industrial strength xml remover and json polishing agent17:57
morganfainbergamakarov, ayoung, is there a reason to make the redelgation count configurable?17:58
morganfainbergamakarov, ayoung, i'm thinking we should just say "can be re-delegated" and it does redelgation_count = --current_count17:59
morganfainbergamakarov, ayoung, the only reason for a max-depth of redelegation is to help limit the expensive lookups.17:59
*** wwriverrat has joined #openstack-keystone18:00
*** wwriverrat has left #openstack-keystone18:01
dstanekstevemar: will those IDs come out in the right place?18:02
ayoungmorganfainberg, it needs to be there18:03
ayoungI want to delegate to solum, which delegates to heat, which delegates to Nova18:03
amakarovmorganfainberg, I can imagine the case: I want to restrict redelegation depth to, say, 3 instead of default 5 in fear that my trust may be redelegated to somebody I don't know and I want him to act on my behalf, but further redelegation is not necessary18:03
ayoungNova should not be able to delegate further18:03
morganfainbergi'm looking at the current UX and it's kindof... bad.18:04
ayoungafaranha, they match property names18:04
ayoungso the rule ^x_.*  matches all propertues that start with x_18:04
amakarovayoung provided more clear case )18:04
morganfainbergayoung, i think the issue here is you can't know solum will delegate to heat then nova and not need to then delegate to cinder18:05
ayoungamakarov, at a quick look it appears correct18:05
ayoungmorganfainberg, it is optional, but limiting the number of redelegations will help us avoid a class of attacks18:06
morganfainbergi'm getting the feeling that this is either a "you can redelegate" or you "can't"18:06
morganfainbergayoung, my issue is the UX is weird. especially that in the middle of a chain i could in theory reduce the amount of redelegation18:07
amakarovmorganfainberg, ++18:07
morganfainbergso 5 -> 4 -> 1 -> not-redelegatable18:07
amakarovmorganfainberg, the question is: who makes the decision?18:08
morganfainbergamakarov, i would think the initial delegator says "this can be redelegated, yes/no" and then that has a max depth - which is to limit DOS-type attacks based on expensive lookups18:08
morganfainbergat anypoint you can choose to disable re-delegation18:09
morganfainbergbut it's not that you change the depth18:09
amakarovmorganfainberg, yes, and this choice is made by unknown app18:09
stevemardstanek, i believe so, for service name they will at least18:10
stevemarservice id*18:10
morganfainbergamakarov, you obviously have trusted the app somewhat because you've handed off a delegation to it already (or at least to something that trusts it)18:10
stevemarhave to check the code18:10
morganfainbergamakarov, and let that $something$ redelegate18:10
amakarovmorganfainberg, on the other hand, he who trusts may explicitely restrict the depth18:10
morganfainbergamakarov, maybe this value is only settable at the first redelegation point?18:11
morganfainbergbut i still think that UX is a bit weird.18:11
ayoungso long as the amount of redelgation cannot be increased, I am prone to support this.  Saying "yeah, I know you said it could be redeletated 10 times, but I know that I need to hand it to Nova nad then we are done"  is OK18:11
morganfainbergayoung , ++ never increase18:11
ayoungWhy not just say "I am consuming more than one redelegations..."18:11
amakarovmorganfainberg, it begins to look like some kind of policy...18:12
morganfainbergayoung, the only reason i advocated for the max-depth (global) was to circumvent DOS attacks based on an unbounded/expensive lookup18:12
amakarovayoung, ++18:12
morganfainbergsame concept as a max recursion depth.18:13
morganfainbergamakarov, ayoung, so a cleaner UX [and tell me if this then matches] is you can toggle the boolean at any point in the chain preventing further redelegation. but you can't ever set/change the depth value.18:14
ayoungmorganfainberg, so do you really have a problem with the spec as written?  So long as they redelegation count can't increase, I can't see how this is a bad feature18:14
amakarovmorganfainberg, for now one can do both18:14
ayoungmorganfainberg, I think that is just the degenerate case18:14
morganfainbergayoung, i'm worried about adding in superfluous things that make the UX weird or add surface area for potential issues18:14
ayoungit keeps you from saying " I can delegate to you, but you can't delegate further"18:15
morganfainbergespecially because once we mint the API as stable we're stuck with it18:15
morganfainbergayoung, my only concern is the changing depth on the chain, changing the boolean at anypoint is totally valid (boolean from true -> false that is)18:15
ayoungI'd go with the more flexible API in this case18:15
morganfainbergayoung, we could always add that bit back in if there is a real case for it (reducing max delegation at any point)18:16
amakarovmorganfainberg, you can toggle redelegation off and never back on18:16
morganfainbergayoung, i'd rather be conservative in the API unless we really have a case for it.18:17
ayoungmorganfainberg, we have use cases for it18:17
morganfainbergamakarov, ++ yes. and that should stay.18:17
morganfainbergayoung, please elaborate18:17
ayoungmorganfainberg, when a user creates a trust, they redelegate to solum18:17
ayoungsolum uses that trust to talk to several services18:17
morganfainbergayoung, i'm not sold that handing off to solum and having to know solum will only delegate 2 more times down the chain.18:17
morganfainbergayoung, you're *already* trusting solum to do what it needs to do.18:18
morganfainbergif you don't trust solum to redelegate twice, or four times, you don't trust it once.18:18
ayoungunless we make the user define different trusts for each redelegation, solum needs to provide the exact same redelegation to heat and to nova18:18
ayoungo...user delegates to solum. solum delegates to two or more services18:18
morganfainbergbecause it could already do bad things™ with one redelegation18:18
ayoungone gets redeleagation=2, one gets it =118:19
morganfainbergayoung, i think this is over-engineering it.18:19
morganfainberguser wont know how many times solum needs to redelegate... only that it does.18:19
ayoungmorganfainberg, David Chadwick brought this up back when I was proposing trusts...redelgation and counts.  I deferred for just this reason, and now solum needs the redelegation.  Lets engineer it right this time18:20
ayoungBut...ask Chadwick.  I would argue that he has the perspective on this one.18:20
morganfainbergayoung, the only bit of it i'm against is mid-stream change of the max-redelegation.18:20
morganfainbergall the rest makes perfect sense.18:21
morganfainbergi would *probably* argue that we don't need to change the max-redelegation at anypoint. but i'm willing to leave that18:21
morganfainbergi don't see why Nova would change the max delegation depth18:22
morganfainbergin a chain of user -> solum -> heat -> nova -> glance18:22
morganfainbergit *could* turn off redelegation if it doesn't trust the app for redelegation, that is a boolean and should be able to be turned off18:22
morganfainbergayoung, so, proposed change to the SPEC: don't allow mid-chain max-depth change (reduction)18:23
morganfainbergonce a max depth is set, you're stuck with it.18:23
morganfainbergbut you can halt further redelegation via the boolean.18:24
amakarovmorganfainberg, so if user just trusts solum for depth 10, and solum knows there shouldn't be anything behind glance, what will it do?18:25
morganfainbergamakarov, how does solum know that18:25
morganfainbergand how does solum know that wont change?18:25
morganfainbergonce you hand off to heat solum wont always know - heck heat may not know nova needs to redelegate to glance. heat is asking nova to do things. nova will either say "can't do this without redelegation or can"18:26
lbragstadmorganfainberg: yeah, I'm curious to see what the outcome of that thread will be18:27
*** richm has joined #openstack-keystone18:27
openstackgerritAndrey Pavlov proposed openstack/keystone: Handle SSL termination proxies for version list
morganfainbergbut it's one of those cases you're trusting an service to do the right thing - either you trust it or you don't. If you trust it, you're trusting it's trusted apps. [they could just push the token around and do bad things™ as well)18:27
*** viscious is now known as vishy18:29
morganfainbergvishy, welcome back to non-friday nic day!18:29
amakarovmorganfainberg, so we can leave max depth to be "stop-loss" limit and forgen about it in API calls?18:29
openstackgerritgordon chung proposed openstack/keystonemiddleware: Adding audit middleware to keystonemiddleware
*** NM has quit IRC18:30
morganfainbergamakarov, thats my general view. but I'm not opposed to the initial trust being able to set a max limit (lower than global), I just don't see a benefit for changing that limit mid-chain artifically18:30
amakarovmorganfainberg, 1 less validation :)18:32
lbragstadmorganfainberg: or probably, when the XML tempest stuff will be removed :)18:33
openstackgerritgordon chung proposed openstack/keystonemiddleware: Adding audit middleware to keystonemiddleware
amakarovmorganfainberg, and what harm can this feature bring?18:34
*** NM has joined #openstack-keystone18:35
ayoungmorganfainberg, I think you are picking at nits.  I really don't think it is an issue.  But if you care so strongly,  I can abide.18:36
*** diegows has joined #openstack-keystone18:37
ayoungIt means that the call for an initial trust creation is different than a redelegation.  I'd argue it is the same degree of complication either way.18:37
openstackgerritgordon chung proposed openstack/keystonemiddleware: documentation for audit middleware
*** aix has joined #openstack-keystone18:39
*** _cjones_ has quit IRC18:40
afaranhaayoung: Could you read this spec about the policy?
ayoungafaranha, sure18:42
ayoungafaranha, So you've seen the chain of policy related work I've been putting forth, right?18:43
afaranhaayoung: Due to some reviews on the sample policy patch, we prefer to modify the current one to this new18:43
ayoungafaranha, for example, the unified policy file, etc?18:43
afaranhaayoung: Its better to maintain18:43
afaranhaayoung: No, I didn't18:43
ayoungafaranha, go to that link, and look at the set of related specs18:44
*** ajayaa has quit IRC18:44
ayoungafaranha, also:
*** NM has quit IRC18:44
*** amcrn has joined #openstack-keystone18:44
ayoungyep, that one too18:44
*** NM has joined #openstack-keystone18:45
ayoungafaranha, the idea is that we need a progressive approach on policy.  I'm not certain if yours is essential. In a vacuum, yes, it would be a good idea18:45
ayoungI think a better one would be to focus at least on the unified policy file18:45
ayoungthe old one is a known quantity18:45
ayoungwe leave it in place, and do the work to mitigate the effects of going to a dynamic policy  approach18:46
*** jaosorior has joined #openstack-keystone18:46
afaranhaayoung: At the spec that I read, I'm don't know how this will work18:46
ayoungso... yeah, the unified policy should pull in the improvements that you  want for the base policy file18:46
ayoungafaranha, which spec?18:47
afaranhaayoung: We have a single file in Keystone, and for every service, we change the rules at that file, tight?18:47
*** _cjones_ has joined #openstack-keystone18:47
ayounglook through the series of specs18:47
ayoungwe make it possible for the services to fetch policy from Keystone, and then they all get the same default file18:47
ayoungwhich has a deconflicted set of rules18:47
afaranhaayoung: Sure18:47
ayoungnow, yeah, we could replace etc/policy.json  but then we break everyone depending on that (admittedly poor) approach18:48
afaranhaI think it's great for common rules, like context_is_admin, user_or_owner, service, etc18:48
afaranhaayoung: But, we already discuss this here, and there's the problem that the context of the services aren't the same18:48
afaranhafor example, we don't have domain_id in the context of all the services18:49
ayoungso work with samuelms and Henrique rodrigods etc to divvy up the work from the set of specs, and rip them apart, and tell me where things are messed up, and lets unify our efforts18:49
afaranhaSure :)18:50
ayoungafaranha, ah,  god point.  so we say that projects *are* domains.  Hierarchical multitenancy actually gives us some leeway there18:50
*** NM has quit IRC18:53
*** jorge_munoz has quit IRC18:55
morganfainbergayoung, i'm trying to keep the surface area of our API spec as sane as possible / limit what we're having to carry forward. if there isn't a good case for it i don't want to carry it.18:56
morganfainbergespecially if it's a weird UX.18:56
*** NM has joined #openstack-keystone18:56
ayoungmorganfainberg, I understand that.  Just that "enable"  and "set non 0"  are the same thing18:56
morganfainbergayoung, so lets pick one or the other, not both. and "enable" is the better UX18:57
ayoungmorganfainberg, the initial was not true or false.  It was 0  means fals,   >0  means redelgate that many times,  < 0 means infiniate times18:58
afaranhaayoung, THIS topic number 118:58
afaranhaayoung: We need it to fiz the security issue we are having in the policy we are providing18:58
ayoungafaranha, assume that I never have the context of what you are talking about, and you will not be far off18:58
ayoungafaranha, I wish that were possible18:58
ayoungafaranha, we can't toy with the default policy without breakin a lot of things18:59
ayoungthe cloud sample was Henrynash's first take on an improved solution, but we found it doesn't yet work due , as you pointed outk to domain scoped tokens etc18:59
ayoungso we really need HMT before we can improve policy19:00
afaranhaayoung: In a domain context, we want to create an admin that would also be responsible for role assignment on projects19:00
ayoungafaranha, I get it.  We just can't do that today19:01
ayoungit breaks horizon19:01
ayoungand...the real question is who assigns what roles...but that is further down my list19:01
afaranhaAs domain is the container of users and projects, the domain admin should be responsible for assignment, agree?19:02
*** NM has quit IRC19:04
*** afazekas has joined #openstack-keystone19:05
* ayoung just got the joke about nose.run19:08
ayoungafaranha, look at the hierarchical role spec19:08
*** thiagop has joined #openstack-keystone19:09
*** afazekas has quit IRC19:10
stevemarayoung, bknudson is it not advisable to dynamically switch from an SQL backend to an LDAP one? like... just changing the keystone.conf file and restarting? Would anything else break?19:12
*** packet has joined #openstack-keystone19:12
*** NM has joined #openstack-keystone19:12
ayoungstevemar, role assignments would be all wrong19:12
ayoungwe see that all the time19:12
ayoungnone of the LDAP users have role assignments19:12
stevemarayoung, yeah thats what I was not hoping for...19:12
stevemarbut figure19:13
bknudsonstevemar: I manually switched a small devstack from SQL to LDAP for a while but it wasn't easy19:13
bknudsonthis was before devstack supported LDAP19:13
ayoungstevemar, sorry, I keep my magic wand in my back pocket, and I sat on it.  Its in the shop.  Miracles will have to wait until I get it fixed.19:13
stevemarbknudson, yeah but I want to use a different LDAP than the one used by devstack19:14
ayoungthat is why we want the multi backend storage approch:  leave your service users in SQL, new assignments come in for users in LDAP19:14
* bknudson wonders if stevemar has been smoking something19:14
bknudsonstevemar: y, that's what the multi-domain backend is for.19:15
*** htruta has quit IRC19:15
*** gyee_ has joined #openstack-keystone19:15
stevemarbknudson, i meant something other than
bknudsonstevemar: you don't need that specific ldif but your directory will need to have places for users and groups.19:17
*** gothicmindfood has quit IRC19:19
*** htruta has joined #openstack-keystone19:20
amakarovmorganfainberg, ayoung, so what do I do about redelegation depth? Just forbid modifying by anyone but trustor?19:21
morganfainbergamakarov, sec will be back [had a phone call and am chasing a bug down w/ nova guys]19:21
*** gothicmindfood has joined #openstack-keystone19:22
*** dougwig has joined #openstack-keystone19:27
*** Haneef has quit IRC19:29
dougwighi keystone.  my third-party CI has started spewing this in a tight loop, to the tune of 2GB of compressed logs per run, in the last 2-3 days.  ring any bells?  "[Sun Aug 24 21:38:25.027602 2014] [:info] [pid 11760:tid 140155513620352] mod_wsgi (pid=11760): Python has shutdown."19:29
*** jistr has quit IRC19:31
morganfainbergdougwig, i have not seen that before.19:34
dougwigmorganfainberg: thanks, i'll dig into it.19:34
*** carlosmarin has quit IRC19:34
amakarovdougwig, I'd start with log format: it looks quite specific19:35
uvirtbotLaunchpad bug 1394995 in keystone "Test allow creation of service without type" [Undecided,New]19:35
morganfainberglbragstad, also... related19:35
*** diegows has quit IRC19:35
lbragstadlol sweet19:35
morganfainbergi think we're closing that as wont fix19:35
* morganfainberg reads.19:35
dstanekdougwig: are you seeing that same pid in all of the messages?19:35
lbragstadI feel like this was opened *everywhere*19:35
morganfainberglbragstad, ok this is the *same* bug19:35
dougwigamakarov: it looks like mod_wsgi is recycling the interpreter in a tight loop for some reason.  only appears in the keystone log file.19:35
dstanekmorganfainberg: do you have any experience using the infra yaml stuff to run the devstack tests in a test environment?19:36
morganfainberglbragstad, mark that as a dupe of the one you're fixing with validation19:36
dstanekmorganfainberg: i have some changes i want to test that setup devstack for functional tests19:36
morganfainbergdstanek, >.> yes. somewhat >> should i deny having that knowledge?19:36
morganfainbergdstanek, so you're looking at devstack-gate changes?19:36
morganfainbergdstanek, or what do you need to change. and i can probably point you at the place to do it19:37
morganfainbergdougwig, i've seen that happen if there is a crash in python19:37
morganfainbergdougwig, mod_wsgi keeps trying to spawn.19:37
lbragstadthis is a dup of
uvirtbotLaunchpad bug 1394995 in keystone "Test allow creation of service without type" [Undecided,New]19:37
morganfainberglike compile-time crash19:37
morganfainberglbragstad, yeah19:38
dstanekmorganfainberg: i know the place in openstack-infra/project-config/tree/jenkins/jobs to make the change to create a functional just template, but i don't know how to test the changes19:38
morganfainbergdstanek, ah. you make the change, mark the new "job" as expirimental19:38
morganfainbergthen you run [i think] "check expirimental" on a patchset19:38
morganfainbergit'll run all expirimental jobs.19:39
morganfainbergwork on fixing things that are broken, once it is stable we can move it to "non-vote" or "voting"19:39
morganfainbergbroken could be broken in devstack, devstack-gate, or keystone19:39
dstanekmorganfainberg: ah, so i can create a patchset and make it run those jobs without a commit?19:39
morganfainbergdstanek, you will also likely need to edit the zuul.yaml file19:39
amakarovdougwig, try to start it manually (keystone-all)19:40
morganfainbergdstanek, or pick any patchset ;)19:40
morganfainbergonce infra has merged the new jobs19:40
dstanekmorganfainberg: but before i run 'check experimental' my experimental job has to be merged?19:41
amakarovdougwig, I guess if there is a general problem then you'll see something more informative. Also try --debug key19:41
dougwigit looks like somehow debugging logging got enabled.  was that an accidental commit maybe?19:42
morganfainbergbknudson, not sure what we need to do about this.19:42
uvirtbotLaunchpad bug 1393920 in keystone "I18n" [Undecided,New]19:42
morganfainbergbknudson, will chase it down, i18n stuff doesn't map anywhere for middleware apparantly19:43
dstanekthat's a cool bug19:44
rodrigodsmarekd, nice, will review it as soon as I have some time19:44
bknudsonmorganfainberg: document that the client now supports I18n.19:44
morganfainbergbknudson, there is something wrong in the infra config19:45
bknudsonI mean the auth_token middleware19:45
morganfainbergbknudson, sure, i meant the error19:46
openstackgerritMarek Denis proposed openstack/keystone-specs: Service Provider for K2K
*** carlosmarin has joined #openstack-keystone19:47
dougwignot a keystone issue, something enabled debug.  i'll wager devstack or one of the ci repos.  i'll check there.  thank you!19:47
bknudsonmorganfainberg: what error?19:47
amakarovmorganfainberg, do we have a conclusion about trust API change?19:48
bknudsondougwig: keystone has always had debug enabled in tests.19:48
afaranhaayoung, do you know about other services whats the current status of recognizing keystone v3?19:51
ayoungafaranha, issue aside from Horizon is in keystonemiddleware.auth_token.  jamielennox has a series of patches that need review/merging to support19:51
*** diegows has joined #openstack-keystone19:51
*** Ctina_ has joined #openstack-keystone19:52
amakarovmorganfainberg, an idea: we can go with "allow_redelegation" flag and max depth in config thus totally excluding redelegation count from API19:52
amakarovmorganfainberg, and add it later if needed19:52
*** Ctina has quit IRC19:55
morganfainbergamakarov, i'm thinking we go simple and expand as needed19:56
morganfainbergamakarov, so yeah i'd support that change. and then it's easy to expand on it - we're not exluding the capability down the line19:57
morganfainbergjust not out the gate.19:57
dougwigbknudson: thanks.  *something* changed about 3 days ago that enabled lots of useless logs.  will look further.19:57
*** jorge_munoz has joined #openstack-keystone19:58
amakarovmorganfainberg, so we have: max depth in config, allow_redelegation in API?19:58
morganfainbergamakarov, ok so, let me see19:58
morganfainbergwe have 2 options:19:58
morganfainberg1) boolean "allow_redelegation"19:59
*** htruta has quit IRC19:59
morganfainbergor 2) "max depth"[whatever it is called], where 0 = no redelegate, > 0 = that many levels, and [future] -1 could mean unlimited19:59
morganfainberg#2 is more flexible, and not a bad UX. as long as the max depth isn't "changed" by a user when redelegating20:00
morganfainbergamakarov, the first option is a bit more straight forward.20:00
morganfainbergthe only issue with #2 is if a user sets max depth to > global max depth, does it just set ot he global or 400?20:01
morganfainbergand how does a user know the max depth20:01
amakarovmorganfainberg, for now it's an error20:02
morganfainbergamakarov, then how does a user know the max depth?20:02
morganfainbergthey have to try and fail.20:02
*** joesavak has quit IRC20:02
amakarovmorganfainberg, no, there are 2 sources20:02
amakarov1) config itself20:02
morganfainbergassume the user can't read the config20:02
*** joesavak has joined #openstack-keystone20:02
amakarov2) if user redelegates, it uses trust containing remaining depth20:03
morganfainbergbut the user can't know what the max is when creating the original trust20:03
morganfainbergso it's a guessing game20:03
amakarovmorganfainberg, and if user omits the depth, the default is used20:03
morganfainbergok so here i think are the right changes:20:04
amakaroveach redelegation decrements remaining depth by default20:04
amakarovI handle that case when user passes depth as uncommon20:05
morganfainbergso 1st off, i think a user should opt into redelegation vs opt-out20:05
morganfainbergtrusts should *not* be redelegatable by default20:06
amakarovthey are not20:06
morganfainbergno change in our current default behavior20:06
morganfainbergyou said if redelegate depth is not passed it is set to max?20:06
morganfainbergor it *also* requires the boolean20:06
amakarovone must explicitly set the flag and may even forget about depth20:07
openstackgerritLance Bragstad proposed openstack/keystone: Require `name` when creating Services
morganfainbergamakarov, so i think the fix is minor20:07
lbragstadmorganfainberg: ^ service['name'] required patch20:07
morganfainbergno one can change the max depth later in the chain20:07
morganfainbergeverything else can remain the same.20:08
morganfainbergso a redelegate you're stuck with the chain length you started with.20:08
amakarovmorganfainberg, so trustor don't set it too?20:08
morganfainbergamakarov, so initial trust creation sets it.20:08
morganfainbergamakarov, if it's a redelegatable trust. after that, it can't be changed in subsequent redelegations20:09
morganfainbergeverything else stays the same20:09
amakarovmorganfainberg, we may avoid that special case at all since we have default from config20:09
amakarovtrustor don't need to specify it20:09
morganfainbergamakarov, right, but to ayoung's point they want to limit redelegation. so it can be specified, but not need to be specified20:09
morganfainbergnot specified = use global20:09
morganfainbergif specified it must be <= global20:10
amakarovmorganfainberg, understood. I'll do it tomorrow.20:10
morganfainbergso we're just avoiding the special case of "if you are redelegating a trust, you can't change the depth"20:10
bknudsonis the number of roles in the token a limit to redelegation?20:10
afaranhaayoung: There's a lot of efforts on these blueprint20:10
morganfainbergminor change, but it keeps the UX cleaner.20:10
ayoungafaranha, which is why I don't want more effort than necessary.20:11
morganfainbergbknudson, right now, you would need to use a trust to limit roles in the token20:11
afaranhaayoung,  I didn't understand what's the improvement from usgint he policy for this bp
morganfainbergbknudson, redelegated or not.20:11
amakarovmorganfainberg, this way trustor ability to set it becomes the special case )20:11
morganfainbergamakarov, ++20:11
morganfainberglbragstad, you could just mark the bugs in LP as duplicates20:12
morganfainberglbragstad, as well vs. needing to do related-bug20:12
*** gyee_ has quit IRC20:12
lbragstadmorganfainberg: the ones I marked as related are the dup ones I believe. just double checking that is the one that we want to track everything with.20:13
uvirtbotLaunchpad bug 1259425 in keystone "service-create allows 2 services with the same name" [Medium,In progress]20:13
morganfainberglbragstad, yeah20:13
lbragstadsweet, I'll remove20:13
lbragstadthe Related tags20:13
openstackgerritLance Bragstad proposed openstack/keystone: Require `name` when creating Services
amakarovmorganfainberg, 1 more: please see HA fix. It's tiny but proved necessary:
morganfainberglbragstad, we also need a SQL migration for that20:15
morganfainberglbragstad, or are you doing that as separate?20:15
*** thedodd has joined #openstack-keystone20:16
*** amakarov is now known as amakarov_away20:16
lbragstadI'll do that as a dep of the one above ^20:16
morganfainberglbragstad, ok -1 since it doesn't close the bug (partial-bug)20:16
morganfainbergthe dep should closes-bug20:17
openstackgerritLance Bragstad proposed openstack/keystone: Require `name` when creating Services
morganfainbergand likely we need KSC / OSC updated to be smarter.20:17
morganfainbergbut that is different than this bug.20:17
*** kobtea has joined #openstack-keystone20:18
*** htruta has joined #openstack-keystone20:18
lbragstadmorganfainberg: just double checking, but I can use for adding a name column to the service table20:21
lbragstadand it shouldn't be nullable and should be unique20:22
morganfainberglbragstad, only if it's a backport20:22
morganfainberglbragstad, don't use the placeholders otherwise20:22
morganfainberglbragstad, if it's a backport the real migration comes after placeholders and is idempotent, then the backport is added to the placeholder in master *then* backported to /stable20:22
lbragstadso does that mean I start at 6120:22
morganfainbergyeah start at the next non-placeholder20:22
*** kobtea has quit IRC20:23
*** gyee_ has joined #openstack-keystone20:25
*** carlosmarin has quit IRC20:28
*** carlosmarin has joined #openstack-keystone20:43
*** NM has quit IRC20:47
dolphmbknudson: the one sound that no one knows, what does the bot say?
uvirtbotLaunchpad bug 1393920 in keystonemiddleware "I18n" [Medium,Confirmed]20:50
bknudsonfoxes make little yipping noises like little dogs.20:52
lbragstadankle biters?20:58
* lbragstad got bit by a little dog running the other day... 21:00
bknudsonnot running fast enough21:00
lbragstadapparently not21:00
stevemarlbragstad just got burned by bknudson21:12
* lbragstad starts shopping on Amazon for aloe vera21:13
stevemari think we can just add keys/values there21:14
stevemari'll try it out soon21:15
*** Ctina_ has quit IRC21:18
*** carlosmarin has quit IRC21:19
*** carlosmarin has joined #openstack-keystone21:20
ayoungdstanek, how do I run unit tests from the command line in the debugger?  I can't do python -m pdb  since I need to do pythom -m nose ....21:23
dstanekayoung: if you are using nose then 'python -m nose --pdb' should work21:24
*** htruta_ has joined #openstack-keystone21:24
ayoungdstanek, trying21:24
dstanekayoung: we also have a debug target in tox that uses testr21:24
ayoungdstanek, I had other issues with that.21:25
ayoungtrying to limit it to one degree of frustration at a time21:25
lbragstadayoung: you can also use
stevemarnot sure how that guy works with unit tests, never tried it before with that21:25
ayounglbragstad, since the whole set of unit tests take so long I want to run just a single named test21:25
ayoungsick of reinventing how I do this21:26
lbragstadayoung: you can match the tests you want to run with a regex21:26
lbragstadand it generates a list of tests to run21:26
dstaneklbragstad: yeah, but that's stupid :-)21:26
dstaneklbragstad: with nose i can tell it what to run and it runs it21:26
lbragstaddstanek: has better solutions I'm sure21:27
lbragstadI just happen to stumble across that one day and I've held on to it since because it worked :)21:27
dstaneklbragstad: i can see the value of testr for CI style testing, but i run my tests on just about every save and it's too slow for that21:28
lbragstadlike ayoung, I was frustrated beyond belief and need to debug a test21:28
lbragstaddstanek: so how do you do it?21:28
dstaneklbragstad: i use nose21:29
ayoungseems to be having a problem setting breakpoints21:30
ayoungdstanek, when I run nose that way, it gets through the failing test and then hits a break point in the test_runner21:32
dstanekayoung: it doesn't stop on your breakpoint?21:32
ayoungthis is about as simple a test as can be. Does not inherit from any of the Oslo?keystonetest classes21:32
ayoungdstanek, how do I set one?  If I do it using import pdb; pdb.set_trace()  it just hangs21:33
ayoungrunnning -m pdb will stop before main21:33
ayoungbut this way...yeah, debugger is running, but it is kinda useless21:33
ayoungI'm running like this21:33
ayoung/opt/stack/keystone/.tox/py27/bin/python -m nose  --pdb keystone.tests.test_access_info21:34
ayoungclass AccessTestCase(testtools.TestCase)21:34
ayoungshould be straight forward...21:35
dstaneklet me give it a try21:36
dstanekayoung: what patch are you testing?21:37
ayoungdstanek, nothing I've submitted yet21:38
ayoungI can post, but it looks like the issue is how I am running pdb, no?21:38
*** topol has joined #openstack-keystone21:39
*** ChanServ sets mode: +v topol21:39
ayoungdstanek,   --pdb                 Drop into debugger on errors21:41
ayoungI need to be in the debugger up front21:41
dstanekayoung: lol, just looked at my vim config and i use -s --pdb21:43
dstanekthe -s tells nose to string stdout21:43
dstaneki actually use 'nosetests -s --pdb {args}'21:43
*** dnalezyt has joined #openstack-keystone21:44
ayoungdstanek, is there a more intelligent way to do this?  It should not be that hard:  run python  in debug mode and kick off a test runner...anytest runner, that honors the debugger21:45
ayoungSo I can step through my code;21:46
*** zzzeek has quit IRC21:46
dstanekayoung: python -m nose -s --pdb - to run through nose21:47
dstanekayoung: the link lbragstad posted does the same in testr21:47
ayoungdstanek, that givesme no way to set a breakpoint, just dumps to the debugger on failure21:47
dstanektestr is more complicated because it is multiprocess21:47
dstanekimport pdb; pdb.set_trace() drops you into the debugger21:48
lbragstadin the link I posted, you can set "traditional" breakpoints21:48
lbragstadwhat dstanek said21:48
*** carlosmarin has quit IRC21:49
ayoungdstanek, not for is hanging.  Maybe due to the venv python?21:50
dstanekayoung: did you add the -s?21:50
ayoungdstanek, ah...missed getting the right combination of args... ok think that worked21:51
dstanekayoung: try running 'nosetests -s --pdb keystone.tests.test_accessinfo'21:51
dstanekayoung: k21:51
ayoungdstanek, it was21:51
dstanekayoung: now create a shell alias!21:51
ayoung/opt/stack/keystone/.tox/py27/bin/python -m nose -s  --pdb keystone.tests.test_access_info21:52
ayoungdstanek, well, I am actually doing it from inside of emacs now21:52
dstanekayoung: ah...i was just going to tell you to add it to your vim snippets :-)21:52
ayoungso I can see what line the debugger stops on without havving to list every time21:52
dstaneklbragstad: messing with this gate stuff is crazy21:55
lbragstaddstanek: yeah, makes you appreciate the infra guys...21:56
lbragstadthat stuff makes my head hurt..21:56
*** packet has quit IRC21:57
lbragstaddstanek: are you making progress with it?21:58
*** zzzeek has joined #openstack-keystone21:59
dstaneklbragstad: yes, i'm trying to get a barebones federation setup going22:02
*** harlowja is now known as harlowja_away22:05
*** jsavak has joined #openstack-keystone22:06
*** mikedillion has joined #openstack-keystone22:06
*** joesavak has quit IRC22:09
*** harlowja_away is now known as harlowja22:16
*** jsavak has quit IRC22:17
*** dims has quit IRC22:25
*** dims has joined #openstack-keystone22:26
*** dims has quit IRC22:26
*** dims_ has joined #openstack-keystone22:27
*** dims_ has quit IRC22:28
*** _cjones_ has quit IRC22:33
stevemardstanek, whatcha trying to progress on?22:35
dstanekstevemar: playing with functional testin22:36
*** _cjones_ has joined #openstack-keystone22:36
stevemardstanek, functional tests and federation stuff? now we're cookin with fire22:37
dstanekstevemar: yessir, fun stuff22:37
stevemardstanek, i was going to write a spec about that, you have an etherpad or something for ideas?22:37
*** dims has joined #openstack-keystone22:39
lbragstadmorganfainberg: if isn't going to be nullable, what should it default to?22:40
morganfainbergRequired not null?22:41
morganfainbergDon't think we can default it.22:41
lbragstadso don't make it nullable?22:41
morganfainbergI mean we could.22:41
morganfainbergunique constraint would still work iirc22:42
lbragstadso, I think if we make it non-nullable we should default to an ''22:42
*** afazekas has joined #openstack-keystone22:42
lbragstadbut, that case would fast fail in the jsonschema validator22:42
morganfainbergIt could still be null. I guess so we don't break anyone.22:43
*** diegows has quit IRC22:43
lbragstadmorganfainberg: ok, running testing against that22:44
morganfainberg(Null, type-not-null) should be a fine unique constraint.22:44
morganfainbergI think.22:44
lbragstadyeah, type is still required and can't be null22:44
*** junhongl has quit IRC22:46
*** afazekas has quit IRC22:47
*** junhongl has joined #openstack-keystone22:55
*** diegows has joined #openstack-keystone23:00
*** topol has quit IRC23:01
*** jaosorior has quit IRC23:03
*** nkinder has quit IRC23:06
openstackgerritLance Bragstad proposed openstack/keystone: Migration script for adding name to service table
*** lhcheng has quit IRC23:10
*** ncoghlan has joined #openstack-keystone23:14
jamielennoxmorganfainberg, lbragstad: i don't think we need to make name a unique value23:20
jamielennoxi think if it's just a new column with either a default empty string or Null value it's fine23:22
*** edmondsw has quit IRC23:23
morganfainbergjamielennox, name + type = unique constraint23:28
jamielennoxmorganfainberg: type == unique23:29
jamielennoxpretty much the catalog will fail otherwise23:30
jamielennoxthere was a bug about this opened not long ago23:30
morganfainbergjamielennox, so you can never have type duplicated ever?23:30
*** thedodd has quit IRC23:31
jamielennoxtrying to find the bug23:32
jamielennoxbut i can say that OpenStack today just wont work if you do, and i don't see the point in adding the ability23:33
jamielennoxdamnit i can't find it23:35
morganfainbergi'm fine with it if that is the real limitation23:35
jamielennoxbut essentially if you have multiple service_types the way the catalog searches if it can't find a suitable endpoint in the service_type it will fail early23:35
morganfainbergok so type unique and nothing else unique23:35
jamielennoxso if you have two 'compute' types it will only ever check one23:35
morganfainbergwell not name23:36
morganfainbergmaybe in the future we can expand that constraint if we have multiple types23:36
morganfainbergerm support for multiples of a type23:36
jamielennoxi can't see a reason to allow multiple types - isn't that what the endpoints should be filtered on?23:36
jamielennoxif we had a semi-sane catalog i would have made the service_type the dictionary key23:37
jamielennoxso it doesn't work now - and i'm not sure if it's something we should try to make work23:37
jamielennoxmorganfainberg: might have been
uvirtbotLaunchpad bug 1377080 in python-keystoneclient "Stale endpoint selection logic in keystone client" [Wishlist,Invalid]23:39
morganfainberglbragstad, ^23:39
morganfainbergall makes sense to me23:39
morganfainbergname in this case seems superfluous23:39
jamielennoxi would agree with keystone putting a unique constraint on type23:40
jamielennoxmorganfainberg: ++23:40
jamielennoxi'm pretty sure this stuff was never designed, just stuff was thrown in as people thought it might be useful23:40
*** jamielennox has left #openstack-keystone23:40
*** jamielennox has joined #openstack-keystone23:40
*** ChanServ sets mode: +v jamielennox23:40
morganfainbergjamielennox, haha23:43
jamielennoxmorganfainberg: if it comes to it name + type unique will work i just don't see the point23:47
*** oomichi has joined #openstack-keystone23:48
*** kobtea has joined #openstack-keystone23:55
jamielennoxi have found the common factor in all these gate issues that people think are transitive.... me23:59

Generated by 2.14.0 by Marius Gedminas - find it at!