Sunday, 2016-01-24

*** ngupta has joined #openstack-keystone00:00
*** dims has quit IRC00:11
*** shoutm has joined #openstack-keystone00:14
*** shoutm has quit IRC00:14
*** ngupta has quit IRC00:23
*** ngupta has joined #openstack-keystone00:23
*** ngupta has quit IRC00:28
*** su_zhang has joined #openstack-keystone00:33
*** dims has joined #openstack-keystone00:39
ayoungsomething just broke.  Running tox against master: keystone.tests.unit.test_token_provider.TestPKIProviderWithStdlib.test_get_token_id_error_handling fails00:39
stevemarayoung: yep00:40
ayounghttp://fpaste.org/314145/45359601/00:40
stevemarayoung: i opened a bug about it this morning https://bugs.launchpad.net/keystone/+bug/153740100:40
openstackLaunchpad bug 1537401 in OpenStack Identity (keystone) "keystone gate failing test_get_token_id_error_handling" [Undecided,New]00:40
stevemareventlet broke it00:40
stevemarayoung: subscribe to keystone bugs :O00:41
ayoungstevemar, I see that...giving it a test now00:41
stevemarayoung: dstanek wrote "Eventlet 0.18.0 was released to PyPI today and seems to be what broke it. When I constrained by environment to installing 0.17.4 things were fine"00:41
ayoungstevemar, I have them going to a subfolder...I was unable to keep up with the flow of email from OpenStack and something had to give00:41
stevemarah00:41
*** topol_ is now known as topol00:43
*** ChanServ sets mode: +v topol00:43
stevemarayoung: fwiw, looks like henry and notmorgan pushed my skip through: https://review.openstack.org/#/c/271723/00:44
stevemarlogging off for a bit00:45
*** browne has quit IRC01:06
*** EinstCrazy has quit IRC01:14
*** EinstCrazy has joined #openstack-keystone01:15
*** EinstCrazy has quit IRC01:18
ayoungyou can tell whenever I've been working in another language.  I got back to using " for all strings01:23
*** shoutm has joined #openstack-keystone01:24
ayoungstevemar, yeah. I saw that, but I think we'll need to use dstanek 's approach of constraining eventlet to the earlier version or we will break people using PKI.01:29
* dstanek glares at eventlet01:29
openstackgerritMerged openstack/keystone: skip test_get_token_id_error_handling to get gate passing  https://review.openstack.org/27172301:38
ayoungdstanek, guess what?  We are getting eventlet out of Tripleo as we speak.  One less place we'll have to deal with, and that is a big one01:39
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Updating sample configuration file  https://review.openstack.org/26947901:40
openstackgerritayoung proposed openstack/keystone: Implied Roles API  https://review.openstack.org/24261401:40
*** EinstCrazy has joined #openstack-keystone01:53
*** agireud has quit IRC01:55
*** agireud has joined #openstack-keystone01:57
notmorganayoung: basically unbreak gate until requirements fix can land02:06
ayoungnotmorgan, yep...works just fine02:07
ayoungand I am not super worried about PKI people chasing top of tree02:07
notmorganhah02:08
notmorgantrue enough02:08
notmorganhad a limequat today. Holy crap tart.02:08
notmorganpainfully so.. like sour candy... but even better02:09
*** EinstCrazy has quit IRC02:14
*** lhcheng has joined #openstack-keystone02:42
*** ChanServ sets mode: +v lhcheng02:42
*** EinstCrazy has joined #openstack-keystone02:45
*** phalmos has joined #openstack-keystone02:47
*** markvoelker has joined #openstack-keystone03:00
*** EinstCrazy has quit IRC03:01
*** EinstCrazy has joined #openstack-keystone03:05
*** markvoelker has quit IRC03:07
*** phalmos has quit IRC03:12
*** lhcheng has quit IRC03:12
*** phalmos has joined #openstack-keystone03:13
*** phalmos has quit IRC03:16
*** EinstCrazy has quit IRC03:29
*** EinstCrazy has joined #openstack-keystone03:32
*** EinstCrazy has quit IRC03:38
*** EinstCrazy has joined #openstack-keystone03:43
*** EinstCrazy has quit IRC03:47
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Updated from global requirements  https://review.openstack.org/27164203:58
*** lhcheng has joined #openstack-keystone04:01
*** ChanServ sets mode: +v lhcheng04:01
*** markvoelker has joined #openstack-keystone04:03
*** markvoelker has quit IRC04:07
openstackgerritMerged openstack/keystone: Remove keystone/common/cache/_memcache_pool.py  https://review.openstack.org/27148504:27
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Updating sample configuration file  https://review.openstack.org/26947904:29
*** dims has quit IRC04:42
*** alex_xu has quit IRC04:54
*** alex_xu has joined #openstack-keystone04:55
*** fawadkhaliq has joined #openstack-keystone04:58
*** markvoelker has joined #openstack-keystone05:04
*** alex_xu has quit IRC05:04
*** alex_xu has joined #openstack-keystone05:05
*** markvoelker has quit IRC05:09
openstackgerritMerged openstack/keystone: Updated from global requirements  https://review.openstack.org/27164206:08
*** fawadkhaliq has quit IRC06:28
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Updating sample configuration file  https://review.openstack.org/26947906:37
*** davechen has joined #openstack-keystone06:44
*** lhcheng has quit IRC06:54
*** markvoelker has joined #openstack-keystone07:05
*** shoutm has quit IRC07:09
*** henrynash has joined #openstack-keystone07:38
*** ChanServ sets mode: +v henrynash07:38
*** markvoelker has quit IRC07:39
*** ajayaa has joined #openstack-keystone07:41
*** rcernin has joined #openstack-keystone08:23
*** su_zhang has quit IRC08:33
*** rcernin_ has joined #openstack-keystone08:36
*** rcernin_ has quit IRC08:36
*** lhcheng has joined #openstack-keystone08:43
*** ChanServ sets mode: +v lhcheng08:43
*** lhcheng has quit IRC08:48
*** hogepodge has quit IRC09:32
*** davechen has left #openstack-keystone10:52
*** markvoelker has joined #openstack-keystone11:36
*** markvoelker has quit IRC11:41
*** fawadkhaliq has joined #openstack-keystone12:03
*** fawadkhaliq has quit IRC12:25
*** alex_xu has quit IRC12:42
*** alex_xu has joined #openstack-keystone12:43
*** EinstCrazy has joined #openstack-keystone13:01
*** dims has joined #openstack-keystone13:11
*** brad[] has quit IRC13:18
*** markvoelker has joined #openstack-keystone13:37
*** markvoelker has quit IRC13:41
*** lhcheng has joined #openstack-keystone14:09
*** ChanServ sets mode: +v lhcheng14:09
*** lhcheng has quit IRC14:14
*** ayoung has quit IRC14:18
*** jasondotstar_ has joined #openstack-keystone14:22
*** jasondotstar has quit IRC14:23
*** jasondotstar_ is now known as jasondotstar14:23
*** rdo has quit IRC14:25
*** rdo has joined #openstack-keystone14:27
openstackgerritBrant Knudson proposed openstack/keystone: keystonemiddleware is not a requirement  https://review.openstack.org/27148014:31
*** rcernin has quit IRC14:40
*** phalmos has joined #openstack-keystone14:47
*** shoutm has joined #openstack-keystone14:49
*** shoutm has quit IRC14:51
*** rcernin has joined #openstack-keystone15:11
*** huats_ has quit IRC15:18
*** phalmos has quit IRC15:24
*** EinstCrazy has quit IRC15:27
*** ayoung has joined #openstack-keystone15:30
*** ChanServ sets mode: +v ayoung15:30
*** markvoelker has joined #openstack-keystone15:37
*** markvoelker has quit IRC15:42
*** su_zhang has joined #openstack-keystone15:44
*** jsavak has joined #openstack-keystone15:52
*** lhcheng has joined #openstack-keystone15:55
*** ChanServ sets mode: +v lhcheng15:55
*** fawadkhaliq has joined #openstack-keystone16:14
*** jsavak has quit IRC16:32
*** jsavak has joined #openstack-keystone16:33
*** EinstCrazy has joined #openstack-keystone16:33
*** jsavak has quit IRC16:37
*** fawadkhaliq has quit IRC16:40
*** fawadkhaliq has joined #openstack-keystone16:40
*** EinstCrazy has quit IRC16:41
*** lhcheng has quit IRC16:43
*** ayoung has quit IRC16:43
*** dims has quit IRC17:05
*** fawadkhaliq has quit IRC17:29
openstackgerrithenry-nash proposed openstack/keystone: Allow project domain_id to be nullable at the manager level  https://review.openstack.org/26453317:38
*** markvoelker has joined #openstack-keystone17:38
*** markvoelker has quit IRC17:43
openstackgerrithenry-nash proposed openstack/keystone: Verify project unique constraints for projects acting as domains  https://review.openstack.org/15837217:43
*** EinstCrazy has joined #openstack-keystone17:56
*** EinstCrazy has quit IRC18:02
*** huats_ has joined #openstack-keystone18:17
*** huats_ has quit IRC18:17
*** huats_ has joined #openstack-keystone18:17
*** phalmos has joined #openstack-keystone18:44
*** phalmos has quit IRC18:59
*** ayoung has joined #openstack-keystone19:01
*** ChanServ sets mode: +v ayoung19:01
ayoungnot sure if the others went through:  jamielennox|away gyee, please look at implie roles api when you have a chance:  https://review.openstack.org/#/c/242614/19:06
*** lhcheng has joined #openstack-keystone19:09
*** ChanServ sets mode: +v lhcheng19:09
*** phalmos has joined #openstack-keystone19:14
*** ajayaa has quit IRC19:18
openstackgerrithenry-nash proposed openstack/keystone: Add tests for role management with v3policy file  https://review.openstack.org/26184619:22
openstackgerrithenry-nash proposed openstack/keystone: Add CRUD support for domain specific roles  https://review.openstack.org/26187019:23
openstackgerrithenry-nash proposed openstack/keystone: Modify rules in the v3 policy sample for domain specifc roles  https://review.openstack.org/26207819:25
openstackgerrithenry-nash proposed openstack/keystone: Modify implied roles to honor domain specific roles  https://review.openstack.org/26306419:26
openstackgerrithenry-nash proposed openstack/keystone: Modify rules for domain specific role assignments  https://review.openstack.org/26354919:27
*** su_zhang has quit IRC19:31
*** rcernin has quit IRC19:37
*** markvoelker has joined #openstack-keystone19:39
*** markvoelker has quit IRC19:43
*** lhcheng has quit IRC19:44
openstackgerrithenry-nash proposed openstack/keystone: Projects acting as domains  https://review.openstack.org/23128919:50
*** henrynash has quit IRC19:55
*** henrynash has joined #openstack-keystone19:59
*** ChanServ sets mode: +v henrynash19:59
*** dims has joined #openstack-keystone20:00
*** dims has quit IRC20:05
*** su_zhang has joined #openstack-keystone20:05
*** henrynash has quit IRC20:06
*** brad[] has joined #openstack-keystone20:09
*** henrynash has joined #openstack-keystone20:09
*** ChanServ sets mode: +v henrynash20:09
*** dims has joined #openstack-keystone20:22
*** simondodsley has joined #openstack-keystone20:23
*** EinstCrazy has joined #openstack-keystone20:38
*** markvoelker has joined #openstack-keystone20:40
*** EinstCrazy has quit IRC20:42
*** markvoelker has quit IRC20:44
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Updated from global requirements  https://review.openstack.org/27183120:49
*** dslev has joined #openstack-keystone20:50
*** dslev_ has joined #openstack-keystone20:50
*** dslev_ has quit IRC20:50
openstackgerrithenry-nash proposed openstack/keystone: Projects acting as domains  https://review.openstack.org/23128921:02
*** su_zhang has quit IRC21:11
*** dims has quit IRC21:17
*** phalmos has quit IRC21:28
*** dims has joined #openstack-keystone21:38
*** rcernin has joined #openstack-keystone22:01
*** amit213 has quit IRC22:09
*** amit213 has joined #openstack-keystone22:10
*** mattt has left #openstack-keystone22:12
*** dims has quit IRC22:13
*** jamielennox|away is now known as jamielennox22:27
*** dims has joined #openstack-keystone22:30
openstackgerritJamie Lennox proposed openstack/keystoneauth: Use positional library instead of our own copy  https://review.openstack.org/26730022:32
jamielennoxnotmorgan: i had ^ from a while ago as well22:34
notmorganjamielennox: heh.22:36
notmorgansure. i was aiming for "use it" then play cleanup22:37
notmorganjust so we didn't make anything grumpy, but for KSA it's not as importnat22:37
notmorganabandoned mine22:38
*** lhcheng has joined #openstack-keystone22:38
*** ChanServ sets mode: +v lhcheng22:38
*** EinstCrazy has joined #openstack-keystone22:39
*** markvoelker has joined #openstack-keystone22:40
*** markvoelker has quit IRC22:45
*** LukeH has joined #openstack-keystone22:47
*** LukeH has quit IRC22:48
*** dims has quit IRC22:50
jamielennoxnotmorgan: yea, for ksc we need to keep it for a while because of auth_token, but ksa i think we just replace it22:55
openstackgerritJamie Lennox proposed openstack/python-keystoneclient: Replace TestResponse with requests_mock  https://review.openstack.org/25874122:57
notmorganright22:57
notmorganwe should also propose the fix to ATM for future / next release22:57
notmorgansince positional is now "in"22:57
notmorganalso, yay we can use it in Keystone22:57
notmorgani want to start making all the cachable interfaces use positional to prevent kwargs22:57
notmorganactually i have an idea for kwarg keygen first22:58
notmorgannow that i think about it22:58
jamielennoxnotmorgan: is there an oslo deprecations thing or is it just debtcollector?23:01
notmorgandebtcollector23:01
notmorganoooor the versionutils thing23:01
notmorganbut debtcollector is the "right" choice iirc23:01
*** rcernin has quit IRC23:01
openstackgerritSteve Martinelli proposed openstack/keystone: Revert "skip test_get_token_id_error_handling to get gate passing"  https://review.openstack.org/27185123:03
notmorganstevemar: i assume this means the requirements pin landed?23:03
jamielennoxnotmorgan: so the versionutils thing was mentioned in one of my reviews, but i don't know it23:04
notmorganwe use versionutils all over23:04
jamielennoxi wasn't sure which was supposed to be used23:04
notmorganit just logs23:04
jamielennoxclient?23:04
notmorgandebtcollector uses warning23:04
notmorgani'd use debtcollector if you can make it work23:04
notmorganbut versionutils is fine imo otherwise23:04
stevemarnotmorgan: no, new eventlet version23:10
notmorganstill will want a pin to blacklist the broken version before we undo that23:12
notmorganfwiw23:12
notmorgans/pin/exclude23:12
*** EinstCrazy has quit IRC23:13
stevemarnotmorgan: sure, i was gonna wait and see if jenkins passes23:13
stevemarnotmorgan: we also need tor release keystonemiddleware with the caching stuff brought back in23:13
stevemarforgot about that, lemme do that23:13
openstackgerrithenry-nash proposed openstack/keystone: Allow project domain_id to be nullable at the manager level  https://review.openstack.org/26453323:16
notmorganjamielennox: so, cache_key generator. i think this is doable [check my logic]: we already have the argspec to the FN, and we can determine default values by doing a reverse ordering of the argspec and lookin at defaults, so deepcopy(argspec.args) then .reverse() it. we then store it, when we get a *args **kwargs passed in we look for the args and kwargs and populate the defaults in our23:16
notmorgankey-generator-list. aphabatize the list and we have a consistent key23:16
notmorganjamielennox: i'm thinking of proposing this to dogpile upstream. so let me draw up pseudo code.23:16
notmorganstevemar: ^ if you want to weigh in too23:16
notmorganstevemar: also i think sdague proposed a blacklist to requirements23:16
notmorgansince it was also affecting nova23:16
jamielennoxnotmorgan: your going to have to backtrack a little and tell me what you're trying to solve23:16
notmorganjamielennox: dogpile does not handle kwarg caching23:17
* jamielennox has never used @MEMOIZE23:18
notmorganjamielennox: because the key_fn_generator can't determine the values since it is a decorator, so *args, **kwargs miss defaults23:18
notmorganalso kwargs are not-order specific23:18
notmorganso.. they just punted on handling kwargs23:18
notmorganhttps://bitbucket.org/zzzeek/dogpile.cache/src/c6913eb143b24b4a886124ff0da5c935ea34e3ac/dogpile/cache/util.py?at=master&fileviewer=file-view-default#util.py-76:8523:18
jamielennoxok, then yes you can absolutely use inspect to get the default values of a function, sort the dict and use that as part of your key23:19
notmorganthey already use inspec23:19
notmorgant23:19
jamielennoxthat works so long as you have defaults, it gets risky when taking **kwargs23:19
notmorganbut they still punt because cache keys need to be absolutely consistent23:19
notmorganright. now. since we know the underlying function will explode if it gets unknown values23:20
notmorganwe can just populate a list for non-passed kwargs, and do an alphabatize of the kwargs [or even all args]23:20
notmorganto generate the key23:20
notmorganthat way we don't care about the ordering23:20
notmorganuser can do func(a=1,c=2,b=3) and we'll still end up with func|a|b|c23:21
stevemarnotmorgan: https://review.openstack.org/#/c/271853/23:21
notmorganthe only issue is we break compat with currently populated cache-keys23:21
notmorgani guess we could just do the alpha work if **kwargs is populated23:21
notmorganso.. we only use the new code path that way23:22
notmorganso, func(a, c, b=12) would be func|a|c|b=1223:22
jamielennoxnotmorgan: so i would expect a user to specify which args are important23:22
notmorganbut func(a, c=10, b=12) would be func|a|b=12|c=1023:22
jamielennox(no real knowledge of dogpile)23:23
notmorganugh, we always need to alpha though.23:23
jamielennoxso i would expect so say23:23
jamielennox@MEMOIZE(keys=['a', 'b', 'c'])23:23
notmorganotherwise we have different cache keys between func(a, b, c) and func(a=a, b=b, c=c)23:23
jamielennoxso that you can say that those are the args that i care about23:23
jamielennoxie, d=4 is not important to the cache23:23
notmorganjamielennox: sure, that is in our code. i'm trying to avoid carrying that23:23
notmorgana generic way would be nice.23:23
notmorganand this is pretty deep in the dogpile code fwiw23:24
jamielennoxyou could to keys=dogpile.ALL or something23:24
*** dslev has quit IRC23:24
jamielennoxbut i'm not sure i want too much magic in this23:24
notmorgani could write a simple keygen for oslo.cache that does this but i know this has been a general complaint in dogpile.cache23:24
notmorgan"why can't i handle kwargs"23:24
notmorgani guess i could just make it a new function_key_generator in utils and let people opt into it23:24
jamielennoxwith keys=[] you should be able to23:24
notmorganthen openstack can default23:24
jamielennoxkeys doesn't care if it's arg or kwargs23:25
jamielennoxshouldn't - again i'm coming from theory23:25
notmorgansure, but in 99% of the cases we never want to not capture all args23:25
jamielennoxsure so you can default to an ALL23:25
notmorganbecuase otherwise we have a cachekey that is duplicated between (a=1, b=2, c=3, d=10) and (a=1, b=2, c=3, d=12)23:26
notmorgani'm thinking we don't really have a solid case where that is normal in openstack23:26
jamielennoxagain - not always a problem23:26
jamielennoxlog=False23:26
jamielennoxumm23:26
jamielennoxdebug=XX23:26
jamielennoxanything that doesn't actually impact what is calculated23:26
notmorganthat is pretty far down and often not part of the memoization23:27
notmorganmaybe i'd rather do an @memoize(ignore_keys=[list])23:27
jamielennoxagain, i haven't used it like that, in keystone i trust you did a good job, in auth_token i want/have to use the object directly23:27
jamielennoxnotmorgan: either way works, you can do both if you put some sanity checking in23:28
notmorganyou're not going to @memoize in auth_token23:28
*** rcernin has joined #openstack-keystone23:28
notmorganin almost any case23:28
jamielennoxi can't23:28
notmorganwell there are a couple cases we could23:28
notmorganbut they are all pure calculation stuff that isn't worth it23:28
jamielennoxfetching tokens from keystone23:28
notmorgani mean, we *could* @memoize the whole fetch. and rehydrate the object as though it came from keystone23:29
notmorganbut ... that is something i'd rather move towards down the line if we want rather than start there23:29
notmorgansince invalidation is much much harder with @memoize23:29
jamielennoxnotmorgan: that's what memcache is doing in auth_token now23:29
notmorganright you're using the memcache-like interface23:30
notmorganthe right answer initially23:30
jamielennoxbut i can't use memoize because i don't have a full CONF object to construct the memoize globally23:30
notmorganjamielennox: oh i've solved that before23:30
notmorganwe can lazy config by passing a region into our decorator23:30
notmorganit's a little more expensive and much harder to follow23:31
jamielennoxnotmorgan: i don't particularly want to use memoize, just saying i can't23:31
notmorganbut i don't see a huge win23:31
notmorganbasically we do a lazy region.cache_on_arguments(func)(thing)23:31
notmorganit's ugly23:31
jamielennoxanyway, i'd default to keys=ALL23:31
jamielennoxif you do ignore_keys then inspect and do all but those23:31
notmorganright if ignore_keys=[] then we can just use keys=all23:32
jamielennoxand some sanity checking that keys and ignore_keys aren't both passed23:32
notmorganlike i said i think i have a solution here for generic kwarg handling if so, i'm going to push to make this the default in openstack (oslo.cache)23:32
*** gildub has joined #openstack-keystone23:33
notmorganthe @memoize stuff is very much part of keystone's code base still23:33
notmorganit does some behind the scenes offloading of config etc23:33
*** hogepodge has joined #openstack-keystone23:33
notmorganoslo.cache is a much rawer interface but still not as raw as dogpile.cache directly23:33
*** hogepodge has quit IRC23:38
*** markvoelker has joined #openstack-keystone23:41
*** henrynash has quit IRC23:43
*** markvoelker has quit IRC23:46
openstackgerritMerged openstack/keystonemiddleware: Group common PKI validation code - Tests  https://review.openstack.org/26855623:54
*** dims has joined #openstack-keystone23:54
openstackgerritMerged openstack/keystonemiddleware: Group common PKI validation code - Refactor  https://review.openstack.org/26855723:54
openstackgerritMerged openstack/keystone: Updated from global requirements  https://review.openstack.org/27183123:58
jamielennoxnotmorgan: interesting  https://review.openstack.org/#/c/271357/23:58
notmorgani don't agree with that in the slightest23:59
notmorganbut that is a result of an issue that i am fixing here: https://review.openstack.org/#/c/271536/23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!