Wednesday, 2016-08-03

*** code-R has quit IRC00:00
*** thumpba has joined #openstack-keystone00:04
*** code-R_ has quit IRC00:05
*** code-R has joined #openstack-keystone00:05
*** ravelar159 has joined #openstack-keystone00:09
*** thumpba has quit IRC00:12
*** ravelar159 has quit IRC00:14
*** ravelar159 has joined #openstack-keystone00:19
*** pgbridge has quit IRC00:20
*** browne has quit IRC00:28
*** jdennis has quit IRC00:29
*** ravelar159 has quit IRC00:30
*** jdennis has joined #openstack-keystone00:32
*** ravelar159 has joined #openstack-keystone00:42
*** ravelar159 has quit IRC00:49
*** aastha has quit IRC00:49
*** ravelar159 has joined #openstack-keystone00:49
*** richm has quit IRC00:50
*** code-R has quit IRC00:54
*** ravelar159 has quit IRC00:54
*** spzala has quit IRC00:56
*** sdake has quit IRC00:57
*** sdake has joined #openstack-keystone00:59
*** code-R has joined #openstack-keystone01:02
stevemarbreton: ahh i miss those hours01:04
*** tqtran has quit IRC01:23
*** spzala has joined #openstack-keystone01:24
*** browne has joined #openstack-keystone01:25
*** adu has joined #openstack-keystone01:25
*** spzala has quit IRC01:27
*** spzala has joined #openstack-keystone01:27
*** davechen has joined #openstack-keystone01:38
*** ravelar159 has joined #openstack-keystone01:40
*** sdake has quit IRC01:44
*** EinstCrazy has joined #openstack-keystone01:44
*** browne has quit IRC01:44
stevemarlbragstad: is bug 1607553 a dupe of bug 1590779 ?01:47
openstackbug 1607553 in OpenStack Identity (keystone) "Revocation event caching is broken across processes" [High,New] https://launchpad.net/bugs/160755301:47
openstackbug 1590779 in OpenStack Identity (keystone) "Cache region invalidation works for local CacheRegion object only" [High,In progress] https://launchpad.net/bugs/1590779 - Assigned to Boris Bobrov (bbobrov)01:47
*** ravelar159 has quit IRC01:47
*** itisha has quit IRC01:50
openstackgerritSteve Martinelli proposed openstack/keystone: Move Assertion API to its own file  https://review.openstack.org/35015801:58
openstackgerritDolph Mathews proposed openstack/keystone: Add basic upgrade documentation  https://review.openstack.org/35034102:12
openstackgerritDolph Mathews proposed openstack/keystone-specs: Fix the name of the "manage-migration" spec  https://review.openstack.org/35034302:13
dolphmstevemar: yes, i think so02:14
*** ravelar159 has joined #openstack-keystone02:14
stevemardolphm: yeah, pretty sure too02:14
stevemardolphm: i'll mark it as such02:14
stevemardolphm: you missed out on a career as a writer02:16
dolphmstevemar: who says i'm missing out02:17
*** ravelar159 has quit IRC02:19
*** code-R has quit IRC02:36
*** marekd2 has joined #openstack-keystone02:37
*** adrian_otto has joined #openstack-keystone02:39
*** markvoelker has joined #openstack-keystone02:39
*** dave-mccowan has joined #openstack-keystone02:41
*** marekd2 has quit IRC02:42
openstackgerritColleen Murphy proposed openstack/keystone: Skip middleware request processing for admin token  https://review.openstack.org/34449602:47
openstackgerritMerged openstack/keystone: Don't include openstack/common in flake8 exclude list  https://review.openstack.org/34998802:56
*** adu has quit IRC02:56
*** adu has joined #openstack-keystone02:58
stevemarcrinkle: i was just gonna fix that comment for ya03:02
*** code-R has joined #openstack-keystone03:03
crinklestevemar: the self.assertNotIn ?03:03
stevemarcrinkle: aye03:04
* crinkle too fast for stevemar03:04
stevemarcrinkle: had it in my env, ran the test and walked away03:04
crinkleha03:04
stevemarcrinkle: i'll take your speed any day!03:04
*** dave-mccowan has quit IRC03:08
*** code-R has quit IRC03:14
*** code-R has joined #openstack-keystone03:14
openstackgerritMerged openstack/keystone: Adds test for SecurityError's translation behavior  https://review.openstack.org/34808503:15
openstackgerritMerged openstack/keystone: Adds a custom deepcopy handler  https://review.openstack.org/34808603:15
*** adu has quit IRC03:18
*** adrian_otto has quit IRC03:18
*** adu has joined #openstack-keystone03:21
*** adu has quit IRC03:22
*** Nakato has quit IRC03:27
*** Nakato has joined #openstack-keystone03:32
*** roxanaghe has joined #openstack-keystone03:35
*** spzala has quit IRC03:39
*** code-R has quit IRC03:41
*** code-R has joined #openstack-keystone03:41
*** ayoung has quit IRC03:48
*** sdake has joined #openstack-keystone03:50
*** sdake_ has joined #openstack-keystone03:54
*** markvoelker has quit IRC03:55
*** sdake has quit IRC03:58
*** sdake_ has quit IRC04:05
*** links has joined #openstack-keystone04:10
*** davechen has quit IRC04:13
jamielennoxah, never look at our auth pipeline04:17
stevemarjamielennox: never read the comments04:24
stevemarjamielennox: you can take a crack at https://bugs.launchpad.net/keystone/+bug/1600393 if you want :P04:25
openstackLaunchpad bug 1600393 in OpenStack Identity (keystone) "AttributeError: 'list' object has no attribute 'items'" [Critical,Confirmed]04:25
jamielennoxstevemar: do we have any further clues?04:25
stevemarmfisch said he would get us middleware logs and his config tomorrow04:25
stevemarbut not so much04:26
stevemarwe have confirmation from mfisch that hes hitting this in his mitaka dev cloud and it's terrrrribad, its causing things to choke04:26
openstackgerritSteve Martinelli proposed openstack/keystone: Improve domain configuration API docs  https://review.openstack.org/34859104:29
*** markvoelker has joined #openstack-keystone04:33
openstackgerritMerged openstack/python-keystoneclient: Fix other-requirements.txt for deb based distros  https://review.openstack.org/34100704:35
*** markvoelker has quit IRC04:39
*** dikonoor has joined #openstack-keystone04:45
jamielennoxstevemar: i'm looking at how to do the long running token thing and i don't think i can get it done for newton04:52
jamielennoxstevemar: i'm away next week, and this is uglier than i expected04:52
openstackgerritMerged openstack/keystone: Move Mapping API to its own file  https://review.openstack.org/35011704:57
openstackgerritMerged openstack/keystone: Move Service Provider API to its own file  https://review.openstack.org/35012804:57
*** julim has quit IRC05:02
*** jaosorior has joined #openstack-keystone05:03
openstackgerritMerged openstack/keystone: Move List Projects and Domains API to its own file  https://review.openstack.org/35014305:03
openstackgerritMerged openstack/keystone: Move Federation Auth API to its own file  https://review.openstack.org/35015105:03
openstackgerritMerged openstack/keystone: Test number of queries on list_users  https://review.openstack.org/34796705:03
*** Guest53941 has joined #openstack-keystone05:04
*** roxanaghe has quit IRC05:05
*** nisha_ has joined #openstack-keystone05:25
*** markvoelker has joined #openstack-keystone05:28
*** Guest53941 has quit IRC05:28
*** rcernin has joined #openstack-keystone05:28
*** lamt_ has quit IRC05:31
*** markvoelker has quit IRC05:33
*** spzala has joined #openstack-keystone05:40
*** spzala has quit IRC05:44
openstackgerritMerged openstack/keystone: PCI-DSS Password history requirements  https://review.openstack.org/32833905:46
*** julim has joined #openstack-keystone05:57
*** sdake has joined #openstack-keystone06:07
*** sdake has quit IRC06:07
*** sdake has joined #openstack-keystone06:07
*** code-R has quit IRC06:11
*** code-R has joined #openstack-keystone06:12
*** tqtran has joined #openstack-keystone06:21
*** markvoelker has joined #openstack-keystone06:22
*** sdake has quit IRC06:23
*** nishaYadav has joined #openstack-keystone06:24
*** nishaYadav is now known as Guest4397806:24
*** tqtran has quit IRC06:25
*** nisha_ has quit IRC06:26
*** markvoelker has quit IRC06:26
*** code-R has quit IRC06:32
*** code-R has joined #openstack-keystone06:32
*** belmoreira has joined #openstack-keystone06:42
*** tesseract- has joined #openstack-keystone06:45
*** sorrison has quit IRC06:53
openstackgerritMerged openstack/keystone-specs: Fix the name of the "manage-migration" spec  https://review.openstack.org/35034306:54
*** tangchen_ has quit IRC07:00
*** brancal has joined #openstack-keystone07:14
*** jaosorior is now known as jaosorior_brb07:15
*** markvoelker has joined #openstack-keystone07:16
*** jpena|off is now known as jpena07:16
*** code-R has quit IRC07:20
*** pcaruana has joined #openstack-keystone07:21
*** markvoelker has quit IRC07:21
*** links has quit IRC07:23
*** dikonoor has quit IRC07:23
*** code-R has joined #openstack-keystone07:28
*** ekarlso has quit IRC07:29
*** links has joined #openstack-keystone07:36
*** ekarlso has joined #openstack-keystone07:37
*** dikonoor has joined #openstack-keystone07:37
*** brancal has quit IRC07:43
*** brancal has joined #openstack-keystone07:45
*** Guest43978 has quit IRC07:50
openstackgerritMerged openstack/keystone: Use %()d for integer substitution  https://review.openstack.org/35006907:52
*** marekd2 has joined #openstack-keystone07:53
*** daemontool has joined #openstack-keystone07:57
*** jistr is now known as jistr|training07:59
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-keystone08:02
openstackgerritDavanum Srinivas (dims) proposed openstack/keystone: [WIP] Testing latest u-c  https://review.openstack.org/31843508:10
*** markvoelker has joined #openstack-keystone08:10
*** jaosorior_brb has quit IRC08:12
*** jaosorior_brb has joined #openstack-keystone08:13
*** markvoelker has quit IRC08:14
*** code-R has quit IRC08:22
*** jaosorior_brb is now known as jaosorior08:23
*** tqtran has joined #openstack-keystone08:23
*** brancal has quit IRC08:25
*** danpawlik has joined #openstack-keystone08:27
*** tqtran has quit IRC08:27
*** brancal has joined #openstack-keystone08:28
*** pnavarro has joined #openstack-keystone08:37
*** gb21 has joined #openstack-keystone08:40
*** gb21 has quit IRC08:48
*** gb21 has joined #openstack-keystone08:48
*** jistr|training is now known as jistr08:49
*** code-R has joined #openstack-keystone08:54
*** code-R has quit IRC08:59
*** amitkqed has quit IRC09:04
*** markvoelker has joined #openstack-keystone09:04
*** amitkqed has joined #openstack-keystone09:04
*** gb21 has quit IRC09:05
*** markvoelker has quit IRC09:09
*** gb21 has joined #openstack-keystone09:10
*** EinstCrazy has quit IRC09:15
*** EinstCrazy has joined #openstack-keystone09:16
*** gb21 has quit IRC09:25
*** code-R has joined #openstack-keystone09:49
openstackgerritMerged openstack/oslo.policy: Allow policy file to not exist  https://review.openstack.org/34173209:49
*** code-R has quit IRC09:53
*** Jaison has joined #openstack-keystone09:58
*** itsuugo has quit IRC09:58
*** markvoelker has joined #openstack-keystone09:58
*** links has quit IRC09:59
*** itsuugo has joined #openstack-keystone10:00
*** markvoelker has quit IRC10:04
*** serverascode has quit IRC10:15
*** serverascode has joined #openstack-keystone10:17
*** gb21 has joined #openstack-keystone10:18
*** alexander__ is now known as amakarov10:19
*** rodrigods has quit IRC10:30
*** rodrigods has joined #openstack-keystone10:30
*** marekd2 has quit IRC10:30
openstackgerritMikhail Nikolaenko proposed openstack/keystone: Retry revocation on MySQL deadlock  https://review.openstack.org/34492410:30
*** EinstCrazy has quit IRC10:39
*** EinstCrazy has joined #openstack-keystone10:39
openstackgerritMerged openstack/keystone: Improve domain configuration API docs  https://review.openstack.org/34859110:41
openstackgerritMerged openstack/keystone: Move Assertion API to its own file  https://review.openstack.org/35015810:42
*** code-R has joined #openstack-keystone10:43
openstackgerritMikhail Nikolaenko proposed openstack/keystone: Retry revocation on MySQL deadlock  https://review.openstack.org/34492410:43
*** EinstCrazy has quit IRC10:44
*** code-R has quit IRC10:47
*** Jaison has quit IRC10:49
*** links has joined #openstack-keystone10:53
*** dikonoor has quit IRC10:57
openstackgerritMerged openstack/keystone: Bump API version number and date  https://review.openstack.org/35028911:03
*** gb21 has quit IRC11:17
*** gb21 has joined #openstack-keystone11:23
*** code-R has joined #openstack-keystone11:37
*** code-R has quit IRC11:42
*** markvoelker has joined #openstack-keystone11:52
*** raildo has joined #openstack-keystone12:11
*** adu has joined #openstack-keystone12:11
*** sigmavirus_away is now known as sigmavirus12:14
*** mnikolaenko has joined #openstack-keystone12:17
*** jpena is now known as jpena|lunch12:18
*** gb21 has quit IRC12:19
*** edmondsw has joined #openstack-keystone12:19
*** adu has quit IRC12:23
*** gordc has joined #openstack-keystone12:25
*** spzala has joined #openstack-keystone12:25
*** tqtran has joined #openstack-keystone12:25
*** tqtran has quit IRC12:30
*** daemontool_ has joined #openstack-keystone12:35
*** sdake has joined #openstack-keystone12:37
*** adu has joined #openstack-keystone12:37
*** daemontool has quit IRC12:37
samueldmqoh well, couple of minutes writting a bug description, and launchpad gives me timeout error12:40
samueldmqok12:40
*** julim has quit IRC12:48
openstackgerritSamuel de Medeiros Queiroz proposed openstack/python-keystoneclient: Correct test_implied_roles  https://review.openstack.org/35056212:50
samueldmqdstanek: you around ?12:50
samueldmqor henrynash12:50
samueldmqI'd like someone to check if bug 1609398 sounds sane12:54
openstackbug 1609398 in OpenStack Identity (keystone) "test_implied_roles fails intermittently" [Medium,New] https://launchpad.net/bugs/1609398 - Assigned to Samuel de Medeiros Queiroz (samueldmq)12:54
*** jsavak has joined #openstack-keystone12:56
*** pauloewerton has joined #openstack-keystone13:04
stevemarsamueldmq: any reason that bug isn't against ksc?13:06
samueldmqstevemar: the reason is that I messed up :p13:07
samueldmqit should be13:07
stevemarhenrynash: you still west coatin' ?13:07
stevemarsamueldmq: fixed it for ya :)13:07
samueldmqstevemar: thanks, that's why the patch didn't show up in launchpad automatically13:07
samueldmqstevemar: thanks13:07
stevemarsamueldmq: or as the cool kids on the internet say FTFY13:07
samueldmqstevemar: :D13:08
*** jmlowe1 has joined #openstack-keystone13:08
samueldmqstevemar: does the bug description sound sane ?13:09
*** jmlowe has quit IRC13:10
*** dave-mccowan has joined #openstack-keystone13:10
dstaneksamueldmq: yes, i'm here13:10
*** lamt_ has joined #openstack-keystone13:10
samueldmqdstanek: hey13:10
stevemarsamueldmq: didn't actually look at the bug description, i assumed it was just around the failing test13:11
samueldmqstevemar: yes13:11
samueldmqdstanek: I wanted some eyes on bug 160939813:11
openstackbug 1609398 in python-keystoneclient "test_implied_roles fails intermittently" [Medium,In progress] https://launchpad.net/bugs/1609398 - Assigned to Samuel de Medeiros Queiroz (samueldmq)13:11
samueldmqif the description is sane, the fix is very simple13:12
samueldmqand will unblock nisha so her patches will continue to merge13:12
samueldmqin keystoneclient13:12
samueldmqoutreachy round ends this month, so it's important to keep moving13:13
*** agireud has quit IRC13:14
dstaneksamueldmq: yuck...i always hate it when tests assert counts13:16
*** jpena|lunch is now known as jpena13:16
dstaneksamueldmq: and it looks like it's using a shared user and not it's own13:17
stevemarsamueldmq: dstanek why not assert the interence count <=13:17
samueldmqSpecially when the count assertion is not related to what the test is supposed to test13:17
*** agireud has joined #openstack-keystone13:17
samueldmqdstanek: does that matter ? (shared user)13:18
dstaneki actually don't understand what that test is actually testing13:19
samueldmqthat test isn't really meaningful13:19
*** markvoelker has quit IRC13:19
samueldmqit's just creating role inferences, and testing they were created13:19
samueldmqnot really testing the behavior of them (which is the important part)13:19
samueldmqbut writting more meaningful tests is a separate thing13:19
samueldmq(to role inference rules)13:20
dstanekat first glace it looks like it's just testing fixtures13:20
*** adu has quit IRC13:20
samueldmqthe test was the same before fixtures13:20
samueldmqthere were intermittent failures happening when mixing role creation with direct calls and fixtures (for other tests)13:21
samueldmqthen nisha changed this test to use fixtures as well13:21
samueldmqso for me, a good would be: make that work so we unblock the other work going on in the tests13:22
samueldmqthen fix those tests to make them more meaningful13:22
dstaneksamueldmq: no, i don't think it's a fixtures thing. the tests just look terrible. it's a case of refactoring tests too much IMO13:23
samueldmqdstanek: that test for role inference that is terrible ? or all the tests?13:23
*** akscram has quit IRC13:23
*** akscram has joined #openstack-keystone13:25
dstaneksamueldmq: i'm just not a fan of the fixtures being used that way. in order to see what is being tested i have to go somewhere else to look13:28
dstanekjust a preference13:28
dstaneki like fixtures for the non-essential setup, like logging, setting up infra like DB, etc13:28
samueldmqdstanek: I think for those tests you'd need to just look at the fixtures code once for all the entities13:29
dstaneki like to see explicit setup of the things i am asserting against13:29
samueldmqbecause all the fixtures do there for functional tests is to create/cleanup the entities13:29
samueldmqthat's all13:29
dstaneksamueldmq: that's all right now :-)13:30
samueldmqdstanek: :)13:30
dstanekand then someone adds a boolean switch that changes the passed in args and yada, yada confusion13:30
* samueldmq watch outs for args in fixtures13:31
*** belmoreira has quit IRC13:31
samueldmqdstanek: I agree with you, if one does that, then the thing starts being very confusing13:31
samueldmqand hard to read13:32
*** tonytan4ever has joined #openstack-keystone13:32
samueldmqdstanek: we will probable ending up rewriting that test for implied roles13:33
samueldmqnisha asked if she should do that (from scratch), I told her that is a good idea after finishing the others that have no test13:34
*** brancal has quit IRC13:34
*** richm has joined #openstack-keystone13:43
*** brancal has joined #openstack-keystone13:43
*** markvoelker has joined #openstack-keystone13:46
*** brancal has quit IRC13:49
stevemarwhere's ayoung hiding at13:51
stevemarjamielennox: i assume you know13:51
stevemarhenrynash: ping me when you're online, i need your help triaging https://bugs.launchpad.net/keystone/+bug/1588190 and https://bugs.launchpad.net/keystone/+bug/160765513:51
openstackLaunchpad bug 1588190 in OpenStack Identity (keystone) "policy.v3cloudsample.json broken in mitaka" [High,Triaged]13:51
openstackLaunchpad bug 1607655 in OpenStack Identity (keystone) "domain admin cannot create implied role in default v3 policy" [Medium,Triaged] - Assigned to Henry Nash (henry-nash)13:51
stevemaramakarov or dolphm around?13:53
amakarovstevemar, o/13:53
stevemaramakarov: there you are13:53
stevemaramakarov: so, i don't recall discussing this one, even though i wrote a note in the "whiteboard" area: https://blueprints.launchpad.net/keystone/+spec/pre-cache-tokens13:53
amakarovstevemar, me neither )13:54
stevemaramakarov: i don't remember if we approved it for newton or not, sounds like it was13:55
amakarovstevemar, last thing I remember about it - dolphm wanted performance comparison13:56
amakarovand I can't get our qa do that for me ))13:56
stevemaramakarov: i think that's still a good idea13:56
amakarovstevemar, me too13:56
amakarovaccording to our stress tests validation takes 2x of issue time13:57
amakarovand we can get rid of the entire operation13:58
amakarovAndrew Grebennikov laughed that it's a return to token persistency )13:59
stevemaramakarov: hehe13:59
stevemaramakarov: not entirely, but that statement is not entirely inaccurate :P14:00
amakarovstevemar, I'd ask you you to pay attention to this improvement too: https://review.openstack.org/#/c/285521/14:00
patchbotamakarov: patch 285521 - keystone - Closure table for HMT14:00
*** krot_sickleave is now known as krotscheck14:00
stevemaramakarov: i'll keep an eye on it14:02
amakarovstevemar, what should be in the spec :)14:02
amakarov?14:02
stevemaramakarov: for pre-cache?14:02
amakarovit's completely internal change14:02
amakarov++14:02
stevemaramakarov: i don't think it's necessary, i just dont remember approving it14:02
stevemaramakarov: i approved it now in launchpad14:02
stevemarsorry for the confusion14:03
*** akrzos has joined #openstack-keystone14:03
amakarovstevemar, so what's to be done to merge that?14:03
stevemaramakarov: my lazy butt has to review it14:04
stevemaramakarov: removed my -114:04
amakarovHave you asked me for performance check or it's just a reminder for yourself?14:05
*** diazjf has joined #openstack-keystone14:06
stevemaramakarov: oh that's the comment to activate lbragstad's performance tests14:06
stevemaramakarov: https://github.com/lbragstad/keystone-performance#how-do-i-run-performance-on-a-patch-in-review14:08
stevemarIf a comment is left on an openstack/keystone review containing check performance in the message, the bot will performance test the patch against master and leave a comment on the review when it is finished.14:08
stevemaramakarov: considering it just does auth calls this is the perfect test and should see a change14:09
amakarovstevemar, wow!14:09
amakarovLooks like I'm missing interesting things happening (14:10
stevemaramakarov: its cool cause it's run on the same machine always14:10
amakarovstevemar, does this bot runs its tests with caching enabled?14:11
lbragstadamakarov it runs however openstack-ansible deploys keystone14:12
stevemarlbragstad or dstanek one of you want to take a quick look at https://review.openstack.org/#/c/344496/ ? it'll close out a bug14:12
patchbotstevemar: patch 344496 - keystone - Skip middleware request processing for admin token14:12
amakarovlbragstad, hi! Where can I find now openstack-ansible deploys keystone? )14:12
lbragstadamakarov https://github.com/lbragstad/keystone-performance#osic-performance-bot14:12
*** ayoung has joined #openstack-keystone14:12
*** ChanServ sets mode: +v ayoung14:12
lbragstadamakarov the role that I'm consuming to stand up keystone can be found here - https://github.com/openstack/openstack-ansible-os_keystone14:13
*** narengan has joined #openstack-keystone14:14
rodrigodsstevemar, ayoung is in an endless battle with tripleo, he seems to be winning now14:16
stevemarrodrigods: is the battle to the death?14:17
ayoungrodrigods, shh you will jinx me14:17
amakarovlbragstad, are settings in defaults/main.yml?14:17
lbragstadamakarov most of them, yes14:17
stevemarayoung got a few minutes to look at policy bugs?14:17
ayoungstevemar, in a few minutes I will14:18
stevemarayoung: okay, i just need opinions on https://bugs.launchpad.net/keystone/+bug/1588190 and https://bugs.launchpad.net/keystone/+bug/160765514:18
openstackLaunchpad bug 1588190 in OpenStack Identity (keystone) "policy.v3cloudsample.json broken in mitaka" [High,Triaged]14:18
openstackLaunchpad bug 1607655 in OpenStack Identity (keystone) "domain admin cannot create implied role in default v3 policy" [Medium,Triaged] - Assigned to Henry Nash (henry-nash)14:18
* lbragstad amakarov default/main.yml is the standard defaults that osa deploys keystone with - I have to override some of them here https://github.com/lbragstad/keystone-performance/blob/master/setup_perf_host.yml14:18
*** narengan1 has joined #openstack-keystone14:18
mfischstevemar: when does browne typically come on?14:20
mfischhe's probably PSD14:20
mfischPST14:20
*** narengan has quit IRC14:23
*** links has quit IRC14:23
*** jaosorior has quit IRC14:25
openstackgerritMerged openstack/keystone: Remove configuration references to eventlet  https://review.openstack.org/34855514:25
*** dkehn_ has quit IRC14:25
*** stevemar changes topic to "Newton Deadlines: http://releases.openstack.org/newton/schedule.html | Meeting Agenda https://etherpad.openstack.org/p/keystone-weekly-meeting"14:26
*** stevemar changes topic to "Newton Deadlines: http://releases.openstack.org/newton/schedule.html | Meeting Agenda https://etherpad.openstack.org/p/keystone-weekly-meeting"14:26
*** stevemar changes topic to "Newton Deadlines: http://releases.openstack.org/newton/schedule.html | Meeting Agenda https://etherpad.openstack.org/p/keystone-weekly-meeting"14:26
ayoungstevemar, 1588190  sounds like role assignment inheritance14:26
openstackgerritRon De Rose proposed openstack/keystone: Drop EPHEMERAL user type  https://review.openstack.org/29663914:28
openstackgerritRon De Rose proposed openstack/keystone: Drop EPHEMERAL user type  https://review.openstack.org/29663914:29
stevemarrderose: poke14:31
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Lockout requirements  https://review.openstack.org/34007414:34
stevemarrderose: just to clarify, i think bug 1601929 should be the next one on your list after PCI is done :)14:36
openstackbug 1601929 in OpenStack Identity (keystone) "Relax the requirement for mappings to result in group memberships" [High,Triaged] https://launchpad.net/bugs/1601929 - Assigned to Ron De Rose (ronald-de-rose)14:36
stevemarIMO that's the nastiest of the federation bugs14:36
samueldmqayoung:14:37
samueldmqayoung:  can you look at https://review.openstack.org/#/c/350562/ ?14:37
patchbotsamueldmq: patch 350562 - python-keystoneclient - Correct test_implied_roles14:37
*** code-R has joined #openstack-keystone14:37
samueldmqayoung: I didn't approve so you have a chance to take a look  :)14:37
*** dkehn_ has joined #openstack-keystone14:38
*** code-R_ has joined #openstack-keystone14:38
*** michauds has joined #openstack-keystone14:40
ayoungsamueldmq, I always like to write tests that check a precondition when possible14:40
ayoungso, just because the number of roles match at the end is not proof that the code executed in between did anything14:41
samueldmqayoung: why is that a precondition? other tests ensure the roles are created appropriately14:41
ayounginstad, check what it is without the rule, add the rule, check what it is with the rule14:41
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Minimum password age requirements  https://review.openstack.org/34331414:41
*** code-R has quit IRC14:41
ayoungsamueldmq, think of it as a control in an experiment14:41
samueldmqayoung: I see tests as being specific on what they want to test14:42
ayoungsamueldmq, and this is specific14:42
samueldmqayoung: let's check roles are created in the roles tests, inference rules in their own tests14:42
ayoungit wants to test that adding an implied role to a user increases the number of roles that user has14:42
samueldmqayoung: nah, that one is just checking against global roles, no?14:43
samueldmqayoung: it's checking the global roles ahve been increased14:43
samueldmqayoung: which is not the intent of that test14:43
samueldmqayoung: I will work with nisha to make that test more meaninful later, just want to get things moving again14:43
ayoungsamueldmq, er, yeah, this is checking that the set of roles are created14:43
samueldmqayoung: :)14:44
ayoungsamueldmq, so, keep the intention of checking the precondition/postcondition14:44
ayoungyou can do it by checking specific role names or what not14:44
ayoungif you want it as two separate tests, that is ok14:45
samueldmqayoung: yes, but checking the roles are created is part of the roles tests14:45
samueldmqnot the inference rules tests14:45
samueldmqand that part is just checking the roles were created (the part I removed)14:46
samueldmqayoung: do you agree?14:46
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Password expires validation  https://review.openstack.org/33336014:46
ayoungsamueldmq, true.  However, in this case, those roles are used to check the new inference rules.  I don't know what it would mean if the roles were not created14:46
samueldmqayoung: the role tests would fail14:46
samueldmqtoo14:46
*** knikolla has joined #openstack-keystone14:47
*** code-R_ has quit IRC14:48
*** code-R has joined #openstack-keystone14:48
knikollahello everyone o/14:48
knikollaback from vacations14:48
stevemarknikolla: you were gone forever :O14:48
stevemaro_O14:48
ayoungsamueldmq, I'm OK with your change.  I would move the role create to before any of the testing, then.14:49
ayoungcreate_roles;  check rule count;  create rules;  check rule count;14:49
knikollastevemar: nice to see i was missed :)14:50
samueldmqayoung: ins't just a test creating roles and checking they were created enough?14:50
samueldmqI don't see what that test buy us (checking counts)14:50
ayoungsamueldmq, this test really should be testing the rules14:50
ayoungthe role creation is already tested14:50
samueldmqexactly, so why you want to check your calls to create role effectively worked?14:51
samueldmqthat's an assumption of the test14:51
ayoungrule14:51
ayoungsamueldmq, rewrite however you see fit.  This was just a create test, and I would not object to clearer naming and separate of concernts14:52
*** ravelar159 has joined #openstack-keystone14:52
ayoungbut importnat part here is test creating the rules, not the roles14:52
* ayoung sghould have come up with more different nouns14:52
samueldmqayoung: exactly14:53
ayoungOK14:53
samueldmqthat's the reason I am removing the ROLES part14:53
samueldmqand only leaving the rules cehcks14:53
stevemarknikolla: whered you go?14:55
knikollastevemar: french alps. beautiful landscapes.14:57
*** amakarov has quit IRC14:58
*** diazjf has quit IRC14:58
stevemarknikolla: nice14:59
stevemardolphm: do you think 1601929 should wait until https://review.openstack.org/#/c/324055/ lands?14:59
patchbotstevemar: patch 324055 - keystone-specs - Mapping shadow users into projects and roles14:59
*** amakarov has joined #openstack-keystone15:00
stevemari agree that https://review.openstack.org/#/c/324055/ will solve the bug -- i guess i was hoping we could rid ourselves of that requirement sooner15:00
patchbotstevemar: patch 324055 - keystone-specs - Mapping shadow users into projects and roles15:00
dolphmstevemar: i was just re-thinking the bug, actually. i think there's two pieces to it, and for the second half-- yes.15:00
dolphmstevemar: to break it down...15:00
stevemardolphm: jam with me15:00
dolphmstevemar: the first part in my mind is the "constraint" that you referred to, that we basically 401 if you don't receive a group membership, right?15:01
openstackgerritGage Hugo proposed openstack/keystone: Add schema validation to create user v2  https://review.openstack.org/34853115:01
stevemardolphm: correct15:01
dolphmstevemar: you should still be able to auth and receive manual concrete role assignments at any time15:01
dolphmstevemar: that's the bug15:01
dolphmstevemar: right?15:01
stevemardolphm: correct, and that is what I tried to state in my bug report15:01
dolphmstevemar: the second part is that you should be able to auth and receive authorization via some means other than group memberships, which is the spec and is not a bug, just a feature gap that we have the opportunity to fill15:02
dolphmstevemar: alright, let me comment on it again15:02
stevemardolphm: in my mind, i want me mapping to just be: local: "user: {0}", remote: "SOME_HEADER" -- so they can authenticate and i can assign them roles15:02
*** pgbridge has joined #openstack-keystone15:02
stevemardolphm: correct15:02
stevemardolphm: now, to put a wrinkle in it...15:03
stevemardolphm: is a user being able to auth all that useful if they don't have any role assignments ? :(15:03
dolphmstevemar: your bug description seems to describe the spec pretty well though: "we should allow for mappings to result in per-user assignments"15:03
samueldmqayoung: you okay with that then?15:04
dolphmstevemar: yep, the only utility is that they get a local user ID and they can receive manual concrete assignments15:04
ayoungsamueldmq, yep15:04
stevemardolphm: maybe it's more logical to wait until the spec lands, cause if we remove the constraint then -- yeah ^15:04
stevemardolphm: it's definitely scaffolding for the spec, not sure how much it actually buys us15:04
samueldmqayoung: thanks, I will add you to other changes (adding tests) for implied,15:04
dolphmstevemar: the biggest immediate benefit though is that i don't have to manage a complicated mapping in all cases.... like you suggested, a really simple mapping will get federation working for me, and i can do things the old fashioned way15:05
stevemardolphm: yes, but the flow would be super wonky -- "auth, get a 401 since i have no roles, contact admin, get a role assigned to me, then re-auth"15:06
stevemarthe admin can't assign me a role earlier since i don't have a user id yet, cause i haven't authed15:06
stevemarunless you have some middleware that could do auto assignment15:06
dolphmstevemar: but, you'd still be able to get an unscoped token15:07
dolphmstevemar: so, no 40115:07
stevemartru dat15:07
*** roxanaghe has joined #openstack-keystone15:07
stevemarokay15:07
dolphmstevemar: new bug description? "With the introduction of shadow users, we should not require mappings to result in group memberships. This should not require an API change, but would allow for much simpler mappings to be used (literally just assigning a unique ID, and nothing more), which would be sufficient to allow federated users to receive manually assigned concrete role assignments (a process that operators are15:07
dolphm already familiar with)."15:07
stevemardolphm: ++15:08
stevemarrderose: ^15:08
dolphmstevemar: cool, one sec15:08
stevemardolphm: thanks bud15:08
stevemareveryone is gonna hate me after a few weeks of this pestering15:09
rderosedolphm++15:09
dolphmstevemar: rderose: added another comment as well15:10
rderosedolphm: okay, will take a look.  thx15:10
dolphmstevemar, professional cat herder15:10
dolphmstevemar: and i agree with the High in this case15:11
stevemardolphm: coolio15:16
*** julim has joined #openstack-keystone15:16
*** ravelar159 has quit IRC15:16
*** diazjf has joined #openstack-keystone15:16
openstackgerritDolph Mathews proposed openstack/keystone: Clean up the introductory text in the docs  https://review.openstack.org/35063915:16
*** ravelar has joined #openstack-keystone15:16
stevemarrderose: are your PCI patches ready for review or still -W?15:20
samueldmqstevemar: dolphm: just have a comment/question on 350639 before approving15:21
*** jcalcote has quit IRC15:21
rderosestevemar: this one is ready https://review.openstack.org/#/c/340074/15:22
patchbotrderose: patch 340074 - keystone - PCI-DSS Lockout requirements15:22
rderosestevemar: just fixed merge conflict15:22
*** dkehn_ has quit IRC15:22
*** sheel has joined #openstack-keystone15:23
*** dave-mccowan has quit IRC15:24
samueldmqstevemar: so when it says 'distributed multi-tenant authorization', is it talking about projects/domains/assignments15:25
samueldmqstevemar: why distributed then?15:25
stevemarsamueldmq: just ignore the word :P15:26
stevemarsamueldmq: hehe, i'll let dolphm answer you then15:26
samueldmqhehe lol15:26
samueldmqstevemar: ok, just trying to understand it completely. I like the change15:27
dolphmsamueldmq: i replied in the review15:27
dolphmsamueldmq: just expanded on steve's answer a bit15:27
samueldmqdolphm: kk I thought we were only considering keystone server in that, but it includes other bits like middleware and oslo policy15:28
samueldmqstevemar: dolphm: agreed then, thanks for clarifying15:28
dolphmsamueldmq: in my view, clients authenticate with keystone (federation, password auth, tokens), use the service catalog to discover the rest of openstack, and then almost the rest of our ecosystem is to facilitate authorization and tenant isolation across a whole bunch of services that we have no actual control over (distributed)15:29
openstackgerritGage Hugo proposed openstack/keystone: Add schema validation to update user v2  https://review.openstack.org/34502215:29
stevemarbreaking for lunch, see you folks in a bit15:29
samueldmqdolphm: perfect, totally agree15:30
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Minimum password age requirements  https://review.openstack.org/34331415:31
*** dkehn_ has joined #openstack-keystone15:32
*** itlinux has joined #openstack-keystone15:39
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Minimum password age requirements  https://review.openstack.org/34331415:40
rderosestevemar: this one is ready as well: https://review.openstack.org/#/c/343314/15:41
patchbotrderose: patch 343314 - keystone - PCI-DSS Minimum password age requirements15:41
*** nishaYadav has joined #openstack-keystone15:42
*** nisha_ has joined #openstack-keystone15:43
samueldmqayoung: deleting a role will cause the deletion of all the roles implied by that first?15:43
samueldmqayoung: or is it just going to delte the role inferences ? (I 'd expect this)15:43
ayoungno15:44
ayoungdeleting a role should have no impace on other roles, just on the inference rules15:44
*** pece has joined #openstack-keystone15:45
mnasersorry for jumping in with a crazy idea right away.  has the idea of adding some sort of compatibility layer in v2 to use domains ever been discussed?  we want to move to a domain per user model and the fact a lot of customers depend on v2.0 makes it hard :(15:46
*** nishaYadav has quit IRC15:46
mnasermaybe something crazy like OS_TENANT_NAME=tenant@domain15:46
mnaser(yes I know this is crazy but the # of tools built for keystone 2.0 only make it quite hard)15:47
*** nisha_ is now known as nishaYadav15:47
samueldmqayoung: ok, so docs in https://review.openstack.org/#/c/334546/11/keystoneclient/v3/roles.py are worng (delete role docs)15:48
patchbotsamueldmq: patch 334546 - python-keystoneclient - Improve docs for v3 roles15:48
dstanekmnaser: i think that would require lots of changes to the v2 code. not sure if it's possible, but it's an interesting idea15:48
samueldmqayoung: thanks for confirming15:48
mnaserdstanek, that way we can slowly start using domains without telling customers "your software doesnt work anymore"15:48
mnaserex: jcloud doesnt support v3 auth afaik, so we have customers that do CI and they wouldn't be able to do that anymore15:50
* nishaYadav waves hello o/15:50
dstanekmnaser: sure. at a mimimum you'd have to change all the code that uses the default domain to parse the tenant info for the actual domain. i'm not sure if we have any assumptions in v2 about the default domain that would break15:53
dstanekmnaser: off the top of my head this feels like an invasive change15:54
mnaserdstanek, it is very invasive indeed!  however, looking at our stats, it's kinda crazy how low the # of v3 api requests that come in15:54
mnaseri took a sample of ~7300 external keystone requests (not the ones from our services).. only 300 were towards v3 endpoint15:54
*** danpawlik has quit IRC15:57
*** nishaYadav has quit IRC15:58
*** nishaYadav has joined #openstack-keystone15:58
samueldmqnishaYadav: hi15:59
*** nisha_ has joined #openstack-keystone16:00
samueldmqnisha_: hi16:00
*** nishaYadav has quit IRC16:00
*** nisha_ is now known as nishaYadav16:01
nishaYadavsamueldmq, hey16:01
samueldmqnishaYadav: how are you?16:01
samueldmqnishaYadav: I just left a couple of comments in patch 33454616:01
patchbotsamueldmq: https://review.openstack.org/#/c/334546/ - python-keystoneclient - Improve docs for v3 roles16:01
dolphmsamueldmq: stevemar: fixed a tiny nit and re-approved https://review.openstack.org/#/c/350639/16:02
patchbotdolphm: patch 350639 - keystone - Clean up the introductory text in the docs16:02
samueldmqdolphm: nice16:02
samueldmqnishaYadav: https://review.openstack.org/#/c/350562/ will fix the intermittent errors. see the description of the bug it fixes to understand what was going on16:03
patchbotsamueldmq: patch 350562 - python-keystoneclient - Correct test_implied_roles16:03
*** Gorian_ has joined #openstack-keystone16:03
nishaYadavsamueldmq, I am good thanks16:03
nishaYadavsamueldmq, looking at them :)16:04
*** ddieterly has joined #openstack-keystone16:04
*** itisha has joined #openstack-keystone16:04
*** esp has joined #openstack-keystone16:04
*** jistr is now known as jistr|biab16:06
nishaYadavsamueldmq, thanks for reviewing, doing the changes in role docs :)16:12
samueldmqnishaYadav: cool, also just left a couple of comments in the role tests too16:13
nishaYadavsamueldmq, nice16:13
*** adrian_otto has joined #openstack-keystone16:13
openstackgerrithenry-nash proposed openstack/keystone: Add support for rolling upgrades to keystone-manage  https://review.openstack.org/34971616:14
*** roxanaghe has quit IRC16:20
*** jistr|biab is now known as jistr16:22
*** diazjf has quit IRC16:24
*** tqtran has joined #openstack-keystone16:27
*** jpena is now known as jpena|off16:30
*** nisha_ has joined #openstack-keystone16:31
*** tqtran has quit IRC16:31
*** jsavak has quit IRC16:33
*** jsavak has joined #openstack-keystone16:33
*** ddieterly is now known as ddieterly[away]16:34
*** nishaYadav has quit IRC16:34
*** roxanaghe has joined #openstack-keystone16:35
*** nisha_ is now known as nishaYadav16:36
*** rcernin has quit IRC16:38
*** ddieterly[away] is now known as ddieterly16:40
*** pece has quit IRC16:44
*** roxanaghe has quit IRC16:45
*** roxanaghe has joined #openstack-keystone16:47
openstackgerritMerged openstack/python-keystoneclient: Correct test_implied_roles  https://review.openstack.org/35056216:47
*** krotscheck is now known as kro_focused16:53
*** adrian_otto has quit IRC16:53
*** roxanaghe has quit IRC16:56
*** browne has joined #openstack-keystone16:56
*** adrian_otto has joined #openstack-keystone16:56
*** roxanaghe has joined #openstack-keystone16:57
*** roxanaghe has quit IRC16:57
*** roxanaghe has joined #openstack-keystone16:57
bretonfg17:00
*** pcaruana has quit IRC17:00
*** tqtran has joined #openstack-keystone17:00
*** pnavarro has quit IRC17:01
*** tqtran_ has joined #openstack-keystone17:01
*** tesseract- has quit IRC17:03
*** tqtran has quit IRC17:04
*** openstackgerrit_ has joined #openstack-keystone17:06
*** openstackgerrit_ has quit IRC17:08
*** nisha_ has joined #openstack-keystone17:11
*** spzala has quit IRC17:12
*** nishaYadav has quit IRC17:12
*** nisha_ is now known as nishaYadav17:12
*** KevinE has joined #openstack-keystone17:15
*** jsavak has quit IRC17:18
*** jsavak has joined #openstack-keystone17:18
openstackgerritNisha Yadav proposed openstack/python-keystoneclient: Improve docs for v3 roles  https://review.openstack.org/33454617:20
*** narengan1 has quit IRC17:20
*** spzala has joined #openstack-keystone17:20
*** spzala has quit IRC17:20
*** spzala has joined #openstack-keystone17:20
*** dave-mccowan has joined #openstack-keystone17:20
KevinEandreykurilin__: Hey, I was wondering what happened to the change? I was going to test it against our env but I saw someone added some changes and now it can't pass a single test17:21
*** dave-mcc_ has joined #openstack-keystone17:22
*** amakarov is now known as amakarov_away17:24
*** ddieterly is now known as ddieterly[away]17:24
andreykurilin__@KevinE: it is only about db migrations, so you check this patch without it17:24
*** dave-mccowan has quit IRC17:27
*** browne1 has joined #openstack-keystone17:27
*** browne has quit IRC17:27
openstackgerritLance Bragstad proposed openstack/keystone: Make all token provider behave the same with trusts  https://review.openstack.org/35070417:29
lbragstadbreton dstanek ^17:31
lbragstadbreton let me know if that works for you17:31
lbragstadi'm going to break for lunch17:31
bretonbut... they behaved the same for me already17:32
bretontest_validate_v3_trust_scoped_token_against_v2_succeeds is new though17:32
*** daemontool_ has quit IRC17:38
stevemarbreton: just looking through open bugs, isn't https://bugs.launchpad.net/keystone/+bug/1406314 resolved now that we run migration tests on mysql and postgres?17:39
openstackLaunchpad bug 1406314 in OpenStack Identity (keystone) "db migration tests falsely succeed" [Wishlist,Triaged]17:39
*** jsavak has quit IRC17:40
bretonstevemar: looks like resolved. Thank you, closing it.17:40
stevemarbreton: thanks17:40
*** jsavak has joined #openstack-keystone17:40
*** code-R has quit IRC17:45
openstackgerritEric Brown proposed openstack/keystone: Use URIOpt for endpoint URL options  https://review.openstack.org/35071317:50
*** julim has quit IRC17:53
mordredstevemar: heya - any chance you have a timeframe for a ksa release in mind?17:55
stevemarmordred: already requested one this morning ;O https://review.openstack.org/#/c/350571/17:56
patchbotstevemar: patch 350571 - releases - release keystoneauth 2.11.017:56
mordredneat!17:56
openstackgerritMonty Taylor proposed openstack/keystoneauth: Add tests for YamlJsonSerializer  https://review.openstack.org/34494317:56
stevemarmordred: just waiting for dims or dhellmann17:56
*** julim has joined #openstack-keystone17:56
mordredstevemar: soo .... any chance we can get that ^^ in before hand?17:56
stevemarmordred: you need 344943 in there?17:56
stevemaraye17:56
stevemarlemme take a quick look17:57
mordred(I found a bug in the code while adding tests ... fancy that)17:57
stevemari was going through the open reviews in the morning and figured it was OK to relase cause that was just test related, didn't realize it was changing a fixture17:57
mordredyah - it was a one-liner. I should probably have written a better commit message17:58
*** harlowja has quit IRC17:58
mordrednotmorgan: ^^ have a sec for a quick review?17:58
*** haplo37__ has joined #openstack-keystone17:59
stevemarmordred: can you fix one little typo in the sample URL17:59
mordredyes!17:59
openstackgerritNisha Yadav proposed openstack/python-keystoneclient: Add role functional tests  https://review.openstack.org/33511817:59
stevemarits bugging me way too much that keystone is missing an e17:59
*** ddieterly[away] is now known as ddieterly18:00
*** harlowja has joined #openstack-keystone18:00
openstackgerritMonty Taylor proposed openstack/keystoneauth: Add tests for YamlJsonSerializer  https://review.openstack.org/34494318:00
mordredstevemar: without an e, we're just a ton of keys18:01
stevemarmordred: i think you have to update the json too18:01
mordredpiddle18:01
stevemarmordred: exactly!18:01
openstackgerritMonty Taylor proposed openstack/keystoneauth: Add tests for YamlJsonSerializer  https://review.openstack.org/34494318:02
mordredstevemar: how's that?18:02
*** esp has quit IRC18:02
stevemarmordred: poifect!18:03
mordredstevemar: if you haven't seen the output of this in action: https://review.openstack.org/#/c/344397/2/cassettes/citycloud/test_compute/TestCompute/test_set_and_delete_metadata.yaml18:06
patchbotmordred: patch 344397 - openstack-infra/shade - Migrate functional tests to betamax18:06
stevemarmordred: whoa thats a big file18:06
mordredyah. our API interactions may be a bit chatty18:07
stevemarcity cloud is the cool new cloud provider18:07
*** diazjf has joined #openstack-keystone18:08
*** tonytan4ever has quit IRC18:08
mordredthey're doing pretty well - although they are a floating ip cloud, which makes me a little sad18:08
*** dave-mcc_ has quit IRC18:08
*** narengan has joined #openstack-keystone18:10
*** jaugustine has quit IRC18:13
*** gagehugo has quit IRC18:13
*** esp has joined #openstack-keystone18:13
*** nk2527 has quit IRC18:14
*** ddieterly is now known as ddieterly[away]18:16
*** ddieterly[away] is now known as ddieterly18:17
*** code-R has joined #openstack-keystone18:17
*** gagehugo has joined #openstack-keystone18:18
*** code-R_ has joined #openstack-keystone18:20
*** adrian_otto has quit IRC18:21
*** code-R has quit IRC18:23
*** sheel has quit IRC18:26
*** jaosorior has joined #openstack-keystone18:29
*** jaosorior has quit IRC18:30
*** jaugustine has joined #openstack-keystone18:33
notmorganahahahahahaha somehow my OTP matched one of my gerrit reviews *boggle*18:34
notmorganso copy/paste fail.18:34
notmorganand i was wondering why you were wanting an infra change in ksa18:34
*** jmlowe1 has quit IRC18:36
*** ayoung has quit IRC18:38
*** ddieterly is now known as ddieterly[away]18:40
*** ametts has joined #openstack-keystone18:45
*** nk2527 has joined #openstack-keystone18:45
*** julim has quit IRC18:46
mordred:)18:48
bknudsonMaybe notmorgan can help me with something I'm looking at...18:51
bknudsonor anyone here.18:52
bknudsonhere's part of a stack trace: http://paste.openstack.org/show/547832/18:52
bknudsonso it's getting revocation event from the cache, and it's failing to parse whatever memcache returns.18:53
bknudsonso 1) maybe something overwrote the line on us, or 2) maybe it was evicted and memcache returns a special value for that?18:54
bknudsonCould use some more debug in this case.18:54
bknudsonnotmorgan: any idea? I'll look for docs.18:55
*** haplo37__ has quit IRC18:55
*** jed56 has quit IRC18:55
notmorganbknudson: uhm, hold on.18:55
*** adrian_otto has joined #openstack-keystone18:56
notmorganbknudson: you're running into something where the __init__ func is *not* a string. not sure what is being returned though. is this in master? some change you're working on?18:56
bknudsonnotmorgan: it's in master. We've got a team running rally on a test deployment.18:57
bknudsonand they've been hitting this issue (it causes keystone to 500 on token validation)18:57
notmorganbknudson: honestly, I have no idea what has been done to the keystone code base in the last 2+ months18:58
notmorganso, i can look, but I'd also need to know what is being returned to revoke_model18:58
stevemarbknudson: that with liberty?18:58
notmorganstevemar: master he says18:58
bknudsonstevemar: no, this is master.18:58
stevemarbknudson: k18:58
samueldmqdstanek: asserting that a list is empty with assertFalse(list) is correct?18:59
samueldmqdstanek: it seems weird to me18:59
bknudsonnotmorgan: can you think of a reason why memcache wouldn't return what was stored in it?18:59
notmorganbknudson: something stomping on memcache19:00
notmorganbknudson: or a legit bug in the code, but we need to know what the data returned is.19:00
bknudsonnotmorgan: I wonder if we have to have code to protect ourselves from this?19:00
notmorganmostly we assume keystone owns memcache keys19:00
notmorganso if something is stomping on memcache, we don't really protect ourselves19:00
bknudsonI'm trying to get on the system to see what it is... haven't done this before so not sure what the problem is.19:00
stevemari was hoping you were also running into https://bugs.launchpad.net/keystone/+bug/160039319:00
openstackLaunchpad bug 1600393 in OpenStack Identity (keystone) "AttributeError: 'list' object has no attribute 'items'" [Critical,Confirmed]19:00
notmorganwell actually19:01
bknudsonstevemar: don't hope for us to run into bugs!19:01
notmorganno this isn't memcache19:01
notmorganthis is in-memory dict19:01
stevemarbknudson: better for me to have 1 bug instead of 219:01
notmorganthis is in the request_local cache19:01
notmorganso def. not memcache19:01
bknudsonnotmorgan: oh, good catch!19:01
bknudsonmight make this a little easier.19:02
notmorganit's something that is occuring when msgpack is deserializing19:02
*** gagehugo_ has joined #openstack-keystone19:02
notmorganit should make this a LOT easier :)19:02
*** ravelar has quit IRC19:02
stevemaryeah this one: https://github.com/openstack/keystone/blob/0c2fbef08c21f39998965379a9e2953c237735e4/keystone/common/cache/_context_cache.py#L24-L6219:02
stevemarbknudson: maybe lbragstad's commit: https://github.com/openstack/keystone/commit/510f00f365cd988ca2a5d22413d488e8d78c9a89 ?19:02
*** jsavak has quit IRC19:03
notmorganthat shouldn't have broken things.19:03
notmorganafaict19:03
notmorganit's getting... oh wait ...19:03
stevemarthe refactor could have changed a method signature19:03
bknudsonWish I could say if this just started happening or if it's been going on for a while.19:03
bknudsonBut I think they just started with the testing when they reported this bug and that's only been a couple weeks.19:04
notmorganuhm. i ... wow this code has changed a bit.19:04
*** dave-mccowan has joined #openstack-keystone19:04
stevemarbknudson: revert that change i suggested, see if you still hit it19:05
bknudsonstevemar: you make that sound so simple.19:05
notmorgani think msgpack is storing something (object?) as a reference in **revoke_event_data19:06
bknudsonnotmorgan: is it possible for get() to return api.NO_VALUE?19:06
*** fifieldt has quit IRC19:07
notmorganbknudson: which line?19:07
notmorganalso, yes.19:07
*** haplo37__ has joined #openstack-keystone19:07
bknudsonI'll see if I can mock that up.19:07
stevemarbknudson: lol19:07
notmorganbknudson: when the error occurs raise up a repr of revoke_event_data19:08
notmorganbknudson: you'll know exactly what is being passed to __init__19:08
notmorganand we can more easily trace what is going on19:08
*** tonytan4ever has joined #openstack-keystone19:08
notmorganbknudson: https://github.com/openstack/keystone/commit/510f00f365cd988ca2a5d22413d488e8d78c9a89#diff-b4c4265719691b41cde418bcfa152ee9R345 wherever that line is now.19:09
bknudsony, I need to figure out how to get on this system and change the code a bit.19:09
bknudsonok, thanks!19:09
notmorgansomething is being stored incorrectly now in the dict, so you have keys that aren't strings, possibly an object, and the revokeevent is cranky (more specifically python is) about it19:10
bknudsonnotmorgan: do you know if there's some way for deserialize to indicate that the data it got was just bad / unexpected?19:13
*** tonytan4ever has quit IRC19:13
bknudsonand just return that there's no object instead of failing?19:13
bknudsonmaybe just catch the exception and return api.NO_VALUE19:14
*** nisha_ has joined #openstack-keystone19:15
*** nishaYadav has quit IRC19:15
*** nisha_ is now known as nishaYadav19:15
mordredstevemar: don't be mad k?19:16
openstackgerritMonty Taylor proposed openstack/keystoneauth: Add tests for YamlJsonSerializer  https://review.openstack.org/34494319:17
mordredstevemar: we missed a place with keystonauth19:17
*** ddieterly[away] is now known as ddieterly19:17
stevemarmordred: those damn tons of keys!19:18
*** diazjf has quit IRC19:18
*** fifieldt has joined #openstack-keystone19:18
mordredstevemar: I git grepped this time :)19:19
notmorganbknudson: msgpack doesn't know anything about the data besides what that serializer does19:19
*** jsavak has joined #openstack-keystone19:19
notmorganbknudson: so the smarts have to be in the msgpack serializer/deserializer19:19
notmorganbknudson: if RevokeEvent wasn't so weird, it would just work w/o magic code19:19
mordrednotmorgan: we're msgpacking now?19:20
bknudsonwhat's weird about it?19:20
*** diazjf has joined #openstack-keystone19:20
notmorganmordred: only in the request local cache, becuase revokeevent is ... icky19:20
mordredah - interesting19:20
notmorganbknudson: well, json just throws it's hands up assuming circular deps among other things19:20
mordredI was going to suggest protobuf instead ... but maybe not for that :)19:20
bknudsonI thought we changed it to a list...19:20
notmorganbknudson: that is the issue.19:21
notmorganit's not *just* a list19:21
notmorganit has references to references to references in the list19:21
notmorganand has added properties that make it so msgpack can't just deserialize19:21
notmorganhttps://github.com/openstack/keystone/blob/0c2fbef08c21f39998965379a9e2953c237735e4/keystone/models/revoke_model.py#L7519:21
notmorganbasically it is a bad object design at this point19:22
bknudsonhere's the commit https://github.com/openstack/keystone/commit/75abc21ecfc2a37c10be61289204b5056403dd5c19:23
bknudsonI don't see any refs in the current revoke model.19:23
openstackgerritNisha Yadav proposed openstack/python-keystoneclient: Improve docs for v3 roles  https://review.openstack.org/33454619:23
notmorganwe still use the revokeevent object19:24
notmorganaccording to that commit19:24
notmorganit's not the "tree" it's the actual event that is na issue19:24
bknudsonso msgpack might be useful here but not necessary, could use json19:24
notmorganyou can try and make json work again.19:24
bknudsonit's not referencing other complex objects.19:25
notmorgani gave up on it19:25
notmorganthe only reason to use msgpack was the inability to serialize the revokeevent & tree19:25
notmorganjson is ~13usec vs 60-100useq19:25
notmorganiirc19:25
notmorganfor serialize/deserialize19:26
bknudsonwhat does the standard serialization work with? only dicts?19:26
notmorganpython primitives19:26
notmorganif you only use the primitives, json will work19:26
notmorganso no custom classes19:26
bknudsonso if we MEMOIZE something it can only return primitives?19:27
bknudsonotherwise we need custom serdes.19:27
notmorganwell memcache uses pickle... but that aside19:27
bknudsonyaa!19:27
notmorgani had everything *but* revoke events working with json19:28
notmorganand the only reason we're using msgpack / json is because deepcopy is wonky as hell and inconsistent19:29
notmorganotherwise i'd just have deepcopied the objects fotr the request_local cache19:29
notmorganbut msgpack is easiest, if we make RevokeEvent work without the custom serializer, yay.19:29
notmorganit would mean we have a lot less magic code to maintain19:30
notmorganit *may* just work now to remove the handler19:30
bknudsoneasy enough to try it.19:30
*** jsavak has quit IRC19:30
bknudsonI'll give it a shot.19:30
*** jsavak has joined #openstack-keystone19:31
notmorgani think the reason we couldn't json serialize is we have datetime objects19:31
notmorganand those get rehydrated as strings.19:31
bknudsonshoot19:31
*** tesseract- has joined #openstack-keystone19:31
openstackgerritNisha Yadav proposed openstack/python-keystoneclient: Add role functional tests  https://review.openstack.org/33511819:32
nishaYadavsamueldmq, ^19:32
notmorgani'm guessing at some of these because i've been trying to be not-involved in keystone things for a bit so brain has swapped most of this out19:32
*** ametts has quit IRC19:32
bknudsonwe'll keep dragging you back in -- https://www.youtube.com/watch?v=UPw-3e_pzqU19:32
*** clenimar has quit IRC19:34
*** clenimar has joined #openstack-keystone19:35
*** tesseract- has quit IRC19:37
*** ametts has joined #openstack-keystone19:45
*** ayoung has joined #openstack-keystone19:45
*** ChanServ sets mode: +v ayoung19:45
*** roxanaghe has quit IRC19:48
*** tonytan4ever has joined #openstack-keystone19:49
*** gagehugo has quit IRC19:52
*** narengan1 has joined #openstack-keystone19:58
*** gagehugo_ has quit IRC20:00
*** narengan has quit IRC20:00
lbragstaddstanek ayoung stevemar breton thoughts on my comment here - https://review.openstack.org/#/c/350704/1/keystone/tests/unit/test_auth.py20:00
patchbotlbragstad: patch 350704 - keystone - Make all token provider behave the same with trusts20:00
lbragstadit's about another inconsistency between the fernet and uuid providers20:01
*** narengan has joined #openstack-keystone20:01
*** gagehugo has joined #openstack-keystone20:02
*** permalac has quit IRC20:02
*** permalac has joined #openstack-keystone20:03
*** narengan1 has quit IRC20:04
*** daemontool has joined #openstack-keystone20:10
*** nishaYadav has quit IRC20:13
stevemarlbragstad: meant to look at it, got distracted, will do now20:14
stevemarlbragstad: line 250 here makes me sad: https://review.openstack.org/#/c/350704/1/keystone/models/revoke_model.py20:15
patchbotstevemar: patch 350704 - keystone - Make all token provider behave the same with trusts20:15
lbragstadstevemar what - the fact that it was completely broken?20:16
stevemarlbragstad: yeah, it gave me a sad20:17
stevemarlbragstad make my sad go away20:17
lbragstadstevemar yeah - that will start to happen when we get jamielennox's view stuff implemented for auth20:17
stevemarbiab, gonna straight my legs20:17
stevemarlbragstad: yep20:18
*** adrian_otto has quit IRC20:24
jamielennoxo/20:25
jamielennoxoh, i was trying to play with models as well - that's going to be a hard change20:26
jamielennox(code models)20:26
jamielennoxdamnit that whole sentence is terrible, might get tea and try and start the day again in a few minutes20:26
stevemarjamielennox: that last sentence was so aussie20:28
*** lamt_ is now known as help20:29
stevemardayuum lbragstad you did some serious digging around20:29
*** help is now known as Guest2334420:29
stevemarlbragstad: must be those new glasses that make you extra smart20:29
lbragstadstevemar yeah - i'm fried20:29
lbragstadwhat started as a "just delete this test and fix the failure" turned into a huge spiraling rabbit hole20:31
*** Guest23344 has quit IRC20:31
*** Guest23344 has joined #openstack-keystone20:32
*** ddieterly is now known as ddieterly[away]20:32
*** Guest23344 is now known as LamT20:34
dstaneklbragstad: good catch20:34
bknudsonlbragstad: stevemar: is there another security bug with revocation events? (regarding https://review.openstack.org/#/c/350704/1/keystone/models/revoke_model.py )20:38
patchbotbknudson: patch 350704 - keystone - Make all token provider behave the same with trusts20:38
*** roxanaghe has joined #openstack-keystone20:39
bknudsoneven though I don't work on security bug handling anymore I'm still paranoid.20:39
lbragstadbknudson well - it was completely broken20:39
lbragstadthe build_token_values_v2 method seemed to be complete broken20:39
*** ddieterly[away] is now known as ddieterly20:40
jamielennoxyea, i've been looking through the auth stuff as well, i'm not sure how it manages to work at all20:41
bknudsonluck.20:41
bknudsonmagic?20:41
lbragstadyes20:41
lbragstadand meth20:41
jamielennoxbknudson: history, and bad unit tests20:41
stevemarrule #1 of keystone, don't look at keystone.auth and keystone.token20:42
bknudsonif we ran it through a code obfuscator, could we tell the difference?20:42
dstanekbknudson: yes, it would get better20:43
stevemarmaybe a pinch, just cause we know the original from painful memories20:43
bknudsonlol20:43
*** jsavak has quit IRC20:46
*** jsavak has joined #openstack-keystone20:46
*** daemontool has quit IRC20:48
openstackgerritLance Bragstad proposed openstack/keystone: Make all token provider behave the same with trusts  https://review.openstack.org/35070420:51
lbragstadstevemar dstanek jamielennox bknudson updated with comments20:51
*** ametts has quit IRC20:52
*** ntpttr- is now known as ntpttr20:56
jamielennoxlbragstad: i don't know if i want to understand that20:56
lbragstadjamielennox which one/20:57
jamielennoxtoken providers and trusts20:57
lbragstadjamielennox I have a couple short-story length comments in there that attempt to help20:58
*** jsavak has quit IRC20:58
*** pauloewerton has quit IRC21:13
*** fifieldt has quit IRC21:15
*** ddieterly is now known as ddieterly[away]21:16
*** michauds has quit IRC21:23
*** fifieldt has joined #openstack-keystone21:25
*** narengan has quit IRC21:26
*** adriant has joined #openstack-keystone21:33
*** ddieterly[away] is now known as ddieterly21:36
openstackgerritDolph Mathews proposed openstack/keystone: Add rolling upgrade documentation  https://review.openstack.org/35079321:37
openstackgerritDolph Mathews proposed openstack/keystone: Introduce read-only mode for the database  https://review.openstack.org/34970021:41
*** sdake has quit IRC21:43
openstackgerritBrant Knudson proposed openstack/keystone: Ignore errors deserializing revocation events  https://review.openstack.org/35079921:47
openstackgerritBrant Knudson proposed openstack/keystone: Add debug logging for RevokeEvent deserialize problem  https://review.openstack.org/35080421:56
*** markvoelker has quit IRC21:58
*** markvoelker has joined #openstack-keystone21:59
mfischbrowne1: you around?22:01
browne1yep22:01
mfischbrowne1: which cache did you turn off? Keystone or middleware?22:03
*** markvoelker has quit IRC22:03
browne1mfisch: turned off keystones cache22:03
mfischare you using middleware cache?22:04
browne1not sure, where is it set? in keystone.conf?22:05
mfischso you have in effect 2 layers of cache22:05
mfisch1 is keystone's own cache which it sounds like you use22:05
mfisch2 is that nova (for example) can cache that a token is valid - middleware cache22:05
browne1i changed [cache]/enabled=false in keystone.conf22:05
mfischand then it won't need to hit keystone22:05
mfischL2 and L1 cache in effect22:05
mfischthats keystone cache22:05
*** adrian_otto has joined #openstack-keystone22:06
mfischelse would be in nova|cinder|glance|etc.conf22:06
mfischbtw - once this works, I'd recommend enabling that one ^22:06
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Minimum password age requirements  https://review.openstack.org/34331422:06
jamielennoxmiddleware cache in auth_token middleware?22:06
mfischy22:06
jamielennoxthat shouldn't be in play for keyston22:06
mfischjust confirming22:06
mfischI'm getting dirty looks from the team so need to disable now22:07
jamielennoxdid you get any further on how this info got into cache?22:07
browne1doesn't look like we're taking advantage of any caching in keystone_authtoken for the various services22:07
*** spzala has quit IRC22:08
*** diazjf has quit IRC22:08
jamielennoxoh, are you sharing the memcache backend between authtoken services and keystone?22:08
mfischno22:08
mfischI'm asking browne1 which he meant when he said he turned off caching to make this issue go away22:08
*** spzala has joined #openstack-keystone22:08
mfischseparate boxes, separate memcache servers for me22:08
browne1we only noticed problems in the caching when we ran performance/scale testing using Rally22:09
browne1although the role caching we noticed broke for us right away22:10
mfischif you look closely I bet its causing other issues22:10
mfischwe catch it on every 3rd deploy22:10
browne1our ansible playbooks to configure keystone and grant roles failed. authorization failed for glance right after granted the role to the user22:10
mfischpuppet making api calls and 1/1000 fails and its enough22:10
rodrigodsstevemar, kinda cool using openstackclient for SAML http://rodrigods.com/what-about-ecp/22:13
*** spzala has quit IRC22:13
*** adrian_otto has quit IRC22:16
* dstanek is interested and needs to read the backlog22:16
dstanekbrowne1: what are the problems you are seeing?22:17
browne1dstanek: basically, we saw various places where caching was used and received invalid data from it22:19
browne1https://bugs.launchpad.net/keystone/+bugs?search=Search&field.bug_reporter=ericwb22:19
browne1i opened these 3 bugs.  they aren't easy to recreate. only our scale/perf testing found them.22:20
dstanekbrowne1: oh, cool, bugs. i can start taking a look. i'm actually working on a cache bug now anyway22:20
dstanekbreton: have you run the same tests with caching off too?22:21
browne1saw crazy things like v3 tokens with v2 service catalog22:21
jamielennoxrodrigods: hey - can you test that with OSC 3?22:24
jamielennoxrodrigods: i haven't had a usable saml environment for a while, but with the switch to keystoneauth the plugins have changed22:24
dtroyerrodrigods, jamielennox: (re OSC 3) that would be this review: https://review.openstack.org/346113 + current osc-lib and os-client-config22:26
dtroyerwhere current == master22:26
jamielennoxdtroyer: oh, i thought the keystoneauth stuff was already merge22:27
jamielennoxd22:27
dtroyerjamielennox: nearly done, I finished fixing o-c-c this morning.  osc-lib should be ready, and that osc review needs a rebase22:27
dtroyerthen it's test like mad ;)22:28
dtroyero-c-c 1.19.0 release will be submitted in a few minutes ;)22:28
rodrigodsjamielennox, of course, will do that today22:29
rodrigodsjamielennox, do i need an updated keystoneauth as well?22:29
rodrigodsand keystoneclient?22:29
jamielennoxdtroyer: ahh o-c-c,22:29
*** gordc has quit IRC22:29
dtroyerI've been working with current release ksa and ksc, but ksc should be a don't care for this22:30
jamielennoxrodrigods: i think most recent release of KSA and maybe os-client-config22:30
rodrigodsjamielennox, got it22:30
rodrigodswas worried about the entry points in ksc22:30
jamielennoxthe auth_type will change, but i want to make sure you can get a scoped token directly and do operations with it22:30
jamielennoxrodrigods: your example uses v3unscopedsaml which is not really useful because you can't do anything with it22:31
*** ddieterly has quit IRC22:31
rodrigodsjamielennox, i scope it later22:32
rodrigodsusing v3scopedsaml22:32
rodrigodsthat was the "workflow" that worked: first get unscoped -> scope22:32
openstackgerritEric Brown proposed openstack/keystone: Removal of deprecated direct driver loading  https://review.openstack.org/35081522:32
*** ddieterly has joined #openstack-keystone22:32
jamielennoxrodrigods: yep - i never liked that workflow because it involves the user having to do multiple manual steps22:33
jamielennoxin ksa we fixed it22:33
rodrigodsyeah22:33
rodrigodsjamielennox, what i really don't like is that we have no idea of what auth plugins are available22:34
jamielennoxthe new one is v3samlpassword and you should be able to use it with a --project-name directly22:34
jamielennoxhmm so i expose it with ksa22:34
jamielennoxdtroyer: ^22:34
rodrigodswould be nice to have a osc command to display them22:34
jamielennoxdtroyer: what about an osc auth-info command22:34
jamielennoxi don't know how os-c-c would be involved, but you can easily list available plugins and the Opts they all provide22:35
dtroyerI thought we had something, in previous incarnations the list of plugins was in the help22:35
*** spzala has joined #openstack-keystone22:35
dtroyerit does need to be easier to find22:35
dtroyerand I'm not sure if the current incarnation still gets the help list right22:35
jamielennoxdtroyer: we do that horrible thing where all the auth opts are mashed together22:35
*** ddieterly has quit IRC22:35
dtroyerI'm trying to dump that22:36
jamielennoxdtroyer: i've had a few reviews to remove that, but i don't know if i proposed it this time around22:36
dtroyermay not make it for 3.0, but have gotten closer22:36
jamielennoxyea, i don't consider that a breaking change so just whenever22:36
dtroyereverything is different in the arg handling now with o-c-c and osc-lib in the mix22:36
jamielennoxyea, but o-c-c does so much now i don't know how to fix it22:37
dtroyerslowly, one refactor at a time ;)22:37
*** julim has joined #openstack-keystone22:37
jamielennoxwe can merge all the os-cloud stuff back into keystoneauth :)22:37
dtroyerI hacked through the major bits and use a subclass for the rest right now22:37
*** spzala has quit IRC22:38
*** spzala has joined #openstack-keystone22:38
jamielennoxso i merged some stuff to keystoneauth to help with all this22:38
jamielennoxOpt now has prompt so we can kill that guessing from osc22:39
jamielennoxi did a thing where you can have a loader choose different plugins based on what options are provided22:39
jamielennoxwhich should help abstract the use token_endpoint if --os-url/--os-token else use password22:40
jamielennoxbut i was going to say - can we just use straight 'password' plugin by default for OSC 3?22:40
jamielennoxtoken_endpoint should be less useful with doing keystone bootstrap22:40
dtroyerI think we've already done that, can't remember if it was in the 2.6.0 release or not, so yes ;)22:41
dtroyerall of the mucking about with options in OSC has been consolidated into the o-c-c- subclass in osc-lib, so now we just sort out what belongs where and we'll have this mess cleaned up22:41
jamielennoxi'm not touching it till you've decided you're finished22:43
jamielennoxtoo much stuff happening at the same time22:43
dtroyerwe're getting close to that point.  I'm sure you'll find things that can be simplified once this settles down22:50
*** KevinE has quit IRC22:54
*** sdake has joined #openstack-keystone23:03
*** Gorian_ has quit IRC23:27
*** tonytan4ever has quit IRC23:33
bretondstanek: which tests?23:34
bretondstanek: the ones with trusts?23:35
*** hoonetorg has quit IRC23:39
*** hoonetorg has joined #openstack-keystone23:44
*** roxanaghe has quit IRC23:47
*** jamielennox is now known as jamielennox|away23:48
*** jrist has quit IRC23:49
*** sdake has quit IRC23:51
*** ravelar has joined #openstack-keystone23:53
*** spzala has quit IRC23:55
*** code-R_ has quit IRC23:55
*** sdake has joined #openstack-keystone23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!