Tuesday, 2016-07-26

*** ravelar159 has joined #openstack-keystone00:02
*** ravelar159 has quit IRC00:10
*** nkinder has quit IRC00:11
*** woodburn1 has joined #openstack-keystone00:11
*** code-R has quit IRC00:12
*** woodburn has quit IRC00:13
*** roxanaghe has joined #openstack-keystone00:14
*** dan_nguyen has quit IRC00:14
*** roxanaghe has quit IRC00:18
*** ddieterly has joined #openstack-keystone00:22
henrynashstevemar: you around?00:29
stevemarhenrynash: you betcha00:29
henrynashstevemar: struggling with getting openstack client to run.....getting th e'ol Discovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL00:30
henrynashstevemar: which I thought had something to do with clouds.yaml00:30
*** ravelar159 has joined #openstack-keystone00:30
henrynashstevemar: but try as I might, can't seem to fix it00:31
stevemarhenrynash: paste the output of the same command using --debug?00:32
stevemarhenrynash: using devstack i assume?00:32
henrynashstevemar: devstack, yes00:32
henrynashstevemar: on devstack I seem to have version 2.2 of osc00:33
henrynashstevemar: whcih works fine00:33
henrynashstevemar: trying to run my dev version of osc in a another VM, attaching to the devstack VM00:33
*** adrian_otto has quit IRC00:35
henrynashstevemar: http://paste.openstack.org/show/541672/00:35
henrynashstevemar: is debug ouput from osc00:35
henrynashstevemar: ok, sorry, fixed it!00:38
*** ravelar159 has quit IRC00:38
henrynashstevemar: it actually was a mismatch...but the problem was actually in my env variables not in cloud.yaml...00:38
*** sdake has joined #openstack-keystone00:39
stevemarhenrynash: woo hoo00:40
henrynashstevemar: IP adress *.*.*.149 != *.*.*.249 :-)00:41
henrynashstevemar: shocking, that00:41
*** ravelar159 has joined #openstack-keystone00:44
*** ayoung has joined #openstack-keystone00:45
*** ChanServ sets mode: +v ayoung00:45
*** code-R has joined #openstack-keystone00:47
*** code-R_ has joined #openstack-keystone00:48
openstackgerritAdrian Turjak proposed openstack/keystone: adding totp support to password auth plugin  https://review.openstack.org/34342200:49
*** sdake_ has joined #openstack-keystone00:50
notmorganhenrynash: i did say it is often env var vs cli/etc ;)00:50
*** spandhe has quit IRC00:51
*** code-R has quit IRC00:51
*** sdake has quit IRC00:52
*** ravelar159 has quit IRC00:54
henrynashnotmorgan: yep!!!! I thought I had check them all....but then, whatdoyaknow.....00:54
*** sdake has joined #openstack-keystone00:55
*** sdake_ has quit IRC00:55
*** ravelar159 has joined #openstack-keystone00:56
*** adrian_otto has joined #openstack-keystone00:57
*** ayoung has quit IRC01:00
*** ravelar159 has quit IRC01:03
*** code-R_ has quit IRC01:06
*** code-R has joined #openstack-keystone01:06
*** tqtran has quit IRC01:09
*** clenimar_ has joined #openstack-keystone01:14
*** clenimar_ has quit IRC01:15
*** dan_nguyen has joined #openstack-keystone01:21
*** adrian_otto has quit IRC01:27
*** ddieterly has quit IRC01:31
*** hoonetorg has quit IRC01:35
*** ddieterly has joined #openstack-keystone01:38
openstackgerritClenimar Filemon proposed openstack/keystone: Improve os-federation docs  https://review.openstack.org/34709101:39
*** dan_nguyen has quit IRC01:39
*** ravelar159 has joined #openstack-keystone01:41
*** dikonoor has joined #openstack-keystone01:48
*** ravelar159 has quit IRC01:48
*** hoonetorg has joined #openstack-keystone01:49
*** sdake has quit IRC01:59
*** roxanaghe has joined #openstack-keystone02:02
*** KevinE has quit IRC02:02
*** KevinE has joined #openstack-keystone02:02
*** KevinE has quit IRC02:03
*** KevinE has joined #openstack-keystone02:03
*** julim has joined #openstack-keystone02:04
*** edmondsw has quit IRC02:05
*** catintheroof has quit IRC02:06
*** roxanaghe has quit IRC02:06
*** KevinE has quit IRC02:06
*** browne has quit IRC02:06
*** tqtran has joined #openstack-keystone02:07
*** ddieterly has quit IRC02:08
*** ravelar159 has joined #openstack-keystone02:09
*** tqtran has quit IRC02:11
*** davechen has joined #openstack-keystone02:19
*** code-R has quit IRC02:22
*** code-R has joined #openstack-keystone02:22
*** ravelar159 has quit IRC02:29
*** jistr has quit IRC02:32
*** jistr has joined #openstack-keystone02:33
openstackgerritTang Chen proposed openstack/keystone: Use assertEqual() instead of assertDictEqual()  https://review.openstack.org/34709702:35
*** gagehugo has quit IRC02:52
*** richm has quit IRC02:56
openstackgerritLance Bragstad proposed openstack/keystone: Switch fernet to be the default token provider.  https://review.openstack.org/34568803:11
openstackgerritMerged openstack/keystone: Fix v2-ext API enabled documentation  https://review.openstack.org/34696503:16
*** dikonoor has quit IRC03:29
*** dan_nguyen has joined #openstack-keystone03:37
*** josdotso has quit IRC03:39
*** dan_nguyen has quit IRC03:47
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Password expires validation  https://review.openstack.org/33336003:49
*** roxanaghe has joined #openstack-keystone03:50
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Password expires validation  https://review.openstack.org/33336003:51
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Password history requirements  https://review.openstack.org/32833903:52
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Password history requirements  https://review.openstack.org/32833903:54
*** roxanaghe has quit IRC03:54
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Minimum password age requirements  https://review.openstack.org/34331403:54
sorrisonHi keystone people, we just did an upgrade to mitaka and ended up rolling back04:02
sorrisonRaised a bug https://bugs.launchpad.net/keystone/+bug/160642604:02
openstackLaunchpad bug 1606426 in OpenStack Identity (keystone) "Upgrading to Mitaka casues significant slow down on user-list " [Undecided,New]04:02
sorrisonOne of those bugs you only see in prod due to the scale of our deployment04:03
*** links has joined #openstack-keystone04:03
*** roxanaghe has joined #openstack-keystone04:22
*** spandhe has joined #openstack-keystone04:31
stevemarsorrison: are there 2 bugs?04:32
stevemarsorrison: i assume changing the apache didn't hurt, otherwise you would have noticed this in project list too04:34
*** pcaruana has quit IRC04:34
sorrisonno just the one bug04:34
sorrisonyeah I don't suspect apache04:34
stevemarsorrison: this is a weird one04:35
sorrisonwe are going through other api calls to see if anything else affected04:35
sorrisonJust adding 10000 users to our dev instance04:35
stevemarmaybe its the shadow user stuff we added in mitaka04:36
sorrisonMy guess is there is now some iteration happening after the initial SQL query to get all users04:37
sorrisonHave seen similar mistakes in other projects04:37
sorrison(as opposed to doing a join)04:37
*** GB21 has joined #openstack-keystone04:40
*** dikonoor has joined #openstack-keystone04:42
stevemarsorrison: hmm, this is one of the few changes to the sql backend between kilo and mitaka https://github.com/openstack/keystone/blob/312a041862dc48b776715ccb2585d21cc479f5fb/keystone/identity/backends/sql.py#L20104:43
stevemaryou can use git blame to see the whole commit04:43
stevemarlots of things changed04:43
sorrisonIs there an index on the join key I wonder04:46
sorrisonforeign key is on it, I don't know enough about sql to know if that is enough04:46
*** nk2527 has quit IRC04:49
*** GB21 has quit IRC04:51
*** kaszkiet has quit IRC04:54
sorrisonOK I've PDB'd it to confirm and it's in the sql backend layer somewhere04:55
stevemarsorrison: :sadface:04:56
sorrisonthe call in identity/core.py "ref_list = driver.list_users(hints)" is the slow point04:56
sorrisonI'll step in there and get my hands dirty04:56
stevemarsorrison: that would be super appreciated04:56
*** davechen has quit IRC04:57
*** ntpttr- has quit IRC04:59
*** ntpttr- has joined #openstack-keystone05:03
*** GB21 has joined #openstack-keystone05:09
sorrisonOK the slow bit is the for loop at https://github.com/openstack/keystone/blob/master/keystone/identity/backends/sql.py#L7905:10
sorrisonstevemar: taking out the filter_user call doesn't make a difference eg just calling [x.to_dict() for x in user_refs] is slow05:10
sorrisonhints are {'limit': None, 'filters': [], 'cannot_match': False}05:11
sorrisonstevemar: Still not sure why as that code is in Kilo too so nothing much has changed there05:12
*** roxanaghe has quit IRC05:12
sorrisonexcept user_ref is now much bigger05:12
*** code-R has quit IRC05:14
*** code-R has joined #openstack-keystone05:14
sorrisonyeah I'm pretty sure it's just the converting them all to dicts that is slowing things down05:15
*** ntpttr- has quit IRC05:15
*** ntpttr- has joined #openstack-keystone05:17
sorrisonfor i in user_refs: foo = i runs quick but05:19
sorrisonfor i in user_refs: foo = i.to_dict()     slows down significantly05:19
openstackgerritAdrian Turjak proposed openstack/keystone: adding totp support to password auth plugin  https://review.openstack.org/34342205:42
*** GB21 has quit IRC05:49
openstackgerritTin Lam proposed openstack/keystone: Add schema validation to v2 create tenant  https://review.openstack.org/34659405:52
*** adriant has quit IRC05:57
*** GB21 has joined #openstack-keystone06:02
*** code-R has quit IRC06:09
*** code-R has joined #openstack-keystone06:12
*** nishaYadav has joined #openstack-keystone06:16
*** davechen has joined #openstack-keystone06:17
*** nishaYadav is now known as Guest879406:17
*** woodster_ has quit IRC06:19
*** ntpttr- has quit IRC06:24
*** ntpttr- has joined #openstack-keystone06:26
*** ntpttr- has quit IRC06:37
*** tesseract- has joined #openstack-keystone06:39
*** ntpttr- has joined #openstack-keystone06:39
*** spandhe has quit IRC06:39
*** Guest8794 is now known as nisha_06:46
*** tangchen has quit IRC06:46
*** pcaruana has joined #openstack-keystone06:55
*** tangchen has joined #openstack-keystone06:57
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/keystone: [WIP] Testing latest u-c  https://review.openstack.org/31843507:00
openstackgerritTang Chen proposed openstack/keystone: Use assertEqual() instead of assertDictEqual()  https://review.openstack.org/34709707:04
*** rcernin has joined #openstack-keystone07:06
*** tqtran has joined #openstack-keystone07:09
*** tqtran has quit IRC07:13
*** pnavarro has joined #openstack-keystone07:35
*** code-R_ has joined #openstack-keystone07:35
*** code-R has quit IRC07:38
*** tonytan4ever has quit IRC07:44
*** aastha has quit IRC07:49
*** code-R_ has quit IRC07:50
*** code-R has joined #openstack-keystone07:50
*** code-R has quit IRC07:59
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-keystone08:00
*** mvk has quit IRC08:24
*** pblaho has quit IRC08:32
*** pnavarro has quit IRC08:43
*** tonytan4ever has joined #openstack-keystone08:44
*** tonytan4ever has quit IRC08:49
*** mvk has joined #openstack-keystone08:51
*** nisha_ has quit IRC08:51
*** TxGVNN has joined #openstack-keystone08:55
*** pnavarro has joined #openstack-keystone09:02
*** tqtran has joined #openstack-keystone09:10
*** tqtran has quit IRC09:15
openstackgerritTang Chen proposed openstack/python-keystoneclient: Use assertEqual() instead of assertDictEqual()  https://review.openstack.org/34720809:16
*** GB21 has quit IRC09:22
*** TxGVNN has quit IRC09:34
*** TxGVNN has joined #openstack-keystone09:35
*** GB21 has joined #openstack-keystone09:40
*** hwcomcn has joined #openstack-keystone09:49
*** hwcomcn has quit IRC09:52
*** hwcomcn has joined #openstack-keystone09:53
*** mvk has quit IRC09:56
*** mvk has joined #openstack-keystone09:57
*** GB21 has quit IRC10:12
*** NishaYadav has joined #openstack-keystone10:13
*** NishaYadav is now known as nishaYadav10:14
*** nishaYadav is now known as nisha_10:15
*** NishaYadav has joined #openstack-keystone10:16
*** NishaYadav is now known as nishaYadav10:17
*** GB21 has joined #openstack-keystone10:18
*** nisha_ has quit IRC10:19
openstackgerritBoris Bobrov proposed openstack/keystonemiddleware: Fix description of option `cache`  https://review.openstack.org/34723410:30
*** ntpttr- has quit IRC10:30
*** ntpttr- has joined #openstack-keystone10:35
*** TxGVNN has quit IRC10:38
*** tonytan4ever has joined #openstack-keystone10:46
*** davechen has left #openstack-keystone10:49
*** tonytan4ever has quit IRC10:50
openstackgerritMaho Koshiya proposed openstack/python-keystoneclient: Add wrapper classes for return-request-id-to-caller  https://review.openstack.org/26118810:51
*** GB21 has quit IRC10:56
*** GB21 has joined #openstack-keystone11:00
*** rdo has quit IRC11:10
*** tqtran has joined #openstack-keystone11:12
*** tqtran has quit IRC11:16
*** GB21 has quit IRC11:21
*** GB21 has joined #openstack-keystone11:24
openstackgerritMikhail Nikolaenko proposed openstack/keystone: Retry revocation on MySQL deadlock  https://review.openstack.org/34492411:31
*** nishaYadav has quit IRC11:39
*** rdo has joined #openstack-keystone11:49
*** dkehn has quit IRC11:52
*** richm has joined #openstack-keystone11:54
*** GB21 has quit IRC11:57
*** GB21 has joined #openstack-keystone11:58
*** nishaYadav has joined #openstack-keystone12:11
dstanekgood morning keystone12:16
*** GB21 has quit IRC12:16
*** edmondsw has joined #openstack-keystone12:17
*** pauloewerton has joined #openstack-keystone12:23
*** ericksonsantos has joined #openstack-keystone12:27
*** julim has quit IRC12:35
*** rodrigods has quit IRC12:35
*** rodrigods has joined #openstack-keystone12:35
stevemarmorning dstanek12:35
stevemarand breton12:35
nishaYadavhey, morning o/12:36
nishaYadavstevemar, if you have some time, can you please look at this patch https://review.openstack.org/#/c/341612/12:41
patchbotnishaYadav: patch 341612 - python-keystoneclient - Improve implied-role functional tests12:41
*** catintheroof has joined #openstack-keystone12:46
*** ddieterly has joined #openstack-keystone12:51
*** jsavak has joined #openstack-keystone12:51
*** woodster_ has joined #openstack-keystone12:58
*** Xudong has joined #openstack-keystone13:00
*** Xudong has left #openstack-keystone13:01
*** ddieterly has quit IRC13:13
*** rdo has quit IRC13:18
*** julim has joined #openstack-keystone13:24
*** TxGVNN has joined #openstack-keystone13:32
*** ddieterly has joined #openstack-keystone13:43
*** links has quit IRC13:44
*** ayoung has joined #openstack-keystone13:48
*** ChanServ sets mode: +v ayoung13:48
*** tonytan4ever has joined #openstack-keystone13:52
*** ddieterly has quit IRC13:55
*** ddieterly has joined #openstack-keystone13:56
SamYaplemorning lbragstad13:56
*** code-R has joined #openstack-keystone14:02
dstanekgood morning, good afternoon, good evening and good night!14:03
*** rdo has joined #openstack-keystone14:03
openstackgerritMerged openstack/python-keystoneclient: Use assertEqual() instead of assertDictEqual()  https://review.openstack.org/34720814:04
*** nk2527 has joined #openstack-keystone14:04
stevemardstanek: i see what you did there truman14:06
dstanekstevemar: :-)14:07
dstaneki think it may be review time14:07
stevemardstanek: yep14:07
stevemarjamielennox|away: i removed your BPs from the meeting agenda14:07
stevemarfolks need to add to the meeting agenda :O14:08
*** ravelar159 has joined #openstack-keystone14:09
*** nkinder has joined #openstack-keystone14:10
*** ravelar159 has quit IRC14:13
*** tqtran has joined #openstack-keystone14:13
*** sdake has joined #openstack-keystone14:14
*** ravelar159 has joined #openstack-keystone14:14
*** gordc has joined #openstack-keystone14:16
*** sdake_ has joined #openstack-keystone14:18
*** ravelar159 has quit IRC14:18
*** tqtran has quit IRC14:18
*** ravelar159 has joined #openstack-keystone14:20
*** sdake has quit IRC14:20
bretonstevemar: i think i can confirm sorrison's issue14:20
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Minimum password age requirements  https://review.openstack.org/34331414:20
stevemarbreton: its a nasty one14:21
*** gagehugo has joined #openstack-keystone14:21
*** ravelar159 has quit IRC14:22
bretonstevemar: i've just created 10k users on Liberty. It takes 2 seconds to fetch on L and 15 on M.14:22
stevemarbreton: for i in 10000; openstack user create i14:22
*** ravelar159 has joined #openstack-keystone14:22
bretonstevemar: nope, takes 2+ hours14:23
bknudsonbreton: git bisect should help find the problem commit.14:23
stevemarbreton: oh, what did you do?14:23
bretonstevemar: had to curl them14:23
bretonbknudson: yep, will do now14:24
*** raildo has joined #openstack-keystone14:25
bretonbknudson: bisecting is a little hard because of migrations14:27
openstackgerritMerged openstack/keystone: Make it so federated tokens are validated on v2.0  https://review.openstack.org/34568514:28
*** rdo has quit IRC14:28
*** michauds has joined #openstack-keystone14:28
bknudsonbreton: you'd have to recreate 10k users every time :(14:28
bretonone wrong move and have to re-dump db14:28
bretonbknudson: or just dump the db14:29
lbragstaddolphm awesome writeup14:36
*** haplo37__ has joined #openstack-keystone14:36
*** roxanaghe has joined #openstack-keystone14:38
*** michauds_ has joined #openstack-keystone14:39
breton> for relatively minimal cost (no $1,200 tickets required)14:39
bretonyou have not seen plane tickets' price :p14:39
*** pnavarro has quit IRC14:40
*** ravelar159 has quit IRC14:40
*** hwcomcn has quit IRC14:40
*** adrian_otto has joined #openstack-keystone14:42
*** roxanaghe has quit IRC14:42
*** ravelar159 has joined #openstack-keystone14:44
*** slberger has joined #openstack-keystone14:45
*** dikonoor has quit IRC14:45
stevemarbreton: have you seen the cost for the summit entry o_O14:45
stevemarpycon is $600 USD14:46
bretonstevemar: meh, summit is free14:47
stevemarbreton: haha, for you and I :)14:47
stevemari wonder what percentage of the summit gets in for free?14:48
SamYaplestevemar: i have to guess alot14:48
stevemarspeakers + ATC + other / total14:48
SamYapleand you know there is early bird tickets for $300, and then group tickets companies can buy that can be pretty cheap ive heard14:49
stevemarSamYaple: hmm, yeah, at least 2,159 have one commit in newton so far: http://stackalytics.com/?metric=commits14:49
SamYaplethe ATC/Speaker/Alt codes dont associate with a single person and i know people share those too14:50
stevemaraustin had 7000 people14:50
openstackgerritAlexander Makarov proposed openstack/keystone: Unified delegation model  https://review.openstack.org/20848814:50
stevemari'd ballpark it at half?14:50
bretonsummits brag to have 7k+14:50
SamYapleat least half i would guess. the companies at the summit pay for most of it14:50
SamYapleso really its probably just venue costs anyway, not that high overhead14:50
stevemarwould be interesting to see the data14:51
SamYaplei bet tom would have access to it14:51
*** michauds_ has quit IRC14:54
bknudsonmidcycle in moscow!14:55
dstanekbknudson: pass14:55
*** spandhe has joined #openstack-keystone14:56
*** spandhe has quit IRC14:56
openstackgerritAlexander Makarov proposed openstack/keystone: Unified delegation model  https://review.openstack.org/20848814:56
*** rdo has joined #openstack-keystone14:57
*** adrian_otto has quit IRC14:57
bknudsondstanek: moscow, ohio?14:57
*** dikonoor has joined #openstack-keystone14:58
*** ravelar159 has quit IRC14:59
*** ravelar159 has joined #openstack-keystone14:59
amakarovbknudson, in January?15:00
bretonEurope would be ok15:00
amakarovbreton, ++15:02
amakarovwhat about Prague?15:02
*** nishaYadav has quit IRC15:03
*** rdo has quit IRC15:03
dstanekbknudson: i'd be down for ohio15:03
*** rcernin has quit IRC15:05
bretondid we have any problems with migration 89 -> 90?15:05
breton(1005, "Can't create table 'test_keystone.local_user' (errno: 150)")15:05
openstackgerritAlexander Makarov proposed openstack/keystone: Unified delegation assignment driver  https://review.openstack.org/29131815:08
*** Trident has joined #openstack-keystone15:11
openstackgerritLance Bragstad proposed openstack/keystone: Switch fernet to be the default token provider.  https://review.openstack.org/34568815:12
openstackgerritLance Bragstad proposed openstack/keystone: Make AuthWithTrust testable against uuid and fernet  https://review.openstack.org/34568615:12
openstackgerritLance Bragstad proposed openstack/keystone: Allow V2TestCase to be tested against fernet and uuid  https://review.openstack.org/34568715:12
*** danpawlik has quit IRC15:13
*** KevinE_ has joined #openstack-keystone15:17
openstackgerritLance Bragstad proposed openstack/keystone: Switch fernet to be the default token provider.  https://review.openstack.org/34568815:21
*** Guest51132 is now known as med_15:23
*** med_ has quit IRC15:24
*** med_ has joined #openstack-keystone15:24
*** pgbridge has joined #openstack-keystone15:25
*** code-R_ has joined #openstack-keystone15:25
*** code-R has quit IRC15:28
*** nkinder has quit IRC15:29
*** rdo has joined #openstack-keystone15:30
*** samueldmq has joined #openstack-keystone15:34
*** ChanServ sets mode: +v samueldmq15:34
samueldmqg'afternoon all15:34
lbragstadall recovered?15:37
samueldmqlbragstad: almost hehe15:37
lbragstadsamueldmq i'm working through your revocation changes15:37
lbragstadsamueldmq I have https://review.openstack.org/#/c/345688/6 passing locally so that might mean we can us it instead of https://review.openstack.org/#/c/319489/15:38
patchbotlbragstad: patch 345688 - keystone - Switch fernet to be the default token provider.15:38
patchbotlbragstad: patch 319489 - openstack-dev/devstack - Switch fernet back as the default token provider15:38
lbragstadsamueldmq I'm trying to step through your revocation changes to see if there is one that will get https://review.openstack.org/#/c/345688/6 passing by added a depends on from one of your patches15:39
patchbotlbragstad: patch 345688 - keystone - Switch fernet to be the default token provider.15:39
samueldmqlbragstad: hmm, yes ? so we making it as keystone default this cycle?15:39
samueldmqlbragstad: the one that disable caches, I couldn't get  a solution to the issue so far, just identified it15:40
lbragstadsamueldmq we should see the same issues regardless of it being the default in keystone or devstack - right?15:40
lbragstadsamueldmq when you disable the cache - it passed with fernet as the default right?15:40
samueldmqlbragstad: add depends on in 343875 (actually pep8 failing, I will fix in a bit)15:40
samueldmqlbragstad: yes, and yes15:41
lbragstadsamueldmq I can respin it15:41
*** EinstCrazy has joined #openstack-keystone15:41
lbragstadsamueldmq I already have some of the changes locally15:41
samueldmqlbragstad: awesome, thanks15:42
samueldmqlbragstad: also update the commit message, I think we can make it a proposal for now (disabling cache), as it is not affecting performance terribly15:42
samueldmqlbragstad: specially if we want this to merge this cycle... so we can circle back and work on adding caching later (again)15:43
lbragstadsamueldmq we also have to make sure it doesn't degrade performance a lot when there are tons of revocation events15:43
*** tesseract- has quit IRC15:43
lbragstadright now the current performances tests don't really take that into consideration15:43
samueldmqlbragstad: so 'check performance' is not that heavy?15:44
samueldmqlbragstad: I agree, we should do more tests... the ideal would be to us to fix that cache15:44
lbragstadsamueldmq ++15:44
lbragstadsamueldmq check performance will do a bunch of authenticate and validates15:44
lbragstadbut the test criteria doesn't populate keystone with a bunch of revocation events prior to running the test15:45
lbragstadsamueldmq that's actually something we could add here - https://github.com/lbragstad/keystone-performance/issues15:45
samueldmqlbragstad: nice, so tests could be more meaninful15:47
lbragstadsamueldmq right15:47
lbragstadsamueldmq i'm trying to track cases like that in issues so that we can pick them off as we go15:48
lbragstadwithout forgetting about them15:48
*** woodburn has joined #openstack-keystone15:49
lbragstadsamueldmq https://github.com/lbragstad/keystone-performance/issues/1215:49
*** woodburn1 has quit IRC15:50
openstackgerritLance Bragstad proposed openstack/keystone: Switch fernet to be the default token provider.  https://review.openstack.org/34568815:53
openstackgerritLance Bragstad proposed openstack/keystone: Remove cache from revoke subsystem  https://review.openstack.org/34387515:53
openstackgerritLance Bragstad proposed openstack/keystone: Switch fernet to be the default token provider.  https://review.openstack.org/34568815:56
*** BjoernT has joined #openstack-keystone16:01
*** jsavak has quit IRC16:05
*** jsavak has joined #openstack-keystone16:06
*** NishaYadav has joined #openstack-keystone16:09
dstanek /b 2316:09
*** NishaYadav is now known as nishaYadav16:09
*** EinstCrazy has quit IRC16:11
*** EinstCrazy has joined #openstack-keystone16:11
*** EinstCrazy has quit IRC16:16
*** richm has quit IRC16:16
*** nkinder has joined #openstack-keystone16:16
*** richm has joined #openstack-keystone16:16
*** adrian_otto1 has joined #openstack-keystone16:18
*** adrian_otto1 is now known as adrian_otto16:19
*** gyee has joined #openstack-keystone16:21
*** ChanServ sets mode: +v gyee16:21
*** nishaYadav has quit IRC16:23
*** nishaYadav has joined #openstack-keystone16:23
*** code-R_ has quit IRC16:33
*** EinstCrazy has joined #openstack-keystone16:33
*** code-R has joined #openstack-keystone16:33
*** tqtran has joined #openstack-keystone16:37
*** tqtran_ has joined #openstack-keystone16:39
*** tqtran has quit IRC16:41
*** samueldmq has quit IRC16:44
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Minimum password age requirements  https://review.openstack.org/34331416:46
*** code-R has quit IRC16:47
*** itisha has joined #openstack-keystone16:48
*** dan_nguyen has joined #openstack-keystone16:49
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Minimum password age requirements  https://review.openstack.org/34331416:50
*** EinstCrazy has quit IRC16:51
*** samueldmq has joined #openstack-keystone16:52
*** ChanServ sets mode: +v samueldmq16:52
*** mvk has quit IRC16:57
*** browne has joined #openstack-keystone16:57
*** jsavak has quit IRC16:59
*** ddieterly is now known as ddieterly[away]17:00
*** aastha has joined #openstack-keystone17:01
*** roxanaghe has joined #openstack-keystone17:03
*** chrisshattuck has joined #openstack-keystone17:05
*** adrian_otto has quit IRC17:07
*** Gorian_ has joined #openstack-keystone17:09
*** josdotso has joined #openstack-keystone17:10
*** code-R has joined #openstack-keystone17:12
*** dikonoor has quit IRC17:13
*** code-R_ has joined #openstack-keystone17:14
openstackgerritLance Bragstad proposed openstack/keystone: Use quotes consistently in token controller  https://review.openstack.org/34749317:15
*** code-R has quit IRC17:17
*** jsavak has joined #openstack-keystone17:24
henrynashjust checking....are we on for the keystone IRC meeting today?17:26
*** Krishna has joined #openstack-keystone17:28
josdotsoAsked this question over at #opesntack-sdks, but thought I'd post here, too since it's auth related: Hi folks.  I've installed from master branch into a fresh virtualenv clones of keystoneauth and python-openstackclient from review.openstack.org.  I'm getting this auth_ref error whilst attempting v3oidcpassword.  How can I squash this error? http://paste.openstack.org/show/542153/17:28
*** achanda has joined #openstack-keystone17:29
*** TxGVNN has quit IRC17:31
*** amakarov is now known as amakarov_away17:32
*** sdake_ is now known as sdake17:37
stevemarclenimar: want to toss up a new patch for https://review.openstack.org/#/c/319446/ ?17:41
patchbotstevemar: patch 319446 - python-barbicanclient - Use keystoneauth17:41
clenimarstevemar, sure. i'll take a look into it17:43
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Minimum password age requirements  https://review.openstack.org/34331417:43
*** spandhe has joined #openstack-keystone17:44
*** spandhe has quit IRC17:44
*** spandhe has joined #openstack-keystone17:46
bretonstevemar: sorrison: 312a041 introduced the regression :(17:51
patchbotbknudson: patch 278570 - keystone - Shadow users - Separate user identities (MERGED)17:51
stevemarclenimar: its pretty inactive :(17:52
*** mvk has joined #openstack-keystone17:52
*** tonytan4ever has quit IRC17:53
stevemarbreton: it's that damn outerjoin(LocalUser) calls17:54
stevemari actually don't know, but i did suspect shadow users :(17:54
*** jaugustine_ has joined #openstack-keystone17:56
*** shaleh has joined #openstack-keystone17:58
*** ddieterly[away] has quit IRC18:00
*** henrynash_ has joined #openstack-keystone18:02
*** ChanServ sets mode: +v henrynash_18:02
*** henrynash has left #openstack-keystone18:03
*** henrynash_ has left #openstack-keystone18:03
samueldmqstevemar: do we have meeting today?18:05
samueldmqstevemar: nvm, just joined18:05
*** adrian_otto has joined #openstack-keystone18:16
bretonindeed, to_dict takes the most time18:20
*** timcline has joined #openstack-keystone18:23
samueldmqbreton: where ?18:23
*** timcline_ has joined #openstack-keystone18:24
bretonsamueldmq: bug 160642618:24
openstackbug 1606426 in OpenStack Identity (keystone) "Upgrading to Mitaka casues significant slow down on user-list " [Critical,New] https://launchpad.net/bugs/1606426 - Assigned to Ron De Rose (ronald-de-rose)18:24
*** BjoernT is now known as Bjoern_zZzZzZzZ18:24
samueldmqbreton: thanks18:25
*** timcline has quit IRC18:27
*** adrian_otto has quit IRC18:30
*** sdake has quit IRC18:35
*** ddieterly has joined #openstack-keystone18:35
*** sdake has joined #openstack-keystone18:35
*** Bjoern_zZzZzZzZ has quit IRC18:38
*** tonytan4ever has joined #openstack-keystone18:43
*** spandhe has quit IRC18:46
*** adrian_otto has joined #openstack-keystone18:55
*** timcline_ has quit IRC18:56
*** itlinux has joined #openstack-keystone18:56
*** spandhe has joined #openstack-keystone18:59
*** Krishna has quit IRC19:00
jaugustine_Will do19:00
lbragstadjaugustine_ rderose lamt dstanek continue here?19:00
rderoseokay, so are we including LDAP in the lockout feature or not?19:00
lbragstadrderose I think we should hold off until we have more of a usecase?19:01
lbragstadfor the ldap lockout19:01
rderoselbragstad: agree, just want to make sure others are on board19:01
jaugustine_I'm on mobile but I can get the details it was asked how we would accomplish this so if it can be useful to others of course we would like to do it for the entire community19:01
lbragstadper dstanek's concern - it can be confusing if a user is locked out of one, or both systems19:01
lamtI understand that with the current ldap policy, anyone can lock the user out - but the security folks just want to make sure it is not keystone that's the cause of problem.19:02
rderoselbragstad: yeah, I think it's confusing19:02
lbragstadrderose how hard would it be add this in later?19:02
bretonstevemar: fetching attributes `name` and `password` take the most time19:02
lbragstadrefactoring the lockout into the Manager() layer?19:02
dstaneklamt: how do you prevent this in other applications?19:03
dstaneklamt: or why is keystone special?19:03
topollbragstad, dsatnek I agree the am I locked out of keystone or ldap (or both) could get nasty19:03
rderoselbragstad: currently adding new columns to local_user table, moving it up a layer would cause it to be in the user table19:03
topoldstanek ^19:03
*** fifieldt has quit IRC19:03
rderoselbragstad: so adding new columns; migrating the data potentially19:03
lbragstadrderose so it would require a data migration specifically to move it to the manager19:04
lbragstadrderose just curious if it was trivial or not19:04
openstackgerritEric Brown proposed openstack/keystone: Invalid tls_req_cert constant as default  https://review.openstack.org/34752319:04
rderoselbragstad: so yeah, not trivial if we change later19:04
*** haplo37__ has quit IRC19:06
dstanektopol: yeah, not sure this usecase has been fully thought though yet19:06
lamtdstanek: other applications that use LDAP has their own lock out count prevention.  The solution may be change on the LDAP side.19:07
lbragstadlamt which applications?19:07
topollamt do you have any  public examples?19:07
dstaneklamt: so you don't use ldap lockout at all?19:07
lamtThere is LDAP lockout.  E.g. a user signing into Windows desktop login leverages the corporate Active Directory19:08
lamtIf a user signs in a few times incorrectly, it locks19:08
dstaneklamt: so for example your webmail system would have a shorter lockout?19:09
rderoselamt: yeah exactly. I would expect the same behavior if you are logging into an app backed by AD.19:10
rderosedstanek: ++19:10
lamtdstanek: yes19:11
lbragstadlamt so what happens when a user gets locked out of the webmail application but not locked out of AD?19:11
dstaneklamt: do you have an example of what the lockouts would be?19:12
dstanekAD is set to 5 and all other applications are 3 (for an example)19:12
rderoselamt: hmm...  is that really the expected behavior, that all of these apps would implement a separate lockout feature. what's the point of having an IdP19:12
lamtlbragstad: the shorter lock out is time based, after a while it is unlocked.19:13
lbragstadlamt is there way to override that?19:13
lbragstadi believe rderose is providing a lockout override in addition to a time-based lockout for the sql stuff19:13
rderoselamt lbragstad: AD is time-based19:13
dstaneklamt: so you can still lockout a user from AD be using multiple apps?19:13
rderoselbragstad: correct, you can override by re-enabling the user19:14
lamtdstanek: yes, you can do that by just sending in multiple ldapsearch with wrong credentials19:14
lbragstadand that doesn't necessarily mean you've locked the user out of a *single* app19:14
dstaneklamt: then i'm confused...are you just making it a little harder to lockout?19:15
lamtThe idea is that if a lock out is to happen, it is prefer you are locked out of the single app over getting locked out of the active directory - which requires a service call to unlock19:15
*** sdake has quit IRC19:15
*** timcline has joined #openstack-keystone19:16
*** ddieterly is now known as ddieterly[away]19:16
*** timcline_ has joined #openstack-keystone19:17
*** fifieldt has joined #openstack-keystone19:18
lbragstadthere is still the possibility of a user getting locked out of AD if you have 5 applications that have a max auth count of 3 (each) and your AD lockout is 5.19:18
rderoselbragstad: yep19:19
lbragstadyou could auth from each application once and lock your AD account19:19
lbragstadlamt do your applications display different lockout messages for each case?19:20
*** timcline has quit IRC19:20
lamtlbragstad: agreed.  It is just that they do not want keystone to be the sole application that is the problem - but I understand the use case isn't bulletproofing an AD lockout.19:21
rderoseand for internal apps, users typically don't login, but their Windows account is automatically applied. in those cases, lockout happens at Windows login19:22
lamtCurrently, it does not, it returns an Unauthorized if the user is locked out of the AD.19:22
dstaneklamt: if they are locked out of keystone/openstack wouldn't them make a service call to unlock?19:23
lbragstadfor AD - lockout is time-based...19:24
lamtdstanek: yes, but the procedure is less onerous19:24
lbragstaderr... applications use AD only supply a time-based lockout19:24
lbragstadwhere as a lock out from AD requires a service call to re-enable the user19:24
dstaneklamt: how so?19:24
dstaneklamt: sorry about all the questions, but i really want to understand the usecase19:25
lamtit is not a problem - I understand the use case is wonky because the way the corporate security is structured19:25
dstanekfrom a user perspective since i don't know why i'm locked out i will make the same support call/ticket/whatever either way19:26
lbragstadthat works if the same team manages access to all applications19:26
lamtBecause the cloud team that runs the cloud is different from the corporate Active Directory team - to unlock keystone does not require the same # of hoops needed to unlock the corporate AD access.19:27
dstaneklamt: how does a user know what app they are locked out of? and do they have a different way to report the error based on that?19:27
lbragstadbut - it would be up to the application to display the proper error in that case19:27
dstaneki don't know if they is possible, but i want to recommend setting up an ldap server they AD replicates to, but has a different lockout policy19:28
lbragstadif the openstack deployment is managed by a different group than corporate IT, it could get confusing as to which group to go talk to from a user perspective19:29
lamtthat is a potential solution - the interaction with the corporate AD team can be very challenging.19:29
*** jaugustine_ has quit IRC19:30
bretonhow does mysql general log work?19:30
dstanekbreton: general log?19:30
lamtlbragstad: agreed19:30
dstaneklamt: how does it work now?19:31
lbragstadlamt in your case - is all that managed by the same group?19:31
bretondstanek: [mysqld]19:32
bretongeneral_log = on19:32
lamtCurrently, keystone (and the openstack clouds) infrastructure is managed by 1 group, and a separate group (the corporate security) owns the active directory/LDAP.19:32
dstanekbreton: i've never seen that before, so i'm no help19:33
bretonhttp://paste.openstack.org/show/542167/ this is what i see with shadow users19:33
lbragstadlamt so do all locked account requests go to only one group (the corporate security)?19:33
bretonare these actual db queries or just explanation how one big query works?19:33
dstaneklamt: so you know currently have any opps that implement lockout that are managed by other groups?19:33
rderosebreton: is this from the performance bug?19:34
*** gyee has quit IRC19:34
bretonrderose: yeah19:34
lamtlbragstad: yes19:34
rderosebreton: so separate queries for each table looks like19:35
rderosebreton: which is configurable19:35
dstaneklamt: so that team will essentially have to query AD and keystone to see where the user is locked and forward to the correct team?19:35
*** sdake has joined #openstack-keystone19:36
lbragstadlamt so regardlless of which application locks the user out - they all have to go to the same team to get unlocked19:36
lamtdstanek: I need to check.  There are just too many applications in ATT.19:36
bretonzzzeek: any ideas ^?19:36
zzzeekbreton: ideas and an arrow....lets follow....19:36
*** samueldmq has quit IRC19:36
lamtdstanek: Yes.19:38
zzzeekbreton: the general log...logs all the queries.  that paste is what's being sent to the DB.19:38
lamtlbragstad: if it is locked at the AD level, yes.  It is still very possible for a user to be locked out of the AD.19:39
zzzeekbreton: so...this paste shows like seven queries taht are keystone specific, all the "select 1"s are an oslo.db thing, and the stuff above is a handful of queries the SQLA dialect emits on first connect.19:39
dstaneklamt: you can actually be locked out of either or both19:40
lbragstadthat what it sounds like - but in either case it sounds like no matter what you have to go talk to the same team to get unlocked19:40
lamtyes, they are asking to find a way to minimize the chance of a lockout at the AD level.19:41
*** nisha_ has joined #openstack-keystone19:41
*** nishaYadav has quit IRC19:42
*** ddieterly[away] is now known as ddieterly19:42
lamtif one is to maliciously trying to lock you out of the AD, through multiple application, that's something corporate security has to solve.  They are asking for the case of someone writing a loop that repeatedly send incorrect credential to keystone would stop it at the application level , and not the corporate AD level.19:43
dstaneklamt: probably need to be a feature request on launchpad so that the conversation can be better captured19:48
lbragstaddstanek ++19:48
lamtdstanek: Thanks - I will do that.19:49
*** achanda has quit IRC19:49
dstaneklamt: spend more time of why you want to do this and whey the keystone implications are than on what you want to do19:50
openstackgerritNisha Yadav proposed openstack/python-keystoneclient: Improve docs for v3 roles  https://review.openstack.org/33454619:51
*** spandhe has quit IRC19:56
*** timcline_ has quit IRC19:57
*** timcline has joined #openstack-keystone19:58
*** slberger1 has joined #openstack-keystone20:03
*** spandhe has joined #openstack-keystone20:05
*** slberger has quit IRC20:05
*** chrisshattuck has quit IRC20:10
openstackgerritColleen Murphy proposed openstack/keystone: Add dummy domain_id column to cached role  https://review.openstack.org/34754320:10
openstackgerritColleen Murphy proposed openstack/keystone: Add dummy domain_id column to cached role  https://review.openstack.org/34754320:17
*** nisha_ has quit IRC20:18
bretonzzzeek: so these are separate queries? I have 10002 'FROM password' in the log, so there were 10002 queries?20:18
*** woodburn has quit IRC20:19
bretonand all are with the same id 3720:19
*** spandhe has left #openstack-keystone20:25
*** karthikb has joined #openstack-keystone20:28
bretonstevemar: rderose: yep, N queries are made for N users20:30
breton45 Query     SELECT password.id AS password_id, password.local_user_id AS password_local_user_id, password.password AS password_password  FROM password WHERE 3111 = password.local_user_id20:31
*** dkehn has joined #openstack-keystone20:31
*** henrynash has joined #openstack-keystone20:31
*** ChanServ sets mode: +v henrynash20:31
rderosebreton: yeah, I'm testing it out using eager loading instead20:31
zzzeekbreton: yes20:32
rderosebreton stevemar: that should fix it. just want to test it out any edge cases20:33
zzzeekbreton: keystone is big on many small queries.   receives lots of web requests.   this is why there's interest in more caching20:33
*** jsavak has quit IRC20:34
*** jsavak has joined #openstack-keystone20:35
bretonhere a single list_users produces N requests. I think that eager loading will help.20:35
rderosebreton: it should20:36
*** thiagolib has joined #openstack-keystone20:36
bretonzzzeek: thank you!20:38
bknudsonbreton: lbragstad: the list users is a good candidate for performance testing.20:41
*** slberger1 has quit IRC20:42
*** gyee has joined #openstack-keystone20:43
*** ChanServ sets mode: +v gyee20:43
*** karthikb has quit IRC20:43
*** josdotso has quit IRC20:46
*** slberger has joined #openstack-keystone20:47
*** itisha has quit IRC20:50
*** timcline has quit IRC20:52
*** catintheroof has quit IRC20:52
stevemarbreton: rderose what should fix it?20:56
*** adrian_otto1 has joined #openstack-keystone20:56
*** adrian_otto has quit IRC20:58
rderosestevemar: I think changing lazy='subquery' to eager loading would help: https://github.com/openstack/keystone/blob/master/keystone/identity/backends/sql_model.py#L3720:58
rderosestevemar: just trying to test this out first, to see if there is any side effects20:58
*** julim has quit IRC21:02
*** raildo has quit IRC21:03
bretonrderose: it probably won't help21:03
bretonrderose: the issue is in passwords21:03
*** timcline has joined #openstack-keystone21:03
bretonrderose: please see the query above21:04
rderosebreton stevemar: right, I would change to eager loading for all relationships21:04
rderosebreton stevemar: including passwords21:04
rderosebreton: yeah, the query above should be a joined query instead, this is configurable as well21:05
openstackgerritBoris Bobrov proposed openstack/keystone: Eagerly load local users and passwords  https://review.openstack.org/34755221:08
bretonrderose: like this ^?21:08
stevemarbreton: hehe21:09
bretonah dammit, forgot the import21:09
dstanekbreton: nice21:09
stevemarbreton: aren't lines 78 and 79 the same?21:10
rderosebreton: yep21:10
stevemaror, can we pass in query instead on line 79?21:10
stevemarbreton: what kind of gains are you seeing?21:11
bretonthe patch is written for stable/mitaka and it seems it won't work for master, fixing it now21:11
*** adrian_otto1 has quit IRC21:13
openstackgerritBoris Bobrov proposed openstack/keystone: Eagerly load local users and passwords  https://review.openstack.org/34755221:14
bretonstevemar: 30 seconds -> 5 seconds21:14
stevemarbreton: \o/21:14
bretonhttp://paste.openstack.org/show/542177/ and these queries in mysql log21:15
bretoni wonder if these 4 queries can be joined into 121:16
*** adrian_otto has joined #openstack-keystone21:17
bretonafter all apache instances got their first requests, it became ~2.7-2.8 seconds21:18
stevemarwould be nice if we could create a test that ensured none of the list operations took too long for 10K objects21:20
bretonwe could count queries like in https://review.openstack.org/#/c/344924/4/keystone/tests/unit/test_v3_os_revoke.py21:21
patchbotbreton: patch 344924 - keystone - Retry revocation on MySQL deadlock21:21
bretoni'll have some sleep, will be back in 9h21:23
*** rm_work has quit IRC21:25
*** mjb has quit IRC21:25
*** jsavak has quit IRC21:26
*** rm_work has joined #openstack-keystone21:26
*** jsavak has joined #openstack-keystone21:27
*** mjb has joined #openstack-keystone21:28
rderosebreton: you're too quick, I was doing something similar. will stop and let you finish fixing this bug.21:34
*** jsavak has quit IRC21:36
stevemarbreton: i'm jealous of your 9hrs21:38
* breton wanted to go, but ran into https://eng.uber.com/mysql-migration/21:42
*** ayoung has quit IRC21:45
*** ddieterly is now known as ddieterly[away]21:50
*** ddieterly[away] has quit IRC21:54
openstackgerritLance Bragstad proposed openstack/keystone: WIP: refactor views to be used by v2.0 and v3  https://review.openstack.org/34756121:54
*** ravelar159 has quit IRC21:56
*** adriant has joined #openstack-keystone21:57
*** pauloewerton has quit IRC22:00
*** darrenc is now known as darrenc_afk22:01
*** openstackgerrit has quit IRC22:03
*** openstackgerrit has joined #openstack-keystone22:03
*** gordc has quit IRC22:08
*** gagehugo has quit IRC22:09
*** sdake has quit IRC22:09
*** KevinE_ has quit IRC22:15
*** browne has quit IRC22:17
jdennisstevemar: were you able to verify your locale settings as per my response to you in https://review.openstack.org/#/c/343035/22:20
patchbotjdennis: patch 343035 - python-openstackclient - arguments are not locale decoded into Unicode22:20
openstackgerritMerged openstack/keystone: Add performance tuning documentation  https://review.openstack.org/34556622:23
*** jamielennox|away is now known as jamielennox22:23
*** edmondsw has quit IRC22:24
openstackgerritTin Lam proposed openstack/keystone: Add schema validation to v2 create tenant  https://review.openstack.org/34659422:28
*** darrenc_afk is now known as darrenc22:35
*** diazjf has joined #openstack-keystone22:36
*** diazjf has left #openstack-keystone22:36
*** timcline has quit IRC22:45
*** sdake has joined #openstack-keystone22:47
*** openstackgerrit has quit IRC22:48
*** openstackgerrit has joined #openstack-keystone22:48
*** sdake_ has joined #openstack-keystone22:49
*** sdake has quit IRC22:52
*** michauds has quit IRC22:55
*** browne has joined #openstack-keystone22:56
*** tqtran_ has quit IRC22:57
*** tqtran has joined #openstack-keystone22:57
*** thiagolib has quit IRC22:58
openstackgerritColleen Murphy proposed openstack/keystone: Add dummy domain_id column to cached role  https://review.openstack.org/34754322:59
*** slberger has quit IRC23:04
*** slberger has joined #openstack-keystone23:18
stevemarjdennis: i tinkered with it but couldn't get any further with OSC23:19
*** ddieterly has joined #openstack-keystone23:22
*** code-R_ has quit IRC23:27
Gorian_hey stevemar23:28
Gorian_you ever find out why openstack doesn't use a UUID for the region?23:28
*** ddieterly is now known as ddieterly[away]23:32
*** Gorian_ has quit IRC23:35
*** slberger has left #openstack-keystone23:37
*** ddieterly[away] has quit IRC23:42
*** tonytan4ever has quit IRC23:47
stevemarGorian: yo23:49
stevemarGorian: not really, just when making the APIs someone thought it was a good idea to not use them :\23:49
stevemardesign by committee ftw23:49
openstackgerritMerged openstack/keystone: Use quotes consistently in token controller  https://review.openstack.org/34749323:51
stevemarjamielennox: lemme know when you're around23:52
jamielennoxstevemar: i'm still in san jose - so for like 10 more minutes23:52
stevemarjamielennox: ha23:53
stevemarjamielennox: i've got a nasty one for ya: https://bugs.launchpad.net/keystone/+bug/160039323:53
openstackLaunchpad bug 1600393 in OpenStack Identity (keystone) "AttributeError: 'list' object has no attribute 'items'" [High,New]23:53
stevemar(referring to my comment #2)23:54
jamielennoxwhere is that coming from23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!