Wednesday, 2014-04-09

dolphmderek_c: it doesn't really matter where the code lives (although contrib is appropriate) -- you just need to reference it in your paste file (keystone.paste.ini)00:00
dolphmin etc/00:00
dolphmderek_c: and put it into your wsgi pipeline(s) wherever appropriate00:00
derek_cdolphm: great, I will give it a try.  thanks!00:02
*** elmiko_afk is now known as elmiko00:10
nkinderbknudson: I just added two minor comments to https://review.openstack.org/#/c/80401/00:19
nkinderbknudson: I'm fine if you choose not to address them.00:20
nkinderbknudson: the way the tests hash the tokens just seemed odd to me at first00:21
nkinderbknudson: it sounds like you may be changing the approach anyway given your discussion with dolphm above00:24
*** RockKuo has joined #openstack-keystone00:28
*** RockKuo_ has joined #openstack-keystone00:28
*** RockKuo_ has quit IRC00:28
*** browne has quit IRC00:29
*** dims has joined #openstack-keystone00:32
ayoungjamielennox, so...to solve the race condition, I thought the solution before was to do an atomic write of the files, right?  That should still be the case00:34
ayoungwhat is different this time around00:36
*** andreaf has quit IRC00:37
ayoungah...we don't do that for the token revocation list... dolphm ?00:37
*** lbragstad has joined #openstack-keystone00:38
ayoungno, we do atomic write for TRL, but verify it using the certificates which have not been fetched yet...00:38
ayoungTRACE keystoneclient.middleware.auth_token File "/opt/stack/new/python-keystoneclient/keystoneclient/middleware/auth_token.py", line 1275, in verify_signed_token  that is not what I have in my repo...maybe an old keystone client?00:41
ayoungdolphm, we don't keep a tag on the last release python-keystoneclient?00:42
bknudsonnkinder: if I keep that code around then I'll look at making that change.00:42
ayoungdolphm, I think that the problem with the race condition is an old version of python-keystoneclient00:45
ayoungah...disreagrd, that was the old log from an old bug report00:46
*** wchrisj has quit IRC00:50
*** elmiko has quit IRC00:51
dstanekwe have some tests make sure that the domain name unique constraint is case insensitive. is that the expectation of the operators or is that just an artifact of our dev environment?00:52
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: Hash functions support different hash algorithms  https://review.openstack.org/8620200:55
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: Support token hash algorithm  https://review.openstack.org/8039800:55
*** wchrisj has joined #openstack-keystone01:09
ayoungnkinder, check me on this:  http://logs.openstack.org/34/81834/8/check/check-tempest-dsvm-neutron-heat-slow/0976a2d/logs/screen-h-api.txt.gz#_2014-04-08_13_26_47_942  shows01:18
ayoungraise exceptions.CertificateConfigError(err)01:18
ayoungthat should be raised by common/cms.py and caught in auth_token middleware01:19
ayoungI'll link01:19
ayounghttps://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/middleware/auth_token.py#L120901:20
*** wchrisj has quit IRC01:20
ayoungits like it is not matching the exception01:20
ayoungboth are doing from keystoneclient import exceptions01:22
ayoung raise exceptions.CertificateConfigError(err)  on the cms side01:23
ayoungexcept exceptions.CertificateConfigError as err:  on the middleware01:23
*** ilives has joined #openstack-keystone01:24
nkinderayoung: being called to dinner.  Will check afterwards01:24
ayoungnkinder, thanks...01:24
ayoungdolphm, I don't think that http://logs.openstack.org/34/81834/8/check/check-tempest-dsvm-neutron-heat-slow/0976a2d/logs/screen-h-api.txt.gz#_2014-04-08_13_26_47_942    is the race condition.  I think it is a different bug01:25
ayoungdstanek, check me on this:   http://logs.openstack.org/34/81834/8/check/check-tempest-dsvm-neutron-heat-slow/0976a2d/logs/screen-h-api.txt.gz#_2014-04-08_13_26_47_942    shows a raise of an excpetion at   https://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/common/cms.py#L134    that should be caught at  https://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/middleware/auth_to01:29
ayoungken.py#L1211    but it is not01:29
ayounghttps://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/middleware/auth_token.py#L1211   was that last link01:30
jamielennoxayoung: anything i can help with?01:30
ayoungits like Python Class matching got  broken01:30
ayoungjamielennox, sure.  Anyone01:30
ayoungjamielennox, see the above01:30
jamielennoxi haven't been following01:30
ayoungjamielennox, its not a race condition01:30
ayoungits a failed exception handle01:30
jamielennoxthere was a change to exceptions overnight01:30
jamielennoxthey were moved into openstack.common.apiclient01:31
jamielennoxbut they should be compatible01:31
ayoungjamielennox, so an exception thrown from one piece of code is not caught by another01:31
ayoungthrown   https://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/common/cms.py#L13401:31
ayoungcaught https://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/middleware/auth_token.py#L121101:31
ayoungstack trace showing that not happinging  http://logs.openstack.org/34/81834/8/check/check-tempest-dsvm-neutron-heat-slow/0976a2d/logs/screen-h-api.txt.gz#_2014-04-08_13_26_47_94201:32
ayoungjamielennox, one piece of code is out of sync with the other?01:32
*** richm has quit IRC01:32
jamielennoxthat is.. weird01:34
ayoungjamielennox, so let me see the change01:35
jamielennoxayoung: it shouldn't matter cause they are explicitly throwing the same thing01:35
ayoungjamielennox, yeah.01:35
jamielennoxi thought it might have been some HTTP error edge case01:35
jamielennoxi can't think of even a race condition that would be a problem witht that01:37
ayoungjamielennox, It might be an older change.  I think they said they've been seeing this problem since the 31st or so01:37
ayoungbut the two logs on the bug report are both post that merge01:39
jamielennoxayoung: yea, but it's not one of the exceptions that was moved01:40
ayoungjamielennox, yeah01:40
jamielennoxobviously it's not reproducable01:42
*** gokrokve has joined #openstack-keystone01:42
jamielennoxayoung: oh, that makes sense01:42
jamielennoxline 121101:43
jamielennoxit's being re-raised01:43
jamielennoxso it is a race01:43
jamielennoxit does a verify and the files are missing01:43
jamielennoxsomething else is downloading them01:43
jamielennoxit checks to see if signing missing - nope it's there now01:43
jamielennoxit checks to see if cert_file is missing - nope, that's there as well01:44
ayoungAHA!01:44
ayoungyep01:44
jamielennoxprints the CMS verify output then re-raisese01:44
jamielennoxayoung: ages ago i had an object that did this downloding stuff01:45
jamielennoxi don't think it ever got through review01:45
ayoungOK...wonder that we never saw this before.01:46
ayounglike most bugs it goes from "why is this broken" to "how did this ever work?"01:46
jamielennoxhttps://review.openstack.org/#/c/38763/13/keystoneclient/utils.py01:47
jamielennoxhave a look at FetchableFile in there01:47
jamielennoxso maybe if we do a check for an object rather that a file, and let the object do things on __init__01:47
jamielennoxthen you can essentially use the presence of the object as a lock01:47
ayoungthat is too big a fix for this, though.01:48
jamielennoxyea, guess os01:48
ayoungmy code is incorrectly reporting an error when all is well.01:49
jamielennoxthe existing stuff you mean?01:49
ayoungwe need to correctly ID the error, and match the solution to the problem01:49
ayoungyeah01:49
jamielennoxno, it's right most of the time01:49
ayoungnow, we could drop the "raise" at the end.  But that would put us in an endless loop if the exception comes for some other reason01:50
jamielennoxwhich would be the majority of cases01:50
ayoungwhat if we allow it to run through once?01:51
ayoungat line 1220  something like01:51
jamielennoxit's hacky01:51
ayoungif (not rechecked)  : rechecked = True continue01:51
ayoung agreed. And we can do a more elegant solution like you proposed once we have some breathing room01:52
jamielennoxwe could just put in a lock01:54
ayoungwhere?01:54
ayoungin the exception block?01:54
jamielennoxhttp://eventlet.net/doc/modules/event.html01:54
ayoungbut this is not eventlet specific code01:54
jamielennoxsigning_cert = signing_cert_evt.wait()01:55
jamielennoxmeh event is wrong01:55
jamielennoxayoung: it's only going to occur on eventlet though right/01:55
ayoungnope01:56
jamielennoxayoung: no - because they are writing to the same place01:56
jamielennoxdamn01:56
ayoungthis is a multi threading issue for anything01:56
jamielennoxhmm, we could download per pid but that's not nice01:57
jamielennoxhttp://pythonhosted.org//lockfile/lockfile.html is in the global requirements01:59
jamielennoxi don't think the intention is you lock the file you are writing to01:59
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: auth_token middleware hashes tokens with sha256  https://review.openstack.org/8620601:59
jamielennoxbut you could use a temp01:59
jamielennoxat least to solve the immediate problem02:00
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: auth_token middleware hashes tokens with sha256  https://review.openstack.org/8039802:00
*** marcoemorais has quit IRC02:01
bknudsondolphm: ^ checks with both md5 and sha25602:02
ayoungbknudson, interesting02:02
bknudsonayoung: dolphm suggested it. seems to work02:03
ayoungbknudson, people are really bothered by this MD5 thing, huh?02:03
ayoungI mean, I understand it.  Its just not really an issue here.02:03
ayoungBut...yeah02:03
*** browne has joined #openstack-keystone02:05
ayoungbknudson, yeah, it makes sense....and it is very elegant02:06
bknudsonayoung: one enhancement is to add an option to skip the md5.02:06
ayoungfavor sha256 cuz that is what we are going with long term...drop md5 over time....is there a possibility of  sha256 getting the wrong token?02:06
ayounglike, keystone is using md5, but sha256 matches?02:07
bknudsonsha256 is longer02:07
derek_cdolphm: I added my module to keystone-paste.ini but am now getting this error: No section 'tfa_extension' (prefixed by 'app' or 'application' or 'composite' or 'composit' or 'pipeline' or 'filter-app') found in config /etc/keystone/keystone-paste.ini02:08
derek_cI tried do it exactly the same way as the ec2 module02:08
*** mberlin1 has joined #openstack-keystone02:08
derek_cso not sure what I'm supposed to add here02:08
*** mberlin has quit IRC02:08
bknudsonayoung: do you mean a hash collision? that could happen with 2 tokens with md5, too.02:08
ayoungbknudson, yep.02:09
*** zhiyan is now known as zhiyan_02:11
*** zhiyan_ is now known as zhiyan02:12
derek_cdoes anyone know the correct way to modify keystone-paste.ini in order to enable your own API extensions?02:13
bknudsonderek_c: you need to define the filter and then add it to the pipeline02:14
derek_cbknudson: I think I did that, yeah02:15
derek_cbknudson: http://upl.io/zb1vt102:15
derek_c"tfa_extension" is the extension I'm trying to add02:15
derek_cgetting this error when trying to run keystone though: error: No section 'tfa_extension' (prefixed by 'app' or 'application' or 'composite' or 'composit' or 'pipeline' or 'filter-app') found in config /etc/keystone/keystone-paste.ini02:16
ayoungderek_c, I think you want it before admin_service02:16
bknudsonderek_c: it might be because it's the last one in the list.02:17
ayoungjamielennox, tox is still runnning, but I think I am going to submit my hack02:17
derek_cwhy does the order matter?02:18
bknudsonderek_c: http://pythonpaste.org/deploy/#filter-composition02:19
bknudson"pipeline is a list of filters ended by an application"02:19
*** gyee has quit IRC02:47
derek_cI see. thanks guys!02:55
openstackgerritayoung proposed a change to openstack/python-keystoneclient: once more into the breach, good friends  https://review.openstack.org/8621203:01
ayoungbknudson, ^^ is quite possibly the ugliest code I have written this week.03:01
*** harlowja is now known as harlowja_away03:12
dstanekayoung: are you still having that issue?03:16
*** Guest_ has joined #openstack-keystone03:16
*** david-lyle has joined #openstack-keystone03:18
derek_cI'm trying to PATCH something from openstackclient; however the HTTP request apparently doesn't have context['token_id'] with it, resulting in a key error on the server side.  How do I make sure the token_id is part of the request?03:19
dstanekayoung: that exception is getting re-raised https://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/middleware/auth_token.py#L122103:22
*** stevemar has joined #openstack-keystone03:22
*** amcrn has quit IRC03:30
derek_cah, apparently my request does have "X-Auth-Token"03:31
derek_cbut for some reason, the "context" object does not contain a 'token_id' field, but only a 'X-Auth-Token' field03:32
derek_cdoes anyone know why this could happen?03:32
derek_coh!!03:32
derek_cI see, it's the order of the middleware that is wrong03:33
derek_cit seems like unstack and then rejoin-stack is not enough to make use of a new keystone-paste.ini?03:36
*** mjfork has quit IRC03:40
*** chandan_kumar has joined #openstack-keystone04:04
*** zhiyan is now known as zhiyan_04:06
*** topol has joined #openstack-keystone04:06
*** sld has left #openstack-keystone04:35
*** wchrisj has joined #openstack-keystone04:37
*** wchrisj has quit IRC04:52
*** chandan_kumar has quit IRC05:00
*** henrynash_ has joined #openstack-keystone05:06
*** chandan_kumar has joined #openstack-keystone05:14
*** marcoemorais has joined #openstack-keystone05:24
*** marcoemorais1 has joined #openstack-keystone05:26
*** marcoemorais has quit IRC05:28
*** zhiyan_ is now known as zhiyan05:31
*** gokrokve has quit IRC05:32
*** stevemar has quit IRC05:43
*** gokrokve_ has joined #openstack-keystone05:44
*** topol has quit IRC05:54
*** henrynash_ has quit IRC05:55
openstackgerritJenkins proposed a change to openstack/keystone: Imported Translations from Transifex  https://review.openstack.org/8395506:01
*** Guest_ has quit IRC06:05
*** Guest_ has joined #openstack-keystone06:06
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Session Bindings  https://review.openstack.org/8623706:14
derek_cI created several columns for sql.User by using a migration script in common/sql/migrate_repo.  However, seems like the columns I created in the migration script are not directly accessible from a User object06:16
*** inc0 has joined #openstack-keystone06:21
*** inc0 has quit IRC06:26
*** tomoiaga has joined #openstack-keystone06:34
*** jamielennox is now known as jamielennox|away06:35
*** gokrokve_ has quit IRC06:36
*** dims has quit IRC06:48
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Sync test_migrations  https://review.openstack.org/8061807:01
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Redundant unique constraint  https://review.openstack.org/8444707:01
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Corresponding `nullable` value.  https://review.openstack.org/8444607:01
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Compatible server default value in the models.  https://review.openstack.org/8444507:01
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Explicit foreign key indexes.  https://review.openstack.org/8444407:01
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Make it possible to use multiprocess file locks  https://review.openstack.org/8444807:01
openstackgerritIlya Pekelny proposed a change to openstack/keystone: WIP: Comparision of database models and migrations.  https://review.openstack.org/8063007:01
*** dims has joined #openstack-keystone07:02
*** Guest_ has quit IRC07:05
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Sync test_migrations  https://review.openstack.org/8061807:08
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Redundant unique constraint  https://review.openstack.org/8444707:08
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Corresponding `nullable` value.  https://review.openstack.org/8444607:08
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Compatible server default value in the models.  https://review.openstack.org/8444507:08
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Explicit foreign key indexes.  https://review.openstack.org/8444407:09
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Make it possible to use multiprocess file locks  https://review.openstack.org/8444807:09
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Comparision of database models and migrations.  https://review.openstack.org/8063007:09
*** Guest_ has joined #openstack-keystone07:09
*** henrynash has joined #openstack-keystone07:11
*** Guest_ has quit IRC07:16
openstackgerritsong (bruce) zhang proposed a change to openstack/keystone: replace word 'by' with 'be'  https://review.openstack.org/8624607:18
openstackgerritsong (bruce) zhang proposed a change to openstack/keystone: replace word 'by' with 'be' Related-Bug:#1304834  https://review.openstack.org/8624607:22
openstackgerritJuan Antonio Osorio Robles proposed a change to openstack/keystone: Refactor: moved flatten function to utils  https://review.openstack.org/8572107:24
*** Gue______ has joined #openstack-keystone07:26
*** gokrokve has joined #openstack-keystone07:36
*** marekd|away is now known as marekd07:38
*** Gue______ has quit IRC07:39
*** Gue______ has joined #openstack-keystone07:39
*** gokrokve has quit IRC07:41
*** Guest_ has joined #openstack-keystone07:51
*** Gue______ has quit IRC07:53
*** Guest_ has quit IRC07:53
*** Guest_ has joined #openstack-keystone07:54
*** Guest_ has quit IRC07:54
openstackgerritSergey Nikitin proposed a change to openstack/keystone: Some methods in ldap were moved to superclass  https://review.openstack.org/8625008:05
*** leseb has joined #openstack-keystone08:15
*** d0ugal has quit IRC08:18
*** d0ugal has joined #openstack-keystone08:21
*** gokrokve has joined #openstack-keystone08:37
*** gokrokve has quit IRC08:42
*** chandan_kumar has quit IRC08:49
*** mberlin1 is now known as mberlin09:02
*** chandan_kumar has joined #openstack-keystone09:03
*** morganfainberg is now known as morganfainberg_Z09:16
*** david-lyle has quit IRC09:20
*** gokrokve has joined #openstack-keystone09:38
*** gokrokve has quit IRC09:42
*** zhiyan is now known as zhiyan_09:43
*** marcoemorais1 has quit IRC09:43
*** chandan_kumar_ has joined #openstack-keystone10:03
*** chandan_kumar has quit IRC10:06
*** marcoemorais has joined #openstack-keystone10:11
*** marcoemorais has quit IRC10:12
*** marcoemorais has joined #openstack-keystone10:12
*** marcoemorais has quit IRC10:17
*** dims has quit IRC10:29
*** chandan_kumar_ has quit IRC10:32
*** gokrokve has joined #openstack-keystone10:39
*** gokrokve has quit IRC10:43
*** chandan_kumar_ has joined #openstack-keystone10:58
*** topol has joined #openstack-keystone11:08
*** RockKuo has quit IRC11:09
*** marcoemorais has joined #openstack-keystone11:13
*** marcoemorais has quit IRC11:17
*** jamielennox|away is now known as jamielennox11:30
*** jamielennox is now known as jamielennox|away11:33
*** dims has joined #openstack-keystone11:39
*** leseb has quit IRC11:48
*** leseb has joined #openstack-keystone11:50
afaranhaHello, does somebody knows how to retrieve the keystone url from the auth_token in API?11:59
*** dims has quit IRC12:03
*** d0ugal_ has joined #openstack-keystone12:04
*** d0ugal has quit IRC12:07
openstackgerritSergey Nikitin proposed a change to openstack/keystone: Code which get elements of tree in ldap moved to one common method  https://review.openstack.org/8630212:09
*** zhiyan_ is now known as zhiyan12:10
*** diegows has joined #openstack-keystone12:11
afaranhaI want to use keystoneclient in nova api. Should I use "keystoneclient.Client(token=context.auth_token, auth_url=auth_url)"? If so how do I get the "auth_url"? Thank you12:12
*** marcoemorais has joined #openstack-keystone12:14
*** marcoemorais has quit IRC12:18
tomoiagaafaranha: you can check how horizon did it for example: https://github.com/openstack/horizon/tree/master/openstack_dashboard/api12:18
*** d0ugal_ is now known as d0ugal12:23
afaranhatomoiaga: It uses a request to instantiate the keystone, I'll try to use the "request_id" from context, should it work? Thank you12:26
tomoiagarequest is used for context data (conatins different things), however, you should know the keystone url when you authenticate, after that you should request the catalog. In horizon I seem to remember they are doing that (trying to find endpoints where to send requests)12:29
tomoiagaafaranha: as mentioned, you should know the auth_url, if not, nova should have it somewhere and you should be able to use it. See how nova does the token verification12:30
*** gokrokve has joined #openstack-keystone12:40
tomoiagaafaranha: as far as I can see, horizon looks at the settings file for the auth_url which seems a normal thing to do: auth_url = getattr(settings, 'OPENSTACK_KEYSTONE_URL')12:41
tomoiagaafaranha: also this may help, at least it did help me out when I first looked at how to use the clients in a python script: http://www.ibm.com/developerworks/cloud/library/cl-openstack-pythonapis/index.html?ca=drs-12:43
*** wchrisj has joined #openstack-keystone12:44
afaranhatomoiaga: As I'm seeing in Nova for neutro_url it uses this: "CONF.neutron_admin_auth_url" I think I could work with both and see how will work in my case. Thank you :)12:44
*** gokrokve has quit IRC12:45
tomoiagaafaranha: the rest of the urls except keystone's auth_url, can be gathered from the keystone catalog.12:45
dolphmdstanek: did you look into https://bugs.launchpad.net/cinder/+bug/1285833 ?12:46
uvirtbotLaunchpad bug 1285833 in python-keystoneclient "Keystone client racing on certificate lookups causing 401 Unauthorized on API calls" [Critical,In progress]12:46
dolphmtomoiaga: ignoring regions, auth_url is the public identity endpoint12:46
afaranhatomoiaga: Yes, having the keystone it seems really easy to get others informations12:46
tomoiagadolphm: indeed12:47
*** raildo has joined #openstack-keystone12:47
afaranhatomoiaga: I hope now this work. I have many option to try now (I can also try to use os.environ)12:47
dstanekdolphm: i didn't realize there was a bug for it, but i responsed to a query from ayoung last night12:48
dolphmdstanek: that's why i asked12:48
dstanekdolphm: ayoung: that exception is getting re-raised https://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/middleware/auth_token.py#L122112:48
dolphmdstanek: should it not be?12:49
dstanekdolphm: not sure i'll read the bug12:49
raildodolphm: can you explain to me, how policy(not policy.json, the policy service) wokrs in keystone?12:49
dolphmdstanek: the original bug description has been fixed12:50
dolphmdstanek: but there's an elastic recheck query that's suddenly seeing the same backtrace again12:50
dolphmraildo: what do you mean by "policy service" ?12:50
dolphmraildo: /v3/policies ? oslo.policy enforcement?12:51
raildodolphm: http://api.openstack.org/api-ref-identity.html#Policy_Calls12:51
dolphmraildo: it's not really used by openstack (yet), but could be consumed by oslo.policy to centralize policy management https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v3/src/markdown/identity-api-v3.md#policy12:51
*** thiagop has joined #openstack-keystone12:55
raildodolphm: thanks12:55
*** dims has joined #openstack-keystone12:56
ayoungdolphm  I think it says something about our code reviewers that they can even catch misquoted Shakespear.13:00
ayoungShakespeare  that is13:00
dims:)13:00
dstanekdolphm, ayoung: i wonder if the filenames don't match in cert_file_missing...like a symlink or something13:01
ayoungdstanek, no, I think it is a race13:02
dolphmayoung: caused by what?13:02
ayoungdstanek, two threads each download the files.13:02
ayoungor rather,  two threads each get an exception due to a missing file13:02
ayoungthe first one downloads the file  (the second one to be checked) and then the second thread hits the condition where it checks for existence13:03
dstaneksay that happens..won't they both download and write the file in an atomic way?13:03
ayoungbut now it does exist, so it falls through to the 'raise'13:03
dolphmoh.13:03
ayoungdstanek, the problem is in the error recover section13:03
dstanekah13:03
dstaneklol, yeah that would do it13:03
ayoungdstanek, jamielennox|away gets credit for realizing it13:04
ayounghence. https://review.openstack.org/#/c/86212/13:04
dolphmso how do we eliminate the race altogether, rather than just trying to ignore it?13:05
ayoungdolphm, I see ^^ as a stopgap13:05
dolphmunconditionally call self.fetch_signing_cert() and self.fetch_ca_cert() ?13:05
ayoungdolphm, two ways:  either we lock upon fetch, or we fetch at startup13:05
ayoungproblem with fetch at startup is if the keystone server is not up, we can't start13:05
ayoungthis is in some ways an artificial problem.  We see it in testing, but not on a live system.  It happens, once, early on and that is it13:06
dstanekayoung: why check at all if the file exists? if the error message says it doesn't just download it13:06
ayoungdstanek, there are two different files to download.  And then the download could fail, so we need some way to avoid an endless loop13:07
ayoungdstanek, this code was written to make the expected case fast.  And I think we should keep that philosophy13:07
dolphmayoung: why wouldn't it happen in a live system..?13:07
ayoungdolphm, it might happen once, but then the certs would be in place, and all would work13:08
dstanekayoung: it just feels really complicated to me; the loop even13:08
ayoungdstanek, checking for the existence of the file is a blocking operation13:08
ayoungthe popen itself is already expensive13:08
openstackgerritDolph Mathews proposed a change to openstack/python-keystoneclient: eliminate race condition  https://review.openstack.org/8632113:09
ayoungthis code is executed on every request, so we need to streamline the expected case as much as possible13:09
ayoungdolphm, you've just written an infinite loop if the keystone server is down13:09
dolphmayoung: _fetch_cert_file will raise an unhandled exception and break the loop13:10
dolphmayoung: i can remove cert_file_missing() though now13:11
dstaneki don't see the need for the while True13:11
dstanekwe don't really want to try the operation more that twice anyway13:11
dolphmdstanek: ++13:12
openstackgerritDolph Mathews proposed a change to openstack/python-keystoneclient: eliminate race condition  https://review.openstack.org/8632113:13
dolphmeliminated while loop13:13
dolphmalso restored the warning log, but i'm not sure it's necessary since it's raising anyway13:14
dolphmwhoops, one sec13:15
dstanekdolphm: i was thinking something slightly different13:15
dolphmdstanek: ?13:15
dstanekdolphm: ah no, nm - didn't see the recursive call13:16
openstackgerritDolph Mathews proposed a change to openstack/python-keystoneclient: eliminate race condition  https://review.openstack.org/8632113:16
dolphmdstanek: i forgot to actually use 'retry' until ^13:16
dstanekwell, that explains why i didn't see it :-)13:17
dolphmi mean, i meant to replace the race condition with infinite recursion13:17
ayoungdolphm, that last   versions looks about right13:18
ayoungso if there are no certs,  fetch each one, and then retry13:18
*** henrynash has quit IRC13:18
ayoungif the fetch fails, we'll error out13:18
dolphmdamn, i introduced another bug...13:19
dstanekdolphm: what is the value of the 'return True'?13:19
*** topol has quit IRC13:19
ayoungif they succeed, either by this thread or another,  the validate will process as per normal13:19
dolphmdstanek: i wanted to retry if either cert got updated successfully... but i'm not doing that (i'm only refetching the first one, then retrying)13:19
*** henrynash has joined #openstack-keystone13:20
ayoungdolphm, change or to and13:20
ayoungor better yet13:20
ayoung except exceptions.CertificateConfigError as err:13:20
ayoungif retry:13:20
ayoungself.fetch_signing_cert()13:20
ayoung self.fetch_ca_cert())13:21
dolphmayoung: i just wrote that lol13:21
dstanekif either fails an exception will be raised so you can probably just write them as separate statements instead of an if13:21
ayoungdolphm, cool13:21
dolphmdstanek: and about to remove the return True13:21
dolphmdstanek: ++13:21
openstackgerritDolph Mathews proposed a change to openstack/python-keystoneclient: eliminate race condition  https://review.openstack.org/8632113:21
dstanekdolphm: this is what i was originally thinking http://paste.openstack.org/show/75413/13:24
dstanekbut i have an infinte recursion problem :-(13:25
dolphmdstanek: lol is there any advantage to avoiding recursion?13:25
dstanekoh, nm i pasted you the second link13:25
dolphmdstanek: yeah i don't see infinite recursion...13:25
dstanekin that one i use the nested function to avoid the extra param13:26
dstanekdolphm: i pasted a few diff revisions and i thought i gave you the broken one :-)13:26
dolphmdstanek: so you're just not a fan of exposing retry=True in the method signature?13:27
dstanekif we think it'll be useful outside of avoiding infinite recursion, i don't mind13:28
dstanekbut if it's really internal book keeping exposed as an argument, i'm sad13:28
dolphmdstanek: i can get behind that, but we do it in several places in auth_token13:28
openstackgerritSergey Nikitin proposed a change to openstack/keystone: Fixed wrong behavior in method search_s in BaseLdap class.  https://review.openstack.org/8632513:29
dstanekyeah i saw that and the other 'while True'13:30
*** joesavak has joined #openstack-keystone13:30
dolphmdstanek: i don't think you should escalate the log to ERROR though - this will occur on every devstack run at least once13:31
dolphmdstanek: it's just the only place we fetch certs13:31
openstackgerritDolph Mathews proposed a change to openstack/python-keystoneclient: eliminate race condition  https://review.openstack.org/8632113:33
dstanekyeah, that makes sense13:33
dolphmdstanek: do you use dstanek@dstanek.com for gerrit?13:34
dstanekdolphm: yep13:34
dstanekdolphm: that looks good13:34
dolphmdstanek: i'm adding you as a co-author13:35
dolphmayoung: and jamie lennox figured out the root cause?13:35
openstackgerritDolph Mathews proposed a change to openstack/python-keystoneclient: eliminate race condition fetching certs  https://review.openstack.org/8632113:35
ayoungdolphm, don't need the retry param13:36
openstackgerritDolph Mathews proposed a change to openstack/python-keystoneclient: eliminate race condition fetching certs  https://review.openstack.org/8632113:37
dolphmayoung: ^ also didn't need 'as err'13:37
*** vhoward has left #openstack-keystone13:37
ayoungyou are going to have this down to a one liner if we keep this up13:37
ayoungdolphm, I think we want err.output13:38
ayoungthat is what tells you the actual problem13:38
dolphmayoung: it'll get unconditionally raised if it still occurs in the second verify()13:38
dolphmayoung: so you'll get an exception log13:38
dolphmayoung: right?13:39
*** marcoemorais has joined #openstack-keystone13:39
ayoungdolphm, yeah, but I don;t think we actually get the problem message unless we explicitly print output13:39
ayounghttp://logs.openstack.org/34/81834/8/check/check-tempest-dsvm-neutron-heat-slow/0976a2d/logs/screen-h-api.txt.gz#_2014-04-08_13_26_47_94213:40
dstanekwe could add a try around the second verify to log13:40
dstanekand then reraise13:40
ayoungthe real message is ERROR keystoneclient.middleware.auth_token [-] CMS Verify output: Error loading file /var/cache/heat/cacert.pem13:40
ayoung140047624885920:error:02001002:system library:fopen:No such file or directory:bss_file.c:169:fopen('/var/cache/heat/cacert.pem','r')13:40
ayoungand that is in the output string13:40
ayoungbut the exception itself only produces the stack trace underneath13:40
*** marcoemorais1 has joined #openstack-keystone13:41
*** gokrokve has joined #openstack-keystone13:41
dolphmayoung: gotcha, adding another try/except lol13:41
ayoungdolphm, just print it out before reraising, I think13:41
dolphmayoung: ++13:42
openstackgerritDolph Mathews proposed a change to openstack/python-keystoneclient: eliminate race condition fetching certs  https://review.openstack.org/8632113:42
dolphmayoung: `^13:42
*** marcoemorais has quit IRC13:43
ayoungAHHH@!13:43
ayoungI can't keep up13:43
ayoungAh, nevermind, that was the version I +2ed13:44
dolphmayoung: then you were really fast13:44
tomoiagadoes it make sense to create a keystone auth plugin if I want to allow an already authenticated user in a django app, access to services ? (users will live in keystone and in the django app also)13:44
*** gokrokve has quit IRC13:45
ayoungdolphm, well, it is a short patch, and I was just looking for that diff...I think we are good.13:45
*** marcoemorais1 has quit IRC13:45
dstanekdolphm: nice, thanks13:46
*** chandan_kumar_ has quit IRC13:46
dolphmtomoiaga: users shouldn't live in more than one place, but yes13:46
openstackgerritDavid Stanek proposed a change to openstack/keystone: Moves test database setup/teardown into a fixture  https://review.openstack.org/8565113:47
dstanekdolphm: ayoung: gating our unit tests on a real database like mysql/postgres...wishlist or must have?13:48
ayoungdstanek, I would say it is the sign of a mature product?13:49
ayoungSo...somewhere in between13:49
ayoungWe've dealt without it this long, but we've been lucky13:49
dolphmdstanek: high priority wish list?13:49
dstanekok, i'll make a blueprint for it13:50
*** andreaf has joined #openstack-keystone13:50
dolphmdstanek: for bonus points, you can run mysql in memory :)13:50
dstaneki thought i could fix some tests but ran into lots of issues13:50
ayoungIsn't there one already?13:50
dstanekayoung: not sure, but i'll search for it first13:51
ayoungdstanek, thought I wrote one.  I write a lot of blueprints13:51
tomoiagadolphm: that's true. I would like to keep users in keystone to be able to manage the permissions/domains/projects from there and not from the django app.13:51
dolphmdstanek: i take it back, running mysql in memory instead of using innodb is probably just as useless as sqlite....13:51
dstanekdolphm: ayoung: one of my queries from last night:13:51
dstanekwe have some tests make sure that the domain name unique constraint is case insensitive. is that the expectation of the operators or is that just an artifact of our dev environment?13:52
ayoungdstanek, strange.  that is news to me13:52
dolphmdstanek: did i write that? :-/13:52
ayoungdstanek, that whole effort was driven by gyee.13:52
ayoungI'd ask him13:52
dstanekoops i mean case sensitive13:52
ayoungHmmm13:53
dstanekdolphm: it's in a bunch of different places, but mostly test_backend.py13:53
ayoungprobably not what we want, is it13:53
*** ayoung is now known as ayoung-afk13:53
dolphmdstanek: i think those tests were the result of a mailing list conversation complaining that we were case insensitive somewhere13:53
dolphmdstanek: and it turned out to be an issue with their mysql config13:54
*** chandan_kumar_ has joined #openstack-keystone13:54
dolphmor whatever their backend was, i don't recall13:54
dolphmdstanek: the tests simply illustrate the current intent, as we make no effort to handle case insensitivity properly13:54
dstanekmysql by default won't allow 'Default' and 'default' to co-exist as domains, but our code expects that to work13:55
dstanek well test code, our code code doesn't care13:55
dolphmdstanek: i'd care more about user names than domains13:55
dstanekdolphm: are 'dstanek' and 'DStanek' different users?13:55
dolphmdstanek: they are today, right?13:56
dstanekin sqlite yes, in mysql no13:56
dstanekat least in the devstack mysql13:56
dstanekwell i take that back - in the way our models generate the schema during test runs13:56
dstanektesting devstack itself now13:57
dolphmdstanek: aren't they varchar columns?13:57
dolphmdstanek: (maybe i'm out of date on this issue, or misunderstood it)13:57
*** topol has joined #openstack-keystone13:58
dstanekdolphm: yes, they are varchar with a unique constraint13:59
dstanekdolphm: http://paste.openstack.org/show/75420/13:59
dstanekdolphm: our unit tests expect that to be possible13:59
dstanekdolphm: http://git.openstack.org/cgit/openstack/keystone/tree/keystone/tests/test_backend.py#n179114:00
dstaneki'm going to file a bug for this, but i was hoping to get a little insight first14:01
*** samuelmz has joined #openstack-keystone14:04
*** gokrokve has joined #openstack-keystone14:06
tomoiagadolphm: the external authentication seems to be doing something similar. Users are "kept" in two places (httpd and keystone). Not sure how else to approach this yet.14:09
afaranhatomoiaga: It's possible to get the endpoints by the request headers, it's a field with the catalog called "X-Service-Catalog". I could get the keystone url this way: "req.headers.get('X-Service-Catalog', req.headers.get('X_STORAGE_TOKEN'))".14:09
afaranhaThe problem is that the catalog don't show keystone v3, only v2. To use it I just need to supply a v2 token? (I tried it but got error =/)14:09
tomoiagaafaranha: you need to define v3 endpoints (for services that support this) in keystone if you want them to show up14:09
dolphmdstanek: well i'd say mysql's behavior is preferrable then14:10
samuelmzHey guys, I took a look at the multitenancy proposal at https://wiki.openstack.org/wiki/HierarchicalMultitenancy and I'd like to know what's already implemented and what's missing14:17
samuelmzCould you give me some info or say me who I should ask to get this info?14:17
samuelmzI guess vishvananda is the leader of this feature, right?14:17
*** thedodd has joined #openstack-keystone14:22
*** henrynash has quit IRC14:24
dolphmsamuelmz: there's a weekly meeting where i'm sure all the work as been shared and discussed, i'd skim through the meeting logs14:25
dolphmsamuelmz: there's also a mailing list discussion with links to the various efforts, and two summit sessions14:25
*** henrynash has joined #openstack-keystone14:25
*** henrynash has quit IRC14:25
*** saju_m has joined #openstack-keystone14:26
*** gokrokve has quit IRC14:26
*** gokrokve has joined #openstack-keystone14:32
tomoiagaI see that with LDAP, users are also kept in two places if one would like to keep user authorization in keystone14:32
samuelmzdolphm, do you know when this meeting take place? (the day, hour)14:32
samuelmzdolphm, do you have a link to this ML thread?14:32
dolphmsamuelmz: https://wiki.openstack.org/wiki/Meetings#Hierarchical_Multitenancy_Meeting14:32
samuelmzdolphm, great, thank you14:33
dolphmsamuelmz: there's a lot to dig through https://www.google.com/search?q=openstack-dev+archive+hierarchical+multitenancy14:34
*** htruta has joined #openstack-keystone14:36
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Sync test_migrations  https://review.openstack.org/8061814:43
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Redundant unique constraint  https://review.openstack.org/8444714:43
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Corresponding `nullable` value.  https://review.openstack.org/8444614:43
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Compatible server default value in the models.  https://review.openstack.org/8444514:43
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Explicit foreign key indexes.  https://review.openstack.org/8444414:43
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Make it possible to use multiprocess file locks  https://review.openstack.org/8444814:43
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Comparision of database models and migrations.  https://review.openstack.org/8063014:43
htrutahello! i'm working on this bug: https://bugs.launchpad.net/keystone/+bug/1264325 openstackgerrit14:50
uvirtbotLaunchpad bug 1264325 in keystone "API v3 - Unable to perform scope independant operations with unscoped token" [High,Triaged]14:50
htrutai want to know how can I allow this unscoped user to make such operations. One can easily do this by removing de rule on the policy.json. However, it will allow any user to do this. The question is: Who should be able to do these operations?14:52
*** kun_huang has joined #openstack-keystone14:53
*** david-lyle has joined #openstack-keystone14:55
*** saju_m has quit IRC14:58
*** rodrigods has joined #openstack-keystone15:02
*** ilives has quit IRC15:13
*** dstanek is now known as dstanekafk15:19
openstackgerritDolph Mathews proposed a change to openstack/python-keystoneclient: eliminate race condition fetching certs  https://review.openstack.org/8632115:19
*** dstanekafk is now known as dstanek15:20
dolphmayoung-afk: dstanek: deleted cert_file_missing() and never removed the corresponding test until ^15:21
dstanekdolphm: lgtm still15:22
*** dolphm has quit IRC15:29
*** dolphm has joined #openstack-keystone15:30
*** ChanServ sets mode: +o dolphm15:30
*** ayoung-afk is now known as ayoung15:32
*** wchrisj has left #openstack-keystone15:32
*** zhiyan has quit IRC15:45
*** stevemar has joined #openstack-keystone15:46
*** browne has joined #openstack-keystone15:52
*** Chicago has quit IRC15:55
dolphmayoung: this is why we don't backport refactors! https://bugs.launchpad.net/keystone/+bug/130505615:55
uvirtbotLaunchpad bug 1305056 in keystone "Impossible to use method search_s in BaseLdap if attribute 'page_size' is not 0." [Medium,In progress]15:55
*** Chicago has joined #openstack-keystone15:55
*** Chicago has joined #openstack-keystone15:55
ayoungdolphm, yeah, and as I am now unloaded of responsibility for getting that particular bug fixed, I can say I wholeheartedly agree with you15:56
dolphmayoung: i know ;)15:57
ayoungseems we have a good enough workaround, but I still think we'll get a backport for Icehouse in the early May time frame15:57
ayoungwithout the refactor15:57
dolphmayoung: that'd be great to see (could apply to havana too)15:57
ayoungdolphm, to test SAML, would this make sense:  set up Keystone with an LDAP backend.  Use that same LDAP provider for SAML,  enable the SAML plugin, and test if authenticate works?15:58
ayoungWe need to have the user in the identity backend stil, right?15:58
dolphmayoung: are you talking about federation?15:58
ayoungdolphm, yes15:59
dolphmayoung: use the sql identity backend (no need to duplicate users), and ask marekd for docs to configure mod_shib + OS-FEDERATION together16:00
ayoungdolphm, but my SAML provider uses LDAP.  THat is why I am asking16:00
*** jsavak has joined #openstack-keystone16:01
ayoungItwould be the same LDAP source for both SAML and Keystone, so no duplication16:01
ayoungmarekd, ^^ question is really for you, then16:01
dolphmayoung: what do you want out of OS-FEDERATION then?16:01
ayoungdolphm, just a sanity test.16:01
dolphmayoung: you want keystone to use SAML instead of talking to LDAP at all, no?16:01
ayoungAnd to make sure we can support it16:02
ayoungdolphm, eventually, but not this round.  For this round, I want to just test using SAML to create a token16:02
marekddolphm: ayoung  hello.16:02
marekdayoung: you want a unit test that utilized mod_shib?16:02
dolphmayoung: i'm not sure what the in-between solution you're looking for would be16:03
*** zhiyan has joined #openstack-keystone16:03
ayoungmarekd, sure, that would be a good starting point.16:03
*** joesavak has quit IRC16:04
dolphmsamuelmz: hierarchical multitenancy meeting is starting16:04
ayoungdolphm, I thought we needed to have any user in the identity backend.  But that does not mean they have to be in SQL, right?  THey could be in LDAP as the identity backend, and it would all work still, no?16:04
dolphmsamuelmz: false alarm. lol16:04
samuelmzdolphm, ok, I'm gonna join it16:04
samuelmzdolphm, wow D:16:05
samuelmzdolphm, no problem :)16:05
dolphmsamuelmz: i was really confused as to why it was starting (wrong day!)16:05
dolphmayoung: you don't need *any* users in SQL to use OS-FEDERATION16:05
samuelmzdolphm, yes, normally it happens on fridays :)16:05
dolphmayoung: you *should* have service users in SQL, still16:05
ayoungdolphm ah....I thought that the assignment backend still looked for users in identity?16:06
marekdayoung: no, it takes user from mappings.16:06
marekdmore or less.16:06
dolphmayoung: ah, i see... you want to *continue* using the LDAP identity backend. that would be fine, but it's unrelated to what OS-FEDERATION requires16:06
dolphmayoung: OS-FEDERATION needs groups from identity, that's it16:07
dolphmayoung: group membership to federated users is handled by mapping16:07
marekdayoung: I also wonder how you want to make a unit test with mod_shib included....16:07
*** joesavak has joined #openstack-keystone16:07
marekdayoung: I didn't come up with any reasonably good idea, so I would like to hear something if you have one16:08
*** jsavak has quit IRC16:08
*** chandan_kumar_ has quit IRC16:08
ayoungmarekd, I wouldn't call it a unit test.  I'd call it a proof-of-concept16:09
marekdayoung: ah, ok16:09
marekdayoung: you can try this: https://github.com/zaccone/keystone-federation16:10
marekdayoung: of course it need tweaking like adjusting the hostname etc.16:10
ayoungmarekd, cool16:10
marekdayoung: i used testshib.com as an IdP.16:11
ayoungdolphm, OK, so I don't need to continue using LDAP as the Identity backend.  I think I can go with this setup.16:11
marekdayoung: i think mhu was also trying to setup a federated-keystone.16:11
*** joesavak has quit IRC16:11
marekdayoung: what IdP do you want to use? setup your own, any public one, or utilize one you already have access to?16:13
ayoungmarekd, excellent.  We should be able to start by duplicating what you did, and then adding in our own SAML provider16:13
ayoungmarekd, we are working on one as a front to FreeIPA16:13
ayoungamong other things16:13
ayounghttps://git.fedorahosted.org/git/ipsilon.git   marekd16:13
marekdayoung: understood.16:13
ayoungshould work for AD as well, or so I've been told16:14
ayoungits new, and just being developed, but I want to start beating on it16:14
*** ukalifon1 has joined #openstack-keystone16:14
marekdayoung: wait, ipsilon is a IdentityProvider implementing SAML2 ?16:14
marekdayoung: or i am misunderstanding?16:15
marekdayoung: something like IdP from Shibboleth?16:15
ayoungmarekd, it is an IPA client that exposes SAML16:15
ayoungthe IdP is FreeIPA16:15
*** marcoemorais has joined #openstack-keystone16:15
marekdayoung: ok, so push for browserless extensions :-)16:16
ayoungmarekd, oh, indeed.16:16
*** gyee has joined #openstack-keystone16:17
*** richm has joined #openstack-keystone16:20
*** andreaf has quit IRC16:26
dolphmwhoa we have external db2 ci!16:27
dolphmwhen did that happen?16:27
bknudsondolphm: just a couple of days ago they started reporting.16:27
bknudsondolphm: it runs some tempest tests... it's not the unit tests.16:29
htrutahello? anyone can help me with this bug https://bugs.launchpad.net/keystone/+bug/1264325 ?16:32
uvirtbotLaunchpad bug 1264325 in keystone "API v3 - Unable to perform scope independant operations with unscoped token" [High,Triaged]16:32
dolphmhtruta: it's really blueprint work to solve it, via https://blueprints.launchpad.net/keystone/+spec/service-scoped-tokens16:34
*** marcoemorais1 has joined #openstack-keystone16:38
*** marcoemorais1 has quit IRC16:38
*** marcoemorais has quit IRC16:38
*** marcoemorais has joined #openstack-keystone16:39
*** harlowja_away is now known as harlowja16:49
afaranhaDid anybody get this error while using keystoneclient? "Current authorization does not have a known management url"16:49
*** leseb has quit IRC16:49
*** tomoiaga has quit IRC16:49
htrutadolphm: So, the bug will be abandoned, right?16:54
*** marcoemorais has quit IRC16:58
*** marcoemorais has joined #openstack-keystone16:59
*** stevemar has quit IRC17:00
*** gokrokve has quit IRC17:01
*** amcrn has joined #openstack-keystone17:03
*** chandan_kumar_ has joined #openstack-keystone17:24
*** dstanek has quit IRC17:31
*** dstanek has joined #openstack-keystone17:32
*** chandan_kumar_ has quit IRC17:34
*** wchrisj has joined #openstack-keystone17:34
*** thedodd has quit IRC17:36
*** gokrokve has joined #openstack-keystone17:37
*** stevemar has joined #openstack-keystone17:37
*** zhiyan is now known as zhiyan_17:37
ayoungbknudson, I challenge you to get the CI running unit tests in DB2 before we can get them in Mysql17:38
*** marcoemorais has quit IRC17:38
*** vhoward has joined #openstack-keystone17:38
*** marcoemorais has joined #openstack-keystone17:38
diegowswe never finish our discussion about PAM authentication module :)17:39
bknudsonayoung: I don't think it's going to happen... they're been working on DB2 CI for a long time.17:39
openstackgerritA change was merged to openstack/keystone: Fixed wrong behavior in method search_s in BaseLdap class.  https://review.openstack.org/8632517:39
*** kun_huang has quit IRC17:41
*** wchrisj has quit IRC17:45
*** chandan_kumar_ has joined #openstack-keystone17:47
*** rodrigods has quit IRC17:48
*** morganfainberg_Z is now known as morganfainberg17:51
morganfainbergooh db2 ci!17:52
morganfainbergcool17:53
morganfainbergbknudson, is there a way to get a patchset tested through the external ci?17:53
bknudsonmorganfainberg: well, it might be in process already, not sure how would check that.17:54
bknudsonapparently there were some network problems.17:54
morganfainbergbknudson, specifically https://review.openstack.org/#/c/78169/ happy to issue a recheck if that would help.17:54
morganfainbergbknudson, but i'd love to have db2 weigh in on it17:54
bknudsonmorganfainberg: 'recheck db2-test' should do it.17:55
morganfainbergbknudson, cool.17:55
morganfainbergbknudson, suggest you have them do the same thing gate does (if possible) post a comment that it's attempting to run the CI (or is that not allowed?)17:56
bknudsonmorganfainberg: I haven't seen the other cis doing that... like in tempest/nova/neutron17:56
bknudsonalthough it might be a good idea if we're actually watching for it.17:57
morganfainbergbknudson, hm. i really like that the check/gate does it, maybe it would net too much spam on the reviews though for externals to.17:57
morganfainbergbknudson, i personally like knowing what external CIs are running (or attempting to run)17:57
morganfainbergs/like/would like17:57
openstackgerritBrant Knudson proposed a change to openstack/keystone: Include extra attributes in list results  https://review.openstack.org/8104117:58
*** derek_c has quit IRC18:07
openstackgerritMarek Denis proposed a change to openstack/python-keystoneclient: Add CRUD operations for Fedration Mapping Rules.  https://review.openstack.org/8374218:13
afaranhaHello, Does someone knows how to add keystone v3 to service to be list in the keystone catalog? Thank you18:15
*** erecio has joined #openstack-keystone18:16
*** thedodd has joined #openstack-keystone18:20
lbragstadbknudson: looks like https://review.openstack.org/#/c/81041/5 is still going to fail18:24
lbragstadspecifically in here: https://github.com/openstack/keystone/blob/master/keystone/tests/test_revoke.py#L41318:25
lbragstadMismatchError: 8 != 7, so I'm sure sure how relevant this failure is to your change?18:25
bknudsonlbragstad: 1300581 again?18:26
lbragstadbknudson: looks like it18:26
lbragstadfails randomly in the iteration?18:26
lbragstadthreading?18:27
bknudsonlbragstad: we don't have multiple threads, we've got different processes18:27
lbragstadok18:27
bknudsonlbragstad: it could also be an issue with the order the tests are run18:28
bknudsonsince that could change based on what process gets which test18:28
lbragstadright18:28
*** wchrisj_ has joined #openstack-keystone18:30
*** morganfainberg has quit IRC18:31
lbragstadbknudson: looks like everything is going to pass except that, so probably just that bug18:31
*** ukalifon1 has quit IRC18:33
openstackgerritMarek Denis proposed a change to openstack/python-keystoneclient: Add CRUD operations for Identity Providers.  https://review.openstack.org/8333718:36
openstackgerritMarek Denis proposed a change to openstack/python-keystoneclient: Add CRUD operations for Fedration Mapping Rules.  https://review.openstack.org/8374218:42
ayoungnkinder, http://adam.younglogic.com/2014/04/teaching-horizon-to-share/   part one.18:43
ayoungmarekd, I can -1 from here  Fedration18:43
marekdayoung: ?18:43
ayoungFedrations?  Are those like MREs?18:44
marekdayoung: ayoung uf, i thought you found a serious bug. nice catch...18:45
marekdgonna change it now.18:45
openstackgerritMarek Denis proposed a change to openstack/python-keystoneclient: Add CRUD operations for Federation Mapping Rules.  https://review.openstack.org/8374218:48
openstackgerritMarek Denis proposed a change to openstack/python-keystoneclient: Add CRUD operations for Identity Providers.  https://review.openstack.org/8333718:51
openstackgerritMarek Denis proposed a change to openstack/python-keystoneclient: Add CRUD operations for Federation Mapping Rules.  https://review.openstack.org/8374218:52
ayoungmarekd, out of curiosity why req_ref.pop('id')  in the test  https://review.openstack.org/#/c/83742/11/keystoneclient/tests/v3/test_federation.py18:55
marekd1 sec18:56
openstackgerritPriti Desai proposed a change to openstack/keystone: Adding more descriptive error message  https://review.openstack.org/8618718:57
*** ukalifon has joined #openstack-keystone18:59
marekdayoung: if I don't pop that id it will stay in the reference json object18:59
ayoungand the ID is not returned from the server?19:00
*** morganfainberg_Z has joined #openstack-keystone19:01
*** morganfainberg_Z is now known as morganfainberg19:01
ayoungmorganfainberg, http://adam.younglogic.com/2014/04/teaching-horizon-to-share/19:01
marekdayoung: hm, it is: https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v3/src/markdown/identity-api-v3-os-federation-ext.md#create-a-mapping-put-os-federationmappingsmapping_id ok, let me check this.19:02
morganfainbergayoung, reading now, but yay!19:02
*** derek_c has joined #openstack-keystone19:02
morganfainbergayoung, i'd probably go with mod_rewrite vs a meta-refresh.19:03
ayoungmorganfainberg, but this was already in the install19:03
ayoungbut there are many ways to skin that cat19:03
morganfainbergayoung, aye.19:03
morganfainbergayoung, i do like it.19:04
morganfainbergayoung, simple and concise.19:04
ayoungmorganfainberg, I have threee posts.  That one and two others I am still editing19:04
morganfainbergayoung, nice!19:04
ayoungthe next is on Kerberizing Keystone (to include SSL via NSS)19:04
ayoungand then on NSS for horizon19:04
bknudsonmorganfainberg: https://review.openstack.org/#/c/78169/ -- DB2 says succeeded19:05
morganfainbergbknudson, cool. i was concerned about that tbh19:05
bknudsonI had a fix at one point to show the --debug output in devstack but it wound up being abandoned.19:05
morganfainbergbknudson, don't have much experience w/ db2 so wasn't sure what would break / work / not work19:05
bknudsonmorganfainberg: it's always easier when there's a lot less code!19:06
morganfainbergbknudson, :)19:06
morganfainbergbknudson, so the DB2 test isn't voting yet?19:06
morganfainbergoh (non voting) haha19:06
* morganfainberg learns to read19:06
bknudsonmorganfainberg: I'm not sure I've seen any external CI voting?19:07
*** chandan_kumar_ has quit IRC19:07
morganfainbergbknudson, turbo-hipster was.19:07
morganfainbergas was minesweeper i think19:07
bknudsonmorganfainberg: let's give it some time and make it voting if it's stable19:08
morganfainbergbknudson, https://review.openstack.org/#/c/77450/19:08
morganfainbergbknudson, ++19:08
bknudsonmorganfainberg: last I heard minesweeper was broken for weeks.19:08
morganfainbergyeah.19:08
morganfainbergturbo-hipster seems to be working19:08
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: Hash functions support different hash algorithms  https://review.openstack.org/8620219:12
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: auth_token middleware hashes tokens with sha256  https://review.openstack.org/8039819:12
*** ukalifon has quit IRC19:15
dolphmbknudson: i was just about to ping you about that ^19:19
dolphmbknudson: let me do a quick review in case you had the same idea i did...19:20
*** chandan_kumar_ has joined #openstack-keystone19:20
morganfainbergdstanek, dolphm, bknudson, we should move to the sql_upgrade style (migration) testing that nova is using https://github.com/openstack/nova/blob/master/nova/tests/db/test_migrations.py19:20
morganfainbergor similar19:21
bknudsonmorganfainberg: I think chris yeoh tried this once and we didn't take it19:21
dolphmbknudson: (nope) but i think you'll like this -- make the hash algorithm a configurable *list* in auth_token, and default it to ['sha256', 'md5'] and have it attempt them in order exactly like you're doing19:21
dolphmbknudson: that way you provide people a way to migrate to sha256, and then from sha256 to sha512 if they want, etc19:22
dstanekmorganfainberg: i wouldn't mind that19:22
bknudsonmorganfainberg: but I think it was only because we wanted the code in oslo-incubator19:22
dolphmmorganfainberg: it's been proposed against keystone in the past19:22
bknudsondolphm: I like the list. I'll go with that.19:22
dolphmmorganfainberg: we did a terrible job reviewing it (it was late in the grizzly cycle, i suppose) and the author gave up19:22
morganfainbergdolphm, well probably time to resurrect it then :) i like the _check_XXX and _post_<action>_XXX _pre_action_XXX style19:23
morganfainbergit's a bit better than the custom test each migration gets now19:23
marekdayoung: so basically req_ref should not pop id, but the last line should also check manager_ref instead of req_ref as it checks the request, not response and there should be no id in the body.19:23
dolphmmorganfainberg: agree19:24
ayoungmarekd, glad I asked19:24
morganfainbergdolphm, once we get the collapse review through, i think it'll be easier to make this shift19:24
morganfainbergdolphm, so i'll backlog that for later in the cycle19:24
marekdayoung: yes, thank you.19:24
morganfainbergpossibly post j-2 (since at that point we shouldn't be accepting new migrations)19:25
htrutahello! is anyone working on this BP https://blueprints.launchpad.net/keystone/+spec/service-scoped-tokens ?19:27
openstackgerritMarek Denis proposed a change to openstack/python-keystoneclient: Add CRUD operations for Federation Mapping Rules.  https://review.openstack.org/8374219:29
*** openstackgerrit has quit IRC19:34
dstanekmarekd: did you get your git issue taken care of?19:34
bknudsondolphm: qq -- you mentioned you didn't want the hash algorithm in the tokens ...19:34
bknudsonwhat about the revocation list?19:34
marekddstanek: yes! forgot to tell you!19:34
marekddstanek: basically git review -d <base> ; git review -x works perfectly :-)19:35
marekddstanek: thanks!19:35
dstanekmarekd: nice; yw19:35
bknudsondolphm: it would be easy enough to apply the same list of hash algorithms for the revocations.19:37
*** marekd is now known as marekd|away19:37
htrutadolphm: do you know anything about that BP?19:41
afaranhaayoung: Hello, I developing for nova but using keystoneclient. I saw that you a lot about openstack, could you help me? I need to enable keystone v3 in the catalog, do you know how to do it? Thank you19:41
*** leseb has joined #openstack-keystone19:41
ayoungafaranha, you are soaking in it19:42
*** openstackgerrit has joined #openstack-keystone19:42
ayoungv3 is arelady enabled19:42
ayoungalready even19:42
afaranhabut when I list it I dont get it. I get here: "request.headers.get('X-Service-Catalog', req.headers.get('X_STORAGE_TOKEN'))"19:44
afaranhaI'm stuking in this change instance ownership blueprint a long time, I always get some minor problems that I don't know how to deal with and lost a lot of time on it :P Thank you19:44
ayoungafaranha, "when I list it"  how?19:46
ayoungafaranha, the catalog is going to return what you set in it for the endpoints.19:47
ayoungYou need to makes sure that you are making calls on the V3 suburl, and not V219:47
ayoungdid that hack ever get into keystoneclient?  I thought so...19:47
afaranhaI get a request instance from a nova extension (action(self, req, id, body)) and do the following:19:48
afaranhacatalog = req.headers.get('X-Service-Catalog', req.headers.get('X_STORAGE_TOKEN'))19:48
afaranhaLOG.debug("Catalogs: %s" % catalog)19:48
stevemarmorganfainberg, thanks for the help on the trust not found message19:48
morganfainbergstevemar, np19:48
morganfainbergstevemar, was easy :)19:49
*** chandan_kumar_ has quit IRC19:49
*** Krsna has joined #openstack-keystone19:51
KrsnaCould someone point me in the direction to test out federated keystone. Using keystone as the idps instead of saml2, or is this feature still under development?19:52
afaranhaayoung: Since I'm using it in Nova API, It's using V2. To get the catalog for keystone v3 I should not get it in the request header? So I need to list it in keystone? If I need to list it by keystone, it'll be V2, since I need an endpoint and the only endpoint I have is the Keystone V2.19:53
ayoungafaranha, Huh?19:53
KrsnaI followed the few lines of direction here http://docs.openstack.org/developer/keystone/extensions/federation-configuration.html, however I did not see the new create_idp, etc cruds19:53
ayoungafaranha, there are some assumptions in what you are saying that I don't really understand19:54
ayoungNova  needs to start with a Keystone url it gets out of the catalog.  and then it needs to chop /v2.0 off that if it is going to call on the V3 api19:54
*** d0ugal has quit IRC19:54
ayoungI assume you are not using keystoneclient for any of this19:55
afaranhaayoung: Sorry, I didn't understand how could I list V3 version. The only way I know to do it is by the request headers (that I'm currently using) and by keystone.endpoints.19:55
afaranhaayoung: Yes, to get the endpoint I'm not using keystone19:55
ayoungafaranha, "request.headers.get('X-Service-Catalog'  is data populated by auth token middleware into the Nova request.  Nova needs to take that information and make a keystone call19:57
KrsnaFrom what I am reading it seems that some design decisions are still being discussed. However, if there is a clear path ahead and if the federated keystone feature is still under dev, where can I start to jump in help?19:57
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: auth_token middleware hashes tokens with configurable algorithm  https://review.openstack.org/8039819:58
ayoungbut I don't know the code that does that...basically, pull the endpoint out of the service catalog, chop off v2.0 and  make your call on KEystone to get the data you need.19:58
stevemarmarekd|away, yay for getting keystoneclient stuff merged!19:58
bknudsondolphm: ^ that's using the config list19:58
stevemarwell... approved anyway19:59
*** david_lyle_ has joined #openstack-keystone20:01
afaranhaayoung: Sorry, but I need an endpoint to instantiate the keystoneclient. I get the endpoint in the request headers and then instantiate it "keystoneclient.Client(token=context.auth_token, auth_url=auth_url, endpoint=auth_url)" auth_url contains the endpoint information.20:03
afaranhaHow am I able to instantiate the keystoneclient without an endpoint?20:03
*** dklyle has joined #openstack-keystone20:03
ayoungafaranha, so you ARE using keystone client?20:03
afaranhaayoung: yes, I'm using it. I use to list projects and users.20:04
bknudsonwhat's context.auth_token?20:04
*** david-lyle has quit IRC20:04
bknudsondon't you want to use the nova service credentials?20:04
bknudsonalso, is nova service be allowed to list projects and users?20:04
bknudsonis nova service allowed to list projects and users?20:05
bknudsonif move to federation there will be no user list.20:05
afaranhaBut I need keystoneclient V3, because in the currently v2 I'm not able to do "keystoneclient.users.list()" I get an error, and when I change manually to V3 it works fine20:05
bknudsonafaranha: there's a keystoneclient.v3.Client20:05
*** david_lyle_ has quit IRC20:06
dolphmbknudson: that looks like it's going to unnecessarily hash it a second time with md5?20:07
ayoungbknudson, did we merge the hack that lets the client chop off the /v2.020:07
bknudsondolphm: in which case?20:07
bknudsonall cases?20:07
bknudson(this is what it was already doing)20:07
afaranhabknudson: yes, but I need a V3 endpoint also to make it work. When I'm using "from keystoneclient.v3 import client" "keystoneclient.Client(token=context.auth_token, auth_url="http://10.1.0.32:5000/v3", endpoint=http://10.1.0.32:5000/v3)" it works fine20:07
dolphmbknudson: yeah, as a result of token_id = token_id or token_hash20:07
ayoungafaranha, use 3537520:07
ayoung5000 is the auth url, but doesn't have all of the admin functions20:08
bknudsondolphm: it hashes it using sha256, checks cache, if it's not there then tries the same with md520:08
afaranhaayoung: I also tried that, and with keystone.v2 and only keystone, and with "http://10.1.0.32:5000/v2.0" also20:08
ayoungafaranha, chop the /v2.0 off the endpoint in the service catalog.  I think the chang we need to make thing s work is not yet in the client20:09
bknudsondolphm: token_id is None the first time through and the next time through it'll be the sha256 hash so will stay the sha256 hash20:09
dolphmbknudson: but you're still doing the work of creating the md5 hash20:09
*** d0ugal has joined #openstack-keystone20:09
*** d0ugal has quit IRC20:09
*** d0ugal has joined #openstack-keystone20:09
bknudsondolphm: right, we have to check both sha256 and md520:10
dolphmbknudson: well you have to check sha256, and then maybe md520:10
afaranhaayoung: Ok, I'll try it, thank you, at the moment the cloud in working on is down, but when people fix it I'll try. Thank you20:10
dolphmbknudson: you don't have to check *both* every time20:10
bknudsondolphm: if sha256 was in the cache then it would have returned20:10
dolphmbknudson: after wasting cycles producing an md5 hash :)20:11
bknudsondolphm: the code doesn't get to the md5 hash if the sha256 hash was found in the cache.20:11
dolphmbknudson: oh i'm getting myself confused. i read ~1215 first which is producing both hashes20:12
bknudsondolphm: that's the revocation list.20:12
dolphmbknudson: does auth_token not support online validation of PKI tokens?!20:12
bknudsondolphm: I don't think there's any online validation of pki tokens.20:13
bknudsonyou'd have to hash the token first20:13
dolphmbknudson: exactly - that's where the conversation (and bug reported) started20:13
bknudsondolphm: I think the bug was just don't use md5.20:13
bknudsonthere's another bug that was about checking revocation list for token hash.20:14
dolphmbknudson: when it was filed, that's the only place we were using md5: GET /v2.0/tokens/{md5_hashed_pki_token}20:14
*** openstackstatus has quit IRC20:15
bknudsondolphm: auth_token didn't cache pki tokens?20:16
*** openstackstatus has joined #openstack-keystone20:16
bknudsonor maybe it cached them by the full pki token and that was broken20:16
dolphmbknudson: exactly20:16
dolphmbknudson: chasing token_id through this method makes my head hurt...20:16
dolphmbknudson: if you have a PKI token that isn't in the cache, it looks like it's going to write an md5 hash as the key in L879 back to the cache20:18
dolphmbknudson: bah, i'm wrong again :)20:18
bknudsondolphm: it writes token_id.... which should only have been set to the first hash.20:19
bknudsondolphm: I'm sure this could all be made more obvious20:19
dolphmbknudson: i'm sure it could too - can you move L862 after L866? that might help a tiny bit20:20
dolphmbknudson: or even rewrite it as if token_id is None: token_id = token_hash20:20
bknudsondolphm: will do.20:20
bknudsondolphm: I'll do that too20:20
dolphmbknudson: s/first/preferred/ ?20:20
bknudsondolphm: will do.20:21
dolphmi wish we didn't treat PKI tokens any differently from UUID tokens20:23
dolphm(hash them both equally, etc)20:23
morganfainbergdolphm, can we make that change?20:24
dolphmbknudson: logic seems right to me as-is20:24
morganfainbergdolphm, auth_token makes my head hurt when i get into the caching bits20:24
dolphmmorganfainberg: i think it'd be a slight pain to make that transition, but we could20:24
dolphmmorganfainberg: as painful as what bknudson is working on now!20:24
bknudsondolphm: I'll push up the latest version with the other comments.20:24
morganfainbergdolphm, hehe20:24
dolphmthis will require conf updates in other projects, right?20:25
htrutadolphm, ayoung: do you know it there is anyone working on this BP https://blueprints.launchpad.net/keystone/+spec/service-scoped-tokens ?20:25
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: auth_token middleware hashes tokens with configurable algorithm  https://review.openstack.org/8039820:25
htrutaif*20:25
ayounghtruta, topic of discussion for the summit20:25
bknudsondolphm: since it supports md5 they can keep their existing config.20:25
ayounghtruta, but please feel free to take a hack at it20:25
dolphmhtruta: no one is right now. arvind assigned it to himself without knowing what it was, and i proposed my thoughts on the API here https://review.openstack.org/#/c/61869/20:26
ayoungbeen a few people expressing interest, but no work yet20:26
dolphmbknudson: i mean sample conf updates20:26
bknudsondolphm: Yes, I think so... some have got a sample conf generators now.20:27
ayounghtruta, I think that the right solution is going to be to add values to the token request saying "only this service"  or "only this endpoint"  or "only these roles"20:27
dolphmbknudson: i'm planning on cutting 0.8.0 ASAP for https://review.openstack.org/#/c/86321/ but don't want to cause pain to other projects while RC windows are open20:27
ayoungand then enforcement is going to have to be in auth_token middleware20:27
bknudsondolphm: this doesn't need to be in 0.8.020:27
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: auth_token middleware hashes tokens with configurable algorithm  https://review.openstack.org/8039820:33
bknudson^ this one will only hash with the preferred if found in revocation list.20:33
*** harlowja is now known as harlowja_away20:36
*** Mikalv has joined #openstack-keystone20:38
*** jamielennox|away is now known as jamielennox20:43
htrutadolphm, ayoung: hm... thanks! i'll see what i can do20:44
openstackgerritA change was merged to openstack/python-keystoneclient: eliminate race condition fetching certs  https://review.openstack.org/8632120:46
*** G________ has joined #openstack-keystone20:50
*** harlowja_away is now known as harlowja20:55
*** erecio has quit IRC20:56
*** G________ has quit IRC20:56
*** raildo has quit IRC20:59
openstackgerritDavid Stanek proposed a change to openstack/keystone: Adds table and model for storing rotated passwords  https://review.openstack.org/7336820:59
openstackgerritDavid Stanek proposed a change to openstack/keystone: password rotation extension WIP  https://review.openstack.org/7462320:59
*** dklyle is now known as david-lyle21:03
*** diegows has quit IRC21:04
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Add service name to catalog  https://review.openstack.org/7841021:05
*** G________ has joined #openstack-keystone21:14
dstanektopol: i just saw you comment in my review about small patches; you're welcome :-) but it's more for me than you21:14
*** openstackstatus has quit IRC21:14
*** gokrokve has quit IRC21:14
dstaneknow that's wierd - it looks like i lost a commit21:15
topoldstanek, your approach makes reviewing a very pleasant one!21:15
dstanektopol: i start out with a lot of local changes in a bunch of 'wip' commits - when i want to share i break it up sanely so i know what people are looking at21:16
dstanekinteractive rebasing is my best friend21:17
*** openstackstatus has joined #openstack-keystone21:22
morganfainbergtopol, i saw a comment on the summit sessions by btopol, it took me > 5 minutes to figure out who that was.21:23
morganfainbergtopol, i am embarassed21:23
morganfainbergtopol, :P21:23
topolmorganfainberg, its nice to see I am making significant impression on key OpenStack conributors such as yourself :-)21:25
dstanekha ha21:25
topolmorganfainberg I am one chart deck and one blog article away from getting to go back to doing keystone reviews...21:26
morganfainbergtopol, if the name had been "topol" i would have connected it, but who the heck is btopol21:26
dstanektopol: i'm not a key contributor, but you've make an impression on me!21:26
*** gokrokve has joined #openstack-keystone21:27
morganfainbergit's kinda like when stevemar2 appears in the channel21:27
morganfainbergwho the heck is that guy21:27
topoldstanek gets and extra beer when I buy the first night21:27
morganfainbergtopol, oh so he's getting my share of beer... then :(21:28
morganfainbergtopol, :P21:28
topolmorganfainberg, nah... he just gets extra21:28
dstaneknow this is embarrassing - i merged two commits on accident21:28
topolmorganfainberg, dstanek, Thats assuming I have money of course. I travel 5 days to Vegas for IBM Impact conference end of April.  Did I mention everyone in my family has a gambling problem??21:29
morganfainbergdstanek, thats what you get for doing interactive rebasing! :P21:29
*** thedodd has quit IRC21:30
morganfainbergtopol, so... what you're saying is we should bet you for drinks on any given night?21:30
* lbragstad looks up card counting strategies for topol21:30
topolmorganfainberg, i think the addiction only applies to limit and no limit texas holdem poker21:30
*** diegows has joined #openstack-keystone21:30
morganfainbergtopol, hehe21:31
*** thedodd has joined #openstack-keystone21:32
nkinderayoung: that LDAP attriubte case sensitivity thing is a bug in keystone21:33
nkinderayoung: I've got a patch written up that I'm testing out21:33
*** derek_c has quit IRC21:35
stevemarmorganfainberg, was that stevemar2 guy here again? hate him.21:39
*** topol has quit IRC21:40
morganfainbergstevemar, yeah21:42
*** derek_c has joined #openstack-keystone21:48
*** derek_c has quit IRC21:54
*** gokrokve has quit IRC21:58
*** david-lyle has quit IRC22:02
KrsnaWho should I ping about federated keystone?22:04
morganfainbergKrsna, stevemar and marekd|away are two very good resources that worked heavily on it22:07
morganfainbergKrsna, i can try and answer some questions22:07
morganfainbergKrsna, it depends on what you're trying to accomplish22:08
stevemarmorganfainberg, whats up22:08
morganfainbergstevemar, ^22:08
Krsnamorganfainberg: that would be awesome. I am trying to setup and test basic federated keystone. If the full implementation not in place then i would like to help out.22:08
morganfainbergKrsna, so you want keystone to be the IDP and federate to other keystones?22:08
KrsnaI followed the few lines of direction here http://docs.openstack.org/developer/keystone/extensions/federation-configuration.html, however I did not see the new create_idp, etc cruds22:09
morganfainbergKrsna, or use a SAML provider (free IPA? or similar) and have multiple keystones consume that?22:09
Krsnamorganfainberg: that is correct22:09
Krsnamorganfainberg: keystone to be the IDP and federate to other keystones not SAML22:09
morganfainbergKrsna, ah, keystone can consume external IDPs but can only provide identity to itself at the moment22:09
morganfainbergKrsna, short of using a shared (replicateD) ldap or sql backend22:09
*** leseb has quit IRC22:10
morganfainbergKrsna, that exact usecase is on my "can we have that! and how do we get there" short list22:10
morganfainbergKrsna, i really want that functionality22:10
morganfainbergstevemar, i've talked to you about that before actually.22:11
morganfainbergKrsna, i think we even have a summit proposal for the ATL summit on this topic22:11
Krsnamorganfainberg: I don't know if I fully understood that. I would like to have keystone be the IDP and point (federate) to other keystones (some using sql backend others using LDAP)22:11
*** gokrokve has joined #openstack-keystone22:12
morganfainbergKrsna, right now, you can achieve this by replicating the backend data.  This has a lot of issue22:12
Krsnaif that is not possible yet then where would could you point me to start to implement that so that when I come to the ATL summit we can make some real progress?22:12
morganfainbergKrsna, in the future i want keystone (as soon as we can implement it) to be able to be a federated IDP to another keystone server22:12
*** leseb has joined #openstack-keystone22:12
morganfainbergKrsna, right now it isn't possible.22:12
Krsnamorganfainberg: ahhhh i see. basically you have to have the same data in all the different keystone instances?22:13
jamielennoxstevemar: has all that stuff with oauth libraries in stable been finished?22:13
morganfainbergKrsna, correct. keystone doesn't provide data to other keystones (identity data) via a federated protocol. we haven't gotten there.22:13
stevemarKrsna, right now we only support federation when using the SAML federation protocol, not other keystone instances22:14
morganfainbergKrsna, if you're interested in this, i know i've heard from a number of deployers what would love that kind of support (I am one of them)22:14
stevemarjamielennox, not yet, i'm waiting on the library being approved.. https://review.openstack.org/#/c/82929/22:14
Krsnamorganfainberg, stevemar: where can I start to start to implemenet keystones via federated protocol22:15
jamielennoxstevemar: no worries, just looking through what reviews i've missed22:15
stevemarjamielennox, and jogo had to make a bunch of changes to infra so it wouldn't break again22:15
Krsnamorganfainberg: Perfect! in that case where can I start?22:15
*** thedodd has quit IRC22:15
stevemarjamielennox, nah you're good22:15
morganfainbergstevemar, Krsna, not sure where to start, i kindof would like it if keystone spoke SAML to other keystones... but i think we'd need to support some kind of common federation protocol (preferably not invent a new one)22:16
*** leseb has quit IRC22:17
Krsnamorganfainberg: Do we have some kind of documentation of what we got now? That way I can catch up and think of other implementation ideas.22:17
stevemarKrsna, it's all here: https://review.openstack.org/#/c/81022/2/openstack-identity-api/v3/src/markdown/identity-api-v3-os-federation-ext.md22:17
stevemaroops22:17
stevemari meant https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v3/src/markdown/identity-api-v3-os-federation-ext.md22:17
morganfainbergstevemar, lol22:18
stevemari rely on chromes auto-filling waaaay too much22:18
morganfainbergKrsna, i think the first step is to figure out the best way for keystone to communicate to other keystones.  again what kind of protocols make the most sense.22:18
stevemarKrsna, also, a neat ppt -> https://github.com/stevemart/OpenStackFederation/blob/add_os_federation_slides/federated_identity_in_icehouse_4_8_14.pdf22:19
morganfainbergKrsna, i really do want to avoid inventing another "federation" protocol just for keystone (unless there is a damn good reason to)22:19
Krsnamorganfainberg: well, if the saml works and we can leverage that then that is a good idea.22:19
stevemarmorganfainberg, yeah, we really don't want to create our own protocol here22:19
morganfainbergKrsna, pysaml has issues but i started looking at it22:19
Krsnastevemar: thanks for the resources keep them comming. I will read those and come back if I have questions.22:20
morganfainbergKrsna, i think there was another lib that would let python act as a saml provider22:20
morganfainbergKrsna, keep in mind that py33 compatibility is also important22:20
stevemarso we really don't want rely on libraries that are not py33 compatible, like python-oauth2 :(22:21
stevemari learned my lesson from that one22:21
Krsnamorganfainberg: I will read more on pysaml and saml in general and see where I get. Are we making the assumption that all the different instances of keystone live on the same node/cluster/network or they can be on a remote cluster?22:23
*** derek_c has joined #openstack-keystone22:23
morganfainbergKrsna, if the backend data is not shared (e.g. a federated IDP), it shouldn't matter where keystone lives22:23
morganfainbergKrsna, the federated keystone that is22:24
Krsnaright.22:24
morganfainbergKrsna, the whole point would be so that a remote keystone server could provide identity ot your local keystone (e.g. cross businesses, cross AZ, cross deployment, etc)22:25
morganfainbergKrsna, i'm very happy to see more interest in this :)22:25
Krsnamorganfainberg: Let me ask you a question. As a cluster admin I want to have an internal list of service accounts managed via sql (or whatever background) at the same time I would like to be able to have ldap backend that manages users,etc. Does federated keystone sound like the best solution? After reading through my options that seems to be the best bet22:26
*** G________ has quit IRC22:26
morganfainbergstevemar, ^ Krsna, I think this sounds more like a multi-backend approach. we have some stub code (should be fleshed out for Juno) that does exactly this: keystone reads identity information from multiple backends, e.g. multiple ldap servers, ldap servers and mysql, etc22:27
morganfainbergKrsna, you could do it with federation as well.22:28
*** wchrisj_ has quit IRC22:28
*** htruta has quit IRC22:28
*** derek_c has quit IRC22:29
*** openstack has joined #openstack-keystone22:33
morganfainbergKrsna, but there are use-cases for both, likely we will eventually support both.22:34
bknudsondstanek: I thought the new release was imminent when we discussed it last.22:34
Krsnamorganfainberg: Either way sounds interesting. I will need to speak to my higher ups and see which one seems to satisfy the use case I gave you best. Then I should be able to work on one of those solutions.22:34
morganfainbergKrsna, sounds good! let me know22:35
dstanekbknudson: not sure what their release schedule is like, but our check can be disabled easily22:35
Krsnamorganfainberg: Will do. Thank you for clearing things up for me. Is this the best way to stay in touch with you?22:36
morganfainbergKrsna, here on IRC, i'm US Pacific timezone but lurk around a lot22:36
Krsnamorganfainberg: I am in SF, we should be good ;) Thanks again.22:37
morganfainbergKrsna, so often (even weekends) I'll respond (if my name is morganfainberg_Z on IRC, I'm away, but i'll see any messages you leave for me)22:37
* morganfainberg uses a IRC bouncer to capture IRC chatter when computer is off/not on the network.22:37
Krsnamorganfainberg: Will keep that in mind.22:38
*** leseb has joined #openstack-keystone22:40
*** derek_c has joined #openstack-keystone22:42
*** leseb_ has joined #openstack-keystone22:42
Krsnamorganfainberg: Ok setting up a meeting and will let you know the outcome. Just a few more questions if you don't mind. How many people do we have working on this other than you and I? Other than figureing out and implementing how keystone instances talk to one another what else would be needed ?22:43
morganfainbergKrsna, the keystone core team would be involved (review / vetting / etc) and more specifically stevemar marekd|away did a bunch of the other federated work so i expect them heavily involved22:44
*** leseb__ has joined #openstack-keystone22:44
*** leseb has quit IRC22:45
Krsnamorganfainberg: Got it, and for keystone to act as the IDP federated to other keystones, only the protocol for them to speak is what is needed?22:46
morganfainbergKrsna, well, there will need to be the protocol and the supporting implementation22:47
*** leseb_ has quit IRC22:47
morganfainbergKrsna, but i think we should determine what options we have / protocols first rahter than implementing something and then trying to wedge a protocol into that implementaiton22:47
Krsnamorganfainberg: fully agree. I hate throwing away code. Better to take the extra time to determine what we want/where we headed before writing code22:49
*** leseb__ has quit IRC22:49
*** marcoemorais has quit IRC22:50
morganfainbergyep22:50
*** marcoemorais has joined #openstack-keystone22:50
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: auth_token middleware hashes tokens with configurable algorithm  https://review.openstack.org/8039822:58
bknudsonjamielennox: thanks for the feedback on ^ !23:00
jamielennoxbknudson: np - i guess that makes sense with the revocation list23:01
openstackgerritA change was merged to openstack/python-keystoneclient: Hash functions support different hash algorithms  https://review.openstack.org/8620223:02
bknudsonI haven't updated the keystone code for this change yet.23:02
jamielennoxthe time period where you will actually have multiple algorithms configured should be fairly small so there's not a long term overhead for checking both algorithms against the list23:02
bknudsonjamielennox: it should be the token expiration time23:02
*** dims has quit IRC23:02
bknudsonso 60 mins by default23:02
jamielennoxbknudson: so you want to hold it until it's committed to server?23:02
bknudsonjamielennox: that's a good idea23:03
jamielennoxok, put a -2 or a WIP on it23:03
bknudsonI WIP'd it good.23:03
jamielennoxbut it looks fine to me23:03
morganfainbergbknudson, do you have your energy dome hat to go with the comment there?23:06
openstackgerritA change was merged to openstack/keystone: Adding more descriptive error message  https://review.openstack.org/8618723:15
*** diegows has quit IRC23:18
*** gokrokve has quit IRC23:22
*** wchrisj has joined #openstack-keystone23:28
bknudsonmorganfainberg: what comment?23:28
bknudsonoh... I didn't know what an energy dome hat was.23:29
bknudsonI thought they were flower pots23:29
morganfainbergthe WIP'd it good23:29
morganfainberghaha23:29
morganfainbergyeaaaah23:29
*** marcoemorais has quit IRC23:31
*** marcoemorais has joined #openstack-keystone23:32
bknudsonhttps://review.openstack.org/#/c/81041/ is somehow hitting bug 1300581 pretty consistently somehow23:32
uvirtbotLaunchpad bug 1300581 in keystone "test_revoke.RevokeTreeTests.test_cleanup fails" [Critical,Triaged] https://launchpad.net/bugs/130058123:32
bknudsonmaybe it's just bad luck... sometimes on py26 and sometimes py2723:33
bknudsonI haven't been able to recreate23:33
morganfainbergbknudson, same, can't recreate locally23:33
morganfainbergbknudson, i keep trying, even did a tight loop of testing23:33
morganfainbergnever failed23:33
bknudsonmorganfainberg: The only thing I could think of looking at the code was somehow the same timestamp is getting returned...23:34
bknudsonwould require a pretty fast computer.23:34
*** stevemar has quit IRC23:34
morganfainbergit's an odd one23:34
bknudsonor a broken clock23:34
bknudsonalthough I think from the output it's saying there's an extra one23:35
bknudsonmaybe some better output would help23:35
morganfainberginverse23:35
morganfainbergthere aren't enough23:35
morganfainbergwe expect N but found N-123:35
bknudsonthat would make more sense23:35
openstackgerritDavid Stanek proposed a change to openstack/keystone: Adds style checks to ease reviewer burden  https://review.openstack.org/7811923:36
*** browne has quit IRC23:45
*** wchrisj has quit IRC23:47
*** Chicago has quit IRC23:51
openstackgerritBrant Knudson proposed a change to openstack/keystone: More debug output for test  https://review.openstack.org/8647223:54
*** Chicago has joined #openstack-keystone23:54
*** Chicago has joined #openstack-keystone23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!