Monday, 2014-04-21

*** lnxnut has quit IRC00:12
*** wchrisj has joined #openstack-keystone00:13
*** zhiyan_ is now known as zhiyan00:23
*** wchrisj has quit IRC00:34
openstackgerritwanghong proposed a change to openstack/keystone: list_user_ids_for_project returns user multiple times  https://review.openstack.org/8929300:58
openstackgerritWei Wang proposed a change to openstack/keystone: add dependencies of keystone dev-enviroment  https://review.openstack.org/8047401:05
*** ayoung_afk has quit IRC01:10
*** diegows has quit IRC01:33
*** Eric88 has quit IRC02:06
*** zz_dstanek is now known as dstanek02:15
*** dims has quit IRC02:29
*** dims has joined #openstack-keystone02:30
*** dims has quit IRC02:35
*** dims has joined #openstack-keystone02:37
*** mberlin1 has quit IRC02:51
*** mberlin has joined #openstack-keystone02:51
*** zhiyan is now known as zhiyan_03:50
*** zhiyan_ is now known as zhiyan03:52
*** chandan_kumar has joined #openstack-keystone04:32
*** dstanek is now known as dstanek_zzz05:10
*** praneshp has joined #openstack-keystone05:10
*** stevemar has joined #openstack-keystone05:14
*** praneshp_ has joined #openstack-keystone05:48
*** praneshp has quit IRC05:49
*** praneshp_ is now known as praneshp05:49
jzl-ctripHi, guys, I just ran into a issue in keystoneclient's tests,which is due to json.dumps behavior on dicts06:00
openstackgerritOpenStack Proposal Bot proposed a change to openstack/keystone: Imported Translations from Transifex  https://review.openstack.org/8850306:01
jzl-ctripin some tests, we use mox to fake a HTTP server, the requeste data is logged first, and checked against a laater request06:02
jzl-ctripand all request data is seriallized in JSON.But json.dumps may return different results for a group of equal dicts06:03
*** franco34 has joined #openstack-keystone06:17
*** stevemar has quit IRC06:33
jzl-ctripsome tests assumpts json.dumps returning same string for any two equal dicts06:50
jzl-ctripthis paste showes how such tests would be breaked:> and all request data is seriallized in JSON.But json.dumps may return06:50
jzl-ctrip+different results for a group of equal dicts06:50
jzl-ctrip> and all request data is seriallized in JSON.But json.dumps may return06:50
jzl-ctrip+different results for a group of equal dicts06:50
jzl-ctriphttp://paste.openstack.org/show/76484/06:51
*** praneshp has quit IRC06:53
*** franco34 has quit IRC06:57
openstackgerritA change was merged to openstack/python-keystoneclient: Update docs for auth_token middleware config options  https://review.openstack.org/7387507:37
*** chandan_kumar has quit IRC08:50
*** henrynash has joined #openstack-keystone08:57
*** henrynash has joined #openstack-keystone08:57
*** chandan_kumar has joined #openstack-keystone09:08
*** bvandenh has joined #openstack-keystone09:16
*** henrynash has quit IRC09:26
*** zhiyan is now known as zhiyan_09:32
*** leseb has joined #openstack-keystone09:40
*** Daviey has quit IRC09:41
*** Daviey has joined #openstack-keystone09:49
*** LTM has joined #openstack-keystone09:50
LTMhi09:51
LTMi posted a week back on my devstack which exits on error "+ openstack role add --project --user"09:52
LTMdoes anyone here faced this or have a clue what causing this09:52
LTMunder Ubuntu, the stack.sh works, but RHEL it exits09:52
*** LTM has quit IRC09:58
*** chandan_kumar has quit IRC10:43
*** leseb has quit IRC10:44
*** leseb has joined #openstack-keystone10:44
*** leseb has quit IRC10:49
*** chandan_kumar has joined #openstack-keystone10:51
*** diegows has joined #openstack-keystone11:12
*** leseb has joined #openstack-keystone11:15
*** leseb has quit IRC11:19
*** erecio has joined #openstack-keystone12:11
*** leseb has joined #openstack-keystone12:15
*** leseb has quit IRC12:20
*** zhiyan_ is now known as zhiyan12:49
*** ativelkov has joined #openstack-keystone13:02
*** daneyon has joined #openstack-keystone13:06
*** daneyon has joined #openstack-keystone13:07
*** sergmelikyan has joined #openstack-keystone13:09
*** leseb has joined #openstack-keystone13:16
*** wchrisj has joined #openstack-keystone13:17
*** leseb has quit IRC13:21
*** leseb has joined #openstack-keystone13:32
*** dstanek_zzz is now known as dstanek13:42
*** nkinder has quit IRC13:42
*** richm has joined #openstack-keystone13:59
*** dstanek is now known as dstanek_zzz14:01
*** stevemar has joined #openstack-keystone14:08
*** rwsu has joined #openstack-keystone14:11
*** thedodd has joined #openstack-keystone14:11
*** doddstack has joined #openstack-keystone14:14
*** doddstack has quit IRC14:14
*** chandan_kumar has quit IRC14:14
*** doddstack has joined #openstack-keystone14:14
*** doddstack has quit IRC14:14
*** doddstack has joined #openstack-keystone14:15
*** doddstack has quit IRC14:16
*** thedodd has quit IRC14:17
*** dims has quit IRC14:27
*** dims has joined #openstack-keystone14:29
*** topol has joined #openstack-keystone14:33
*** chandan_kumar has joined #openstack-keystone14:35
*** david-lyle has joined #openstack-keystone14:36
*** nkinder has joined #openstack-keystone14:41
*** ayoung has joined #openstack-keystone14:42
*** dstanek_zzz is now known as dstanek14:51
openstackgerritDolph Mathews proposed a change to openstack/keystone: Add detailed federation configuration docs  https://review.openstack.org/8922015:14
openstackgerritDolph Mathews proposed a change to openstack/keystone: Add detailed federation configuration docs  https://review.openstack.org/8922015:15
*** topol_ has joined #openstack-keystone15:15
*** bknudson1 has joined #openstack-keystone15:16
*** topol has quit IRC15:16
*** topol_ is now known as topol15:17
*** bknudson has quit IRC15:17
stevemarthanks dolphm !15:19
*** ativelkov is now known as ativelkov_away15:23
*** ativelkov_away is now known as ativelkov15:23
*** chandan_kumar has quit IRC15:29
*** gyee has joined #openstack-keystone15:30
dstanekis there ever a need to know what plugins are currently enabled?15:33
dstaneki need to know if mine is enabled, but I don't see any generic facilities to do this15:34
*** browne has joined #openstack-keystone15:34
*** leseb has quit IRC15:35
*** daneyon has quit IRC15:36
bknudson1dstanek: extensions are advertised in v2.0/extensions15:36
*** ativelkov has left #openstack-keystone15:36
dstanekbknudson1: is that a list of only active ones?15:37
*** nekron99 has joined #openstack-keystone15:37
bknudson1dstanek: it's whatever extensions have registered themselves... if the extension isn't enabled then it wouldn't register itself15:37
bknudson1although the federation extension seems to register itself even when it's not enabled15:37
dstanekbknudson1: extensions are registered on import right? it's possible that the module is imported, but not in the pipeline15:38
dstaneklike federation, my password rotation extension does that15:39
bknudson1dstanek: they typically register themselves on import15:42
*** nekron99 has left #openstack-keystone15:43
*** gokrokve has joined #openstack-keystone15:53
*** zhiyan is now known as zhiyan_15:57
*** ativelkov has joined #openstack-keystone15:58
openstackgerritSteve Martinelli proposed a change to openstack/keystone: Add detailed federation configuration docs  https://review.openstack.org/8922015:59
bknudson1stevemar: is there a reason ^ is going to keystone and not the config reference -- http://docs.openstack.org/trunk/config-reference/content/16:08
openstackgerritSteve Martinelli proposed a change to openstack/keystone: Add detailed federation configuration docs  https://review.openstack.org/8922016:10
stevemarbknudson1, there were emails exchanged late last week, dolphm suggested placing it at keystone/docs,16:12
stevemarbknudson1, looking at the config reference now, it seems rather bare? just the sample conf and json files?16:12
bknudson1ok, if docs wants it in the config guide at least they have the contents.16:12
bknudson1stevemar: the documentation for keystone in general is pretty weak...16:13
bknudson1I tried to work on the api reference over the weekend a little bit and wound up giving up.16:13
dstanekbknudson1: so i think i'm going to change my extension to register only if it is in the pipeline and then to check if it's active i can just see if it was registered16:15
stevemarbknudson1 at the summit, we should do a session on churning out docs :) i bet we would get pretty far16:15
bknudson1stevemar: we'd need help from the doc team.16:15
bknudson1dstanek: I think that's how most of the extensions work, only register if they're in the pipeline.16:16
bknudson1dstanek: still, there's some oddities in how extension registration works.16:16
bknudson1dstanek: for example, you could potentially register a v3 extension on the admin pipeline and not on public pipeline... so should a GET /v3/extensions on public return a different list than admin?16:18
dstanekbknudson1: hmmm, yeah ... there is something missing here16:19
bknudson1(btw, there's no v3/extensions, was just using that as an example)16:21
bknudson1I think we'll have extensions listed in /v3 response16:21
bknudson1see https://review.openstack.org/#/c/87786/16:21
bknudson1I'll be working on this some more today16:22
*** branen has quit IRC16:24
dstanekbknudson1: on extensions?16:25
bknudson1dstanek: on the extension advertisement for v316:25
bknudson1first need to update the api spec with the current response for /v3 -- it just says TODO16:25
dstanekwell, at least it's accurate16:26
*** dstanek is now known as dstanek_zzz16:29
dolphmbknudson1: v3 doesn't have a separate pipeline for public vs admin16:29
dolphmbknudson1: ... nor should it be deployed that way16:30
*** leseb has joined #openstack-keystone16:36
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/keystone: Discourage use of pki_setup  https://review.openstack.org/8081916:39
*** leseb has quit IRC16:40
*** leseb has joined #openstack-keystone16:45
stevemardolphm, bknudson1, also, i don't think there is any mechanism to see if an extension is v2 or v3 specific right now16:46
dolphmstevemar: GET /v2.0/extensions ?16:46
stevemardolphm, that will include federation16:47
stevemardolphm, which is very v3 specific (groups and all)16:48
*** harlowja_away is now known as harlowja16:48
*** amcrn has joined #openstack-keystone16:49
*** leseb has quit IRC16:49
dolphmstevemar: hrm, that's a bug then. extensions not supported by v2 shouldn't be advertised by v216:55
dolphmstevemar: the v2 implementation of /extensions is basically backwards though... the application should return an empty extensions response, and extensions should inject themselves into that response16:56
*** franco has joined #openstack-keystone16:58
stevemardolphm, see https://bugs.launchpad.net/keystone/+bug/130825216:59
uvirtbotLaunchpad bug 1308252 in keystone "No way to get extensions using V3 API" [Wishlist,In progress]16:59
stevemarthe OS-fed extension is listed there17:00
dolphmstevemar: yeah, i disagree with the premise of the bug17:00
bknudson1dolphm: I thought the v3 pipeline went into the public_api and admin_api but now I see that it's a separate pipeline17:06
bknudson1I think we need a new function to register v3 extensions, then17:06
dolphmbknudson1: that's what paste does17:07
dolphmbknudson1: it composes middleware17:07
bknudson1dolphm: so extensions shouldn't have to register themselves?17:08
*** ativelkov has left #openstack-keystone17:12
dolphmbknudson1: with what?17:14
dolphmbknudson1: and for what purpose?17:14
bknudson1dolphm: so that applications know if the extension is available or not.17:15
dolphmbknudson1: api consumers?17:15
bknudson1dolphm: right, consumers of the API can find out if the extension is there (and what version it is)17:16
dolphmbknudson1: i wouldnt call that "registration"17:18
bknudson1dolphm: I don't call that registration either, registration is how the extension tells keystone that it's available so it can advertise it17:19
bknudson1so keystone can advertise that the extension is available17:19
dolphmbknudson1:the process you're describing is backwards - keystone shouldn't be aware of extensions17:19
*** esmute has left #openstack-keystone17:20
*** chandan_kumar has joined #openstack-keystone17:21
*** esmute has joined #openstack-keystone17:25
ayoungdolphm, "keystone shouldn't be aware of extensions"  why not?17:36
ayoungbknudson1, so paste does the import via the python mechansm for "string to python code"  in the filter entry, and then actually calls that in the pipeline itself.17:42
*** morganfainberg_Z is now known as morganfainberg17:42
ayoungI put in the code that "self registers" with the extensions collection.  I'd be willing to hear a better mechanism?17:42
bknudson1maybe each extension should have its own paste pipeline?17:44
ayoungbknudson1, what would that buy us?17:45
*** leseb has joined #openstack-keystone17:46
bknudson1then the extensions wouldn't be stuck under /v3, they could be /OS-FEDERATION for example17:46
morganfainbergmornin17:48
ayoungbknudson1, or just one extenstion pipeline17:49
*** leseb has quit IRC17:50
openstackgerritBrant Knudson proposed a change to openstack/identity-api: Document GET /v3  https://review.openstack.org/8939417:59
ayoungmorganfainberg, so..we were  talking about extensions, and I started thinkg IofC again, and came across these:    https://pypi.python.org/pypi/injector/  https://pypi.python.org/pypi/PyContainer17:59
morganfainbergayoung, yeah reading up in the conversation now.18:00
ayoungbknudson1, we were discussing whether the links in that GET should be links to the modules.  identity and so forth18:01
*** dstanek_zzz is now known as dstanek18:01
morganfainbergayoung, only thing that makes me cringe is pycontainer "It is configured through XML file."18:01
morganfainbergayoung, in principal i like something generic for this and the concept of self registration18:03
*** gokrokve has quit IRC18:04
dstanekayoung: snakeguice ftw18:05
ayoungdstanek, have you worked with it before?18:06
ayoungmorganfainberg, XML is out18:06
ayoungI want "configured in Python"18:06
dstanekayoung: i wrote it18:07
ayoungwith maybe a few components that can be optionally configured in a flat file like our current paste18:07
ayoungHa!18:07
dstanekit's based on google guice and configured in Pythnon18:07
ayoungdstanek, Um...then that is on the shortlist18:07
dstanekor Python if you prefer18:07
ayoungNo, I prefer Pythnon18:08
dstanekayoung: i started a PoC or replacing our stuff with it18:08
ayoungdstanek, OK...so  lets not start doing decorators as the main way we configure things.18:09
dstanekayoung: if there is growing interest i'll get something workable to push18:09
ayoungI want to say we have 3 pieces of code:18:09
ayoungrequired,  requiring, and the resolution18:09
dstanekayoung: i do use decorator, but as a way to advertise what is required of what is provided18:09
ayoungdstanek, there absolutly is18:09
ayoungdstanek, that is an antipattern, really18:09
dstanekother code is need to do the wiring18:09
dstanekayoung: how so?18:10
ayoungyou want to be able to consume other people's code18:10
ayoungand they don't do hte decorator18:10
ayoungso registering a component needs to be done outside the code you are writing18:10
dstanekayoung: you can still do that easily18:10
ayoungdstanek, yes, but decorator makes that a second class approach.  I want it first class18:11
dstanekif you want to be injected with something you use the decorator to say what, but there are ways around it18:11
ayoungdstanek, it means you write your code to the framework, and that is wrong for reusability18:11
ayounglets keep them separate18:11
dstanekayoung: you are only declaring types in the decorator18:12
ayoungI know, you would never be able to tell that I felt that way based on the code I wrote in Keystone18:12
dstanekayoung: https://code.google.com/p/snake-guice/source/browse/tests/system/test_api.py#13918:13
ayoungdstanek, I realize it is possible, and that it makes of for a lack of type safety in Pyton.  It just means that "my code is more equal than yours"  when it comes to playing with guice18:13
ayoungdstanek, I'd rather have a separate pyton file that states:  here is a component.  It has a request/session/app lifespane.  It gets name=NAME.  It requires ....18:14
ayoungog, and its class is:  pythong.class.18:14
ayoungor pythnon18:14
dstanekayoung: you can do that with snakeguice, but you loose the benefit of see the type in the file itself18:15
ayoungdstanek, understood18:15
ayoungdstanek, I miss type safety.  But we are in python.18:15
ayoungWith types you can do some better things, I agree.    http://adam.younglogic.com/2009/08/cpp-resolver/18:16
dstanekayoung: for me it's not about safety, it's about visibilty18:16
ayoungdstanek, all you get in python is "here are the names of the parameters to create the object"18:16
dstanekayoung: that's the purpose of the decorator - to tell the injector what types you want created for you18:17
ayoungIt really doesn't matter what other operations you provide, because you want to reuest other people's code.  For instance,   SQL Alechmy or LDAP connections18:17
dstanekayoung: you do that with what's called providers18:18
ayoungdstanek, but then the code you write is *only* usable without your framework.18:18
ayoungWrappers?18:18
*** praneshp has joined #openstack-keystone18:18
ayoungOr are they "factories"18:18
dstanekayoung: nope, it's just good OO; yes providers are factories18:18
ayoungdstanek, does a provider always create a new instance?18:18
dstanekno, a provider is a class with a get method. everytime get is called it must return an instance, but not necessarily a new one18:19
dstanekyou almost always want it to return a new one though18:20
dstanekit also has the concept of scopes, which give you singleton, request and custom scopes18:20
dstanekso the code doesn't have to worry about scopes - just the IoC framework18:21
dstaneki tried hard to make it work in a way that is framework agnostic for most the the application code18:21
dstanekthere is really just a few small shims (adapters) needed to work with it18:22
openstackgerritA change was merged to openstack/identity-api: Use --publish for openstack-doc-test  https://review.openstack.org/8919718:23
ayoungdstanek, POK, so those two things should be split up.18:33
ayoungyou want both a pure factory method (creates a new instance) and a method that returns the cached instance, calling the factory if it does not exist18:33
ayoungIt is the registration of the component that specifies the scope, not the code that creates the instance18:34
ayoungthe default creator is "call the constructor with these parameters"18:34
ayoungand "these parameters" are fteched from other accessor functions18:35
dstanekayoung: you are correct and that's how it works18:36
ayoungcool18:36
topoldstanek, in you patch who is setting region.is_configured ???18:36
dstanektopol: what patch?18:36
topoldstanek, https://review.openstack.org/#/c/84052/5/keystone/common/cache/core.py18:36
dstanektopol: ah, that's the dogpile API18:37
dstanekit is doing that18:37
dstanektopol: i think morganfainberg was worried that I was using an API newer that what we required in the requirements.txt18:38
topoldstanek, any idea whi morganfainberg felt like previously he had to use  if 'backend' not in region.__dict__:18:38
morganfainbergdstanek, we're waiting for requirements to merge18:38
morganfainbergonce that goes we can merge that18:38
topoldstanek, OK , that makes sense18:38
dstanekmorganfainberg: nice18:39
morganfainberghttps://review.openstack.org/#/c/84200/18:39
topolcause I figured, morganfainberg would have certainly used the simple check if it was avail :-)18:39
morganfainbergtopol, i contributed the code to dogpile to make the 'backend' not in region.__dict__ a property of the region :)18:39
morganfainbergtopol, so, as soon as we update global reqs, we can get to use the new code18:40
topolmorganfainberg, outstanding!!! Thanks18:40
dstanekis it just me or it gerrit automatically logging users out?18:45
dstanektopol: did my comment here make sense? https://review.openstack.org/#/c/73368/7/keystone/identity/backends/sql.py18:46
topoldtsanek, your response to my comment made sense. Did you update the comment ina subsequent patch or not yet?18:48
topoldstanek, what did you do to piss off jenkins :-). I noticed it like minus oned all your patches to death18:48
dstanektopol: that's what i'm doing now. bknudson1 gave me an idea on how it check for it18:49
ayoungdstanek, morganfainberg OK,  lets use snake-guice as the starting point for any IofC work we do in Keystone.  It looks right to me, and we have a desperate need for something like that18:49
topoldstanek, OK, cool!18:49
dstanektopol: ha, it was that damn neutron bug18:49
topoldtsanek, its like a line from Seinfeld... "Neumann!!" err "Neutron!!!"18:50
dstanekayoung: i'll get you something to start poking at - i started by trying to replace the internals of our dependency module, but just started to rewrite things18:50
ayoungdstanek, token provider is first18:51
ayoungwe need to build a pipeline there18:51
ayoungand it stands out as the odd-man in our current dependency module.18:51
ayoungdstanek, our notifications somehow need it, too.18:53
*** chandan_kumar has quit IRC18:55
*** chandan_kumar has joined #openstack-keystone18:56
dstanekayoung: the one thing i can't model is the circular dependency we have18:58
*** gokrokve has joined #openstack-keystone18:58
ayoungdstanek, that is a problem in most code, too.  Usually, you need a proxy18:58
dstanekothers have asked me to add that like Google Guice's feature, but i think that's bad design and I don't want to allow it :-)18:58
ayounga->b->a  so first create proxy(b), then create a with proxy(b), finally, when proxy(b) tries to access a call through a proxy as well...18:59
ayoungnotifier might help break most of our circulars, too19:00
*** browne has quit IRC19:00
*** browne has joined #openstack-keystone19:00
*** wchrisj has quit IRC19:00
*** browne1 has joined #openstack-keystone19:01
ayoungdstanek, so circular, as I recall, was only necessary because identity and assignment referred to each other.  We should not need that, as identity  should not need to talk directly to assignmnet19:02
dstanekayoung: i don't remember because i brought it up a while ago - at the time morganfainberg said it would probably be going away naturally anyway19:03
ayoungdstanek, I'm looking, but it might even be gone already19:03
*** browne has quit IRC19:04
ayoungbknudson1, dstanek, morganfainberg, this is a prereq to compression  and should be pushed on a head.  https://review.openstack.org/#/c/79411/619:13
*** arunkant has joined #openstack-keystone19:15
morganfainbergayoung, ++ will look at post lunch19:17
*** wchrisj has joined #openstack-keystone19:18
ayoungdstanek, so...the main (only?) thing that identity requires the assignment API for is the Domain enumeration, which you could argue should not be part of the Assignment API anyway.19:22
ayoungand there are comparable things from assignment into identity, revolving mostly around groups.19:24
*** dims has quit IRC19:25
*** dims has joined #openstack-keystone19:27
*** erecio has quit IRC19:33
*** erecio has joined #openstack-keystone19:33
*** chandan_kumar has quit IRC19:34
*** leseb has joined #openstack-keystone19:40
bknudson1Can someone else try this: tox -e py27 keystone.tests.test_sql_upgrade.VersionTests.test_extension_initial19:44
morganfainbergsec will try in a moment19:44
dolphmbknudson1: AttributeError: 'NoneType' object has no attribute 'startswith' http://pasteraw.com/3y16tiiuo562fuavfrmyc7ligk9uqw819:45
morganfainbergdolphm,bknudson1 same19:45
*** leseb has quit IRC19:46
bknudson1dolphm morganfainberg: thanks, wanted to make sure it wasn't just me.19:46
*** dhellmann has joined #openstack-keystone19:47
morganfainbergbknudson1, we must no longer be setting the default connection string in all test-cases19:47
dhellmannmorganfainberg: have a sec for a test question?19:47
morganfainbergdhellmann, sure19:48
dhellmannI'm having some trouble with the logic in http://git.openstack.org/cgit/openstack/keystone/tree/keystone/tests/ksfixtures/database.py#n7919:48
bknudson1maybe CONF isn't getting CONF'd19:48
morganfainbergdhellmann, the auto-find and import sql.py files?19:49
dhellmannI'm trying to package keystone from master, and run the tests. In order to run the tests in the virtualenv I'm creating, I change tox.ini to set a couple of variables differently: http://paste.openstack.org/show/76559/19:49
*** stevemar has quit IRC19:49
dhellmannmorganfainberg: yeah19:49
dhellmannit seems that setting usedevelop=False breaks that function19:49
morganfainbergoh oh i see how it could.19:50
morganfainbergdoh*19:50
*** stevemar has joined #openstack-keystone19:50
*** wchrisj has quit IRC19:50
dhellmannbecause it finds a path for the module with a full path, and the / becomes a . and then import complains that '.home.whatever.some.long.path' has an empty module name19:50
dhellmannat the front there19:50
dhellmannmorganfainberg: so we could fix it a couple of ways, either by doing more smart work with the paths, or by converting those modules to be loaded with stevedore19:51
dhellmannmorganfainberg: before I submit a patch, I thought I'd see which you all would prefer19:51
morganfainbergdhellmann, the latter would be better19:51
morganfainbergdhellmann, i would rather use stevedore for all of our backends eventually, it makes sense that the tests should be the same19:52
dhellmannok, I concur, but didn't know if there was some reason it was done this way to begin with19:52
morganfainbergdhellmann, we haven't converted stuff to stevedore yet, afaict that is the reason19:52
dhellmannmorganfainberg: ok, I'll go see about that -- would it help if I file a bug, too?19:52
morganfainbergdhellmann, please do!19:53
dhellmannmorganfainberg: sure thing, thanks!19:53
morganfainbergdhellmann, awesome :)19:53
dhellmannmorganfainberg: do you mind if I just paste this irc chat into the bug description?19:54
morganfainberggo right ahead19:54
dhellmannmorganfainberg: https://bugs.launchpad.net/keystone/+bug/131076819:54
uvirtbotLaunchpad bug 1310768 in keystone "keystone tests fail unless tox is configured "just so"" [Undecided,New]19:55
*** thedodd has joined #openstack-keystone19:55
morganfainbergdhellmann, looks good to me19:55
ayoungbknudson1, I'm trying to shepherd the comporession patch on through.  I made the changes you suggested in Set 5   https://review.openstack.org/#/c/79411/5  which lead to me having to put in a stub for keeping the old method name around.19:57
dhellmannmorganfainberg: wow, lots of backends :-)19:57
morganfainbergdhellmann, yep19:57
morganfainbergdhellmann, :)19:57
morganfainbergbiggest win with stevedore, custome backends can be installed with the entrypoint logic stuff19:58
*** amcrn has quit IRC19:58
morganfainbergdhellmann, big win for me (personally).19:58
dhellmannmorganfainberg: how about a patch that registers them, without changing the core code to use them yet? I'm not sure I'm up to that big of a change this week...19:59
morganfainbergdhellmann, small steps of course19:59
morganfainbergdhellmann, i wouldn't want it all changed at once19:59
dhellmannmorganfainberg: ok, that much I can do :-)19:59
morganfainbergdhellmann, besides, we shouldn't expect _you_ to be the only one working on it.19:59
bknudson1ayoung: stub?20:01
ayoungbknudson1, yeah...or am I thinking of the follow on patch...wait one20:01
ayoungbknudson1, I was thinking follow on: https://review.openstack.org/#/c/71181/27/keystoneclient/common/cms.py  line 28720:02
ayoungso https://review.openstack.org/#/c/79411/5   should have all of your recommendations covered20:02
bknudson1ayoung: ok, now just have to figure out what it's for.20:03
ayoungbknudson1, the patch?20:04
bknudson1why did we just add universal newlines and now we're removing it again20:04
ayounguniversal newlines implies you are doing text only IPC (pipes between parent and child)20:05
bknudson1oh, it mucks with the input?20:05
ayoungyeah20:05
bknudson1ok, makes sense20:05
ayoungbknudson1, I was burnt by it when I went to do compression on python3320:05
ayoungpy33 does strings very different from py27 and popen assumes that "universal_newlines"  means "it is all string"  but for compression, it is binary data20:06
ayoungI split it out from the follow on patch that does the compression because it was a stand along concept, and easier to understand in its own patch, but really only needed for compression20:07
bknudson1"all line endings will be converted to '\n'" -- so it probably causes a \r\n to switch to \n or something.20:08
*** erecio has quit IRC20:09
*** wchrisj has joined #openstack-keystone20:10
*** erecio has joined #openstack-keystone20:10
*** erecio has quit IRC20:10
*** erecio has joined #openstack-keystone20:11
*** leseb has joined #openstack-keystone20:13
*** erecio has quit IRC20:14
*** erecio has joined #openstack-keystone20:14
*** erecio has quit IRC20:15
*** erecio has joined #openstack-keystone20:15
ayoungbknudson1, that is true, and it corrupts binary data20:15
*** browne1 has quit IRC20:15
morganfainbergayoung, do we have a BP we're using for the work? should this be associated with the compressed tokens one (if we have it)?20:16
bknudson1morganfainberg: the follow-on patch has Blueprint: compress-tokens20:16
ayounghttps://blueprints.launchpad.net/python-keystoneclient/+spec/compress-tokens20:17
morganfainbergbknudson1, ah.20:17
morganfainbergthis probably should be linked to the BP as well, but not going to block on that.20:17
morganfainbergayoung, looks like most of the code is mechanisms around the bytearray useage20:19
morganfainbergayoung, am i wrong?20:19
morganfainbergsome ' vs " changes for consistency20:19
*** daneyon has joined #openstack-keystone20:21
bknudson1ayoung morganfainberg: posted my comments on https://review.openstack.org/#/c/79411/20:21
morganfainbergbknudson1, i agree with the comments 100%20:26
morganfainbergbknudson1, the decode one was the one i was actually going to make (but you beat me to it)20:26
dhellmannmorganfainberg: are the backends under keystone.common different from the ones not in common?20:31
dhellmannmorganfainberg: the names don't seem to follow the same pattern20:31
ayoungthe ' to " was done as a response to your code revewi comments.  Grumble Grumble.20:31
bknudson1I commented on one line to use '20:32
ayoung I agre on the output one, though...let me get that right....20:32
morganfainbergdhellmann, hmm. possibly different20:32
dhellmannmorganfainberg: ok, I can do those by hand20:32
ayoungbknudson1, yes. but the comment indicated that it should be consistant throughout....20:32
morganfainbergdhellmann, the ones in common will (hopefully) this cycle move to the oslocache stuff20:32
dhellmannmorganfainberg: should I just leave them out?20:33
bknudson1ayoung: I agree it should be consistent throughout since that's in the keystone coding guideline.20:33
* ayoung just grumlbing, but actually agrees20:33
morganfainbergdhellmann, i'd do them by hand for now20:33
dhellmannok20:33
morganfainbergdhellmann, if it's not too much extra20:33
morganfainbergwork*20:33
morganfainbergdoens't hurt to be consistent, might justify renaming them and aliasing the old names to be more consistent in either casew20:34
morganfainbergdhellmann, but just adding them now should be sufficient (other changes down the line)20:34
dhellmannmorganfainberg: ok, there are just a few -- the others I have a script to find20:35
morganfainbergdhellmann, cool. thanks for doing this :)20:35
openstackgerritBrant Knudson proposed a change to openstack/keystone: Fix sql_upgrade tests run by themselves  https://review.openstack.org/8941520:37
morganfainbergbknudson1, +2 LGTM ^20:38
bknudson1morganfainberg: oh oh... might have posted that too soon.20:40
bknudson1now the tests run by themselves but other tests don't run.20:40
morganfainbergbknudson1, really?20:41
bknudson1"table credential already exists ..." from keystone.tests.test_sql_upgrade.SqlUpgradeTests.test_upgrade_service_enabled_cols20:41
morganfainbergthat change shouldn't... shouldn't break anything?20:41
morganfainberghmm20:41
morganfainbergoh i know why20:41
morganfainbergthe fixture was designed to create the in-mem db20:42
morganfainbergwe need to call the _initialize_sql_session() function in the database fixture module20:43
morganfainbergnot use the actual fixture20:43
bknudson1so they can't use the fixture20:43
morganfainbergbknudson1, correct.20:43
morganfainbergmake this non-internal https://github.com/openstack/keystone/blob/master/keystone/tests/ksfixtures/database.py#L59 or make a db_setup fixture that just calls that20:44
bknudson1I just added the fixture so that it would call the function.20:44
morganfainbergyeah, the fixture does the whole reflection db creation20:44
dstanekyeah, it does lots of work20:45
bknudson1some kind of composite fixture??20:45
morganfainbergbknudson1, could make a DBSetup fixture that the Database fixture inherits20:46
morganfainbergbknudson1, the dbsetup fixture could just call that setup function20:46
bknudson1I think fixtures can do useFixture.20:46
morganfainbergbknudson1, oh if it can, cool.20:46
bknudson1seems like overkill20:46
morganfainbergbknudson1, not sure which is better, a setup-specific fixture or just directly calling the function20:47
dstanekyeah, they can use useFixture20:47
dstanekbknudson1: not a composite, but creates the full DB schema before each test and removes it after20:47
bknudson1I'll just make initialize_sql_session and call that... should be safe20:48
morganfainbergsounds good to me20:48
bknudson1I'll even try it out first this time20:48
morganfainberghehe20:49
*** erecio has quit IRC20:54
openstackgerritBrant Knudson proposed a change to openstack/keystone: Fix sql_upgrade tests run by themselves  https://review.openstack.org/8941520:54
openstackgerritDoug Hellmann proposed a change to openstack/keystone: Register all backend classes as entry points  https://review.openstack.org/8941921:02
openstackgerritDoug Hellmann proposed a change to openstack/keystone: Register all backend classes as entry points  https://review.openstack.org/8941921:05
openstackgerritDoug Hellmann proposed a change to openstack/keystone: Move stevedore to a production requirement  https://review.openstack.org/8942021:08
*** browne has joined #openstack-keystone21:09
ayoungbknudson1, morganfainberg so....the decode does not make sense.  You might be making this call with binaryd data.  but err is the error message, so you need to decode it to be able to print.21:14
morganfainbergayoung, the default for that method should return the same as current. if you want binary back it should be requested as such... or a separate method21:14
bknudson1ayoung: cms_sign_text decodes output21:15
bknudson1as does cms_verify in auth_token21:15
ayoungbknudson1, right now, yes, but there is going to be a step inbetween in the future21:15
ayounghttps://review.openstack.org/#/c/71181/21:16
bknudson1ayoung: I'd be ok if the docstring was updated to say that the output is bytes or whatever.21:16
ayoungbknudson1, ++ I can get behind that21:16
morganfainbergi guess we are currently the only consumers of this21:18
morganfainbergso sure21:18
ayoungmorganfainberg, bknudson1 so...even err is not decoded unless we are appending our own error message21:23
bknudson1that's weird21:23
morganfainbergayoung, hm.21:23
ayoungbknudson1, yeah21:23
ayounghttps://review.openstack.org/#/c/79411/6/keystoneclient/common/cms.py21:23
morganfainbergayoung, i don't like behavior that changes like that21:23
ayounglook at the21:23
dstaneklooking at that patch now and part of what confuses me is that i don't know what should return bytes vs. string21:23
ayoung _process_communicate_handle_oserror  around line 7521:24
ayoungdstanek, this is the confusion caused by py27 py33 in the same code base21:24
ayoung output, err = process.communicate(data)21:24
*** topol has quit IRC21:24
ayoungin some cases output is text, in others binary, depending on what is processed.  So we assume binary for all cases21:25
ayoungI could do something like:21:25
ayoungif err:  err = err.decode('utf-8')21:25
ayoungright before the return call at line 9621:26
ayoungthen err would always be text21:26
bknudson1ayoung: I like that change, err should always be text.21:27
ayoungbknudson1, OK.  I can make that happen.21:28
dstanekayoung: i like the idea of hiding away all encoding/decoding in _process_communicate_handle_oserror - so nothing else has to change21:29
dstanekwould that be possible?21:29
ayoungdstanek, nope21:29
ayoungdstanek, it needs to return binary21:29
dstanekwho is expecting the binary21:29
ayoungdstanek, in comporessed, we take tex, sign it into a binary format, compress it, then base64.  To verify, reverse the process21:30
ayounghttps://review.openstack.org/#/c/71181/27/keystoneclient/common/cms.py  dstanek21:30
ayoungsee line 175 ish21:31
*** wchrisj has quit IRC21:31
ayoungdstanek, but error comes back as text, and we need to actually inspect the error message for proper error handling.  openssl cms bascially returns a single error code, and we need the text to know if it is a cert missing, or something wrong with the input21:32
*** franco has quit IRC21:32
dstanekwhat uses the output from cms_verify?21:32
*** jamielennox|away is now known as jamielennox21:32
*** franco has joined #openstack-keystone21:32
ayoungOooh, just realized who https://github.com/mathrock  is.  Glad to see he is able to contribute directly.21:33
ayoungdstanek, I think I mix in error processing for that to the big patch.  That is something else that can be split out.21:34
morganfainbergayoung, hahah21:34
*** openstackstatus has quit IRC21:35
*** openstackstatus has joined #openstack-keystone21:36
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Remove _factory methods from auth plugins  https://review.openstack.org/8198521:38
openstackgerritJamie Lennox proposed a change to openstack/keystone: Move mutable parameter checking into federation  https://review.openstack.org/8784921:43
bknudson1global state is a disaster waiting to happen21:44
jamielennoxbknudson1: in relation to?21:44
morganfainbergbknudson1, global state?21:44
bknudson1testing.21:44
bknudson1e.g., the extension registry21:44
bknudson1same with the dependency registry21:45
jamielennoxbknudson1: ok - yea completely agree21:46
openstackgerritayoung proposed a change to openstack/python-keystoneclient: replace double quotes with single.  https://review.openstack.org/8942821:46
openstackgerritayoung proposed a change to openstack/python-keystoneclient: remove universal_newlines  https://review.openstack.org/7941121:46
dstanekbknudson1: extension registry?21:47
ayoungbknudson1, that is only modified at startup21:47
bknudson1ayoung: how about when running tests? then you don't know when it's going to be modified21:47
jamielennoxbknudson1: i'd love to kill that dependency resolution - it's unecessary21:48
dstanekjamielennox: what dependency resolution?21:48
bknudson1dstanek: there's an extension registry -- http://git.openstack.org/cgit/openstack/keystone/tree/keystone/common/extension.py21:48
ayoungjamielennox, so we were talking about using dstanek 's project in its place:  nake-guice21:48
jamielennox@provides @depends21:48
dstanekjamielennox: i like the idea there i just don't like that they are automatically created21:49
jamielennoxdstanek's project?21:49
dstanekjamielennox: https://code.google.com/p/snake-guice/21:49
dstaneki think i need to dust off the docs21:49
*** franco has quit IRC21:56
*** franco has joined #openstack-keystone21:57
openstackgerritayoung proposed a change to openstack/python-keystoneclient: Compressed Signature and Validation  https://review.openstack.org/7118122:00
openstackgerritBrant Knudson proposed a change to openstack/keystone: Allow registering a v3 extension  https://review.openstack.org/8942922:01
openstackgerritBrant Knudson proposed a change to openstack/keystone: Register v3 extensions as v3 extensions  https://review.openstack.org/8943022:01
openstackgerritBrant Knudson proposed a change to openstack/keystone: Advertise extensions for v3  https://review.openstack.org/8778622:01
*** franco has quit IRC22:02
*** franco has joined #openstack-keystone22:02
bknudson1think we could register the extension when the router is created rather than on import?22:03
ayoungjamielennox, so...for the client Kerberos plugin, I prefer "external"  to "kerberos" as the method.  It works without server side changes22:05
ayoungbknudson1, that is fine, but how are we going to know if the router is created?22:06
ayoungright now, when the import happens, we register the extension, and that is what creates the router22:06
jamielennoxayoung: i don't mind - i wanted kerberos because it was explicit and we could fall back to doing kerberos i python22:08
ayoungjamielennox, yeah, but kerberos plugin is much different from external.  I am not certain we are even going to get the python version, and if we don't we end up having to duplicate each of the "eternal" plugins just to change their method strings22:09
ayoungserver side plug in is much different22:09
bknudson1ayoung: the router is referenced in the paste pipeline, e.g., http://git.openstack.org/cgit/openstack/keystone/tree/etc/keystone-paste.ini#n4622:09
dstanekbknudson1, ayoung: i just implemented my own factory method and use that to register my extension - that gets called by paste22:09
bknudson1dstanek: why don't we do that will all the extensions?22:09
dstanekbknudson1: we could probably do that. want me to make a patch?22:10
*** franco has quit IRC22:10
bknudson1dstanek: yes, give it a shot22:10
bknudson1dstanek: I think we'll have to use the 'factory' method since that's what the paste file refs...22:10
bknudson1unless we want to change the paste file22:11
bknudson1dstanek: and I don't know why the v3 extensions are registering as admin and public extensions.22:11
*** leseb has quit IRC22:12
dstanekbknudson1: probably copy/paste code - i'll put together a quick patch to start poking at22:16
bknudson1dstanek: https://review.openstack.org/#/c/89430/ registers v3 extensions as v3 extensions22:17
bknudson1I tried writing unit tests for extension registration but it's global state so affects everything & pretty much untestable22:18
dstanekah, nice - i'll base my work on that review22:18
dstanekhow does copyright work in relation to moving logic around? if you have a file that has a copyright headers and move the contents into a different why what needs to happen (if anything)?22:26
dstaneki'm thinking about this review https://review.openstack.org/#/c/87849/222:26
bknudson1dstanek: those parts were just moved and now they're moving back.22:34
bknudson1like a hot potato22:34
dstanekinteresting...so someone added them to a new file and then added their copyright? now they are moving back?22:38
bknudson1dstanek: https://review.openstack.org/#/c/72581/22:41
*** thedodd has quit IRC22:42
dstanekbknudson1: this code looks be original and copyrighted to CERN https://review.openstack.org/#/c/84389/13/keystone/tests/test_v3_controller.py22:44
dstanekin the new review it is moved is an existing file without CERNs copyright22:45
bknudson1I can't believe anyone could be so callous about intellectual property rights!22:45
openstackgerritBrant Knudson proposed a change to openstack/keystone: Advertise extensions for v3  https://review.openstack.org/8778622:46
dstanekbknudson1: ha ha, i agree!22:46
dstaneki really don't like the fact that the starting point an extension is a router22:50
*** nkinder_ has joined #openstack-keystone22:52
*** nkinder has quit IRC22:55
*** david-lyle has quit IRC23:03
openstackgerritSteve Martinelli proposed a change to openstack/python-keystoneclient: Add request/access token and consumer support for keystoneclient  https://review.openstack.org/8198023:19
openstackgerritSteve Martinelli proposed a change to openstack/python-keystoneclient: Authenticate via oauth  https://review.openstack.org/8198123:20
*** gokrokve has quit IRC23:27
stevemardstanek, i thought lbragstad1 was going to leverage the immutable/mutable param checking23:27
stevemardstanek, bknudson1 could I get eyes on https://review.openstack.org/#/c/81980/5 ? I made a change to not depend on oauthlib being installed, and would appreciate some feedback in the comments23:30
stevemardstanek, bknudson1 now that https://review.openstack.org/#/c/80698/ has merged, we'll be able to avoid gate breaks in the future :)23:30
openstackgerritA change was merged to openstack/keystone: add dependencies of keystone dev-enviroment  https://review.openstack.org/8047423:37
*** nkinder_ has quit IRC23:41
*** topol has joined #openstack-keystone23:42
jamielennoxstevemar: oh - was that there immutable work there on purpose?23:43
jamielennoxbknudson1, dstanek: i didn't realize that those had just been moved23:44
jamielennoxi was looking at moving controllers over to pecan and therefore cleaning them up first and wanted to remove everything that isn't absolutely necessary23:45
jamielennoxif there is a purpose for that work then i'm happy to have that patch -1/-2ed - though if there is a purpose to that work i would really expect there to be at least one dependant patch rather than moving it speculatively23:48
*** openstackgerrit has quit IRC23:50
*** openstackgerrit has joined #openstack-keystone23:50
*** franco has joined #openstack-keystone23:52
*** gokrokve has joined #openstack-keystone23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!