Wednesday, 2015-03-11

samueldmqjamielennox, k makes sense, thanks00:00
*** r-daneel has quit IRC00:01
jamielennoxsamueldmq: the whole interface is an annoying left over from before these things became widespread00:01
*** zzzeek has left #openstack-keystone00:02
stevemarmorganfainberg, lhcheng doing stuff atm, get back to y'all00:03
openstackgerritIan Cordasco proposed openstack/oslo.policy: Publicize oslo_policy.opts.register
*** tsufiev_ has quit IRC00:08
*** tsufiev_ has joined #openstack-keystone00:14
samueldmqjamielennox, ok so the code makes sense, but I am not familiar enough with that workflow00:14
jamielennoxsamueldmq: sure, the auth_plugin stuff?00:15
samueldmqjamielennox, cant review it .. sorry, I need to get more familia with keystonemiddleware00:15
jamielennoxsamueldmq: that's fine - anything you want explained?00:15
samueldmqjamielennox, yes, keystonemiddleware basically holds auth_plugin, right?00:15
jamielennoxsamueldmq: yes it loads a plugin that it holds on to00:15
jamielennoxyou need a valid token to authenticate another token00:16
*** iamjarvo has quit IRC00:16
samueldmqjamielennox, and then the plugin holds that token other services use to validate user's tokens?00:16
jamielennoxsamueldmq: yep, first you load an auth plugin, then you attach it to a session (or adapter) then any requests you make through the session are authenticated00:17
*** tsufiev_ has quit IRC00:17
jamielennoxand it will handle things like refreshing the token if it expires00:17
jamielennoxand service catalog etc00:18
samueldmqjamielennox, so services can use those sessions as tehy had tokens that never expire00:19
samueldmqjamielennox, makes sense, so it holds this common logic that would be replicated over services00:20
jamielennoxservices use that token, but this is the way we are trying to make all the clients work00:20
*** dims_ has joined #openstack-keystone00:20
dstanekjamielennox: i love '!!! - UNDER NO CIRCUMSTANCES COPY ANY OF THIS CODE - !!! '00:20
jamielennoxso instead of passing an auth_url and username/password you pass in a session and auth plugin00:20
jamielennoxdstanek: :) it's about as strong as i could make it00:20
jamielennoxit means all the handling around auth and token management is handled in the one place00:21
samueldmqjamielennox, great! I'll take a deeper look into the code/available docs00:21
samueldmqjamielennox, thank you sir :)00:21
jamielennoxso next time we add a auth format we just make a new plugin00:21
jamielennoxsamueldmq: no problem00:21
dstanekjamielennox: so is this not a real bug?
openstackLaunchpad bug 1411625 in python-keystoneclient "Keystone client (command line) not throwing any exception when there is no proper OS credentails." [Undecided,In progress] - Assigned to Cyril Roelandt (cyril-roelandt)00:21
*** dims has quit IRC00:22
*** dims_ has quit IRC00:22
jamielennoxdstanek: i swear i've reviewed so many versions of this same problem00:23
*** dims has joined #openstack-keystone00:23
jamielennoxdstanek: in that case i think the best we can do is provide a meaningful error if we get back an unscoped token00:24
*** gokrokve has quit IRC00:24
*** tsufiev_ has joined #openstack-keystone00:25
jamielennoxinteresting that nova requires the argument00:25
*** iamjarvo has joined #openstack-keystone00:33
*** david-lyle_afk has quit IRC00:34
*** iamjarvo has quit IRC00:38
jamielennoxstevemar: does help with your horizon federation stuff00:46
jamielennoxi'm hoping it does00:47
jamielennoxlhcheng: you'll be interested in ^ too00:47
openstackgerritwanghong proposed openstack/keystone: add timestamp to project and role
openstackgerritwanghong proposed openstack/keystone: make credential policy check ownership of credential
*** zzzeek has joined #openstack-keystone00:58
*** zzzeek has quit IRC00:59
*** zzzeek has joined #openstack-keystone01:00
openstackgerritwanghong proposed openstack/keystone: move region and service exist checks into manager layer
*** _cjones_ has quit IRC01:11
*** ncoghlan has joined #openstack-keystone01:13
*** samueldmq_ has joined #openstack-keystone01:17
*** samueldmq has quit IRC01:20
stevemarjamielennox, looking soon, i'm behind on like 10 pings01:20
stevemari was apparently popular this evening01:20
*** david-lyle_afk has joined #openstack-keystone01:21
jamielennoxthis is what happens when you get a reputation for being a night owl01:21
*** topol has joined #openstack-keystone01:22
*** ChanServ sets mode: +v topol01:22
*** topol has quit IRC01:27
*** dims has quit IRC01:28
*** dims has joined #openstack-keystone01:29
*** devlaps has joined #openstack-keystone01:31
*** dims has quit IRC01:34
dstanekstevemar: !01:34
dstanekstevemar: what do you think of this: ?01:35
stevemardstanek, i'm here now, everyone can relax01:35
stevemardstanek, :) i saw that in the irc logs, taking a look now01:36
*** dims has joined #openstack-keystone01:40
stevemardstanek, so without looking at the test, i'd say that your mapping should result in the user's name, with groups dev and manager01:43
stevemaroops, negative on that, i thought it was a whitelist, its a blacklist01:43
stevemarso it should just be contractor01:44
stevemarlooks like the user name got matched with 'maybe' instead of the username01:45
stevemarthe group name is still good (that's a plus)01:45
dstanekstevemar: that was to highlight the append vs extend possible issues01:48
stevemardstanek, yep, i see that01:49
dstanekin the case i hacked together is Thing is a single value then i would work, but it if's a list then boom!01:49
stevemari see i see01:49
*** samueldmq_ is now known as samueldmq01:50
samueldmqdolphm, lbragstad any of you around? I'd like to have a quick discussion around Fernet tokens :)01:51
*** devlaps has quit IRC01:51
*** rwsu is now known as rwsu-afk01:52
*** amerine has quit IRC01:54
*** dims_ has joined #openstack-keystone01:55
*** amerine has joined #openstack-keystone01:55
*** diegows has quit IRC01:55
*** dims has quit IRC01:58
openstackgerritJamie Lennox proposed openstack/python-keystoneclient: Split v3 authentication file into module
*** browne has quit IRC02:04
*** spandhe has quit IRC02:06
*** harlowja_ is now known as harlowja_away02:09
jamielennoxmarekd: let me know when you come on02:12
openstackgerritDave Chen proposed openstack/keystonemiddleware: Crosslink to other sites that are owned by Keystone
*** gokrokve has joined #openstack-keystone02:26
*** erkules_ has joined #openstack-keystone02:29
*** zzzeek has quit IRC02:30
*** erkules has quit IRC02:32
openstackgerritDave Chen proposed openstack/python-keystoneclient: Crosslink to other sites that are owned by Keystone
*** david-lyle_afk has quit IRC02:36
openstackgerritDave Chen proposed openstack/python-keystoneclient: Crosslink to other sites that are owned by Keystone
openstackgerritMerged openstack/pycadf: Clean up pycadf's doc landing page
openstackgerritMerged openstack/pycadf: Fix formatting error for geolocation note
*** kaisers3 has joined #openstack-keystone02:42
*** kaisers has quit IRC02:43
DaveChensamueldmq: hi,02:44
samueldmqDaveChen, hi :)02:44
DaveChensamueldmq: thanks for reviewing, you are really a good reviewer!02:44
samueldmqDaveChen, not that much, but thanks02:45
DaveChensamueldmq: seem I am a blind man :)02:45
samueldmqDaveChen, you're welcome, just trying to get things moving :)02:45
samueldmqDaveChen, nah, thanks for submitting this, we appreciate your help02:46
DaveChensamueldmq: just try to make the bug is fully addressed.02:47
openstackgerritJamie Lennox proposed openstack/python-keystoneclient: Extract BaseAuth out of Auth Plugin
openstackgerritJamie Lennox proposed openstack/python-keystoneclient: Add a FederatedBase v3 plugin
openstackgerritDave Chen proposed openstack/keystonemiddleware: Crosslink to other sites that are owned by Keystone
*** jdennis1 has joined #openstack-keystone02:57
*** jdennis has quit IRC02:57
openstackgerritDave Chen proposed openstack/keystonemiddleware: Crosslink to other sites that are owned by Keystone
openstackgerritDave Chen proposed openstack/keystonemiddleware: Crosslink to other sites that are owned by Keystone
stevemaroh thx DaveChen :)03:01
openstackgerritMerged openstack/pycadf: add a new set of release notes
DaveChenstevemar: maybe I still need resubmit your patch to follow samueldma's comments.03:02
DaveChenstevemar:np, sir.03:02
*** dims_ has quit IRC03:02
openstackgerritDave Chen proposed openstack/python-keystoneclient: Crosslink to other sites that are owned by Keystone
*** david-lyle_afk has joined #openstack-keystone03:17
*** gokrokve has quit IRC03:18
openstackgerritDave Chen proposed openstack/keystone: Crosslink to other sites that are owned by Keystone
samueldmqstevemar, DaveChen just to nits on
samueldmqsorry about that, but just take a look to check if it makes sense03:32
*** stevemar has quit IRC03:34
samueldmqDaveChen, well, just look at and make the docs consistent thorugh keystonemiddleware and keystoneclient03:39
DaveChensamueldmq: looks better, it seems like an english testing. :)03:40
samueldmqDaveChen, and then everything would be ok03:40
samueldmqDaveChen, haha yeah03:40
samueldmqDaveChen, I need to go sleep now, thanks for your patches03:40
DaveChensamueldmq: have a good sleep.03:40
DaveChensamueldmq: Good night03:40
samueldmqDaveChen, thanks, same for you03:41
*** samueldmq is now known as samueldmq_away03:41
*** jdennis1 has quit IRC03:42
*** jdennis has joined #openstack-keystone03:43
*** topol has joined #openstack-keystone03:48
*** ChanServ sets mode: +v topol03:48
*** samueldmq_away has quit IRC03:53
*** topol has quit IRC03:59
*** iamjarvo has joined #openstack-keystone04:00
*** dims has joined #openstack-keystone04:03
*** spandhe has joined #openstack-keystone04:08
*** dims has quit IRC04:10
morganfainbergjamielennox, so we have issues with keystonemiddleware and resolving juno/icehouse requirements.04:11
jamielennoxmorganfainberg: yea, it's a gate thing04:11
morganfainbergjamielennox, need me to chase this at all or you got it?04:12
*** spandhe_ has joined #openstack-keystone04:12
openstackLaunchpad bug 1430592 in devstack "testtools-1.7.0 triggering pkg_resources.VersionConflict: (unittest2 0.5.1 (/usr/lib/python2.7/dist-packages), Requirement.parse('unittest2>=1.0.0'))" [Undecided,In progress] - Assigned to Ian Wienand (iwienand)04:12
*** spandhe has quit IRC04:12
*** spandhe_ is now known as spandhe04:12
jamielennoxsomeone said in the meeting earlier that the gate had been really stable recently04:13
jamielennoxnot so much from where i've been sitting04:13
jamielennoxmight just be i'm touching a lot more projects now04:14
openstackgerritMerged openstack/keystone: Refactor: remove dep on trust_api / v3 token helper
*** richm has quit IRC04:38
*** rushiagr_away is now known as rushiagr04:58
*** iamjarvo has quit IRC05:14
*** browne has joined #openstack-keystone05:16
*** jorge_munoz has quit IRC05:29
openstackgerritDave Chen proposed openstack/python-keystoneclient: Crosslink to other sites that are owned by Keystone
openstackgerritDave Chen proposed openstack/keystonemiddleware: Crosslink to other sites that are owned by Keystone
*** panbalag has quit IRC05:46
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Imported Translations from Transifex
*** panbalag has joined #openstack-keystone06:20
lhchengfinally got my keystone running OIDC \o/06:21
lhchengsurprisingly not that bad06:22
lhchenggood job to everyone who worked on it!06:22
*** henrynash has joined #openstack-keystone06:44
*** ChanServ sets mode: +v henrynash06:44
*** browne has quit IRC06:47
*** spandhe has quit IRC06:58
openstackgerritJamie Lennox proposed openstack/python-keystoneclient: Extract BaseAuth out of Auth Plugin
openstackgerritJamie Lennox proposed openstack/python-keystoneclient: Add a FederatedBase v3 plugin
openstackgerritJamie Lennox proposed openstack/python-keystoneclient: Split v3 authentication file into module
*** kaisers3 has quit IRC07:15
lhchenganyone around familiar what should be set in the remote_id in IdP?07:17
lhchengis that an arbitrary string or I need to figure out what entity_id my IdP is using.07:18
openstackgerritAbhishek Kekane proposed openstack/keystone: Eventlet green threads not released back to pool
*** henrynash has quit IRC07:35
marekdlhcheng: i am.07:42
*** henrynash has joined #openstack-keystone07:42
*** ChanServ sets mode: +v henrynash07:42
marekdlhcheng: for sure this is part of the SAML protocol, not 100% sure if also part of OIDC (though it should!)07:42
lhchenggood morning marekd!07:43
marekdlhcheng: remote_id is an identified unique for an IdP, so you know that actually the asserion is issued by a trusted IdP X, not IdP Y07:43
marekdlhcheng: parameter where it is stored in the assertion may vary per IdP type (implementation) and you should know this apriori07:44
marekdyou == cloud admin.07:44
lhchenghmm is the remote_id_attribute is used to look-up the remote_id from the assertion?07:45
marekdlhcheng: yes, remote_id_attbute specified the name of the parameter where IdP's remote_id will be conveyed.07:46
marekdlhcheng: if you leave it empty Keystone will simply not look for this parameter.07:48
lhchengyeah, saw somewhere that for backward compatibility the remote_id can be empty07:48
lhchengso right now, I am trying to setup websso
lhchengI bumped into the remote_id config in here07:49
*** pnavarro has joined #openstack-keystone07:50
lhchengmarekd: ugh, the remote_id setting is required for the websso07:51
marekdlhcheng: ah, websso is a different story. in fact you must have remote_ids configured for websso :/07:51
marekdlhcheng: yes.07:51
marekdlhcheng: i suggest digging in the oidc specification or asking on the ml about that parameter.07:52
lhchengmarekd: okay, at least now I know I need to figure out the remote_id somewhere  and it is not an arbitrary value07:52
marekdlhcheng: no, it is super important param for the security07:53
lhchengmarekd: yeah, I understand that now07:53
lhchengmarekd: the config docs wasn't that clear, but when I read the BP it provided me some context07:53
marekdand it's required for websso as we query for identity_provider object in the backend  with that parameter value.07:53
marekdlhcheng: if the docs are not clear feel free to give me or anybody else some hints (or file a bug) and I am happy to fix it.07:54
*** erkules_ is now known as erkules07:56
lhchengmarekd: yeah, saw that in the code. the remote_id was the only way to pull-up the identifier for the IdP lookup07:56
*** erkules has quit IRC07:56
*** erkules has joined #openstack-keystone07:56
lhchengmarekd: yeah, will definitely do that!07:56
lhchengmarekd: thanks for the help!07:58
lhchengmarekd: will dig into the oidc spec to figure that remote_id07:59
*** chlong has quit IRC08:01
jamielennoxmarekd: so i liked the stuff you did in ksc-federation so much i copied the base of it back08:06
jamielennoxit won't make any difference over there, but it means for federation style plugins (ones that use the mapper) that aren't saml08:07
jamielennoxwe can take base like the kerberos and an x509 plugin on it08:07
*** lhcheng has quit IRC08:16
marekdjamielennox: hey, sorry, now i am available (earlier i was on a tram)08:40
marekdjamielennox: if we want to use it with kerberosx509 or anything that uses mapping engine we could probably rename it to something more generic.08:43
marekdjamielennox: uh, i will shoot you an e-mail either way.08:43
openstackgerrithenry-nash proposed openstack/keystone: Reload drivers when their domain config is updated
openstackgerrithenry-nash proposed openstack/keystone: Reload drivers when their domain config is updated
*** henrynash has quit IRC08:56
openstackgerritMerged openstack/keystonemiddleware: Allow loading auth plugins via overrides
openstackgerritMerged openstack/keystone: Enable use of database domain config
*** afazekas_ has joined #openstack-keystone09:16
*** jistr has joined #openstack-keystone09:25
*** ncoghlan has quit IRC09:33
*** jistr has quit IRC09:50
*** jistr has joined #openstack-keystone09:52
*** MasterPiece has joined #openstack-keystone10:15
*** breton has quit IRC10:16
*** MasterPiece has quit IRC10:17
*** dims has joined #openstack-keystone10:18
*** MasterPiece has joined #openstack-keystone10:18
*** breton has joined #openstack-keystone10:18
*** MasterPiece has quit IRC10:20
*** MasterPiece has joined #openstack-keystone10:21
*** chlong has joined #openstack-keystone10:23
*** jistr has quit IRC10:38
*** jistr has joined #openstack-keystone10:39
*** krykowski has joined #openstack-keystone10:56
*** amakarov_away is now known as amakarov10:58
*** MasterPiece has quit IRC11:20
openstackgerritafazekas proposed openstack/python-keystoneclient: No special expire limit in identity/
*** topol has joined #openstack-keystone11:36
*** ChanServ sets mode: +v topol11:36
*** samueldmq has joined #openstack-keystone11:37
samueldmqayoung, ping - you around?11:37
samueldmqayoung, would like to talk about policy :) shoudl be quick11:38
*** MasterPiece has joined #openstack-keystone11:41
*** topol has quit IRC11:44
*** pnavarro is now known as pnavarro|lunch11:52
*** iamjarvo has joined #openstack-keystone12:04
*** markvoelker has joined #openstack-keystone12:14
*** richm has joined #openstack-keystone12:15
*** panbalag has left #openstack-keystone12:25
*** raildo_away is now known as raildo12:26
openstackgerritMerged openstack/keystone: add oauth authentication to config file
*** dims has quit IRC12:33
*** dims has joined #openstack-keystone12:34
*** htruta has joined #openstack-keystone12:39
*** rushiagr is now known as rushiagr_away12:46
*** gordc has joined #openstack-keystone12:53
*** pnavarro|lunch is now known as pnavarro12:56
*** MasterPiece has quit IRC13:03
*** mattfarina has joined #openstack-keystone13:04
*** rushiagr_away is now known as rushiagr13:05
openstackgerritMerged openstack/keystone: Move pysaml to requirements.txt for py3
*** aix has joined #openstack-keystone13:13
*** iamjarvo has quit IRC13:19
*** rushiagr is now known as rushiagr_away13:20
*** joesavak has joined #openstack-keystone13:23
*** nellysmitt has joined #openstack-keystone13:24
openstackgerritDavid Stanek proposed openstack/keystone: Fixes minor spelling issue
*** iamjarvo has joined #openstack-keystone13:35
*** gordc_ has joined #openstack-keystone13:36
*** nellysmitt has quit IRC13:39
*** gordc has quit IRC13:42
*** nellysmi_ has joined #openstack-keystone13:42
*** nellysmi_ has quit IRC13:44
dolphmsamueldmq: o/ just saw your ping about fernet from yesterday13:45
*** iamjarvo has quit IRC13:46
ayoungsamueldmq, Here I am!13:46
*** krykowski_ has joined #openstack-keystone13:51
*** krykowski has quit IRC13:55
*** r-daneel has joined #openstack-keystone13:55
*** gordc_ is now known as gordc14:00
*** tellesnobrega_ has joined #openstack-keystone14:00
*** zzzeek has joined #openstack-keystone14:03
*** krykowski_ has quit IRC14:04
*** rushiagr_away is now known as rushiagr14:06
*** gokrokve has joined #openstack-keystone14:13
*** ayoung is now known as ayoung_afk14:13
openstackgerritRodrigo Duarte proposed openstack/keystone: Add test to list projects by the parent_id
*** carlosmarin has joined #openstack-keystone14:20
*** timcline has joined #openstack-keystone14:23
*** bknudson has joined #openstack-keystone14:24
*** ChanServ sets mode: +v bknudson14:24
*** sigmavirus24_awa is now known as sigmavirus2414:27
*** timcline_ has joined #openstack-keystone14:31
openstackgerritDolph Mathews proposed openstack/keystone: Remove redundant creation timestamp from fernet tokens
openstackgerritDolph Mathews proposed openstack/keystone: Remove the expiration timestamp from Fernet tokens
*** gokrokve_ has joined #openstack-keystone14:33
*** timcline has quit IRC14:34
*** henrynash has joined #openstack-keystone14:35
*** ChanServ sets mode: +v henrynash14:35
*** radez_g0n3 is now known as radez14:36
*** gokrokve has quit IRC14:36
*** iamjarvo has joined #openstack-keystone14:39
*** iamjarvo has quit IRC14:39
*** iamjarvo has joined #openstack-keystone14:40
openstackgerritMarco Fargetta proposed openstack/keystone: IdP ID registration and validation
openstackgerritMarco Fargetta proposed openstack/keystone: Correct utf8/innodb issues with tables
*** jorge_munoz has joined #openstack-keystone14:42
rodrigodsayoung_afk, morganfainberg, dstanek: re: blacklist and whitelist, what do you think something in the line:
*** henrynash has quit IRC14:44
marekdrodrigods: it's still wrong.14:44
marekdwe have bigger problem with that14:44
marekdand it's not even bl/wh listing14:44
rodrigodsmarekd, hm...14:44
dstanekrodrigods: i started working on a little refactoring there. not sure if it will turn out good14:46
dstanekrodrigods: you have it backwards if your diff14:46
rodrigodsdstanek, ++ thanks14:46
rodrigodsdstanek, can you explain the bug you found?14:46
dstanekyou would have to make direct_map_values a [[v for v ...]] and make it use extend14:47
dstanekthings are not actually treated as positional14:47
marekddstanek: i started working on that today too, and was treating everythin as list, are we going to map a rule "FOO {0}"  ?14:47
dstanekmarekd: not sure exactly yet, that's what i'm trying to figure out now14:48
marekddstanek: i am fearing the design is somewhat fundamentaly wrong and any change to fix that will not be backwards compatible.14:48
marekdstring.format() should probably not be used.14:49
marekdfor a {n} substitution.14:49
dstanekthe original design was fine for what it was doing, but as we've added bolt ons we should have probably made some tweaks14:50
*** htruta has quit IRC14:51
marekddstanek: i think everybody just assumend the direct maps would be single strings and this is not changed anywhere.14:51
marekdand the test you added yesterday would also fail on a master checkout.14:51
dstanekyeah, i have that test working now in what i'm doing, but i broke two others14:51
dstanekmight take me a little while to unwind this14:52
marekddstanek: may i see the code?14:52
marekdjust curious how you fixed that.14:52
dstanekmarekd: yeah, give me a bit to get back to a cleaner state14:52
marekddstanek: sure.14:53
openstackgerritMerged openstack/keystone: move region and service exist checks into manager layer
openstackgerritMerged openstack/keystone: Refactor: move initiator test to cadf specific section
*** lhcheng has joined #openstack-keystone14:54
rodrigodsmarekd, dstanek in a meeting here, but I'm following the discussion14:54
*** lhcheng_ has joined #openstack-keystone14:55
*** lhcheng has quit IRC14:55
*** lhcheng_ is now known as lhcheng14:55
*** stevemar has joined #openstack-keystone15:02
*** ChanServ sets mode: +v stevemar15:02
bknudsonif you want to use a format other than {}, then use some kind of flag to switch.15:02
marekdbknudson: it's still not backward compatible and basically looks like the 'mapping language' is growing (which is not bad actually)15:03
*** topol has joined #openstack-keystone15:04
*** ChanServ sets mode: +v topol15:04
openstackgerritRodrigo Duarte proposed openstack/keystone: Add test to list projects by the parent_id
marekddstanek: one thing we could do is: if the rule matches the regexp: "{n}" than replace with the object (list), if it's part of something like "foo {n}" apparently it meant to be a string.15:05
marekdso do the ";".join(v)15:06
lhchengstevemar: good morning!15:06
stevemarlhcheng, o/15:06
bknudsoncould have a flag "this is a list": true15:07
stevemarlhcheng, lets get your sso setup!15:07
*** tellesnobrega_ has quit IRC15:07
dolphmjorge_munoz: i'm updating all the patches you depend on one at a time -- still working on
lhchengstevemar: heh I was going to ask about that15:07
dolphmjorge_munoz: i'm getting test failures after rebasing that one, which i'm working through now15:07
marekdlhcheng: did you look into oidc spec?15:07
lhchengmarekd: didn't got the chance to look, feel asleep15:08
lhchengmarekd: it was 1am when we were chatting :P15:08
stevemarlhcheng, still stuck on the remote_entity_id part?15:08
*** elarson has left #openstack-keystone15:08
lhchengmarekd: my laziness got ahead of me, thought of just waiting for stevemar15:09
jorge_munozdolphm: Last night I tried adding rebase changes and fix myself but got rejected saying a that a depend patch was closed?15:09
lhchengstevemar: yeah, what value should I set it for google oidc?15:09
lhchengstevemar: is it the OIDC clientId?15:10
dolphmjorge_munoz: i'm fixing a couple other issues along the way - transient failures and whatnot15:10
stevemarlhcheng, give me a few minutes to look it up15:11
stevemarlhcheng, gah! i really should have doc'ed it15:11
jorge_munozdolphm: Let me know if I can help.15:11
lbragstaddolphm: did you still want to hop into -infra?15:12
dolphmlbragstad: let me finish this rebase (which will affect that cherry pick), and then yes, if it's still an issue15:12
lbragstaddolphm: sounds good15:13
*** rodrigods has quit IRC15:13
*** rodrigods has joined #openstack-keystone15:13
dolphmright now i'm having an issue where a bunch of unit tests are failing because /etc/keystone/keystone.conf doesn't exist...15:13
lbragstadmarekd: thanks for the reviews on the federated formatter15:13
lbragstaddolphm: because it uses the Fernet Test Setup Mixin?15:14
lbragstaddolphm: or this
bknudsonfor some reason I get an email now from dolphm that he +2d , but that was 2 days ago.15:15
lhchengstevemar: brb15:16
*** lhcheng is now known as lhcheng_afk15:16
*** lhcheng_afk has quit IRC15:17
dolphmlbragstad: no, it seems to be test_backend_ldap failing in setUp15:17
lbragstadah, weird...15:17
dolphmoslo.config was released yesterday, i'm prepared to blame that15:17
marekdlbragstad: no problem :-)15:17
stevemarlhcheng, try setting the conf.federation.remote_id_parameter to HTTP_OIDC_ISS, and remote_id of the IdP to ""15:18
marekdstevemar: looks logical15:18
lbragstadmarekd: I added a bunch of things to the token in the federated case, but the only thing needed to rebuild the token_ref['user'][federation.FEDERATION] dictionary is the user_id, right?15:18
stevemarmarekd, yeah, i think that's what i did... i'm saying this based on code (and it feels right)15:18
dolphmi'm afraid to ask, but does anyone else see issues with $ tox -r -e py27 keystone.tests.unit.test_backend_ldap.MultiLDAPandSQLIdentityDomainConfigsInSQL15:19
lbragstaddolphm: trying it quick15:20
marekdmee too15:20
marekdlbragstad: what do you mean rebuild?15:21
marekdlbragstad: when the token is validated and keystone wants to rebuild all the data  and compare?15:21
lbragstadmarekd: yes15:21
dolphmmarekd: none of that stuff is stored with fernet, so it has to be recomputed from as little data as possible during token validation15:21
marekdlbragstad: if i understand you correctly you cannot rebuild user_id .... it must be in the token.15:22
lbragstadmarekd: yep, it is15:22
dolphmmarekd: user_id can be in the token, and it is. but the entire user reference is not15:22
lbragstadmarekd: or, the entire user reference doesn't *have* to be, does it?15:23
lbragstaddolphm: yes, that test blew up15:23
lbragstaddolphm: are you running this on master?15:23
marekddolphm: unless you have some magical way to always refere to auth_url (OS-FEDERATION/identity_providers/{...}/auth) we should also keep idp and protocol in token15:23
dolphmlbragstad: yes15:23
dolphmlbragstad: fun.15:23
lbragstadmarekd: ok, so dict(idp_id=ID_ORG,protocol='saml2',groups=group_ids) should be passed into the token as well15:26
*** amerine has quit IRC15:26
marekdlbragstad: next to the user_id, yes.15:27
lbragstadmarekd: ++ awesome,15:27
*** amerine has joined #openstack-keystone15:27
*** gokrokve_ has quit IRC15:27
lbragstadmarekd: then I wouldn't need the federated domain id or name15:27
lbragstaddolphm: reran those tests on master and they passed.15:28
*** lhcheng_afk has joined #openstack-keystone15:28
*** lhcheng_afk is now known as lhcheng15:28
*** hogepodge has quit IRC15:28
marekdlbragstad: are you squeezeing domain in fernet tokens?15:28
dolphmlbragstad: uh oh15:28
lhchengstevemar: back15:28
dolphmlbragstad: why ....15:28
*** david-lyle_afk is now known as david-lyle15:28
marekdlbragstad: if that's the pure federated token  then we can assume the user is in the service domain 'federated'.15:29
lbragstadmarekd: I was having some issues yesterday and I was adding some more things that probably didn't need to be there looking back on it in retrospective15:29
marekddo you pass domain info in the normal tokens?15:29
dolphmlbragstad: they fail for me on master15:29
stevemarlhcheng, read scroll back, i suggested some values :)15:29
lbragstadmarekd: we do here for domain-scoped tokens15:30
marekdi mean for project scoped tokens - you only pass user_id, right?15:31
marekdif so, we will need to properly implement is_mapped_token() for fernet tokens and if that gives true always assume the domain is 'federated'15:31
marekduntil we change this one day in one release.15:32
lbragstadmarekd: no, project scoped tokens have the project_id passed into the token15:32
marekdbut the user is identified by id15:32
marekd(which also determines his domain)15:33
lbragstaddolphm: I just removed all .pyc files and reran
lbragstaddolphm: seems to work15:35
dolphmlbragstad: i do that all the time15:35
dolphmlbragstad: it's part of git-ready :)15:35
lbragstaddolphm: ++15:35
lhchengstevemar: ah, that was the value I was thinking of last night.  Based on what I got from the keystone docs.  But got sleepy to test it out.15:36
lbragstaddolphm: it isn't a specific test that fails for you?15:36
lhchengstevemar: if it works, good job on the keystone doc :)15:36
dolphmlbragstad: every test in that class fails15:36
lbragstaddolphm: something in setUp?  Failed: 20915:36
marekddstanek: did you manage to clean it up?15:36
dolphmlbragstad: you know what? maybe it's because i actually have an /etc/keystone/keystone.conf on this box, but my user can't read it15:36
lbragstaddolphm: sounds probable15:37
*** amerine has quit IRC15:37
dolphmlbragstad: don't know why that would have suddenly popped up as an issue though...15:37
stevemarlhcheng, i hope it'll work :)15:37
dolphmyep, that was it. solution: sudo rm -rf /etc/keystone15:37
stevemarjust patch the idp  :)15:37
dolphmso much for our benchmark server :P15:38
lbragstaddolphm: good thing we have ansible15:38
lhchengstevemar: cool!  just heading out on a meeting, will test it when I get back. :)15:38
lbragstadmarekd: so, how will that effect the is_mapped_token() method?15:39
*** browne has joined #openstack-keystone15:40
marekdlbragstad: hm?15:40
marekdstevemar: -> can you please take a look ?15:41
openstackgerritDolph Mathews proposed openstack/keystone: Drop Fernet token prefixes & add domain-scoped Fernet tokens
*** afazekas_ has quit IRC15:42
stevemarmarekd, i'm scared of it15:42
openstackgerritDolph Mathews proposed openstack/keystone: Convert audit_ids to bytes
*** nellysmitt has joined #openstack-keystone15:45
marekdlbragstad: i meant that if we are sure that the particular fernet token is a federated-fernet we can blindly assume the ephemeral user is in domain 'Federated' and this is probably the only think we can 'rebuild' and not keep in the token :-)15:46
marekdlbragstad: if, on the other hand, one day we decide that ephemeral user can be a member of any domain, basing on whatever mapping rule or so, then...well, we will need to store users domain in the token :(15:47
lbragstadmarekd: but off the bat, we are going to assume the first case?15:48
marekdlbragstad: yes, because we don't support the latter case right now.15:48
lbragstadmarekd: cool, so I'll pull that out the of the review as well.15:49
marekdunless you want to squeeze domain now and have less to do later on.15:49
lbragstadmarekd: it should be a pretty easy change regardless15:49
marekdwouldn't it change the token structure?15:49
*** nellysmitt has quit IRC15:50
dolphmlbragstad: marekd: if it's not something we support today outside of fernet, i don't think we should try to support it in fernet. the cost is just introducing a new payload variant later on15:51
lbragstadmarekd: does this make sense ?15:51
*** amerine has joined #openstack-keystone15:53
openstackgerritDolph Mathews proposed openstack/keystone: Refactor: make Fernet token creation/validation API agnostic
openstackgerritDolph Mathews proposed openstack/keystone: Federated token formatter
dolphmlbragstad: it let me upload a clean cherry pick ^15:54
dolphmlbragstad: cannot explain why.15:54
lbragstaddolphm: me either... and this still exists
*** rwsu-afk is now known as rwsu15:55
openstackgerritDolph Mathews proposed openstack/keystone: Implement Fernet tokens for v2.0 tokens
dolphmjorge_munoz: same ^15:56
dolphmjorge_munoz: that should include your timing fixes, but i haven't run tests yet15:56
lbragstaddolphm: thanks for rebasing, I'm going to go through and rip out some of the extra bloat for federated tokens15:56
*** htruta has joined #openstack-keystone15:56
dolphmlbragstad: cool, disregard the patchset i uploaded if you need to, it's just a cherry pick of the previous one (same for yours, jorge_munoz)15:57
dolphmlbragstad: jorge_munoz: both your patches should be based on 5a333ffc445de351b312578151541ec5e3166294 (my API agnostic refactor) though15:57
lbragstaddolphm: perfect15:57
*** tellesnobrega has quit IRC15:58
dolphmjorge_munoz: you need to get your _get_token_id() refactor up15:58
*** tellesnobrega has joined #openstack-keystone15:59
dolphmjorge_munoz: like, 5a333ffc445de351b312578151541ec5e3166294 <- get_token_id refactor <- your v2 support patch15:59
dolphmjorge_munoz: the 5 tests i see failing are missing that15:59
openstackgerritOpenStack Proposal Bot proposed openstack/keystone: Updated from global requirements
*** iamjarvo has quit IRC16:02
*** _cjones_ has joined #openstack-keystone16:03
*** iamjarvo has joined #openstack-keystone16:03
*** tqtran has joined #openstack-keystone16:03
*** thedodd has joined #openstack-keystone16:04
*** iamjarvo has quit IRC16:05
openstackgerritOpenStack Proposal Bot proposed openstack/python-keystoneclient: Updated from global requirements
*** tqtran_ has joined #openstack-keystone16:11
*** browne1 has joined #openstack-keystone16:11
*** tqtran has quit IRC16:12
*** d34dh0r53 has quit IRC16:13
*** briancurtin has quit IRC16:13
*** darrenc_ has joined #openstack-keystone16:14
*** comstud has quit IRC16:14
*** lbragstad has quit IRC16:14
openstackgerritLance Bragstad proposed openstack/keystone: Federated token formatter
*** browne has quit IRC16:14
*** Nakato has quit IRC16:14
*** cyeoh has quit IRC16:14
*** darrenc has quit IRC16:14
*** gus has quit IRC16:14
*** dolphm has quit IRC16:14
*** zigo_ has quit IRC16:14
*** zigo_ has joined #openstack-keystone16:14
*** d34dh0r53 has joined #openstack-keystone16:15
*** dolphm has joined #openstack-keystone16:15
*** ChanServ sets mode: +o dolphm16:15
*** lbragstad has joined #openstack-keystone16:15
*** comstud has joined #openstack-keystone16:15
*** cyeoh has joined #openstack-keystone16:16
lbragstaddolphm: marekd cut down a lot of the cruft in the federated formatters
*** Nakato has joined #openstack-keystone16:16
*** gus has joined #openstack-keystone16:16
openstackgerritLance Bragstad proposed openstack/keystone: Federated token formatter
*** briancurtin has joined #openstack-keystone16:19
morganfainbergdolphm, just realized you have an issue with not encoding expires at in the fernet token16:20
morganfainbergdolphm, rescoped tokens carry expires of the original tokan16:21
morganfainbergdolphm, with moving expires out of the encoded payload, rescoping the token resets the expiration to the TTL16:21
morganfainbergdolphm, is that something we are intending to change? cc lbragstad16:21
morganfainbergit also tells me we might have a gap in testing coverage16:23
morganfainbergif tests didn't explode with those tokens16:23
marekdlbragstad: yea, i think it makes some sense.16:24
marekdlbragstad: why did you put both domain_name and domain_id ?16:24
lbragstadmarekd: I was hitting a bunch of unrelated validation errors yesterday, I thought it might have something to do with that,16:25
lbragstadmarekd: but I was wrong16:25
*** gokrokve has joined #openstack-keystone16:25
morganfainbergdstanek, ^ cc [since you +2'd the remove expires from fernet]16:25
morganfainbergdolphm, lbragstad, dstanek, i think we can't remove expires encoding from fernet without being clearly aware of this change. and if we're making this change for fernet other token providers will need to mirror.16:26
*** gokrokve has quit IRC16:27
haneeflbragstad:  Does fernet token  exactly follow the spec at or do we change some aspects of it?16:28
lbragstadhaneef: I believe it does, but I'd have to defer to dolphm since he did quite a bit of the work converting from keyczar to cryptography16:29
dolphmhaneef: we don't actually implement the Fernet spec, we use pypi/cryptography (cryptography.fernet) which already implements it16:29
dolphmhaneef: all we do is define the message and let pypi/cryptography do the hard work16:30
dolphmmorganfainberg: hmm...16:31
*** gokrokve_ has joined #openstack-keystone16:31
haneefthanks, I manually tried to base64 decode as per that spec , it did't work. I will check cryptography stuff16:32
dolphmhaneef: it's url-safe base64 encoded, per the spec16:32
dolphmhaneef: python -c "import base64; base64.urlsafe_b64decode(token)"16:32
dolphmhaneef: if the first byte is \x08 or \x80 (can't remember which), then you've decoded it correctly16:33
haneefdolphm:  I believe in uuid/pki we revoke the token if we add/remove roles for an user for a tenant. How does that handled in fernet?16:33
dolphmhaneef: we depend on token revocation events, just like everything else should16:34
dolphm(except for uuid, events are just a perk in that case)16:34
haneefthanks. I will have a look at it16:34
morganfainbergdolphm, so i think changing the expiration behavior is outside the spec/scope for fernet at this point. I'd like to have that discussion separate from the rest of the changes if possible. i know it's a rebase of the chain, but I'm not comfortable with the expiration behavior changing as a side effect here.16:36
dolphmmorganfainberg: i think i buy the rescoping argument16:37
morganfainbergit should be explicitly called out and done for all the token types [that a rescope = expiration extension]16:37
morganfainbergif we want it16:37
openstackgerritMerged openstack/keystone: Fixes minor spelling issue
*** joesavak has quit IRC16:38
dolphmmorganfainberg: i'll try to pull the expiration removal out of the chain, so we can get the rest of the stuff merging, and i'll replace the full timestamp with a ttl in seconds16:38
morganfainbergsounds good.16:38
morganfainbergdolphm, i put a -1 on with the same comment.16:40
dolphmmorganfainberg: ack16:40
morganfainbergdolphm, so it's documented there. thanks.16:40
*** ericksonfgds has joined #openstack-keystone16:40
openstackgerritHenrique Truta proposed openstack/keystone: Add is_domain field in Project Table
openstackgerritHenrique Truta proposed openstack/keystone: Honor domain operations in project table
openstackgerritHenrique Truta proposed openstack/keystone: Change project name constraint
*** ayoung_afk is now known as ayoung16:44
*** openstack has joined #openstack-keystone16:54
*** htruta has quit IRC16:56
*** iamjarvo has joined #openstack-keystone17:03
*** radez is now known as radez_g0n317:05
*** lhcheng_ has joined #openstack-keystone17:06
dstanekmarekd: sorry was lunching17:06
*** spandhe has joined #openstack-keystone17:07
dstanekmorganfainberg: was there a specific review to look at or should i just go over all of the Fernet ones again?17:07
morganfainbergdstanek, the top of the chain, the one you +2'd17:07
morganfainbergdstanek, i commented on it. this is the expiration one17:08
dstanekmorganfainberg: cool, i'll just in it17:08
*** joesavak has joined #openstack-keystone17:08
*** lhcheng has quit IRC17:09
morganfainbergunless anyone has any complaints, i'm going to release KSM today to get swift happy.17:10
*** lhcheng_ is now known as lhcheng17:10
morganfainbergand i thnk someone else who had issues with the oslo_config stuff17:11
*** harlowja_away is now known as harlowja_17:14
*** mgagne_PHL is now known as mgagne17:14
ayoungdstanek, so, as you rework the whitelist/blacklist patch,  can you keep in mind:\ that I want to be able to say "if the group does not existing in the Keystone bakcend, ignore it?"  I think that should only be the case for lists;  and explictly named group in the mapping should have to exist in the keystone identituy keep the current behavior17:18
ayoungyou don't need to work that into the refactoring, just give me a way to match that we are doing a white/black list match instead of an explicit match,17:19
ayoungActually, Whitelist probably should have to exist, just "not blacklist"  but so lone as I have the information at auth time I can work out the logic17:20
*** chlong has quit IRC17:21
dstanekayoung: i wasn't trying to fix the logic much - just encapsulate the direct maps idea17:22
ayoungdstanek, right.17:22
ayoungdstanek, so if I can then use that to deduce we are in a direct map versus list expansion, I can do the logic I need17:23
dstaneki have a real simple change, but there is still a test that was broken17:23
ayounglook at the follow on patch to see what I need to do now17:23
dstanekso i left for lunch :-)17:23
dstanekayoung: ok, i'll see how that fits in too17:23
ayoungand if the answer is "it doesn't"  I'll deal17:24
dstanekhopefully, this experiment works out17:24
lhchengstevemar: the docs mentions about using "remoted_ids" instead of "remote_id" in:
lhchengstevemar: is that the end-goal? or we need to update the doc?17:27
stevemarlhcheng, refer to for remote_id vs remote_ids17:29
*** _cjones_ has quit IRC17:30
lhchengstevemar: ah, coming soon :)17:37
lhchengstevemar: thanks!17:37
*** _cjones_ has joined #openstack-keystone17:38
*** amakarov is now known as amakarov_away17:41
*** devlaps has joined #openstack-keystone17:42
*** nellysmitt has joined #openstack-keystone17:46
*** nellysmitt has quit IRC17:50
dolphmlbragstad: diminishing returns
* dolphm Ran: 5234 tests in 68.0000 sec.17:51
lbragstaddolphm: performance i/o?17:51
dolphmlbragstad: OnMetal I/O17:51
* dolphm Ran: 5234 tests in 59.0000 sec.17:52
dolphmbroke the 1 min barrier! =D17:52
lbragstadthat's ridiculous17:52
openstackgerritMerged openstack/keystone-specs: Mark the domain-config API as experimental
dolphm68s was the first run, so probably had extra overhead somewhere17:53
openstackgerritMerged openstack/keystone-specs: Add domain-config group/option resource relationships
*** ChanServ changes topic to "High Priority Reviews: | Priority for Review should be Fernet Tokens and Domain SQL. Please prioritize reviews for K3."17:55
dolphmoh, now i have the same failing tests i had on the old metal server :(17:55
dolphmanyone else running into: "ConfigFilesNotFoundError: Failed to find some config files: policy.json" on master?17:56
lbragstaddolphm: what are you using to setup these dev boxes?17:56
dolphmlbragstad: skill17:56
stevemarlhcheng, so, whats the verdict17:59
jorge_munozdolphm: Thanks! I’ll fix those issues.17:59
dolphmall it seems to take to make it fails is $ tox -r -e py27 && tox -e py27 # always produces failures on the second run17:59
dolphmdstanek: ^ ?18:00
lhchengstevemar: just got off from the meeting, will be able to test within the next hour18:00
stevemarlhcheng, that was a long meeting18:00
dstanekdolphm: reading18:00
*** radez_g0n3 is now known as radez18:00
dstanekdolphm: your second run always fails?18:01
dolphmdstanek: seems to - trying once more18:01
dolphmdstanek: it only seems to work with -r18:01
dstanekdolphm: sounds like something is caching data in the virtualenv18:02
dstanekdolphm: is this just on master?18:02
dolphmdstanek: yes18:02
dolphm-r succeeded again...18:03
lhchengstevemar: two consecutive meeting, 2nd was with some brazil guys about adding hmt on horizon18:03
ayoungdolphm, revoke_api is slow18:03
ayoungthat should not be the case18:04
lhchengstevemar: they got a lot of good stuff implemented18:04
ayoungWould be interesting to see a profile on keystone.tests.unit.test_v3_auth.TestTokenRevokeApi.test_deleting_role_revokes_token18:04
*** stevemar has quit IRC18:04
*** stevemar has joined #openstack-keystone18:05
*** ChanServ sets mode: +v stevemar18:05
*** lhcheng_ has joined #openstack-keystone18:05
*** henrynash has joined #openstack-keystone18:06
*** ChanServ sets mode: +v henrynash18:06
dstanekdolphm: i'll take a look in a sec18:07
*** lhcheng has quit IRC18:09
dstanekmarekd: some of the group stuff in there is confusing18:13
*** lhcheng_ is now known as lhcheng18:13
dstanekdolphm: test are running18:14
*** gyee has joined #openstack-keystone18:17
*** ChanServ sets mode: +v gyee18:17
*** aix has quit IRC18:21
ayoungdstanek, where is the database stored for sql bases unit testsnow?  We are still using sqlite for the backend, right?18:24
dstanekayoung: i think it's all in memory18:25
ayoungdstanek, we got that to work?  Good..although it means I';ll have to find another way to optimize18:25
iamjarvoif i am using ldap to authenticate, will keystone automatically create an entry in the assignment table? henrynash18:27
*** nellysmitt has joined #openstack-keystone18:27
dstanekayoung: what are you optimizing?18:27
henrynashimajarvo: sorry, in the assignemnt table, no….in the mapping table yes18:27
ayoungdstanek, running the unit tests18:28
ayoungI dstanek although I want to look at the revoke_api slowness.18:28
ayoungjust got sidetracked18:28
henrynashiamjarvo: in the assignemnt table, no….in the mapping table yes18:28
ayoungiamjarvo, absolutely not18:28
stevemarhenrynash, iamjarvo definitely not in the assignment table18:29
ayoungiamjarvo, nothing creates autmatic assignments18:29
henrynashiamjarvo: …but the public ID mapping will be created on-the-fly18:30
iamjarvohenrynash ayoung stevemar awesome works as designed, thought i was missing something. What's the workflow for getting a user into there then? so i connected ldap to keystone and now an admin user does not exist. i created one in ldap18:30
ayoungiamjarvo, are you using LDAP as the only ideneity backend, or as a domain mounted one?18:30
*** bknudson has quit IRC18:30
lhchengstevemar: the login page is not rendering correctly when I pulled Thai's patch18:31
ayoungif you have [identity]  dreiver=....ldap.Identity  then it is the first18:31
iamjarvoidentity. we changed the identity driver from sql to ldap18:31
stevemarlhcheng, what you seeing?18:31
ayoungiamjarvo, OK,  so you need to use the AUTH_TOKEN  in the config file to prime the pump18:31
lhchengstevemar: just the username/password field, signin button.  The openstack logo not rendering18:32
ayoungiamjarvo,  is what I always follow18:32 not the sample data18:32
lhchengstevemar: looking into it..18:33
iamjarvoayoung so im guessing you mean i need to create stuff through the api?18:33
ayoungiamjarvo, basically, you need to set the ADMIN_TOKEN value in the config file, and then use the keystone client to create the role assignement for one of youir uses out of LDAP18:33
dolphmdstanek: do we have a way to call tox without concurrency? or do you have to call testr directly18:33
ayoungiamjarvo, yeah18:33
ayoungiamjarvo, the ADMIN_TOKEN is designed for exactly this18:33
iamjarvoawesome. but what about other users. say if i had a coworker in ldap, how does that person login?18:34
ayoungiamjarvo, as the admin, give them a role.  You just need ADMIN_TOKEN to set up the first admin18:34
dstanekdolphm: there is an env var TEST_RUN_CONCURRENCY you can set to 018:34
dstanekdolphm: or maybe that's 1 you have to set it to18:34
ayoungiamjarvo, so you enable ADMIN_TOKEN,  do a user list and a role slist to make sure it works18:34
ayoungthen create a role assignment for whatever use is going to be your admin18:35
ayoungthen wipe the OS_ vars like:18:35
stevemarlhcheng, enable it in settings?18:35
ayoungas having those set at all will mess you up18:35
ayoungthen log on using OS_USERNAME=<ldap user name>18:35
ayoungiamjarvo, there is an alternative18:36
dolphmiamjarvo: "will mess you up" == "will override all other OS_ auth variables"18:36
ayoungiamjarvo, what release of openstack are you running on?18:36
*** joesavak has quit IRC18:36
*** joesavak has joined #openstack-keystone18:37
lhchengstevemar: yeah, settings are enabled18:37
dstanekdolphm: it didn't fail for me the second time or the third (i rebuilt the venv for the first run)18:39
iamjarvoayoung im going to try using the api and will probably have some follow up questions. thanks for the quick response18:40
dolphmdstanek: i can repro just with tox -e py27 keystone.tests.unit.test_v2_keystoneclient18:42
dolphmdstanek: TEST_RUN_CONCURRENCY=0 is default (workers == vcpus) btw18:42
dolphmdstanek: 1 disables concurrency18:42
dstanekdolphm:  do you think you are having a concurrency issue?18:43
ayoungiamjarvo, want to see the altenrative?18:44
dolphmdstanek: i thought maybe, but i have failures with concurrency=118:44
dstanekdolphm: is it your environment or is it happening to others too?18:44
dolphmdstanek: two different environment. i blew away my old vm and built a new one18:45
dolphmdstanek: trying this with concurrency=1:18:45
dolphmdstanek: tox -r -e py27 keystone.tests.unit.test_v2_keystoneclient && tox -e py27 keystone.tests.unit.test_v2_keystoneclient18:45
dstanekdolphm: what os/python version?18:45
dolphmdstanek: wheezy, python 2.718:45
*** jistr has quit IRC18:46
*** henrynash has quit IRC18:48
dolphmdstanek: send me your pub key if you want18:48
morganfainbergdstanek: send me your private key if you want /s :P18:50
* dolphm runs to meeting18:51
morganfainbergayoung: I'm wondering if we can make the admin token boot strapping a keystone-manage thing not needing an insecure wsgi thing18:51
dstanekdolphm: morganfainberg: sure18:51
morganfainbergayoung: any thoughts.18:51
morganfainbergdstanek: :p18:51
dstanekmorganfainberg: i'll send you both and email you the passphrase18:51
morganfainbergdstanek: perfect! Haha18:52
ayoungmorganfainberg, no, I don't think so.   Unless you want to put a user id in there instead, but you wouldn't be able to add a user18:52
morganfainbergayoung: via keystone manage, sure you could. That can directly interact with the sql layers.18:52
dstanekstevemar: marekd: ok, how do i know in the local rules if something should be a list or a single value?18:53
morganfainbergayoung: vs needing to do rest (it would be able to circumvent the controller / wsgi layer). But it would need access to the keystone.conf18:54
ayoungmorganfainberg, yes.  That would be fine, too18:54
morganfainbergayoung: I don't like that bootstrap requires restarting keystone to remove the insecure admin_token filter.18:54
ayoungmorganfainberg, ah...I reparsed your initial question18:54
stevemardstanek, hmm the local guy is tricky18:54
ayoungso, yes, doing via a CLI utility would always be preferrable18:54
ayoungdstanek, I would like to make a "divider" value18:55
morganfainbergayoung: I might take a stab at that. I would like to see moving that direction. (Stab = sometime by liberty)18:55
ayoungso if there i no divier, it is a single value,  and the divder should be configurable18:55
ayoung;  versus :  actaully came up in a different context18:55
dstanekayoung: the problem right now is that the values are treated in two different ways18:56
ayoungmorganfainberg, what I just proposed ^^ is the sanest way to do it, but would require an additional value on the API.  Is that legal?18:56
stevemardstanek, so you mean in a scenario with the substitution right?18:56
stevemarcause you can have user{name:{ be {0} {1} } }   and a single {0} for 'groups'18:57
morganfainbergayoung: the split would be something passed via the API?18:57
dstanekstevemar: yes, the *list patch does a literal_eval on group_ids because it things it should be a list18:57
ayoungmorganfainberg, when you configure a value to be a list, yes, on the remote sidem, you would specific the character used to divisde the list18:57
morganfainbergAs long as it doesn't break today's users without that configured value18:58
dstanekayoung: how do you know it's a list on the remote side?18:58
morganfainbergBut not sure how you know it's a ... What dstanek said18:58
ayoungdstanek, when someone defines the mapping, they know if they are going to be mapping from a single value or from a list.18:58
ayoungI mean, if the list only has one value,  sure, there is no divider, but still treat it a s a list18:59
*** nellysmitt has quit IRC18:59
*** nellysmitt has joined #openstack-keystone18:59
marekddstanek: ha!18:59
marekdthat's the problem i encountered today too.18:59
marekdi think we can: either constraint that only groups can be a list19:00
marekdor add some string parsing19:00
marekdwhich is BAD19:00
dstanekmarekd: hmmm...let me hack that in and see what happens19:00
dstanekmarekd: right now i am treating everything list a list19:00
marekddstanek: i was doing that too19:00
dstanekayoung: is there something in the mapping that says "this field is a list"?19:01
ayoungmarekd we already do string parsing19:01
marekdbut what happens if the local rule is "Marek {0}"19:01
ayoungdstanek, no, that is what I am propsing19:01
ayoungdstanek, if we put in a field seperator, treat it as a list19:01
ayoungmarekd, how does mod_saml turn the groups into a list19:01
marekdmod saml doesn't - it passses a ';' separated string19:02
ayoungif I am in groups users, admins,  then i would expect to see REMOTE_GROUPS=users;admins19:02
ayoungso specify that the separator is a ;  and know that we have a list19:02
ayoungok..I rthink I am being a dunce19:02
dstanekayoung: yes, so right now it's possible to be broken by the IdP19:03
ayoungwe know that what we are getting is a list...the question is on the local side, not the remote, right?19:03
*** samueldmq_away has joined #openstack-keystone19:03
ayoungdstanek, you mean the IDP could make a group like19:03
ayoungand it should be a single value but shows up as multiple?19:04
dstanekayoung: it could do that to any field and if it's not expected it'll be broken19:04
ayoungdstanek, meh19:04
*** gokrokve_ has quit IRC19:04
* marekd is out now19:04
iamjarvoayoung i see the admin_token in keystone.conf is set but when i try to new up a client i get unauthorized.19:05
dstanekmarekd: ok i pass all the tests now, but it still looks hacky19:05
ayoungiamjarvo, you use different env vars19:07
ayoungkeystone --help19:08
ayoungor openstack --help if you are using the common cli19:08
iamjarvoso im in python repl and doing this... client.Client(token='pass', domain_name='default', auth_url=AUTH_URL)19:09
openstackgerritDavid Stanek proposed openstack/keystone: this is a test to prove broken behavior is possible
openstackgerritDavid Stanek proposed openstack/keystone: Abstract the direct map concept into an object
openstackgerritDavid Stanek proposed openstack/keystone: Implements whitelist and blacklist mapping rules
iamjarvoadmin_token = pass is in keystone.conf19:09
dstanekayoung, marekd, stevemar: that is my attempt at an object19:10
ayoungdstanek, lookijng19:10
*** rushiagr is now known as rushiagr_away19:10
ayoungdstanek, I assmue the important one is
ayoungdstanek, did you change the commit ID?19:12
ayoungcan you resubmit it under the origianl?19:12
openstackgerritDavid J Hu proposed openstack/keystone: Version independent token issuance pipeline
ayoungI want to be able to see the difference between yours and the last proposed19:12
ayoungdstanek, Ah... the important one is,cm19:13
*** bknudson has joined #openstack-keystone19:13
*** ChanServ sets mode: +v bknudson19:13
dstanekayoung: it's not fancy and doesn't fix the fundamental problem19:14
dstanekayoung: yes i changed the commit id for now because i didn't want to change their original for my experiment19:14
ayoungdstanek, exapline the fundamental problem to me,  I've jsut gotten it in pieces19:14
dstanekayoung: the fundamental problem is that we treat the positions in the directmap differently for things we know is a list (like group) vs. everything else19:15
dstanekand the everthing else can be a list, which breaks the world19:15
morganfainbergdstanek: ick!19:16
stevemardstanek, lemme take a look19:16
dstanekrodrigods: why do we need the literal_eval again? i want to find where in the code why turn things into a string19:16
dstanekrodrigods: oh, wait. that's the format()19:17
rodrigodsdstanek, yes...19:17
dstanekso really we have to way to represent {'key': [...]} and the literal eval was a hack to get it to work?19:19
rodrigodsdstanek, yes19:21
ayoungis this due to things coming through as  strings from apache, with the separator?  Or is it already converted by the time this code sees it?19:23
*** diegows has joined #openstack-keystone19:23
dstanekayoung: no is that we '{0}'.format([a,b,c]) which turns into '["a", "b", "c"]' and later want that to actually be a list so we ast.literal_eval it19:25
ayoungWhat is DirectMap? That is saying that a value is explicitly listed in the assertion and should match the value in the group?19:27
ayoung{    "user": {        "name": "{0}"    }}19:28
rodrigodsayoung, the value in the remote field passes through the local19:29
ayoungso order is important, otherwise we match thevalue in the ":remote" section and set the wrong value in the :"local" section?19:30
rodrigodsayoung, yes19:31
rodrigodsthe first value in remote is matched with the second value in local (if I remember correctly)19:32
rodrigodsmarekd, ^please correct me if I'm saying something wrong19:32
*** radez is now known as radez_g0n319:32
dstanekayoung: rodrigods: yes, the bug i highlighted is that getting a list where we expect a single value blows things up19:33
*** timcline_ has quit IRC19:34
bknudsondid we have a keystone rep at the operators meetup?19:34
morganfainbergbknudson, not that i am aware of. but i asked sdague to proxy any feedback for us19:34
dstanekbknudson: i wish i knew about it ahead of time since it's only a few hours drive for me19:34
rodrigodsayoung, the first value in remote is matched with first one in local**19:34
morganfainbergbknudson, as well as mordred.19:34
bknudson -- they've got some burning issues regarding keystone (federation)19:35
bknudsondstanek: they didn't advertise it much... maybe on the operators list which I don't follow.19:35
morganfainbergbknudson, yeah i jumped into the etherpad as much as I could.19:36
morganfainbergbknudson, i would have gone but i just couldn't justify it with the number of HP folks already going.19:37
morganfainbergmordred, ops meetup19:37
mordredmorganfainberg: oh, yeah. it happened19:38
mordredwas big19:38
mordredlots of good people19:38
bknudsonmordred: hard for us to defend ourselves from operators complaints.19:38
morganfainbergmordred, asked for some hp people to keep eyes/ears for keystone feedback/questions19:38
mordredit's the next big thing19:38
mordredmorganfainberg: fwiw, you were not the main target of ire19:38
morganfainbergmordred, i figured as much19:38
morganfainbergbknudson, the complaints are relatively minor afaik [beyond the stuff we've already heard a lot of]19:39
bknudson should help shut them up.19:39
* bknudson needs to rebase19:39
bknudsonwe've been talking in OSSG about testing with TLS... a gate job.19:40
morganfainbergbknudson, there are a lot of gate jobs i want to see19:40
*** timcline has joined #openstack-keystone19:42
morganfainbergnext ops meetup i'll plan to go to19:42
stevemarmordred, do we want to know the main target of ire?19:43
morganfainbergstevemar, you can take a look here:
ayoungwhat is a PHL?19:43
morganfainbergthere is some telling "comments"19:43
morganfainbergayoung, philly19:43
ayoungmorganfainberg, so...did someone mention the amount of effort going in to ssl everywhere?19:43
stevemarbah, i should have bugged topol about this midcycle19:43
morganfainbergayoung, not sure. probably19:43
nkinderayoung: ?19:44
* morganfainberg wasn't there19:44
ayoungline 4619:44
stevemarzzzeek, i have an off-topic openstack request, mind if i email you directly?19:44
topolstevemar, bugged me about what19:44
nkinderyeah, ssl everywhere is so close...19:44
stevemartopol, the ops meetup?19:44
zzzeekstevemar: sure, mike_mp at zzzcomputing.com19:44
stevemarzzzeek, thanks boss :)19:44
ayoungThey have a Keystone working group made up of 5 people I've never heard of.  How wonderful19:44
topolyou wanted to go?19:44
morganfainbergayoung, you've heard of geoff19:44
stevemartopol, i hope we have someone there :)19:44
morganfainbergayoung, he's one of the cisco guys who was supposed to be at the midcycle, has been at the summits, etc19:45
topolI know at least one person went19:45
morganfainbergayoung, he was the one that was sick.19:45
ayounglooks familiar, yeah19:45
topolWe were trying to send more but but they had some personal conflicts19:45
morganfainbergmike i think i've dealt with before19:45
topollets make sure we get it covered for next time!19:45
morganfainbergthe other 3 are new names, good to have them though19:45
nkindermorganfainberg: I have a public SAML IdP demo (running on OpenStack) that we can recommend for people to try things out (soon...)19:45
morganfainbergnkinder, cool.19:46
ayoungI probably know more of them than that...but still...what does that mean, a Keystone working group?19:46
nkinderIt's rebuilt nightly and is based off of an IPA public demo19:46
*** nellysmitt has quit IRC19:46
nkinderwe're just finishing up adding ECP support19:46
morganfainbergayoung, operators meant to communicate ops stuff w/ us and/or dealing with things along those lines19:46
ayoungnkinder, looks a lot like the FreeIPA login screen....19:46
morganfainbergayoung, so it's out line to the operator community19:46
ayoungmorganfainberg, very cool19:46
morganfainbergayoung, yeah. glad geoff is involved with that.19:46
nkinderayoung, morganfainberg: admin/Secret123 will get you in as admin so you can poke around19:46
stevemarnkinder, woot!19:47
morganfainbergnkinder, really... giving out admin creds on irc ... even if it's just testing :P19:47
morganfainbergnkinder, /me cringes out of habit19:47
nkindermorganfainberg: that's what it's designed for19:47
nkinderit's rebuilt from a snapshot every 24 hours19:47
morganfainbergnkinder, like i said... cringes out of habit!19:47
stevemarnkinder, it's not logging in :(19:48
ayoungtook me a couple tries, too19:48
rodrigodsdstanek, kind missed something you said before... do your change pass the test you've submitted?19:48
nkinderstevemar: just worked for me...19:48
stevemarnkinder, maybe too many requests :)19:48
dstanekrodrigods: yes, i'm pretty sure19:49
nkinderstevemar: also has kerberos login support enabled if you use the IPA demo as a KDC (
rodrigodsdstanek, so I guess it looks like the "final" solution to me :)19:50
stevemarbknudson, huh is baically multiple identity backends19:50
stevemarits sql + ldap19:50
rodrigodsdstanek, maybe you should submit that test anyway19:50
nkinderstevemar: I am hopeful that we will have ECP here 2 weeks from now.19:50
morganfainbergnkinder, i'm thinking we might need something a little more locked down than "admin" level credentials for people to use for an official test-bed, and that is because someone could ostensibly change password and prevent others from utilizing it for testing.19:50
dstanekrodrigods: if people are happy with it i can put your change id back in a clean up the docs19:50
lhchengnkinder: nice to have a freeipa demo, been procrastinating setting up one19:50
nkinderhyrid backend is failover (SQL first, then LDAP IIRC)19:51
morganfainbergnkinder, but something to circle back on later.19:51
nkindermorganfainberg: the idea is not for serious testing19:51
nkinderit's to see what it looks like, do some basic playing with it19:51
nkinderhonestly, for testing I'd use a container19:51
stevemarnkinder, you're at the ops meetup?19:52
nkinderstevemar: no19:52
rodrigodsdstanek, and add you as co-author :)19:54
rodrigodsayoung, did you have the chance to look that change? ^19:54
rodrigodsand morganfainberg19:54
dstanekrodrigods: i think there is a lot of redesign that can be done, but i don't think it's wise to jam it into K19:54
rodrigodsdstanek, ayoung, morganfainberg
rodrigodsdstanek, ++19:54
stevemardstanek, yeah ++ for redesign19:54
rodrigodsthat patch was my first one19:55
stevemardstanek, remember when we hacked it up at rax!19:55
rodrigods"inside" federation code19:55
rodrigodshad the same feeling19:55
*** gokrokve has joined #openstack-keystone19:55
dstanekstevemar: not bad for an hour of work!19:55
morganfainbergnkinder, right. i'm thinking we should have a clear documented way for someone to setup a test SAML provider even if it's just us pointing at a recipe to standup a docker container with a running IDP for test in their environment19:55
morganfainbergnkinder, not a public "you can use X for testing out the IDP workflow"19:55
nkindermorganfainberg: +119:55
stevemarmorganfainberg, that sounds all sorts of good19:56
morganfainbergtestshib is sufficient for today, in a very limited manner afict19:56
morganfainbergbut it's not a "good" option19:56
nkinderso as soon as ECP lands in ipsilon, I can update the docker image for it and we can see about getting some steps documented19:57
morganfainbergnkinder, awesomde19:57
morganfainbergnkinder, that'd be super useful for folks testing out federated identity19:57
nkinderyep.  There's an IPA container image too, so it's pretty easy to use kerberos to auth to the IdP as well19:58
iamjarvoayoung so we were able to add a role to the user. can't seem to add a project to the user though?20:01
*** mattamizer has joined #openstack-keystone20:01
ayoungiamjarvo, heh20:03
ayoungyou don't "add a role to a user"  per se.  You assign a role to a user for a project20:03 can always check you work in the database20:03
ayoungiamjarvo, but you should be able to list roles for the user.  And it should show that the role is on the project20:04
dstanekrodrigods: i didn't see any complaints so i'll add your change id back in a push again20:11
dstanekrodrigods: i did a little experiment making a micro langauge like:
*** radez_g0n3 is now known as radez20:13
iamjarvoayoung things seem to line up in mysql but it doesnt reflect in the horizon interface20:13
rodrigodsdstanek, fine by me, and please add yourself as co-author :)20:15
iamjarvoayoung is there a difference with default project and the user only having one project? does it assume the only project is the default project? can the keystone client set the default project?20:15
iamjarvosoo many questions sorry20:15
stevemariamjarvo, try to get away from using the default project concept20:18
*** joesavak has quit IRC20:18
stevemariamjarvo, default project works if there was no project specified in the auth request. if the user has roles on only one project - keystone won't assume it's the default project in v3. and ... keystoneclient can set default project for v220:19
stevemarbut please don't use the default project logic, we want it to die20:19
*** ljfisher has joined #openstack-keystone20:20
iamjarvostevemar ahh i see. i was confused about how that works. I am seeing this in the logs "Unauthorized: User 5251a786b4d90da09a5b045341e73ee12e162d3749bfe7e6d16b88710280c64a has no access to domain default (Disable debug mode to suppress these details.) (HTTP 401)20:21
iamjarvoLogin successful for user "cloud_admin"."20:21
ayoungdstanek, so are you done hacking on the whitelist thing?  I don't want this patch to split between the cracks. I am not yet certai if we have a dealbreaker probklem, or an edgecase that can be dealt with later20:22
ayoungLDAP===no default project20:22
ayoungno where to stick it20:22
stevemarno access to default domain eh20:23
ayoungdomain scoped operation?20:23
ayounglist users?20:23
iamjarvowe are using the multi domain patch20:23
dstanekayoung: i'm fixing it up now to push again20:23
stevemariamjarvo, sounds like user 5251a786b4d90da09a5b045341e73ee12e162d3749bfe7e6d16b88710280c64a needs access to default domain20:23
ayoungdstanek, thanks20:23
ayoungdstanek, so much better if you code and I can review...I'm going to try and make sure I understand the scope of the problem fully20:24
iamjarvostevemar i dont see domain options with the keystone client. how then can we assign a user to a domain? a token?20:27
stevemariamjarvo, use a curl call with the admin token?20:27
dstanekayoung: so i've changed my opinion on the fundamental problem - it's that we don't support lists in our mapping format20:28
ayoungmarekd, stevemar, so...had a discussion with nkinder and jamielennox yesterday about the client plugins.  SInce the Kerberos appaorach is going to use mapping, and it looks like X509 would need it as well, what we are thinking is to move the core of the mapping portion of the federation plugin into the keystoneclient  repo, and keep each of the separate plugin repos light, with only enough code to handel the specifics of SAM20:28
ayoungL, OpenID, Kerberos, and so on.20:29
ayoungIt does mean we shouldrename the federation repo to the SAML repo, as it would not make sense to put that nad openid into the same repo.20:29
ayoungdstanek, ok.20:29
openstackgerritMerged openstack/keystone: Remove unused threads argument
ljfisherAll, could I get some clarification on the deprecation for the v2.0 API. It is stll marked as such in the code, and I found some list discussion from last March that seems to revert the deprecation. From that, my understanding is v2.0 is not currently scheduled for removal in a release, but that it just won’t see any more updates?20:34
*** jistr has joined #openstack-keystone20:35
openstackgerritMerged openstack/keystone: Imported Translations from Transifex
openstackgerritLance Bragstad proposed openstack/keystone: Allow methods to be carried in Fernet tokens.
iamjarvostevemar silly question. i am only seeing examples with creating a user with a domain. cant seem to find one where updating a user takes a domain. can you please point me to the docs with this please20:50
*** lifeless has joined #openstack-keystone20:50
*** trey has quit IRC20:51
bknudsonguess what... tox -epy27 crapping the bed again. :(20:54
bknudsonoops, never mind.20:54
bknudsonthat was me crapping the bed.20:55
*** trey has joined #openstack-keystone20:55
*** bernardo-silva has joined #openstack-keystone20:56
lbragstadbernardo-silva: o/20:57
bernardo-silvao/ :D20:57
*** raildo has quit IRC20:58
*** hogepodge has joined #openstack-keystone20:58
bernardo-silvadolphm: I was reviewing the fernet/spec and I strongly thing that "Timestamp" should'nt be exposed outside. It should be inside the Ciphertext20:58
bernardo-silvadolphm: Can you explain me why you put it outside?20:58
dolphmbernardo-silva: why don't you think it should be in plaintext?20:58
lbragstadbernardo-silva: you're talking about this,
*** joesavak has joined #openstack-keystone21:00
bernardo-silvaWell, first you are exposing that info wich can be used to help a malice person to try to predict the HMAC keys. Imagine that I auth in a predictable timing, the same user and the same credentials a bunch of times. I can try to use the bits on the timestamp to figure it out the HMAC keys somehow, reducing the complexity of a brute force.21:00
bernardo-silvaSecond, timestamp "inside" the cyphertext helps a lot to increase the entropy of the cyphertext binary.21:01
lifelessbernardo-silva: timestamp is only going to be what 10 bits or something21:01
lifelessbernardo-silva: 20 at most21:02
*** samueldmq_away has quit IRC21:03
bernardo-silvalifeless: yes, but that can help a lot the ciphertext checksum entropy. The problem I have is: I see benefits on putting the timestamp inside the cyphertext, and I see no benefit at all to put it ouside.21:03
openstackgerritBrant Knudson proposed openstack/keystone: Document mapping of policy action to operation
openstackgerritSteve Martinelli proposed openstack/keystone: Fix typo in name of variable in resource router
*** r-daneel has quit IRC21:05
*** topol has quit IRC21:05
lbragstadbernardo-silva: I guess in some cases it lightens the work on the Keystone side.21:07
bernardo-silvalbragstad: how? I think if you don't first decrypt a token (as a first thing you do with AE token) properly, any other keystone code shouldn't even be exercised. The keystone code should always have the access to the decrypted text.21:08
bknudsonbernardo-silva: do you have general concerns about the entropy of fernet tokens in general? why isn't there enough entropy coming from elsewhere?21:09
bknudsonyou're definitely going to get a lot of tokens with the same uids in them.21:09
lbragstadbernardo-silva: it just pushes the validation of the token expiration into cryptography I believe21:09
*** tellesnobrega_ has joined #openstack-keystone21:09
bernardo-silvabknudson: entropy comes from the cyphertext content. the only content that I see changing is the audit id (imagine the same user, same credentials, same scope, authing over and over to try to figure it out the keys)21:10
bknudsonbernardo-silva: I can definitely see your concern.21:10
bknudsonwe also can rotate keys.21:10
*** jistr has quit IRC21:10
*** ayoung has quit IRC21:10
*** tellesnobrega_ has quit IRC21:11
bernardo-silvabknudson: we do, but all that just to avoid adding the timestamps inside the cyphertext? why not just just put the ts there? :D21:11
*** mattamizer has quit IRC21:11
bknudsonI don't think rotating keys was added just to avoid adding the timestamps.21:11
bernardo-silvabknudson: again: cyphertext inside == benefits, outside == still don't see any benefit at all.21:11
dolphmbernardo-silva: i don't see any benefit to it being inside considering we have sufficient entropy in the ciphertext from audit IDs21:12
bernardo-silvadolphm: what do you consider "sufficient" entropy?21:12
stevemarzzzeek, (i'm late but...) thanks for replying! my friend was very grateful!21:12
dolphmbernardo-silva: uuid421:12
bernardo-silvadolphm: on what? audit_id?21:13
zzzeekstevemar: hey no prob21:13
dolphmbernardo-silva: audit IDs are uuid421:13
stevemardstanek, needs some comments :)21:13
dolphmbernardo-silva: they have a proprietary encoding scheme (basically compression) but it's uuid4 in terms of entropy21:13
bernardo-silvaok, uuid4's is 128 bit not fully random (some bits are fixed). timestamps are 64 bits (also not fully random), but adding both you have a lot more variation of bits, no?21:14
bernardo-silvadolphm: again, what is the reason/benefit of putting the ts outside?21:14
openstackgerritBrant Knudson proposed openstack/keystone: Cleanup test keeping unnecessary fixture references
*** mattfarina is now known as mfer21:15
*** mfer has quit IRC21:16
dolphmbernardo-silva: you could determine that a token is incredibly old without doing any crypto21:16
bernardo-silvaOK, but, why do you want that? To avoid decrypting it?21:16
lbragstadwhich would help in service side validation21:16
bernardo-silvayou will need to run the HMAC anyway, right?21:16
lbragstadto avoid the round trip21:16
morganfainbergdolphm, is this created or expires?21:16
morganfainberglbragstad, cc&21:16
dolphmmorganfainberg: created21:16
dolphmbernardo-silva: if the creation date is really old, maybe you don't want to bother verifying the hmac at all21:17
bernardo-silvaso, are you adding TTL inside the cyphertext and leave the creation time outside?21:17
morganfainbergi don't care if created is in ciphertext21:17
bernardo-silvawhat if the data is spoofed21:17
bernardo-silvaand that data is changed21:17
morganfainbergi care that it's included in the hash value21:17
dolphmbernardo-silva: yes, ttl is in ciphertext because we're utilizing fernet as specified21:17
*** haneef has quit IRC21:17
morganfainbergincluding created in ciphertext vs outside doesn't really matter that much as long as it's strongly validated.21:17
morganfainbergttl has to be in the payload (ciphertext) in this case21:18
dolphmmorganfainberg: the creation date is included in the HMAC21:18
morganfainbergdolphm, that is sufficient in my view21:18
*** haneef has joined #openstack-keystone21:18
bernardo-silvadolphm: so, still bothers me that a "static" value like TTL gets chosen to go inside the text (decreasing entropy) and a variable one like TS is outside :D21:18
dolphmbernardo-silva: i don't see an attack vector21:19
bernardo-silvaentropy is not based on size, it is based on variation proportion21:19
bernardo-silvagiving X bytes, how many of those can change21:19
bknudsonbernardo-silva: what do you think is the attack time given only audit-ID-bits of entropy?21:19
bernardo-silvabknudson: imagine that I can create tokens for user X21:20
bknudsonthen you have one of their tokens or their password.21:20
bernardo-silvaand I know (based on open source specs :D) the content of my cyphertext and my token21:20
dolphmbernardo-silva: say, 128 bit IV + 128 bit UUID4, plus maybe 128 bit AES key + 128 bit SHA256 key if you're doing rotation faster than an attacker can brute force21:20
bernardo-silvaso, therefore, I can keep generating tokens until I have a bunch where the timestamp difference is in a bit or two21:20
dolphmbernardo-silva: and then all that work is wasted when keys are rotated21:21
bernardo-silvatherefore, I can reduce the amount of effort of a brute force to figure it out KMAC/enc keys21:21
bknudsonthen you can get the key eventually.21:21
dolphmbernardo-silva: and hopefully by that time, the identity team will have blocked your account ;)21:21
bernardo-silvayou don't need many attemps21:21
dolphmbernardo-silva: define "not many"?21:21
bknudsonwe can only generate tokens so fast.21:22
morganfainbergbernardo-silva, what is the attack time [please estimate] based on your assessment of what is in the fernet ciphertext21:22
bernardo-silvayou do some to see the response time (let's say it is very consistent on the 400ms range, which in timestamp is 0sec based.21:22
bknudson(it's python)21:22
bernardo-silvaso, you do auths every 16 seconds21:22
morganfainbergbernardo-silva, i wont hold you to the number if you're wrong btw, just getting a feel for best guess.21:22
bernardo-silvaso, you will have TSs that varies in just 1 bit21:22
bknudsonyou can generate as many tokens as you want given a user and password.21:22
dolphmmorganfainberg: key_rotation_interval = attack_time - 121:22
morganfainbergdolphm, hey! stop stealing my thunder :P21:23
dolphmmorganfainberg: lol21:23
bernardo-silvamorganfainberg: ok: attack time: I run 2 tokens agains keystone and figure it out that it takes around 400ms to answer me.21:23
morganfainbergbernardo-silva, i think we use msec, so it's more than 1bit, but sure.21:23
bernardo-silvaTS is sec based, not msec based21:24
bknudsonbernardo-silva: but if you've got TS barely varying already then they're not giving you any entropy in the cipher either.21:24
bknudsonyou're relying on audit id21:24
bernardo-silvaUUID v4 varies randomly just 122 bits21:24
bknudsontimestamp barely varies at all21:24
morganfainbergso we move away from uuid to a system.urandom for the same number of bytes21:25
dolphmbernardo-silva: if you can guess the next uuid4 i produce i'll give you all the money21:25
bernardo-silvaso, you get the whole message and adds 1 predictable bit (the TS one) to a brute force logic, so you can guess the keys21:25
morganfainbergdolphm, let me get my timemachine21:25
bernardo-silvacurrent implemention: you rely only on audit_id. pushing TS inside the cyphertext you relly on both21:25
dolphmmorganfainberg: uuid4 comes from sys21:25
bernardo-silvaAlso, user CANNOT see the TS of the server21:25
morganfainbergbernardo-silva, so ... if TTL was not ciphertext neither was TS you'd be happy?21:25
bknudsonwe need the server to randomize its timestamp21:26
dstanekstevemar: add added docstrings and update some existing ones - running the last set of tests now21:26
*** darrenc_ is now known as darrenc21:26
* morganfainberg is not saying that is possible.21:26
bernardo-silvawhich can be a big deal, because then is hard to predict if the tokens have a perfect TS21:26
dolphmbknudson: ++ entropy21:26
morganfainbergdolphm, lets make ttl a float, use a urandom X number of digits on the backend of the int21:26
morganfainbergdolphm, >.>21:26
bernardo-silvamorganfainberg: happier* but the best scenario for me would have both on the cyphertext :D21:27
morganfainbergbernardo-silva, the issue is that is duplicating data for very little benefit21:27
bknudsonwe can put as much random junk on the token as we want.21:27
bknudsonmake it a config option.21:27
morganfainbergbernardo-silva, and we already know size of tokens is a real issue21:27
morganfainbergbut we could just wedge random crap in the token too21:27
bernardo-silvamorganfainberg: no! not duplicating data. I am asking to "remove" TS from the token and "moving" it to the cyphertext.21:27
morganfainbergmake any token < 255 characters == 25521:27
bernardo-silvamorganfainberg: not duplicating at all21:27
morganfainbergbernardo-silva, that breaks fernet iirc21:28
dolphmbknudson: entropy_dial = 1121:28
morganfainbergbernardo-silva, since we're using fernet that is there already21:28
bknudsondolphm: mine only goes up to 10!21:28
iamjarvostevemar ahh this was hidden in logs Error getting domain scoped token.21:28
iamjarvofor the user21:28
dolphmbknudson: our entropy is better21:28
bernardo-silvamorganfainberg: oh! so, that is the reason why it is outside (that was my first question, thanks :D)21:28
morganfainbergbernardo-silva, yeah we already had it there21:28
bernardo-silvamorganfainberg: OK, that was my first question, hehehe21:28
morganfainbergbernardo-silva, we didn't want to duplicate it in the ciphertext as well as externally based on the format we're using.21:28
bernardo-silvamorganfainberg: then that is the benefit / requirement for it to be outside21:29
morganfainbergnot because "it's cool to eliminate things from ciphertext" :)21:29
morganfainbergbernardo-silva, aha, glad i could help unwind this a bit21:29
* morganfainberg has been in meetings for a bit and just got back.21:29
morganfainbergbernardo-silva, you need special entropy dials to go to 1121:29
morganfainbergbernardo-silva, whopse21:30
dolphmbernardo-silva: you're looking at the fernet spec, right?21:30
morganfainbergbknudson ^^21:30
morganfainbergdolphm, does the ciphertext use the same key as for the HMAC?21:31
morganfainbergdolphm, i'm spacing on the answer for this atm.21:31
dolphmmorganfainberg: no21:31
morganfainbergdolphm, then i think this is a moot point21:31
dolphmmorganfainberg: 128 bit AES key + 128 bit HMAC key21:31
morganfainbergyeah then i'm really not concerned21:31
morganfainbergbernardo-silva, ^ the HMAC key is the reaaaaaally important part, the ciphertext is convienent for us. we even explored not using the AES layer21:32
dolphmmorganfainberg: "fernet" keys are 256 bit keys, where aes_key = fernet_key[:128]; sha256_key = fernet_key[128:];21:32
*** boris-42 has quit IRC21:32
morganfainbergbernardo-silva, but it's a nice package all around to include both. because fernet gave it to us21:32
bernardo-silvamorganfainberg: OK, I agree21:32
morganfainbergbernardo-silva, if someone cracked the AES key tbh, good on them, have at, fun times..21:33
openstackgerritDavid Stanek proposed openstack/keystone: Abstract the direct map concept into an object
openstackgerritDavid Stanek proposed openstack/keystone: Implements whitelist and blacklist mapping rules
openstackgerritDavid Stanek proposed openstack/keystone: Adds test for federation mapping list order issues
dstanekstevemar: ^21:33
dolphmyou still have to break another 128 bit key to generate your own tokens21:33
morganfainbergi'd be more worried if they got the HMAC key21:33
morganfainbergbecause then they could do some level of extending tokens/replay attacks.21:33
dolphmif you break the hmac key, you can basically refresh your own token until key rotation invalidates the hmac key you discovered21:33
morganfainbergit would still get caught with the same audit id, but it would get weird.21:33
dolphmmorganfainberg: ++21:34
morganfainberglike i said, more worried about HMAC key than AES key. but it's a tall order to break both imo.21:34
morganfainbergand even if you do, rotation should help. unless key generation is weak, then we have a different issue to contend with21:34
dolphmi imagine another attack vector would be easier, like paying someone21:35
dolphmand that'd probably be cheaper21:35
morganfainbergbernardo-silva, so does this solve your concerns?21:35
bknudsondolphm: rubber hoses are cheap.21:35
bernardo-silvamorganfainberg: yes :D21:35
morganfainbergdolphm, i dunno, how many GPUs do you have on those "i work for $HOSTING_COMPANY$ so i don't directly pay for bare metal servers i use for 'development'"? :P21:36
morganfainbergbernardo-silva, awesome :) thnx21:36
morganfainbergdolphm, then again... "hey install my ssh key on your keystone server, here's $1000" goes a long way.21:37
dolphmmorganfainberg: i take personal checks21:38
morganfainbergdolphm, bitcoins?21:38
bknudsonmorganfainberg: if he had that many GPUs he might as well generate his own bitcoins.21:38
bknudsonscrew the fernet key.21:38
dolphmmorganfainberg: no, you'd just take them back because my bit coins are on my keystone server21:38
morganfainbergbknudson, eh, bitcoins are so last year... i heard litecoin is where it's at... or dogecoin21:38
morganfainbergdolphm, damn, my clever plan is foiled again21:39
bknudsoncredentials backend for bitcoins.21:39
morganfainbergbknudson, barbican?21:39
bknudsonwe should have a way to accept bitcoins as tokens.21:39
bknudsonauth_token validates the chain...21:39
morganfainbergbknudson, and use the blockchain for valid... aahhaha21:39
*** radez is now known as radez_g0n321:40
bknudsonautomatic metering21:40
morganfainbergbut you have to be careful about the 50% attacks21:40
morganfainbergdolphm, ok so anything i can do to help get the fernet token chain fixed (since uh ttl has to be encoded) so we can land it this week? *hopeful*21:41
stevemargordc, does commenting 'experimental' trigger an experimental gate/check?21:43
morganfainbergstevemar yes21:43
stevemarmorganfainberg, neato21:43
morganfainbergstevemar, erm check experimental21:43
morganfainbergnot sure if just "experimental" does21:43
stevemargordc, ^21:43
* morganfainberg realizes it's easy to go look at the zuul config to be sure21:44
*** timcline has quit IRC21:44
gordcyeah it's check experimental21:44
gordci'm using your patch as test21:45
gordcand i realised it ain't fully merged yet21:45
gordcstevemar: ^21:45
stevemargordc, happy to be the guinea pig21:46
gordcyour patch will be in limbo21:48
*** bernardo-silva has quit IRC21:52
*** spandhe has quit IRC21:54
jamielennoxmorganfainberg: released?21:55
morganfainbergjamielennox, yes21:55
*** samueldmq_away has joined #openstack-keystone21:55
morganfainbergjamielennox, 1.5.021:55
jamielennoxi'll give it a day or two to make sure there aren't any failures and propose the g-r bump21:56
morganfainbergsounds good21:56
jamielennoxthough i guess i can do the devstack swift fix now21:57
stevemardstanek, you are magical sir21:58
dstanekstevemar: thanks!? i think21:58
*** iamjarvo has quit IRC21:58
*** arif-ali has quit IRC21:58
stevemardstanek, i really like that object rep of directmaps :)21:59
stevemari owe you at least 2 beers for that21:59
morganfainbergjamielennox, don't merge the devstack swift fix until g-r is updated.21:59
*** samueldmq has quit IRC21:59
*** samueldmq_away is now known as samueldmq21:59
jamielennoxmorganfainberg: devstack always runs off the latest upstream AFAIK - it's not linked to g-r21:59
jamielennoxjust needs to be in pypi21:59
morganfainbergjamielennox, i'd still WIP it first22:00
samueldmqdolphm, the priority patch under 'keystonemiddleware' can now have an '(approved)' at the end22:01
samueldmqdolphm, at the list you're maintaining (
morganfainbergsamueldmq, refresh?22:02
morganfainbergi don't see anything under keystonemiddleware section22:02
dolphmmorganfainberg: i broke it again :)22:02
dolphmsamueldmq: this gist is maintained by a bot22:02
morganfainbergdolphm, i see:22:02
morganfainberg(this list is empty!)22:02
samueldmqmorganfainberg, well now it became empty lol22:03
*** _cjones_ has quit IRC22:03
morganfainbergdstanek, ooh so looking at the object22:03
dolphmsamueldmq: i broke that page this morning; it'll be late tonight before i have a chance to fix it22:03
morganfainbergdstanek, way easier to read and know wth is going on22:03
*** _cjones_ has joined #openstack-keystone22:03
openstackgerritBrant Knudson proposed openstack/keystonemiddleware: Move unit tests into tests.unit
stevemarmorganfainberg, ++22:04
samueldmqdolphm, looks like it is working ... I think my page needed a refresh, as morgan suggested22:05
samueldmqdolphm, sorry to make you think it's broken again xD22:05
dolphmsamueldmq: i guarantee it's broken because it was running on a vm that i deleted and haven't set back up22:05
dolphmsamueldmq: you probably needed a refresh too22:06
samueldmqdolphm, ok, it's broken :p22:07
*** csoukup has joined #openstack-keystone22:09
dolphmsamueldmq: morganfainberg: okay fixed it instead of implementing the last bit fernet TTL business, and now i have to RUN! o/22:10
*** arif-ali has joined #openstack-keystone22:10
morganfainbergdolphm, lol need me to rebase the patchchain or anything ?22:10
samueldmqdolphm, k go! o/22:10
*** dims has quit IRC22:11
*** dstanek-troubled has joined #openstack-keystone22:15
*** arif-ali has quit IRC22:15
dstanek-troubledmorganfainberg: re: token expiration; i thought we agreed that having Fernet tokens different was OK if it was documented (or at least someone agreed)22:16
morganfainbergdstanek-troubled, the issue is that we're now changing the expiration behavior22:16
morganfainbergas in, a rescoped token extends the expiration22:16
morganfainbergnot that it's always tied to the keystone conf.22:17
dstanek-troubledmorganfainberg: so what do you think we should be doing?22:17
morganfainbergdstanek-troubled, if we don't mind that a token rescope no longer times out when the original auth times out (so rescopes extend the session) - thats fine, but that wasn't intended22:17
bknudsonmood nicks.22:18
morganfainbergdstanek-troubled, today uuid and pki tokens carry forward the expiration of the initial auth22:18
morganfainbergso when i rescope, my token still expires at the same time as the original token22:18
*** arif-ali has joined #openstack-keystone22:18
samueldmqmorganfainberg, a rescoped token could have its own issued time, but the TTL would be smaller than default, calculated from the time passed since the first issuance22:18
morganfainbergsamueldmq, except with the current proposal TTL is not in the token at all22:19
morganfainbergsamueldmq, it's always based on CONF.token.expiration_time22:19
morganfainbergsamueldmq, for fernet22:19
samueldmqmorganfainberg, ok so we cant :/22:19
samueldmqmorganfainberg, I should be updated of the spec, sorry22:20
samueldmqmorganfainberg, so the idea is to have only the issued_at, and being expired or not depends on comparing that with the expiration_time22:20
morganfainbergdstanek-troubled, so - either we should clearly make this change and commit to it, or we encode TTL.22:21
morganfainbergsamueldmq, that was the proposed change.22:21
morganfainbergsamueldmq, this is not clearly communicated in anyway and changes the behavior so. we need to clearly communicate this or encode TTL.22:21
bknudsonwhy aren't there unit tests for this?22:21
morganfainbergand we've had a pretty hard line stance on rescoping doesn't change the life-span of the token for a while22:22
bknudsonwe don't have unit tests for token from token expiring at the same time?22:22
morganfainbergbknudson, i think there are. just none that are triggered with fernet.22:22
dstanek-troubledmorganfainberg: we could probably encode the ttl in a byte or two22:22
samueldmqmorganfainberg, ttl would be good22:22
dstanek-troubledthat could be bad if you could keep extending a tokens life indefinitely22:23
morganfainbergdstanek-troubled, yeah, it's not a big deal to change. it's that the whole fernet tree is based on a patch that introduces this change in ttl behavior atm22:23
samueldmqmorganfainberg, is there any 'flag' in the rescoped token to say it was rescoped ? so that one could understand why a given token has 'less time' than others22:23
morganfainbergsamueldmq, if you have 2 audit ids, it has been rescoped22:23
morganfainbergsamueldmq, [current_token_audit_id, original_token_in_the_chain_audit_id]22:24
morganfainbergor [current_token_audit_id] <-- not rescoped22:24
samueldmqmorganfainberg, so I think it would be ok to have a reduced ttl in a given token, since there is a way to 'justify' that : rescoped22:24
morganfainbergsamueldmq, so we just need to encode the TTL correctly22:24
*** ayoung has joined #openstack-keystone22:24
*** ChanServ sets mode: +v ayoung22:24
*** hogepodge has quit IRC22:24
morganfainbergsamueldmq, not go with creation + conf.token.expiration_time22:25
samueldmqmorganfainberg, this could even be the default behavior, you're rescoping, so the ttl should be kept, (rescoping, not getting another, by definition)22:25
morganfainbergsamueldmq, it already is the behavior22:25
samueldmqmorganfainberg, ok so just have ttl for fernet, and keep consistent with what we have for others22:25
morganfainbergsamueldmq, expiration_time is kept consistent in a token rescope chain22:25
morganfainbergsamueldmq, fernet does it today, the issue is the current "next" patch to review for fernet changes this:
morganfainbergsamueldmq, so that blocks up the rest of the tree22:26
samueldmqmorganfainberg, what's the main purpose of removing this on that patch? make it still smaller?22:27
*** jorge_munoz has quit IRC22:27
morganfainbergsamueldmq, that was the intent of the patch, we're reversing course on it though and encoding the TTL.22:27
dstanek-troubledhow about we use a new constant, the OpenStack epoch, at a starting point for dates. this would shrink the size needed to hold the offset.22:30
ayoungdstanek-troubled, you trying to write the Y3K bug?22:31
dstanek-troubledlol, i don't think we should do it. was just a random thought.22:31
samueldmqmorganfainberg, rescoping tokens is *really* used by other services? I mean, is it a common scenario or just in specific cases?22:31
morganfainbergsamueldmq, all the time22:32
morganfainbergsamueldmq, unscoped -> scoped, horizon changing projects, etc22:32
morganfainbergdstanek-troubled, i think someone would kill you :P22:32
samueldmqmorganfainberg, k, makes sense. If it wasnt,  a first token could just have no ttl, and once we rescope, we could add a sort of 'passed_time', calculated when rescoping22:33
samueldmqmorganfainberg, and then in the most common case (not rescoped), the token was smaller, but forgot this :p22:34
morganfainbergsamueldmq, lets not get crazy. easier to just always include the TTL22:34
samueldmqmorganfainberg, yeah, I agree it's the best approach, let's keep it simple22:34
samueldmqmorganfainberg, it's just a few bytes, not that worth it to make it complicated22:35
*** dstanek-troubled has quit IRC22:36
* samueldmq would vote for removing ttl if we had dial-up internet connection22:37
ayoungLike a bridge over troubled-stanek....22:37
*** tellesnobrega_ has joined #openstack-keystone22:40
*** tellesnobrega_ has quit IRC22:41
*** ljfisher has quit IRC22:41
*** gordc has quit IRC22:42
morganfainbergsamueldmq, uh.22:42
morganfainbergsamueldmq, 1200baud modem?22:42
*** thedodd has quit IRC22:44
*** dims_ has joined #openstack-keystone22:46
*** pnavarro has quit IRC22:48
samueldmqmorganfainberg, nah, I think 2400 baud.. it already was quick when I born :p22:49
*** ayoung has quit IRC22:50
samueldmqmorganfainberg, I take a bit to understand when you asked dolph: 'lol need me to rebase the patchchain or anything ?'22:50
*** chrisshattuck has quit IRC23:06
*** carlosmarin has quit IRC23:22
*** carlosmarin has joined #openstack-keystone23:27
*** topol has joined #openstack-keystone23:29
*** ChanServ sets mode: +v topol23:29
*** bknudson has quit IRC23:31
*** henrynash has joined #openstack-keystone23:39
*** ChanServ sets mode: +v henrynash23:39
*** chlong has joined #openstack-keystone23:41
*** topol has quit IRC23:47
*** bknudson has joined #openstack-keystone23:52
*** ChanServ sets mode: +v bknudson23:52
*** wwriverrat has joined #openstack-keystone23:53

Generated by 2.14.0 by Marius Gedminas - find it at!