| *** rderose has quit IRC | 00:03 | |
| *** ayoung has joined #openstack-keystone | 00:20 | |
| *** ChanServ sets mode: +v ayoung | 00:20 | |
| *** tqtran has quit IRC | 00:22 | |
| *** rcernin has quit IRC | 00:24 | |
| *** markvoelker has joined #openstack-keystone | 00:27 | |
| *** dave-mccowan has joined #openstack-keystone | 00:27 | |
| *** catintheroof has quit IRC | 00:27 | |
| openstackgerrit | Merged openstack/keystone: Updating sample configuration file https://review.openstack.org/315764 | 00:31 |
|---|---|---|
| *** sdake has quit IRC | 00:33 | |
| *** woodburn1 has joined #openstack-keystone | 00:33 | |
| *** markvoelker has quit IRC | 00:34 | |
| *** dave-mccowan has quit IRC | 00:35 | |
| *** woodburn has quit IRC | 00:35 | |
| *** raddaoui has quit IRC | 00:37 | |
| *** openstackgerrit has quit IRC | 01:02 | |
| *** openstackgerrit has joined #openstack-keystone | 01:02 | |
| *** rbridgeman has joined #openstack-keystone | 01:09 | |
| *** chlong has joined #openstack-keystone | 01:13 | |
| *** dan_nguyen has quit IRC | 01:19 | |
| *** markvoelker has joined #openstack-keystone | 01:29 | |
| *** tonyb has joined #openstack-keystone | 01:29 | |
| tonyb | I'm playing catchup here but when did the keystone command go away? | 01:30 |
| *** alex_xu_ has joined #openstack-keystone | 01:30 | |
| *** EinstCrazy has joined #openstack-keystone | 01:30 | |
| tonyb | tempets-lib on stable branches uses it so I either need to update tempest-lib or pin to an old enough version of python-keystoneclient | 01:31 |
| *** chlong has quit IRC | 01:31 | |
| *** markvoelker has quit IRC | 01:34 | |
| dims | tonyb : a few days ago | 01:35 |
| tonyb | dims: with the 3.0.0 release? | 01:35 |
| dims | y there was a big warning email to -dev list too | 01:36 |
| * tonyb missed that thread :( | 01:36 | |
| dims | tonyb http://markmail.org/message/uhke6kdmhpwuvjbs | 01:37 |
| tonyb | dims: Thanks. | 01:37 |
| morgan | tonyb: 3.0.0 | 01:39 |
| morgan | tonyb: what dims said | 01:39 |
| morgan | tonyb: but basically it's been deprecated for a looong time, openstackclient is the correct path forward. | 01:39 |
| tonyb | morgan: Thanks. | 01:39 |
| morgan | tonyb: np, sorry this one caught you off guard :( - we tried to communicate it as loud as we could. | 01:40 |
| tonyb | s/keystone/openstack identity/ ? | 01:40 |
| morgan | tonyb: unfortunately, the openstackclient commands are quite different :( | 01:40 |
| tonyb | morgan: okay | 01:40 |
| lbragstad | typically in the form of `openstack <entity> <operation> | 01:40 |
| morgan | tonyb: it comes down to "keystoneclient" didn't do things the same way OSC did. OSC is standard across all things. | 01:40 |
| lbragstad | ` | 01:40 |
| morgan | lbragstad: in some cases. in others not even. | 01:41 |
| tonyb | morgan: I knew it was going away but I didn't realise it'd break the gate. | 01:41 |
| morgan | tonyb: but what lbragstad said holds true *most* of the time. | 01:41 |
| morgan | tonyb: /me nods. | 01:41 |
| morgan | tonyb: sorry! :( | 01:41 |
| tonyb | morgan: no problem. | 01:41 |
| morgan | tonyb: at least we've heard of exactly one place things broke now.. from you - this means we *almost* got it right. | 01:41 |
| tonyb | morgan: not cranky at all just tryign to workout the way forward. | 01:42 |
| morgan | tonyb: totally, you didn't come off cranky, still doesn't make me happy to hear we broke ya | 01:42 |
| morgan | *almost* right. | 01:42 |
| morgan | so close. | 01:42 |
| tonyb | :) | 01:42 |
| morgan | hehe | 01:42 |
| *** roxanagh_ has joined #openstack-keystone | 01:44 | |
| *** EinstCra_ has joined #openstack-keystone | 01:44 | |
| *** EinstCrazy has quit IRC | 01:44 | |
| *** roxanagh_ has quit IRC | 01:44 | |
| *** EinstCra_ has quit IRC | 01:52 | |
| *** EinstCrazy has joined #openstack-keystone | 01:52 | |
| tonyb | It doesn't look like it'll be too bad to fix | 01:53 |
| * tonyb wanders off to test it .... | 01:53 | |
| *** EinstCrazy has quit IRC | 01:55 | |
| *** EinstCrazy has joined #openstack-keystone | 01:55 | |
| *** EinstCrazy has quit IRC | 01:56 | |
| stevemar | morgan thanks for answering tonyb's questions! | 01:57 |
| *** EinstCrazy has joined #openstack-keystone | 01:57 | |
| morgan | stevemar: shhhh. | 01:57 |
| *** morgan is now known as notmorgan | 01:57 | |
| stevemar | morgan: you're such a fountain of information | 01:57 |
| * notmorgan is totally not morgan | 01:57 | |
| stevemar | notmorgan: i think folks should ask you all the questions | 01:58 |
| notmorgan | heyyyy i'm not PTL anymore | 01:58 |
| *** EinstCra_ has joined #openstack-keystone | 01:58 | |
| *** josecastroleon has joined #openstack-keystone | 02:00 | |
| notmorgan | stevemar: can we make someone else answer all the questions? | 02:00 |
| notmorgan | ;) | 02:00 |
| *** EinstCrazy has quit IRC | 02:01 | |
| *** EinstCrazy has joined #openstack-keystone | 02:02 | |
| *** josecastroleon has quit IRC | 02:02 | |
| tonyb | so I assume y'all will be ok with me cappign python-keystoneclient <3.0.0 on stable/* | 02:03 |
| notmorgan | yep | 02:03 |
| notmorgan | i'm good with it | 02:03 |
| notmorgan | and it makes *a lot* of sense to do so | 02:03 |
| notmorgan | Tonight the role of Steve Martinelli the PTL will be played by "notmorgan" :P | 02:04 |
| *** EinstCra_ has quit IRC | 02:04 | |
| notmorgan | >.> | 02:04 |
| notmorgan | tonyb: in all seriousness though, capping makes sense for this | 02:04 |
| tonyb | :) | 02:04 |
| tonyb | Yeah it's a major release for a reason .... | 02:04 |
| notmorgan | yep | 02:04 |
| notmorgan | Semver and break things | 02:05 |
| *** furface has quit IRC | 02:13 | |
| *** furface has joined #openstack-keystone | 02:13 | |
| *** iurygregory has quit IRC | 02:13 | |
| openstackgerrit | Colleen Murphy proposed openstack/ldappool: Fix pool_full race condition https://review.openstack.org/315839 | 02:16 |
| openstackgerrit | Steve Martinelli proposed openstack/ldappool: Raise an explicit BackendError on TLS failures https://review.openstack.org/315840 | 02:16 |
| crinkle | stevemar: notmorgan ^ | 02:16 |
| notmorgan | crinkle: ack. | 02:17 |
| * crinkle learned about --until-failure today | 02:17 | |
| notmorgan | crinkle: oooooh wait, is that a tox thing? | 02:17 |
| crinkle | notmorgan: ya | 02:17 |
| crinkle | or | 02:17 |
| crinkle | maybe a testr thing | 02:17 |
| notmorgan | holy crap. that is useful. | 02:17 |
| * notmorgan learned something awesome today. | 02:17 | |
| stevemar | wait wait, i wanna learn | 02:17 |
| crinkle | tox -epy27 -- --until-failure <--- run for a while until you're convinced 315839 works | 02:18 |
| notmorgan | that is fantastic. | 02:19 |
| stevemar | lol does it keep looping? | 02:19 |
| stevemar | crinkle: ^ | 02:19 |
| crinkle | stevemar: ya | 02:19 |
| stevemar | yep | 02:20 |
| stevemar | just tried | 02:20 |
| stevemar | hehe | 02:20 |
| stevemar | i wonder if it failed in my env... | 02:20 |
| stevemar | op, yeah, it sure did | 02:20 |
| stevemar | nice | 02:20 |
| stevemar | crinkle: nice, i pulled it down and it worked | 02:21 |
| crinkle | yay | 02:22 |
| notmorgan | crinkle: i always liked -- --failing | 02:23 |
| notmorgan | crinkle: but this so much better for chasing races down. i've been doing it the hard way :( | 02:23 |
| lbragstad | speaking of races... | 02:23 |
| lbragstad | https://bugs.launchpad.net/keystone/+bug/1578866 fills me with hate | 02:24 |
| openstack | Launchpad bug 1578866 in OpenStack Identity (keystone) "test_user_update_own_password failing intermittently" [High,In progress] - Assigned to Lance Bragstad (lbragstad) | 02:24 |
| notmorgan | lbragstad: round up! | 02:24 |
| notmorgan | lbragstad: round tokens and events up? :P oh wait... that doesn't solve it either | 02:24 |
| lbragstad | i'm running all the same versions of mysql bits that devstack is and i can't seem to recreate it | 02:25 |
| notmorgan | lbragstad: what verison of mysql? | 02:25 |
| lbragstad | 5.5.49-0ubuntu0.14.04.1 | 02:26 |
| notmorgan | also it is super edge-case racy | 02:26 |
| lbragstad | http://logs.openstack.org/30/314330/3/check/gate-tempest-dsvm-neutron-dvr/3d9272f/logs/dpkg-l.txt.gz | 02:26 |
| *** EinstCrazy has quit IRC | 02:26 | |
| lbragstad | it's racy but frequent enough to be a pain in the gate | 02:26 |
| notmorgan | yep | 02:27 |
| notmorgan | lbragstad: the real issue is that our tests do not mirror reality | 02:27 |
| notmorgan | lbragstad: a token and password change in the same second is pretty narrow | 02:28 |
| lbragstad | notmorgan agreed - but it's still a problem | 02:28 |
| openstackgerrit | Steve Martinelli proposed openstack/ldappool: Use standard-library logging to record errors https://review.openstack.org/315844 | 02:28 |
| notmorgan | lbragstad: i would be ok with forcing the rev. events to round up to the next second, but that likely is going to expose other issues | 02:28 |
| lbragstad | notmorgan right | 02:28 |
| notmorgan | lbragstad: frankly, this might actually be a sleep(1) | 02:29 |
| lbragstad | notmorgan we already have one of those in the test | 02:29 |
| notmorgan | lbragstad: if this is a tempest thing, it has to mirror what we support | 02:29 |
| notmorgan | lbragstad: if it's in our unit tests, we can control the clock | 02:29 |
| notmorgan | lbragstad: so since it's in tempest... | 02:30 |
| *** markvoelker has joined #openstack-keystone | 02:30 | |
| stevemar | notmorgan: any brilliant idea how to make ldappool work with both pyldap and python-ldap? | 02:31 |
| notmorgan | stevemar: at the same time? | 02:31 |
| notmorgan | no. | 02:31 |
| notmorgan | stevemar: to test it? yes, provide a test that explicitly overrides the library installed | 02:32 |
| notmorgan | stevemar: (a different requirements.txt) | 02:32 |
| notmorgan | stevemar: possibly we need to use "extras" and make one the default one ... another not default? | 02:32 |
| *** EinstCrazy has joined #openstack-keystone | 02:33 | |
| *** markvoelker has quit IRC | 02:34 | |
| *** edtubill has quit IRC | 02:35 | |
| stevemar | notmorgan: if we swap out python-ldap for pyldap as the default, we're impacting a lot of folks that may upgrade their existing projects | 02:41 |
| stevemar | notmorgan: OTOH, we could keep pyldap as an "extra", and specify that in keystone's requirements | 02:42 |
| stevemar | but hmm... testing will be hard | 02:42 |
| *** fangxu has quit IRC | 02:42 | |
| *** lhcheng_ has quit IRC | 02:57 | |
| notmorgan | stevemar: we'll need to see how an "extra" works as the default vs non-default | 02:58 |
| notmorgan | mutual exclusive extra? | 02:58 |
| notmorgan | stevemar: you know. lets just do pyldap for 2.0.0 and have a compat test job | 02:59 |
| notmorgan | for python-ldap | 02:59 |
| notmorgan | hmm. actually i bet we can do some setuptools magic | 02:59 |
| notmorgan | ugh | 02:59 |
| *** phalmos has joined #openstack-keystone | 03:01 | |
| stevemar | notmorgan: crinkle i posted 2 other changes: https://review.openstack.org/#/c/315844/ and https://review.openstack.org/#/c/315840/ | 03:02 |
| patchbot | stevemar: patch 315844 - ldappool - Use standard-library logging to record errors | 03:02 |
| patchbot | stevemar: patch 315840 - ldappool - Raise an explicit BackendError on TLS failures | 03:02 |
| stevemar | they are old pull requests | 03:02 |
| *** phalmos_ has joined #openstack-keystone | 03:02 | |
| notmorgan | stevemar: not sure of the legalities...but you may want to set the author on those commits? | 03:03 |
| notmorgan | to the original author | 03:03 |
| notmorgan | of the PR | 03:03 |
| stevemar | notmorgan: i was wondering that... | 03:03 |
| notmorgan | also commenting on the logging one. | 03:03 |
| stevemar | but they wouldn't have signed the Openstack CLA? | 03:03 |
| notmorgan | ldappool doesn't require CLA | 03:03 |
| notmorgan | i explicitly set it up to not. | 03:03 |
| openstackgerrit | Merged openstack/ldappool: Fix pool_full race condition https://review.openstack.org/315839 | 03:04 |
| notmorgan | stevemar: i would set the authors, the PRs could have been merged and then it'd be included here anyway | 03:05 |
| notmorgan | this way they at least get credit in the history | 03:05 |
| stevemar | ah okay | 03:05 |
| *** phalmos has quit IRC | 03:06 | |
| notmorgan | stevemar: you know how to set the author right? (some magic git invocation) | 03:06 |
| stevemar | yep | 03:06 |
| notmorgan | okie | 03:06 |
| notmorgan | also look at the logging one. | 03:06 |
| notmorgan | i think one of the debug stantments should be .info | 03:06 |
| openstackgerrit | Steve Martinelli proposed openstack/ldappool: Use standard-library logging to record errors https://review.openstack.org/315844 | 03:07 |
| stevemar | fixed one of them | 03:07 |
| *** chlong has joined #openstack-keystone | 03:08 | |
| stevemar | notmorgan: hmm, kinda hard to find the author on this one: https://github.com/mozilla-services/ldappool/pull/3 | 03:10 |
| notmorgan | i have a trick to find it | 03:10 |
| stevemar | notmorgan: you have too many tricks | 03:12 |
| notmorgan | stevemar: Lorenzo M. Catucci <lorenzo@sancho.ccd.uniroma2.it> | 03:13 |
| notmorgan | retrieved from https://github.com/lmctv/deform/commit/6ba3cef7380cee33f39beaba5d926522e7cde49d | 03:13 |
| stevemar | notmorgan: probably should have PMed that :P | 03:13 |
| notmorgan | stevemar: eh. *shrug* | 03:13 |
| notmorgan | it's going into a git log | 03:13 |
| notmorgan | it is public info on github | 03:13 |
| notmorgan | if it was more than trivial to find i'd PM it | 03:14 |
| openstackgerrit | Steve Martinelli proposed openstack/ldappool: Raise an explicit BackendError on TLS failures https://review.openstack.org/315840 | 03:14 |
| stevemar | anyway, ^ | 03:14 |
| openstackgerrit | Rodrigo Duarte proposed openstack/keystone: Add protocols integration tests https://review.openstack.org/307508 | 03:14 |
| openstackgerrit | Rodrigo Duarte proposed openstack/keystone: Add mapping rules integration tests https://review.openstack.org/305444 | 03:14 |
| notmorgan | stevemar: sorry... -1 on that.. Tests? | 03:14 |
| notmorgan | stevemar: if not possible or not reasonable, +2 | 03:15 |
| stevemar | notmorgan: probably should have a test, just havne't looked into it yet | 03:16 |
| notmorgan | stevemar: also, yay ldappool becoming a thing. | 03:17 |
| notmorgan | that is usable again | 03:17 |
| stevemar | :) | 03:17 |
| notmorgan | so... we have one more major issue with py23 | 03:18 |
| notmorgan | py3* | 03:18 |
| notmorgan | ... python-memcached | 03:18 |
| notmorgan | is STILL a trainwreck | 03:18 |
| notmorgan | maybe we make the default for keystone (and recommendation) pylibmc? | 03:18 |
| *** agrebennikov has quit IRC | 03:20 | |
| stevemar | didn't realize there was a big issue there | 03:20 |
| notmorgan | yeah.. it "loads" but doens't work | 03:20 |
| *** pushkaru has joined #openstack-keystone | 03:21 | |
| tonyb | Are you thinking to switch out python-memcached or only use pylibmc on python_version>3.0 ? | 03:22 |
| stevemar | tonyb: thats the thought | 03:25 |
| stevemar | looking at http://sendapatch.se/projects/pylibmc/ -- it seems to be a drop-in replacement, almost? | 03:25 |
| stevemar | tonyb: i think on either python version would be nice | 03:25 |
| tonyb | stevemar: if it's better maintained then doing the full switch would be the best approach, but that impacts {'requirement': <Requirement('python-memcached>=1.56')>, 'used_by': set(['openstack/keystone', 'openstack/keystonemiddleware', 'openstack/designate', 'openstack/oslo.cache', 'openstack/zaqar'])} | 03:26 |
| tonyb | so you'd probably want buy-in / help for the non keystone side | 03:26 |
| tonyb | and doign it early in newton would be best :) | 03:27 |
| stevemar | oooof oslo.cache, that'll be a toughie | 03:27 |
| stevemar | tonyb: for sure | 03:27 |
| notmorgan | tonyb: pylibmc in newton would be the target | 03:27 |
| * tonyb probably isn't saying anythign surprising | 03:27 | |
| stevemar | *early* newton | 03:27 |
| notmorgan | stevemar: oslo.cache isn't an issue really | 03:27 |
| notmorgan | tbh | 03:27 |
| notmorgan | no one really uses that poart but us (keystone) | 03:27 |
| notmorgan | but pylibmc is likely the best option | 03:27 |
| notmorgan | bmemcache is... weird | 03:27 |
| notmorgan | tonyb: ^ cc (correct me if i'm wrong) | 03:28 |
| tonyb | notmorgan: ISTR patches for nova but they may not have landed | 03:28 |
| notmorgan | tonyb: oh nice. but still not the worst thing to use pylibmc if we can do it seemlessly | 03:28 |
| stevemar | tonyb: i don't think they ever landed | 03:28 |
| notmorgan | tonyb: but i think nova isn't landing those yet | 03:28 |
| tonyb | okay that's probably fine then | 03:29 |
| stevemar | i dont see where zaqar is using it | 03:30 |
| notmorgan | sec | 03:30 |
| tonyb | This is the kind of thing that the 'requirements' team shoudl be able to help with but it doesn't exist yet :( | 03:30 |
| stevemar | tonyb: mind if i start the convo on the ML? | 03:30 |
| jamielennox | i though oslo.cache was going to let us punt these decisions? | 03:31 |
| *** markvoelker has joined #openstack-keystone | 03:31 | |
| tonyb | stevemar: go nuts! | 03:31 |
| notmorgan | keystone, zaqar, nova imports it, celiometer does, heat, kolla, fuel | 03:31 |
| notmorgan | anyway. | 03:31 |
| notmorgan | we can make devstack use pylibmc for keystone | 03:31 |
| notmorgan | and in unit tests it doesn't matter | 03:32 |
| notmorgan | and we can document use pylibmc | 03:33 |
| stevemar | tonyb: notmorgan i don't see how zaqar uses it: http://codesearch.openstack.org/?q=memcache&i=nope&files=&repos=zaqar | 03:33 |
| notmorgan | stevemar: they are importing it. | 03:34 |
| notmorgan | stevemar: and doing things with it | 03:34 |
| notmorgan | i assume that means they are using it | 03:34 |
| tonyb | stevemar: it's possible it's a left over from befoer they used oslo.cache | 03:34 |
| stevemar | notmorgan: i don't see an import statement there | 03:34 |
| notmorgan | oh hah are they "import zaqar.cache as oslo_cache" facepalm | 03:35 |
| *** markvoelker has quit IRC | 03:35 | |
| notmorgan | stevemar: in zaqar.common.cache | 03:35 |
| notmorgan | from oslo_cache import core | 03:36 |
| notmorgan | and zaqar.common.decorator | 03:36 |
| notmorgan | they default to conf.cache.backend = 'dogpile.cache.memory'... oh man i need to go poke at flaper87 | 03:36 |
| notmorgan | about that | 03:36 |
| tonyb | Any chance y'all can drop a +1 on https://review.openstack.org/#/q/I6f31ece2c7b6290abd219f5bf2236718e9bd53f2,n,z ? | 03:36 |
| stevemar | wuuut | 03:36 |
| stevemar | tonyb: about that, i commented on the bug, can we not backport https://github.com/openstack/python-novaclient/commit/d133a664ae19385ded69ee416f04f6243c26285e ? | 03:37 |
| notmorgan | tonyb: done. | 03:37 |
| notmorgan | stevemar: ... i dont' want to rebase this https://review.openstack.org/#/c/271948/ :( | 03:38 |
| patchbot | notmorgan: patch 271948 - keystone - Deprecate keystone.common.kvs | 03:38 |
| tonyb | stevemar: I'll look but it looked like several places in the tests they were shelling out to keystone commands rather than using the keystoneclient module | 03:40 |
| openstackgerrit | Steve Martinelli proposed openstack/keystone: Deprecate keystone.common.kvs https://review.openstack.org/271948 | 03:41 |
| stevemar | notmorgan: the rebase button worked *shrugs* | 03:41 |
| notmorgan | stevemar: cool? | 03:41 |
| notmorgan | stevemar: still needs reno or something... i think | 03:42 |
| stevemar | notmorgan: oh yeah | 03:42 |
| openstackgerrit | Steve Martinelli proposed openstack/ldappool: Use standard-library logging to record errors https://review.openstack.org/315844 | 03:45 |
| *** tqtran has joined #openstack-keystone | 03:47 | |
| *** tqtran has quit IRC | 03:48 | |
| *** pushkaru has quit IRC | 03:52 | |
| *** jamielennox is now known as jamielennox|away | 03:58 | |
| openstackgerrit | Merged openstack/ldappool: Use standard-library logging to record errors https://review.openstack.org/315844 | 04:02 |
| *** jamielennox|away is now known as jamielennox | 04:03 | |
| jamielennox | more than you wanted to know about ldap ... | 04:04 |
| *** phalmos_ has quit IRC | 04:05 | |
| *** ayoung has quit IRC | 04:07 | |
| *** lhcheng has joined #openstack-keystone | 04:11 | |
| *** ChanServ sets mode: +v lhcheng | 04:11 | |
| *** edtubill has joined #openstack-keystone | 04:16 | |
| *** links has joined #openstack-keystone | 04:23 | |
| *** doug-fish has joined #openstack-keystone | 04:25 | |
| jamielennox | notmorgan, stevemar: so i'm trying to write up this spec for srevice token passing user headers rather than user token | 04:26 |
| jamielennox | and i just want reassuring there's not a security problem here | 04:27 |
| jamielennox | because it seems like if you have service role you can do whatever you like as anyone | 04:27 |
| notmorgan | jamielennox: there isn't | 04:27 |
| notmorgan | not anymore than any other thing with bearer tokens | 04:27 |
| notmorgan | make it an option to disable | 04:27 |
| notmorgan | *shrug* | 04:28 |
| jamielennox | yea, i'm going to write it up anyway | 04:28 |
| jamielennox | also it looks like it would make sense to make a function that validates both | 04:28 |
| notmorgan | but in short we want to move away from "pass user authz" between services and need that validation | 04:28 |
| notmorgan | eh | 04:28 |
| notmorgan | no | 04:28 |
| jamielennox | because it seems stupid to validate your user service auth token | 04:28 |
| notmorgan | make it an either or. | 04:28 |
| jamielennox | and then immediately use it again to validate the user headers | 04:28 |
| notmorgan | don't revalidate user authz if the service->service thing is in place | 04:28 |
| jamielennox | rather than pass the service token + user headers and get back a reply with both | 04:28 |
| notmorgan | make someone ask for that | 04:28 |
| notmorgan | i really don't want to make it a "must have both" unless there is a damn good reason | 04:29 |
| notmorgan | i think it's better if we push to "validate on the edge" | 04:29 |
| jamielennox | so must have both is how we know that we can trust the X-User etc | 04:29 |
| *** doug-fish has quit IRC | 04:29 | |
| jamielennox | we don't just blindly trust those headers | 04:29 |
| notmorgan | no trust those headers if the role XXXX is on the service token | 04:30 |
| notmorgan | but i wouldn't make it validate both | 04:30 |
| notmorgan | if you pass a service token and a user token... ???? | 04:30 |
| notmorgan | i'd always defer to service token and then know if header XXXX is trusted | 04:31 |
| *** markvoelker has joined #openstack-keystone | 04:31 | |
| jamielennox | but we want the joint ownership for many things | 04:33 |
| notmorgan | you still rely on the user authz | 04:33 |
| notmorgan | and data | 04:33 |
| openstackgerrit | Steve Martinelli proposed openstack/ldappool: make ldappool py3 compatible https://review.openstack.org/315728 | 04:33 |
| notmorgan | just don't need to validate the user token | 04:33 |
| jamielennox | we need to pass the user headers to keystone | 04:33 |
| notmorgan | nope | 04:33 |
| jamielennox | oh | 04:34 |
| notmorgan | if it is validated at the edge | 04:34 |
| jamielennox | ergh - i know where you're going | 04:34 |
| notmorgan | user -> serivce | 04:34 |
| notmorgan | it's good. | 04:34 |
| notmorgan | :) | 04:34 |
| jamielennox | we had discussed to only pass essentially the fernet data around | 04:35 |
| jamielennox | then have keystone rebuild that info | 04:35 |
| jamielennox | rather than pass the entire env around | 04:35 |
| notmorgan | pass the minimal data around | 04:35 |
| notmorgan | but i'd rather avoid asking keystone each step | 04:35 |
| notmorgan | if that makes sense? | 04:35 |
| jamielennox | so we need to validate service token anywway | 04:36 |
| notmorgan | roles also don't really matter | 04:36 |
| notmorgan | just validate the service token | 04:36 |
| notmorgan | but it saves a dual token validate | 04:36 |
| jamielennox | so my thing is - combine it into one | 04:36 |
| jamielennox | validate the service token and user headers in one go | 04:36 |
| jamielennox | resp={'service': {...}, 'user': {...}} | 04:37 |
| notmorgan | if we're doing that lets force that to be on moving auth to /AUTH | 04:37 |
| *** markvoelker has quit IRC | 04:37 | |
| notmorgan | and make it a type of auth validation | 04:37 |
| notmorgan | i don't want to wedge it into v3 | 04:37 |
| jamielennox | i'd be happy to put it somewhere new | 04:37 |
| notmorgan | i think next week i'm gonna hack the auth change | 04:37 |
| notmorgan | :( | 04:37 |
| jamielennox | but i want to make at least hte keystone part of this doable this cycle | 04:38 |
| jamielennox | i've got a few changes up for context stuff | 04:38 |
| notmorgan | yeah | 04:38 |
| jamielennox | still haven't figured out messaging | 04:38 |
| notmorgan | and they'r epretty good. | 04:38 |
| jamielennox | so all that is going to take a while | 04:38 |
| notmorgan | just need to land/message them | 04:38 |
| notmorgan | so i think /AUTH plus wire /v3/auth up to /auth | 04:38 |
| notmorgan | we win | 04:38 |
| notmorgan | and make ksa smart | 04:38 |
| jamielennox | but i semi promised it to nova at summit so i need to do it now | 04:39 |
| jamielennox | we can do that, just add 'auth': '/auth' or something to GET / | 04:40 |
| jamielennox | all the generic passwords should hit / first for information anyway | 04:40 |
| jamielennox | so we just build that into discovery | 04:40 |
| jamielennox | i've always wanted to do more with the / response - like what auth plugins are enabled, what extensions etc | 04:40 |
| * notmorgan nods | 04:41 | |
| notmorgan | well def. ping me on any of those reviews | 04:41 |
| jamielennox | anyway - unified response? | 04:41 |
| jamielennox | validate service token and user headers in one? | 04:42 |
| openstackgerrit | Steve Martinelli proposed openstack/ldappool: use standard docstring convention for parameters https://review.openstack.org/315879 | 04:45 |
| jamielennox | it's been a while since i looked at fernet's formatting - oh god | 04:47 |
| openstackgerrit | Steve Martinelli proposed openstack/ldappool: use standard docstring convention for parameters https://review.openstack.org/315879 | 04:57 |
| *** itsmee has quit IRC | 05:04 | |
| *** sdake has joined #openstack-keystone | 05:05 | |
| *** itsme_ has joined #openstack-keystone | 05:07 | |
| itsme_ | Hello all | 05:08 |
| *** hoonetorg has quit IRC | 05:11 | |
| *** jaosorior has joined #openstack-keystone | 05:21 | |
| *** hoonetorg has joined #openstack-keystone | 05:24 | |
| *** sdake has quit IRC | 05:26 | |
| *** edtubill has quit IRC | 05:27 | |
| *** sdake has joined #openstack-keystone | 05:28 | |
| *** chlong has quit IRC | 05:29 | |
| *** markvoelker has joined #openstack-keystone | 05:32 | |
| *** rbridgeman has quit IRC | 05:33 | |
| *** edtubill has joined #openstack-keystone | 05:33 | |
| *** sdake has quit IRC | 05:33 | |
| jamielennox | stevemar: why does everything with oauth have admin_required for policy? | 05:34 |
| *** markvoelker has quit IRC | 05:37 | |
| stevemar | jamielennox: because thats what i copy and pasted 3 years ago when i didn't know any better | 05:38 |
| stevemar | jamielennox: also, your change really doesn't want to merge | 05:38 |
| jamielennox | stevemar: god damn it, again | 05:38 |
| jamielennox | ? | 05:38 |
| stevemar | jamielennox: i recheck'ed | 05:38 |
| jamielennox | stevemar: i want to do some testing with that in early in the cycle | 05:39 |
| jamielennox | merge damnit! | 05:39 |
| jamielennox | stevemar: have you ever heard of anyone using oauth? at least the request token stuff should be < admin | 05:40 |
| jamielennox | i think | 05:40 |
| jamielennox | it doesn't seem like anyone "owns" a consumer | 05:41 |
| *** edtubill has quit IRC | 05:41 | |
| jamielennox | but i almost think it should be unprotected and assume that oauth will validate the consumer key | 05:41 |
| jamielennox | i was trying to clean some of this up for a blog post - but i can't see how anyone could reasonably use it | 05:42 |
| jamielennox | yep, authorize_request_token requires admin - so this is broken for essentially everyone | 05:45 |
| *** fangxu has joined #openstack-keystone | 05:46 | |
| openstackgerrit | Ryosuke Mizuno proposed openstack/keystone: Add the validation rules when create token https://review.openstack.org/315894 | 05:48 |
| *** edtubill has joined #openstack-keystone | 05:48 | |
| *** rcernin has joined #openstack-keystone | 05:50 | |
| *** edtubill has quit IRC | 05:50 | |
| *** edtubill has joined #openstack-keystone | 05:54 | |
| *** jamielennox is now known as jamielennox|away | 05:56 | |
| *** jamielennox|away is now known as jamielennox | 06:10 | |
| *** fangxu_ has joined #openstack-keystone | 06:15 | |
| *** fangxu has quit IRC | 06:15 | |
| *** fangxu_ is now known as fangxu | 06:15 | |
| stevemar | jamielennox: just the rule by default? | 06:17 |
| *** woodster_ has quit IRC | 06:18 | |
| stevemar | jamielennox: if the policy rule was changed, would it be less broken? | 06:18 |
| *** fangxu has quit IRC | 06:21 | |
| *** lhcheng has quit IRC | 06:28 | |
| *** furface has quit IRC | 06:29 | |
| openstackgerrit | Steve Martinelli proposed openstack/keystone: Deprecate keystone.common.kvs https://review.openstack.org/271948 | 06:30 |
| jamielennox | the mechanism works, it's not going to give me the subcredentials that dolphm suggested, but it's actually a much better idea for the likes of heat | 06:33 |
| jamielennox | stevemar: so i think create_consumer is right as admin | 06:33 |
| *** markvoelker has joined #openstack-keystone | 06:33 | |
| jamielennox | but i think the rest of it can be just member, and maybe validate that the project you have permissions in the project you authorize (but that should be checked anyway) | 06:34 |
| jamielennox | actually for everything overcloud like heat it's completely right, i'm just not sure how we would deal with consumer keys | 06:36 |
| *** markvoelker has quit IRC | 06:38 | |
| jamielennox | stevemar: maybe request_token should be service | 06:42 |
| jamielennox | maybe access_token as well, but if they're protected by the consumer secret then so long as you're careful with that distribution it should be the same thing | 06:43 |
| *** ninag has joined #openstack-keystone | 06:51 | |
| *** tesseract has joined #openstack-keystone | 06:56 | |
| *** ninag has quit IRC | 06:56 | |
| openstackgerrit | Merged openstack/keystone-specs: Improve example of project acting as a domain https://review.openstack.org/315544 | 06:57 |
| stevemar | jamielennox: service meaning the service role? | 07:01 |
| *** edtubill has quit IRC | 07:05 | |
| jamielennox | stevemar: yep | 07:18 |
| jamielennox | stevemar: sorry, notifications playing up | 07:18 |
| *** fawadkhaliq has joined #openstack-keystone | 07:20 | |
| stevemar | np | 07:23 |
| jamielennox | so there's a bunch of stuff missing from policy - which i can't quite remember if it defaults to any role or admin only | 07:29 |
| jamielennox | this is way too sensitive for past 5 on a friday | 07:30 |
| *** chaithu has joined #openstack-keystone | 07:30 | |
| *** markvoelker has joined #openstack-keystone | 07:34 | |
| stevemar | jamielennox: ++ | 07:36 |
| stevemar | roll out, have a good one | 07:36 |
| stevemar | jamielennox: http://img.izismile.com/img/img7/20141015/1000/daily_gifdump_719_13.gif | 07:38 |
| *** markvoelker has quit IRC | 07:39 | |
| *** pnavarro has joined #openstack-keystone | 07:42 | |
| *** frickler has quit IRC | 07:54 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-keystone | 08:02 | |
| *** rha has joined #openstack-keystone | 08:04 | |
| *** nkinder has quit IRC | 08:06 | |
| *** josecastroleon has joined #openstack-keystone | 08:10 | |
| stevemar | tonyb: finally sent that pylibmc note to the mailing list \o/ | 08:14 |
| stevemar | i need to sleep | 08:15 |
| stevemar | jamielennox: have a good weekend | 08:15 |
| tonyb | stevemar: yes you do! | 08:15 |
| stevemar | :) | 08:15 |
| tonyb | stevemar: I was surprised that you were awake enough to comment on my squashed backport | 08:15 |
| jamielennox | stevemar: yea, i'm supposed to be done - it's gotta be late | 08:15 |
| stevemar | my body lives east coast, but my mind is clearly a fan of pacific time | 08:15 |
| tonyb | :) | 08:16 |
| stevemar | *yawn* see y'all in a few hours :) | 08:16 |
| *** lhcheng has joined #openstack-keystone | 08:17 | |
| *** ChanServ sets mode: +v lhcheng | 08:17 | |
| *** jorge_munoz has quit IRC | 08:19 | |
| *** ozialien10 has joined #openstack-keystone | 08:20 | |
| *** lhcheng has quit IRC | 08:22 | |
| *** alex_xu_ has quit IRC | 08:33 | |
| *** ramishra has quit IRC | 08:34 | |
| *** markvoelker has joined #openstack-keystone | 08:35 | |
| *** fhubik has joined #openstack-keystone | 08:36 | |
| *** alex_xu has joined #openstack-keystone | 08:37 | |
| *** markvoelker has quit IRC | 08:39 | |
| *** mhickey has joined #openstack-keystone | 08:42 | |
| *** jistr has joined #openstack-keystone | 08:46 | |
| *** frickler has joined #openstack-keystone | 08:47 | |
| chaithu | Hi All | 09:06 |
| chaithu | We am trying to do Keystone to Keystone Federation. We configured one keystone as Service Provider and other keystone as Identity Provider | 09:07 |
| chaithu | We got unscoped token but we are facing issue when trying to access federated projects list through the unscoped token. | 09:08 |
| chaithu | Here is the log from sp http://paste.openstack.org/show/497014/ | 09:09 |
| chaithu | Any help is highly appreciated. | 09:09 |
| *** dmk0202 has joined #openstack-keystone | 09:09 | |
| chaithu | This is the blog we are following http://blog.rodrigods.com/it-is-time-to-play-with-keystone-to-keystone-federation-in-kilo/ | 09:09 |
| samueldmq | morning | 09:13 |
| *** nkinder has joined #openstack-keystone | 09:17 | |
| *** henrynash has quit IRC | 09:18 | |
| *** mvk has joined #openstack-keystone | 09:25 | |
| *** markvoelker has joined #openstack-keystone | 09:36 | |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: Add API Change Tutorial https://review.openstack.org/302789 | 09:37 |
| *** markvoelker has quit IRC | 09:40 | |
| *** ninag has joined #openstack-keystone | 09:51 | |
| *** ninag has quit IRC | 09:56 | |
| chaithu | We got scoped token, but we are getting an error when we are trying to single sign on. Here is the error http://paste.openstack.org/show/497022/ | 09:58 |
| *** TxGVNN has joined #openstack-keystone | 10:00 | |
| *** EinstCrazy has quit IRC | 10:02 | |
| *** mhickey has quit IRC | 10:09 | |
| *** d0ugal has quit IRC | 10:20 | |
| *** d0ugal has joined #openstack-keystone | 10:21 | |
| chaithu | We are trying to do Keystone to Keystone Federation. We configured one keystone as Service Provider and other keystone as Identity Provider | 10:23 |
| chaithu | We got scoped token, but we are getting an error when we are trying to single sign on. Here is the error http://paste.openstack.org/show/497022/ | 10:23 |
| chaithu | This is how we are doing single sign on http://paste.openstack.org/show/497032/ | 10:24 |
| chaithu | Did anyone tried this before ? | 10:24 |
| chaithu | Did anyone tried this before ? | 10:25 |
| *** lhcheng has joined #openstack-keystone | 10:31 | |
| *** ChanServ sets mode: +v lhcheng | 10:31 | |
| samueldmq | chaithu: morning | 10:33 |
| samueldmq | chaithu: most of us are USA-based; so expect people to show up in the next few hours | 10:33 |
| samueldmq | chaithu: I don't have experience with k2k, maybe rodrigods may help you | 10:34 |
| chaithu | Oh.... Thank you for info :) | 10:34 |
| samueldmq | yw | 10:34 |
| *** lhcheng has quit IRC | 10:36 | |
| chaithu | samueldmq: Yes, I have hope | 10:36 |
| *** markvoelker has joined #openstack-keystone | 10:36 | |
| *** markvoelker has quit IRC | 10:41 | |
| *** josecastroleon has quit IRC | 10:43 | |
| *** itsme_ has quit IRC | 11:14 | |
| *** NikitaKonovalov has joined #openstack-keystone | 11:16 | |
| NikitaKonovalov | Hi, I've got a question about trusts behavior. If the trustor user is deleted will the trustee be able to get a token with a trust issued earlier? | 11:18 |
| *** mvk has quit IRC | 11:19 | |
| NikitaKonovalov | does implersonation flag matter in this case? | 11:21 |
| *** jaosorior has quit IRC | 11:30 | |
| *** jaosorior has joined #openstack-keystone | 11:30 | |
| rodrigods | chaithu, hi... you mean web sso? | 11:31 |
| chaithu | Yes | 11:32 |
| chaithu | rodrigods: remote-ids missing here http://paste.openstack.org/show/497049/ | 11:32 |
| rodrigods | so why are you trying via CLI? you should try via horizon | 11:33 |
| rodrigods | ? | 11:33 |
| chaithu | rodrigods: even from horizon same error too | 11:33 |
| rodrigods | chaithu, k2k, right? We can't do k2k websso using what is upstream yet | 11:34 |
| rodrigods | it is doable, but with some customization | 11:34 |
| chaithu | rodrigods: Should we set remote-ids like this https://keystone.idp/v3/OS-FEDERATION/saml2/idp_1 | 11:34 |
| rodrigods | chaithu, this is to improve the security (a must do, actually) | 11:35 |
| rodrigods | but for k2k websso, we need further work on horizon and keystoneclient | 11:35 |
| chaithu | We did some customization to horizon settings http://docs.openstack.org/developer/keystone/federation/websso.html#setup-web-single-sign-on-sso | 11:36 |
| *** josecastroleon has joined #openstack-keystone | 11:37 | |
| chaithu | is that remote-ids is correct ? | 11:37 |
| *** markvoelker has joined #openstack-keystone | 11:37 | |
| rodrigods | chaithu, what I mean is: you can't do k2k federation using websso with the current code upstream. You need custom code added. Websso is only possible with "regular"federation -> using an IdP that is not keystone | 11:38 |
| chaithu | Oh understand | 11:39 |
| chaithu | rodrigods: I have question then with the scoped token what do we do ? | 11:40 |
| *** markvoelker has quit IRC | 11:42 | |
| rodrigods | chaithu, from the point you have an openstack token, you can use the openstack resources | 11:47 |
| *** mvk has joined #openstack-keystone | 11:49 | |
| *** rodrigods has quit IRC | 11:52 | |
| *** rodrigods has joined #openstack-keystone | 11:52 | |
| chaithu | rodrigods: We are unble to access openstack resources. Do we need to role add group federated to admin ? | 12:07 |
| chaithu | rodrigods: http://paste.openstack.org/show/497052/ | 12:08 |
| rodrigods | chaithu, that depends what you are trying to do and also on the policy files. That particular request (list users) is admin only | 12:09 |
| rodrigods | chaithu, https://github.com/openstack/keystone/blob/master/etc/policy.v3cloudsample.json#L49 or https://github.com/openstack/keystone/blob/master/etc/policy.json#L45 | 12:10 |
| *** links has quit IRC | 12:14 | |
| *** ninag has joined #openstack-keystone | 12:16 | |
| *** BrAsS_mOnKeY has quit IRC | 12:16 | |
| *** darrenc_ has joined #openstack-keystone | 12:17 | |
| *** darrenc has quit IRC | 12:17 | |
| *** belmoreira has joined #openstack-keystone | 12:21 | |
| *** belmoreira has quit IRC | 12:21 | |
| *** raildo-afk is now known as raildo | 12:22 | |
| *** BrAsS_mOnKeY has joined #openstack-keystone | 12:22 | |
| *** darrenc_ is now known as darrenc | 12:22 | |
| *** henrynash has joined #openstack-keystone | 12:25 | |
| *** ChanServ sets mode: +v henrynash | 12:25 | |
| chaithu | rodrigods: Thanks a lot rorigods :) . Now we are able to access resources. I missed to add admin role to federated group. | 12:33 |
| *** markvoelker has joined #openstack-keystone | 12:38 | |
| *** iurygregory has joined #openstack-keystone | 12:41 | |
| *** rcernin_ has joined #openstack-keystone | 12:42 | |
| *** rcernin has quit IRC | 12:42 | |
| *** markvoelker has quit IRC | 12:43 | |
| *** edtubill has joined #openstack-keystone | 12:45 | |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: Create unit tests for endpoint policy drivers https://review.openstack.org/212006 | 12:48 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: Create unit tests for endpoint policy drivers https://review.openstack.org/212006 | 12:50 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: Create unit tests for endpoint policy drivers https://review.openstack.org/212006 | 12:54 |
| *** phalmos has joined #openstack-keystone | 12:55 | |
| *** edtubill has quit IRC | 12:59 | |
| *** ayoung has joined #openstack-keystone | 13:00 | |
| *** ChanServ sets mode: +v ayoung | 13:00 | |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: Create unit tests for the policy drivers https://review.openstack.org/212957 | 13:03 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: Create unit tests for endpoint policy drivers https://review.openstack.org/212006 | 13:04 |
| *** doug-fis_ has joined #openstack-keystone | 13:10 | |
| *** pnavarro has quit IRC | 13:11 | |
| *** doug-fis_ has quit IRC | 13:11 | |
| *** doug-fis_ has joined #openstack-keystone | 13:11 | |
| *** sdake has joined #openstack-keystone | 13:14 | |
| openstackgerrit | Merged openstack/keystone: switch to tempest instead of deprecated tempest-lib https://review.openstack.org/311901 | 13:15 |
| *** rcernin_ has quit IRC | 13:16 | |
| *** dave-mccowan has joined #openstack-keystone | 13:16 | |
| *** timcline has joined #openstack-keystone | 13:19 | |
| *** fawadkhaliq has quit IRC | 13:20 | |
| *** fawadkhaliq has joined #openstack-keystone | 13:20 | |
| *** edtubill has joined #openstack-keystone | 13:20 | |
| *** timcline has quit IRC | 13:21 | |
| *** odyssey4me has quit IRC | 13:21 | |
| *** odyssey4me has joined #openstack-keystone | 13:21 | |
| *** timcline has joined #openstack-keystone | 13:22 | |
| *** fawadkhaliq has quit IRC | 13:25 | |
| *** rcernin has joined #openstack-keystone | 13:27 | |
| *** spzala has joined #openstack-keystone | 13:33 | |
| *** edtubill has quit IRC | 13:34 | |
| *** markvoelker has joined #openstack-keystone | 13:39 | |
| *** jsavak has joined #openstack-keystone | 13:40 | |
| *** fhubik has quit IRC | 13:42 | |
| *** openstack has joined #openstack-keystone | 13:43 | |
| *** markvoelker has quit IRC | 13:44 | |
| *** openstack has joined #openstack-keystone | 13:44 | |
| chaithu | Hi Stevemar | 13:52 |
| *** openstack has joined #openstack-keystone | 13:52 | |
| chaithu | Stevemar: Hi Stevemar. What is sso-demo.test.ibmcloud.com here https://gist.github.com/stevemart/4b41bd5437048a7fdfab What you are referring. Could you please tell us ? | 13:53 |
| samueldmq | lbragstad: do you have any news about failing jobs after fernet was made default? | 13:53 |
| *** markvoelker has joined #openstack-keystone | 13:55 | |
| samueldmq | lbragstad: http://logs.openstack.org/31/197331/21/check/gate-tempest-dsvm-full/7342a5a/console.html#_2016-05-13_13_26_40_086 | 13:55 |
| lbragstad | samueldmq I've been working on it for a few days | 13:56 |
| lbragstad | https://bugs.launchpad.net/keystone/+bug/1578866 | 13:56 |
| samueldmq | lbragstad: have you verified that the deleted token is the same used in the next request ? | 13:57 |
| samueldmq | lbragstad: for example: https://github.com/openstack/tempest/blob/16863a1b4b16b53c3a73813d5dc6c3122f2d8cbc/tempest/api/identity/admin/v2/test_roles_negative.py#L41-L46 | 13:57 |
| samueldmq | lbragstad: token in L45 is the same that is revoked in L44 | 13:58 |
| *** ninag has quit IRC | 13:58 | |
| openstack | Launchpad bug 1578866 in OpenStack Identity (keystone) "test_user_update_own_password failing intermittently" [High,In progress] - Assigned to Lance Bragstad (lbragstad) | 13:58 |
| samueldmq | lbragstad: this is test_list_roles_request_without_token (another test), and it's failing sometimes as well :/ | 13:58 |
| lbragstad | samueldmq yeah - that makes sense | 13:59 |
| lbragstad | samueldmq are you saying the test is wrong? | 13:59 |
| samueldmq | lbragstad: I don't think so, otherwise it would be failing 100% of the time | 13:59 |
| samueldmq | :( | 13:59 |
| lbragstad | samueldmq right | 13:59 |
| samueldmq | lbragstad: but yes, I was wondering that, but doesn't make sense | 14:00 |
| *** ninag has joined #openstack-keystone | 14:00 | |
| samueldmq | lbragstad: what do you think is causing the issue ? | 14:00 |
| samueldmq | lbragstad: is it always related to revocations ? | 14:00 |
| lbragstad | samueldmq I not quite sure anymore.. | 14:00 |
| lbragstad | I have a strong feeling that it is related to second precision | 14:01 |
| lbragstad | but we've accounted for that in the tempest tests | 14:01 |
| samueldmq | lbragstad: fernet tokens don't have subsecond precision at all right ? | 14:02 |
| lbragstad | samueldmq right - that's why we have some time.sleep(1) statements in temepst | 14:02 |
| *** ninag has quit IRC | 14:03 | |
| samueldmq | lbragstad: so token validation check does: IF user_token.expiry > now(): fail | 14:03 |
| *** ninag has joined #openstack-keystone | 14:04 | |
| *** ametts has joined #openstack-keystone | 14:04 | |
| samueldmq | lbragstad: is that right? | 14:04 |
| lbragstad | samueldmq yes - but I think it's the opposite | 14:04 |
| lbragstad | if token.expires < now(): fail | 14:04 |
| samueldmq | lbragstad: if we changed > by >= that should be fixed | 14:05 |
| lbragstad | samueldmq i'm not sure it's related to token expiration | 14:05 |
| samueldmq | lbragstad: and the current second would be taken into account, which does make sense to me | 14:05 |
| lbragstad | samueldmq i have a feeling it's the issued_at time of the token and issued_before time of the revocation event | 14:05 |
| samueldmq | lbragstad: it's explicity revocation ? | 14:05 |
| samueldmq | lbragstad: yes, when we check for validity of the token against the revocation event | 14:06 |
| samueldmq | lbragstad: we need to include the current second, something like that I was saying | 14:06 |
| samueldmq | the comparison should be >= vs > (same applies to <= and < if that's the case) | 14:07 |
| *** ninag has quit IRC | 14:08 | |
| *** gordc has joined #openstack-keystone | 14:08 | |
| samueldmq | lbragstad: makes sense ? | 14:09 |
| *** EinstCrazy has joined #openstack-keystone | 14:09 | |
| lbragstad | samueldmq we do | 14:09 |
| lbragstad | samueldmq for example | 14:09 |
| lbragstad | if we get a token at 9:10:12.000001 which is rolling into the threshold of a new second, the token issued_at time will be 9:10:12.000000 | 14:10 |
| *** ninag has joined #openstack-keystone | 14:11 | |
| *** ninag has quit IRC | 14:11 | |
| lbragstad | when we search the revocation events - we compare it to the issued_before time of the revocation event with a less than or equal to statement. | 14:11 |
| lbragstad | https://github.com/openstack/keystone/blob/0f579aa362f021b17f7c2931ffca309e395bd51f/keystone/models/revoke_model.py#L223 | 14:12 |
| lbragstad | so - if we revoke our token at 9:10:12.000524 and get a new token at 9:10:12.009827 | 14:13 |
| samueldmq | lbragstad: ok | 14:13 |
| samueldmq | lbragstad: https://github.com/openstack/keystone/blob/0f579aa362f021b17f7c2931ffca309e395bd51f/keystone/models/revoke_model.py#L225 | 14:13 |
| lbragstad | the token we *just* got would still be revoked | 14:13 |
| samueldmq | lbragstad: may this be happening ? | 14:13 |
| lbragstad | hence the time.sleep(1)s in temepst | 14:13 |
| lbragstad | samueldmq what do you mean? | 14:14 |
| samueldmq | lbragstad: what happens if 'issued_at' is not in the token data? | 14:15 |
| samueldmq | lbragstad: it isn't a required field, and is_revoked would return False | 14:17 |
| samueldmq | lbragstad: https://github.com/openstack/keystone/blob/0f579aa362f021b17f7c2931ffca309e395bd51f/keystone/models/revoke_model.py#L247-L252 | 14:17 |
| samueldmq | lbragstad: however not sure this is the case for those failures | 14:17 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:19 | |
| *** jaosorior has quit IRC | 14:19 | |
| *** links has joined #openstack-keystone | 14:19 | |
| openstackgerrit | Dolph Mathews proposed openstack/keystone: Add feature support matrix to documentation https://review.openstack.org/316118 | 14:20 |
| openstackgerrit | henry-nash proposed openstack/keystone: Drop the (unused) domain table https://review.openstack.org/316119 | 14:20 |
| *** edtubill has joined #openstack-keystone | 14:23 | |
| lbragstad | yeah - issued_at should always be passed into the revoke api I believe | 14:24 |
| lbragstad | I can't think of a case where it wouldn't be | 14:24 |
| lbragstad | samueldmq do you have any thoughts on https://bugs.launchpad.net/keystone/+bug/1578866/comments/11 | 14:25 |
| openstack | Launchpad bug 1578866 in OpenStack Identity (keystone) "test_user_update_own_password failing intermittently" [High,In progress] - Assigned to Lance Bragstad (lbragstad) | 14:25 |
| lbragstad | ? | 14:25 |
| *** dan_nguyen has joined #openstack-keystone | 14:25 | |
| samueldmq | lbragstad: if I do DELETE v3/tokens/abc123 | 14:26 |
| samueldmq | lbragstad: what does the revocation event look like? | 14:26 |
| * samueldmq 's looking | 14:26 | |
| *** d0ugal has quit IRC | 14:29 | |
| *** links has quit IRC | 14:29 | |
| lbragstad | samueldmq this is an example of what the revocation events look like in sql http://cdn.pasteraw.com/5x9hfswrfn0g91kmrf2wzdn7j2uxace | 14:29 |
| samueldmq | lbragstad: that can be a possibility | 14:29 |
| samueldmq | lbragstad: the other is that our revocation code is wrong ? (very unlikely?) | 14:29 |
| *** EinstCrazy has quit IRC | 14:30 | |
| lbragstad | samueldmq possibkly | 14:30 |
| samueldmq | lbragstad: so something must match before comparing issued_before | 14:30 |
| *** d0ugal has joined #openstack-keystone | 14:30 | |
| lbragstad | samueldmq the part that throw me is that if the logic in the revocation api were wrong we would hopefully be seeing more consistent failures | 14:31 |
| *** EinstCrazy has joined #openstack-keystone | 14:32 | |
| samueldmq | lbragstad: ++ | 14:32 |
| samueldmq | lbragstad: look at this failure http://paste.openstack.org/show/OBUGE3Ep7SBerrPiO81l/ | 14:32 |
| samueldmq | lbragstad: search for 13:16:18 | 14:32 |
| *** jorge_munoz has joined #openstack-keystone | 14:32 | |
| lbragstad | samueldmq i see 8 occurrences | 14:33 |
| samueldmq | lbragstad: don't look at the ones logged by tempestt | 14:33 |
| samueldmq | lbragstad: overall yo ucan see 3 blocks of log | 14:33 |
| lbragstad | samueldmq "issued_at": "2016-05-13T13:16:18.000000Z" | 14:33 |
| samueldmq | lbragstad: the first is POST /tokens at 13:16:18 | 14:34 |
| samueldmq | lbragstad: the second is DELETE /tokens at 13:16:18 | 14:34 |
| samueldmq | lbragstad: the third is GET /roles and keystone still says it's 13:16:18 at its time | 14:34 |
| lbragstad | samueldmq and that third GET shouldn't work | 14:36 |
| *** jorge_munoz has quit IRC | 14:36 | |
| *** jorge_munoz has joined #openstack-keystone | 14:36 | |
| samueldmq | lbragstad: exactly, because token should be invalid | 14:36 |
| lbragstad | samueldmq yup | 14:37 |
| samueldmq | lbragstad: and that's not related to user changing password, so yes it's indeed an issue with the revocation events + token recovations | 14:37 |
| lbragstad | samueldmq that's what i'm thinking | 14:37 |
| samueldmq | nothing specific to password change workflow | 14:38 |
| *** can8dnSix has joined #openstack-keystone | 14:38 | |
| *** d0ugal has quit IRC | 14:38 | |
| lbragstad | right - but something that is still requires the revocation API | 14:38 |
| *** gyee_ has joined #openstack-keystone | 14:38 | |
| *** gyee has quit IRC | 14:39 | |
| openstackgerrit | Michael Bayer proposed openstack/keystone: Don't set None for ldap.OPT_X_TLS_CACERTFILE https://review.openstack.org/316129 | 14:40 |
| *** ksavich has quit IRC | 14:42 | |
| samueldmq | lbragstad: tell me, when we do the revoke check in https://github.com/openstack/keystone/blob/0f579aa362f021b17f7c2931ffca309e395bd51f/keystone/models/revoke_model.py#L223 | 14:44 |
| samueldmq | lbragstad: are we comparing the time strings | 14:44 |
| samueldmq | ? | 14:44 |
| lbragstad | samueldmq i can double check | 14:45 |
| *** mou has quit IRC | 14:48 | |
| *** mou has joined #openstack-keystone | 14:49 | |
| samueldmq | lbragstad: I am asking because sometimes I see "2016-05-13T13:16:18Z", and sometimes I see "2016-05-13T13:16:18.000000Z" | 14:49 |
| samueldmq | lbragstad: and: | 14:49 |
| lbragstad | samueldmq for what keys? | 14:49 |
| *** jorge_munoz_ has joined #openstack-keystone | 14:49 | |
| lbragstad | "2016-05-13T13:16:18.000000Z" should be the token data's issued_at time | 14:49 |
| samueldmq | lbragstad: always with the subsecond precision right? | 14:50 |
| samueldmq | lbragstad: even if it is always 0 for fernet | 14:50 |
| *** jorge_munoz__ has joined #openstack-keystone | 14:50 | |
| lbragstad | samueldmq should be | 14:51 |
| samueldmq | lbragstad: yes, should be, see: | 14:51 |
| samueldmq | >>> "2016-05-13T13:16:18.000000Z" >= "2016-05-13T13:16:18.000000Z" | 14:51 |
| samueldmq | True | 14:51 |
| samueldmq | >>> "2016-05-13T13:16:18.000000Z" >= "2016-05-13T13:16:18Z" | 14:51 |
| samueldmq | False | 14:51 |
| lbragstad | .... | 14:51 |
| lbragstad | wtf | 14:51 |
| lbragstad | really?! | 14:51 |
| lbragstad | ok - so... how would that be a race condition? | 14:52 |
| edmondsw | ayoung, I've got a customer who mentioned using cn as the user passward attribute with LDAP because "i want to use for authentication kerberos"... does that make sense? will it work? | 14:52 |
| *** jorge_munoz has quit IRC | 14:53 | |
| *** jorge_munoz__ is now known as jorge_munoz | 14:53 | |
| *** slberger has joined #openstack-keystone | 14:53 | |
| samueldmq | lbragstad: if the token was generated exatcly at 13:16:18 (0 subseconds) and keystone showed it without the .000000 | 14:53 |
| samueldmq | ? | 14:53 |
| *** jorge_munoz_ has quit IRC | 14:54 | |
| openstackgerrit | Merged openstack/keystone: Move the federation abstract base class out of core https://review.openstack.org/314137 | 14:54 |
| lbragstad | samueldmq but... sql can truncate the revocation datetimes | 14:54 |
| *** BjoernT has joined #openstack-keystone | 14:54 | |
| lbragstad | so if something was revoked at 9:10:12.093452 it would be stored as 9:10:12 | 14:55 |
| samueldmq | lbragstad: so it's always stored without subsecond precision anyways | 14:56 |
| lbragstad | samueldmq but - that can change depending on the version of sql you're using | 14:56 |
| samueldmq | lbragstad: I'd need to do some testing and see how dates are handled in token generaiton / revocation events | 14:57 |
| lbragstad | some versions of sql will truncate extra precision from the datetime object | 14:57 |
| lbragstad | and some will perform rounding based on the precision | 14:57 |
| samueldmq | lbragstad: we'd need to debug the revocation engine, and see what's coming from sql | 14:58 |
| lbragstad | samueldmq dropping some log statements in it now | 14:58 |
| samueldmq | lbragstad: nice, bad part of it is that we can't mke our patches fail :( | 14:58 |
| *** BAKfr has quit IRC | 14:59 | |
| samueldmq | lbragstad: would we be allowed to merge some temporary logging in keystone ? so we can debug this issue ? | 14:59 |
| lbragstad | samueldmq possibly - dstanek was talking about that yesterday | 15:00 |
| lbragstad | the revocation api doesn't have much for logging | 15:00 |
| samueldmq | because it would be too verbosing right ? :/ | 15:00 |
| samueldmq | verbose* | 15:01 |
| *** ninag has joined #openstack-keystone | 15:01 | |
| *** ninag has quit IRC | 15:01 | |
| *** woodster_ has joined #openstack-keystone | 15:01 | |
| lbragstad | samueldmq probably | 15:01 |
| *** phalmos has quit IRC | 15:02 | |
| *** jsavak has quit IRC | 15:02 | |
| *** BAKfr has joined #openstack-keystone | 15:02 | |
| *** jsavak has joined #openstack-keystone | 15:02 | |
| *** ninag_ has joined #openstack-keystone | 15:03 | |
| lbragstad | samueldmq my diff of the logging i'm adding http://cdn.pasteraw.com/hpertp6wqfbtmc160owayzodpqjd623 | 15:03 |
| *** phalmos has joined #openstack-keystone | 15:03 | |
| samueldmq | lbragstad: maybe loggig all the token data is more useful ? | 15:05 |
| samueldmq | lbragstad: so taht we can identify what token is being checkd | 15:05 |
| *** d0ugal has joined #openstack-keystone | 15:05 | |
| *** julim has quit IRC | 15:05 | |
| samueldmq | lbragstad: I will apply this and run the test test_list_roles_request_without_token so many times tht it will need to fail sometime | 15:05 |
| samueldmq | heheh | 15:05 |
| lbragstad | samueldmq ha - we already tried that | 15:06 |
| lbragstad | samueldmq looks like they are compared as datetime objects | 15:07 |
| lbragstad | 2016-05-13 15:03:31.862 1946 INFO keystone.models.revoke_model [req-97f4542a-f634-4f02-ada1-d4c9a601d12b - - - - -] revoke_map.issued_before: datetime.datetime(2016, 5, 13, 15, 3, 30) | 15:07 |
| lbragstad | 2016-05-13 15:03:31.862 1946 INFO keystone.models.revoke_model [req-97f4542a-f634-4f02-ada1-d4c9a601d12b - - - - -] token_data.issued_at: datetime.datetime(2016, 5, 13, 15, 3, 30) | 15:07 |
| *** jsavak has quit IRC | 15:07 | |
| *** julim has joined #openstack-keystone | 15:08 | |
| lbragstad | http://cdn.pasteraw.com/sytc6xrjsnytufmzi5erthorld6jyo7 | 15:08 |
| samueldmq | lbragstad: yep | 15:12 |
| ayoung | edmondsw, password? | 15:12 |
| ayoung | it does not make sense to me. password should not be readable, so I would think that would break no matter what. | 15:12 |
| edmondsw | ayoung, yeah, I couldn't figure what they might be thinking, but thought maybe I was missing something | 15:13 |
| ayoung | We had a way to do Kerberos with LDAP with the same general setup as a simple bind | 15:13 |
| *** dmk0202 has quit IRC | 15:13 | |
| *** rha has quit IRC | 15:13 | |
| edmondsw | ayoung, how? I've not worked with kerberos before | 15:14 |
| mfisch | dolphm: ready | 15:14 |
| ayoung | edmondsw, if they want to do Kerberos, front the /auth section with mod_auth_kerb or mod_auth_gssapi and set the auth method to kerberos | 15:14 |
| ayoung | I have not tested it in a while, it might well have bitrotted, but I don't think so | 15:14 |
| *** rha has joined #openstack-keystone | 15:14 | |
| *** rha has quit IRC | 15:14 | |
| ayoung | edmondsw, you need to have the kerberos auth plugin | 15:14 |
| *** haplo37 has joined #openstack-keystone | 15:14 | |
| ayoung | we had that in an separate repo, keystoneclient-kerberos. Which might be an issue with keystoneauth1 now. | 15:15 |
| *** rha has joined #openstack-keystone | 15:15 | |
| *** rha has quit IRC | 15:15 | |
| *** rha has joined #openstack-keystone | 15:15 | |
| ayoung | edmondsw, http://adam.younglogic.com/2014/07/kerberos-for-horizon-and-keystone/ is the last step. Let me find an earlier | 15:15 |
| *** chaithu has quit IRC | 15:16 | |
| ayoung | http://adam.younglogic.com/2014/05/keystoneclient-s4u2proxy/ | 15:16 |
| mfisch | dstanek: also here | 15:16 |
| *** pnavarro has joined #openstack-keystone | 15:16 | |
| ayoung | edmondsw, but to be honest, I like the combo of Kerberos and LDAP via SSSD. | 15:16 |
| dstanek | mfisch: k | 15:17 |
| ayoung | http://adam.younglogic.com/2015/03/key-fed-lookup-redux/ | 15:17 |
| *** openstackgerrit has quit IRC | 15:17 | |
| *** openstackgerrit has joined #openstack-keystone | 15:17 | |
| ayoung | edmondsw, jamielennox and did a proof of concept for that last summer, albeit using FreeIPA as the Kerb backend, and it worked quite well. | 15:18 |
| ayoung | https://github.com/admiyo/rippowam/tree/master/roles/packstack/tasks | 15:18 |
| *** rha has quit IRC | 15:19 | |
| edmondsw | ayoung, tx, I'll do some reading | 15:20 |
| ayoung | edmondsw, ++ | 15:20 |
| *** rha has joined #openstack-keystone | 15:21 | |
| *** rha has quit IRC | 15:21 | |
| *** rha has joined #openstack-keystone | 15:21 | |
| *** edtubill has quit IRC | 15:24 | |
| *** edtubill has joined #openstack-keystone | 15:24 | |
| *** henrynash has quit IRC | 15:26 | |
| *** spzala has quit IRC | 15:27 | |
| *** roxanagh_ has joined #openstack-keystone | 15:28 | |
| *** phalmos_ has joined #openstack-keystone | 15:29 | |
| *** phalmos has quit IRC | 15:32 | |
| openstackgerrit | Elvin Tubillara proposed openstack/keystone: Config changes to support PCI-DSS https://review.openstack.org/314679 | 15:32 |
| samueldmq | lbragstad: have you applied those LOGs to your local deploy? | 15:32 |
| *** ericksonsantos has joined #openstack-keystone | 15:33 | |
| lbragstad | samueldmq yeah - i have a local devstack with tempest + kesytone | 15:33 |
| samueldmq | lbragstad: are you logging all the token data? | 15:33 |
| lbragstad | samueldmq not yet | 15:33 |
| lbragstad | I can modify it though | 15:33 |
| samueldmq | lbragstad: could you do a DELETE token and then GET /something | 15:33 |
| samueldmq | so we can see what it looks like ? | 15:34 |
| samueldmq | lbragstad: please log all the token data and all the revoke_map if possible | 15:34 |
| *** jsavak has joined #openstack-keystone | 15:34 | |
| lbragstad | samueldmq ok | 15:34 |
| samueldmq | lbragstad: I am thinking of writting some unittests | 15:34 |
| samueldmq | lbragstad: to test the revocation engine behavior | 15:34 |
| *** tonytan4ever has joined #openstack-keystone | 15:37 | |
| lbragstad | samueldmq http://cdn.pasteraw.com/k2weyf0z12vptw5w6glxzd1vekb4wm9 | 15:38 |
| lbragstad | with this diff http://cdn.pasteraw.com/oytc7i7y88qji5cd2zig2g2ur7skibl | 15:38 |
| *** roxanagh_ has quit IRC | 15:40 | |
| samueldmq | lbragstad: thanks | 15:40 |
| *** yarkot has quit IRC | 15:43 | |
| *** haplo37 has quit IRC | 15:47 | |
| *** henrynash has joined #openstack-keystone | 15:52 | |
| *** ChanServ sets mode: +v henrynash | 15:52 | |
| openstackgerrit | henry-nash proposed openstack/keystone: Drop the (unused) domain table https://review.openstack.org/316119 | 15:54 |
| *** GB21 has joined #openstack-keystone | 15:55 | |
| *** tesseract has quit IRC | 15:56 | |
| *** EinstCrazy has quit IRC | 15:56 | |
| *** gyee_ has quit IRC | 15:56 | |
| *** spzala has joined #openstack-keystone | 15:58 | |
| *** spzala_ has joined #openstack-keystone | 15:59 | |
| *** jsavak has quit IRC | 16:01 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/keystone: Updated from global requirements https://review.openstack.org/316170 | 16:02 |
| *** gyee has joined #openstack-keystone | 16:02 | |
| *** ChanServ sets mode: +v gyee | 16:02 | |
| *** spzala has quit IRC | 16:02 | |
| *** rcernin has quit IRC | 16:02 | |
| *** jsavak has joined #openstack-keystone | 16:02 | |
| *** eandersson has quit IRC | 16:03 | |
| lbragstad | samueldmq not seeing much in those logs | 16:04 |
| lbragstad | samueldmq well - not much interesting stuff | 16:04 |
| *** spzala_ has quit IRC | 16:04 | |
| lbragstad | it does look like issued_at and issued_before are both datetime objects when they are compared | 16:04 |
| *** lhcheng has joined #openstack-keystone | 16:11 | |
| *** ChanServ sets mode: +v lhcheng | 16:11 | |
| *** rbridgeman has joined #openstack-keystone | 16:11 | |
| *** spzala has joined #openstack-keystone | 16:11 | |
| *** GB21 has quit IRC | 16:15 | |
| *** spzala has quit IRC | 16:16 | |
| *** roxanagh_ has joined #openstack-keystone | 16:17 | |
| *** markvoelker has quit IRC | 16:18 | |
| samueldmq | lbragstad: yes, same here | 16:18 |
| *** josecastroleon has quit IRC | 16:20 | |
| dolphm | lbragstad: what were the edge cases that fernet didn't *originally* support? or what didn't it support previous to mitaka, anyway? | 16:20 |
| lbragstad | dolphm wrt subsecond accuracy? | 16:21 |
| dolphm | lbragstad: i'm just thinking API operations | 16:21 |
| dolphm | lbragstad: like, you couldn't get a fernet token using an oauth access token, right? | 16:21 |
| lbragstad | dolphm yeah, oauth was one | 16:22 |
| lbragstad | trusts on v2.0 were another | 16:22 |
| *** agrebennikov has joined #openstack-keystone | 16:22 | |
| lbragstad | using bind | 16:22 |
| dolphm | lbragstad: federation was an issue in kilo | 16:22 |
| *** markvoelker has joined #openstack-keystone | 16:22 | |
| dolphm | ooh, bind | 16:22 |
| dolphm | and we still dont' support that in master, right? | 16:22 |
| lbragstad | right | 16:23 |
| lbragstad | bind has never made it into the token payload | 16:23 |
| lbragstad | because it can technically be unbound | 16:23 |
| *** spzala has joined #openstack-keystone | 16:23 | |
| samueldmq | lbragstad: http://paste.openstack.org/show/497105/ I can't even revoke my own fernet token with my fernet token here in my devstck | 16:23 |
| openstackgerrit | Dolph Mathews proposed openstack/keystone: Add feature support matrix to documentation https://review.openstack.org/316118 | 16:26 |
| lbragstad | samueldmq that's strange | 16:26 |
| lbragstad | samueldmq dolphm I can confirm that with my devstack the sql backend is truncating datetimes for revocation events, not rounding them | 16:27 |
| lbragstad | revocation event before going into sql {'user_id': u'ca2c5cb035b8443ba79c9560d938dd00', 'issued_before': '2016-05-13T16:12:10.745395Z'} | 16:27 |
| *** spzala has quit IRC | 16:27 | |
| lbragstad | data in sql after the revocation event is persisted - | 2292 | ca2c5cb035b8443ba79c9560d938dd00 | 2016-05-13 16:12:10 | 2016-05-13 16:12:10 | | 16:28 |
| dolphm | lbragstad: on mysql version what? | 16:30 |
| lbragstad | 5.5.49-0ubuntu0.14.04.1 | 16:30 |
| lbragstad | dolphm which is the same one used by devstack | 16:30 |
| samueldmq | tokens keep being valid is the issue right ? | 16:31 |
| agrebennikov | hey ayoung, you have any idea why in v3 I cannot set my id for a project, while in v2 I could do it? | 16:31 |
| samueldmq | so revocation events always happen *after* token issue in our scenario | 16:31 |
| samueldmq | rounding vs trucanting shouldn't matter in this case | 16:31 |
| samueldmq | lbragstad: ^ | 16:31 |
| lbragstad | yeah - if it rounded up I think it would only benefit us | 16:31 |
| samueldmq | exactly | 16:32 |
| lbragstad | since the expression is issued_before >= issued_at | 16:32 |
| lbragstad | https://github.com/openstack/keystone/blob/0f579aa362f021b17f7c2931ffca309e395bd51f/keystone/models/revoke_model.py#L223 | 16:32 |
| lbragstad | regardless - our app should be handling time correctly so we're not prone to that kind of stuff anyway | 16:32 |
| lbragstad | or - | 16:32 |
| lbragstad | our app should be handling time such that we aren't prone to differences in backends | 16:33 |
| lbragstad | i.e. removing microsecond precision before storing the revocation event | 16:33 |
| *** spzala has joined #openstack-keystone | 16:35 | |
| *** sdake_ has joined #openstack-keystone | 16:36 | |
| lbragstad | dolphm samueldmq actually | 16:36 |
| lbragstad | dolphm samueldmq does this sound crazy? | 16:37 |
| lbragstad | in keystone when a user changes their password we send a notification to handle the revocation event | 16:37 |
| dolphm | samueldmq: "set my id for a project" ? | 16:37 |
| lbragstad | could it be that the validation of the "revoked" token is beating the queue to keystone? | 16:38 |
| *** spzala has quit IRC | 16:39 | |
| *** sdake has quit IRC | 16:39 | |
| *** mvk has quit IRC | 16:41 | |
| samueldmq | dolphm: what's that? | 16:41 |
| dolphm | samueldmq: i'm quoting you, wondering what you're referring to | 16:41 |
| samueldmq | lbragstad: keep in mind it's happening for other cases too, like explicitly deleting a token | 16:42 |
| dolphm | lbragstad: i doubt it | 16:42 |
| *** jistr has quit IRC | 16:42 | |
| lbragstad | samueldmq right - but for fernet we have to rely on the revocation api for everything | 16:42 |
| dolphm | samueldmq: with fernet? | 16:42 |
| dolphm | lbragstad: ++ | 16:42 |
| samueldmq | dolphm: yes | 16:43 |
| samueldmq | dolphm: I didn't remember to sya that ? lol | 16:43 |
| samueldmq | dolphm: for example (failing with explicity token delete) | 16:43 |
| samueldmq | http://logs.openstack.org/31/197331/21/check/gate-tempest-dsvm-full/7342a5a/console.html#_2016-05-13_13_26_40_086 | 16:43 |
| dolphm | samueldmq: that produces a revocation event, just like changing your password | 16:43 |
| samueldmq | dolphm: exactly, so it's nothing to do with the password change specifically | 16:44 |
| *** raddaoui has joined #openstack-keystone | 16:44 | |
| *** daemontool_ has joined #openstack-keystone | 16:45 | |
| lbragstad | i'm standing up a new devstack and i'm going to run with http://logs.openstack.org/30/314330/3/check/gate-tempest-dsvm-neutron-dvr/3d9272f/logs/reproduce.sh | 16:45 |
| *** spzala has joined #openstack-keystone | 16:47 | |
| *** daemontool has quit IRC | 16:48 | |
| *** josecastroleon has joined #openstack-keystone | 16:49 | |
| *** spzala has quit IRC | 16:51 | |
| lbragstad | dolphm samueldmq ping me your public keys if you want to poke at it | 16:52 |
| *** roxanagh_ has quit IRC | 16:52 | |
| *** sdake_ has quit IRC | 16:54 | |
| *** tpeoples has quit IRC | 16:56 | |
| *** nikhil has quit IRC | 16:56 | |
| *** auggy has quit IRC | 16:56 | |
| *** jed56 has quit IRC | 16:56 | |
| *** spzala has joined #openstack-keystone | 16:59 | |
| *** catintheroof has joined #openstack-keystone | 16:59 | |
| *** catintheroof has quit IRC | 17:00 | |
| *** catintheroof has joined #openstack-keystone | 17:00 | |
| *** spzala has quit IRC | 17:03 | |
| samueldmq | lbragstad: I can't reproduce the error even with a bash script | 17:05 |
| samueldmq | lbragstad: http://paste.openstack.org/show/497108/ run under 1 sec, and it always work as expected | 17:05 |
| lbragstad | samueldmq yeah - bknudson and i both saw the same thing | 17:06 |
| *** jsavak has quit IRC | 17:07 | |
| openstackgerrit | Dolph Mathews proposed openstack/keystone: Add feature support matrix to documentation https://review.openstack.org/316118 | 17:07 |
| *** jsavak has joined #openstack-keystone | 17:07 | |
| samueldmq | lbragstad: I'd like to see logs showing X-Auth-Token in the succeding request is the same as the revoked one | 17:08 |
| samueldmq | lbragstad: so we'd be sure the issue is within keystone | 17:08 |
| samueldmq | lbragstad: tempest deletes self.client.token, but what happens if it changes that value by a valid token somehow before the next request | 17:09 |
| lbragstad | samueldmq jordanP had a patch up to add more logging | 17:09 |
| lbragstad | specifically for that case | 17:09 |
| lbragstad | https://review.openstack.org/#/c/314121/ | 17:10 |
| patchbot | lbragstad: patch 314121 - tempest - WIP : also log Auth-Token | 17:10 |
| samueldmq | lbragstad: in the password test, specifically, it does something like: revoke self.client.token and use self.users_client in the next request | 17:10 |
| openstackgerrit | henry-nash proposed openstack/keystone: Drop the (unused) domain table https://review.openstack.org/316119 | 17:10 |
| samueldmq | lbragstad: what's the guarantee self.client.token is always equal to self.users_client.token ? | 17:10 |
| *** spzala has joined #openstack-keystone | 17:10 | |
| samueldmq | lbragstad: what I mean is that it can be a race condition within tempest tests, maybe unlikely to happen, but how can we (keystone side) be sure of that? | 17:11 |
| lbragstad | samueldmq I'd have to dig around in the tempest code to answer that one | 17:11 |
| lbragstad | but if it wasn't - it wouldn't be a race condition, would it? | 17:11 |
| samueldmq | lbragstad: not sure, it would be something like tempest replacing the revoked token with another very quickly | 17:13 |
| lbragstad | hmm - that seems like it would either fail all the time or not at all? | 17:13 |
| samueldmq | lbragstad: like having a tokenpool and geting another token from there | 17:13 |
| samueldmq | lbragstad: not sure, that's very unlike to happen, but we need to start from the beginning, and make sure it isn't anything related to tempest | 17:14 |
| samueldmq | before crazily digging into keystone | 17:14 |
| samueldmq | and it's something none of us got to reproduce, not even once | 17:14 |
| samueldmq | :( | 17:14 |
| lbragstad | samueldmq yeah - that makes sense | 17:14 |
| *** spzala has quit IRC | 17:14 | |
| lbragstad | but that's why i'm trying to do it with devstack and tempest | 17:15 |
| lbragstad | which should technically reproduce it | 17:15 |
| lbragstad | which makes me think it might be something that environmental between the two environments | 17:15 |
| samueldmq | I tried running tempest tests, but maybe it's a bit slower running that test in isolation, with all the setup, etc | 17:15 |
| samueldmq | lbragstad: jordanP has a "wip do not merge" patch, I think we should merge something, and then expect someone to break | 17:17 |
| *** roxanagh_ has joined #openstack-keystone | 17:17 | |
| samueldmq | rather than rechecking and expecting that specific patch to fail | 17:17 |
| *** josecastroleon has quit IRC | 17:19 | |
| *** tonytan4ever has quit IRC | 17:19 | |
| *** can8dnSix has quit IRC | 17:19 | |
| arunkant | stevemar, dstanek: anyone to do review on this..https://review.openstack.org/#/c/279828/ . | 17:20 |
| patchbot | arunkant: patch 279828 - keystonemiddleware - Adding audit middleware specific notification driv... | 17:20 |
| *** tpeoples has joined #openstack-keystone | 17:25 | |
| *** gordc has quit IRC | 17:27 | |
| samueldmq | lbragstad: dolphm: if I mark a patch as workflow-1, jenkins will still run jobs on it right? | 17:29 |
| *** jed56 has joined #openstack-keystone | 17:29 | |
| lbragstad | samueldmq yes - it should | 17:29 |
| dolphm | samueldmq: absolutely | 17:30 |
| dolphm | samueldmq: it won't merge though, even with another WF+1 | 17:30 |
| samueldmq | lbragstad: dolphm thanks, gonna try something | 17:30 |
| *** auggy has joined #openstack-keystone | 17:30 | |
| samueldmq | dolphm: nice | 17:30 |
| *** gordc has joined #openstack-keystone | 17:37 | |
| *** jsavak has quit IRC | 17:39 | |
| *** nikhil_ has joined #openstack-keystone | 17:39 | |
| *** nikhil_ is now known as Guest53089 | 17:40 | |
| *** mvk has joined #openstack-keystone | 17:43 | |
| *** josecastroleon has joined #openstack-keystone | 17:44 | |
| *** spzala has joined #openstack-keystone | 17:44 | |
| samueldmq | lbragstad: https://review.openstack.org/#/q/topic:token-revocation-issue | 17:45 |
| samueldmq | dolphm: ^ | 17:46 |
| samueldmq | hopefully we can catch a failing gate in one of those | 17:46 |
| *** henrynash has quit IRC | 17:46 | |
| lbragstad | samueldmq why remove the X-Auth-Token? | 17:47 |
| lbragstad | samueldmq isn't that what you wanted to know? | 17:47 |
| samueldmq | lbragstad: I am removing the part that ommited ait | 17:47 |
| samueldmq | it* | 17:47 |
| lbragstad | oh - right | 17:48 |
| lbragstad | nvm | 17:48 |
| dolphm | lbragstad: other way around | 17:48 |
| samueldmq | :) | 17:48 |
| samueldmq | we could also add some keystone logging | 17:48 |
| dolphm | samueldmq: that's awesome | 17:48 |
| *** mvk_ has joined #openstack-keystone | 17:48 | |
| samueldmq | and add a Depends_On on those patches | 17:48 |
| samueldmq | lbragstad: dolphm ^ so we could really track everything is going on when it fails | 17:49 |
| samueldmq | both tempest and keystone sides | 17:49 |
| dolphm | samueldmq: ping the tempest channel - that's a lot of reviews | 17:49 |
| samueldmq | dolphm: yes, I left a question to mtreinish there, but submitted wihtout a response, I will leave a message there | 17:50 |
| samueldmq | thanks for the heads up | 17:50 |
| *** mkrcmari__ has joined #openstack-keystone | 17:50 | |
| *** roxanagh_ has quit IRC | 17:50 | |
| *** lhcheng has quit IRC | 17:51 | |
| *** mvk has quit IRC | 17:52 | |
| samueldmq | done | 17:52 |
| *** dan_nguyen has quit IRC | 17:52 | |
| *** spzala has quit IRC | 17:52 | |
| *** ksavich has joined #openstack-keystone | 17:52 | |
| dolphm | samueldmq: instead of IGNORE, DO NOT MERGE would make it more clear who should do the ignoring. otherwise, well done lol | 17:52 |
| *** roxanagh_ has joined #openstack-keystone | 17:52 | |
| *** mvk has joined #openstack-keystone | 17:53 | |
| openstackgerrit | Merged openstack/keystone: Deprecate keystone.common.kvs https://review.openstack.org/271948 | 17:53 |
| *** mvk_ has quit IRC | 17:53 | |
| samueldmq | dolphm: does changing commit message restart jenkins jobs ? | 17:55 |
| dolphm | samueldmq: no, i think jenkins recognizes that as trivial | 17:55 |
| *** mkrcmari__ has quit IRC | 17:55 | |
| *** stingaci has joined #openstack-keystone | 17:56 | |
| samueldmq | dolphm: nice, I will change the title in the next 'recheck', so I don't cause a flood in qa again right now | 17:56 |
| samueldmq | o/ | 17:56 |
| notmorgan | uhm. iirc commit message changes do re-run check jobs | 17:57 |
| notmorgan | you don't lose (in most caseS) the review scores. | 17:57 |
| dolphm | hrm | 17:58 |
| dolphm | test it with one | 17:58 |
| *** mvk_ has joined #openstack-keystone | 17:59 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/keystone: Updated from global requirements https://review.openstack.org/316170 | 18:02 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: morgan and dolphm, let's see if this re-runs jobs https://review.openstack.org/212957 | 18:02 |
| samueldmq | it was notmorgan, actually | 18:02 |
| notmorgan | lol | 18:02 |
| *** mvk has quit IRC | 18:02 | |
| samueldmq | and yes, it does lol | 18:03 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: Create unit tests for the policy drivers https://review.openstack.org/212957 | 18:03 |
| samueldmq | fixing it ^ o/ | 18:03 |
| samueldmq | yep, I see jobs starting for the new patchsets in zuul.o.o | 18:05 |
| samueldmq | anyways | 18:05 |
| samueldmq | lbragstad: do you have a patch up with logs everywhere around revocation events + token validation | 18:05 |
| samueldmq | lbragstad: so we can post it X times and wait for one to fail :) and change 197331 did | 18:06 |
| samueldmq | as* | 18:06 |
| lbragstad | samueldmq I don't | 18:07 |
| samueldmq | lbragstad: k I will create one | 18:07 |
| lbragstad | I started working on a patch yseterday to make all datetime formats in keystone truncate microsecond precision | 18:07 |
| samueldmq | lbragstad: nice, maybe useful depending on the issue | 18:10 |
| *** josecastroleon has quit IRC | 18:11 | |
| samueldmq | lbragstad: but for now we just need to understand what the heck is going on lol | 18:11 |
| lbragstad | yeah - something we might move forward with regardless | 18:11 |
| lbragstad | yeah - exactly | 18:11 |
| *** ninag_ has quit IRC | 18:11 | |
| *** ninag has joined #openstack-keystone | 18:12 | |
| *** doug-fish has joined #openstack-keystone | 18:12 | |
| *** doug-fi__ has joined #openstack-keystone | 18:15 | |
| *** mkrcmari__ has joined #openstack-keystone | 18:15 | |
| *** tonytan4ever has joined #openstack-keystone | 18:15 | |
| *** doug-fis_ has quit IRC | 18:15 | |
| *** doug-fish has quit IRC | 18:17 | |
| *** spandhe has joined #openstack-keystone | 18:17 | |
| *** doug-fi__ has quit IRC | 18:18 | |
| *** spzala has joined #openstack-keystone | 18:18 | |
| *** mvk_ has quit IRC | 18:19 | |
| *** dan_nguyen has joined #openstack-keystone | 18:19 | |
| *** sdake has joined #openstack-keystone | 18:20 | |
| *** josecastroleon has joined #openstack-keystone | 18:20 | |
| *** spzala has quit IRC | 18:20 | |
| *** spzala has joined #openstack-keystone | 18:21 | |
| *** ninag has quit IRC | 18:21 | |
| *** doug-fish has joined #openstack-keystone | 18:23 | |
| openstackgerrit | Arun Kant proposed openstack/keystonemiddleware: Adding audit middleware specific notification driver conf https://review.openstack.org/279828 | 18:25 |
| *** sdake_ has joined #openstack-keystone | 18:26 | |
| stevemar | arunkant: nice patch, gyee: ^ feel free to kick it off | 18:26 |
| gyee | stevemar, thank you Sir! | 18:26 |
| arunkant | stevemar and gyee: thanks | 18:26 |
| *** sdake has quit IRC | 18:28 | |
| stevemar | ayoung: can you review https://review.openstack.org/#/c/311203/2 edmondsw is poking 968696 again :) | 18:29 |
| patchbot | stevemar: patch 311203 - keystone - admin gets is_admin_project by default | 18:29 |
| samueldmq | lbragstad: you still around ? | 18:31 |
| lbragstad | samueldmq yep | 18:31 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316238 | 18:31 |
| samueldmq | lbragstad: could you test ^ in your install ? | 18:31 |
| lbragstad | samueldmq yeah | 18:31 |
| lbragstad | first node is trashed, i'm rebuilding | 18:31 |
| samueldmq | lbragstad: kk | 18:33 |
| lbragstad | samueldmq thoughts? https://review.openstack.org/#/c/316238/1 | 18:37 |
| patchbot | lbragstad: patch 316238 - keystone - DO NOT MERGE: LOG revocation mechanism | 18:37 |
| *** mkrcmari__ has quit IRC | 18:39 | |
| *** ninag has joined #openstack-keystone | 18:39 | |
| *** mkrcmari__ has joined #openstack-keystone | 18:39 | |
| *** ninag has quit IRC | 18:42 | |
| *** ayoung has quit IRC | 18:42 | |
| *** ninag has joined #openstack-keystone | 18:43 | |
| bknudson | why can't we merge logging fixes? | 18:43 |
| *** ninag has quit IRC | 18:44 | |
| *** roxanagh_ has quit IRC | 18:44 | |
| *** ninag has joined #openstack-keystone | 18:44 | |
| *** TxGVNN has quit IRC | 18:45 | |
| *** ninag has quit IRC | 18:47 | |
| *** roxanagh_ has joined #openstack-keystone | 18:48 | |
| stevemar | this change really doesn't want to merge: https://review.openstack.org/#/c/255686/8 | 18:49 |
| patchbot | stevemar: patch 255686 - keystone - Make AuthContext depend on auth_token middleware | 18:49 |
| samueldmq | lbragstad: why not, will add that too | 18:49 |
| *** josecastroleon has quit IRC | 18:50 | |
| samueldmq | bknudson: we can, but not sure if doing at that level wouldn't be too verbose | 18:50 |
| samueldmq | lbragstad: were you able to run that patch ? or not yet? | 18:51 |
| bknudson | if it's needed to fix this problem then it's not too verbose. | 18:51 |
| *** doug-fish has quit IRC | 18:51 | |
| lbragstad | samueldmq not yet - still reproducing the environment for devstack | 18:51 |
| *** doug-fish has joined #openstack-keystone | 18:52 | |
| *** mhickey has joined #openstack-keystone | 18:52 | |
| *** sdake_ has quit IRC | 18:52 | |
| *** doug-fish has quit IRC | 18:53 | |
| *** sdake has joined #openstack-keystone | 18:53 | |
| *** sdake has quit IRC | 18:53 | |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316238 | 18:54 |
| samueldmq | lbragstad: ^ updated, I also added depends-on that tempest chnge, so we can get both logs | 18:54 |
| lbragstad | samueldmq sweet - i'll pull that one | 18:54 |
| *** doug-fish has joined #openstack-keystone | 18:54 | |
| *** ninag_ has joined #openstack-keystone | 18:55 | |
| *** doug-fish has quit IRC | 18:59 | |
| samueldmq | lbragstad: what log do you see those info ? | 19:00 |
| samueldmq | lbragstad: I am looking at /var/log/apache2/keystone.log in my devstack | 19:00 |
| lbragstad | samueldmq https://review.openstack.org/#/c/316238/2 | 19:00 |
| patchbot | lbragstad: patch 316238 - keystone - DO NOT MERGE: LOG revocation mechanism | 19:00 |
| *** doug-fish has joined #openstack-keystone | 19:02 | |
| lbragstad | samueldmq i got a syntax error with the patch | 19:03 |
| lbragstad | samueldmq here is my diff | 19:03 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316238 | 19:03 |
| samueldmq | lbragstad: ^ | 19:03 |
| *** mvk_ has joined #openstack-keystone | 19:03 | |
| lbragstad | samueldmq http://cdn.pasteraw.com/httod9dtcszzf7gole5ava8053myb4a | 19:03 |
| samueldmq | lbragstad: nice, I just fixed it, and used %r everywhere | 19:03 |
| samueldmq | :) | 19:03 |
| lbragstad | samueldmq awesome, thanks! | 19:04 |
| samueldmq | lbragstad: could you try http://paste.openstack.org/show/497121/ locally? | 19:05 |
| samueldmq | lbragstad: tokenv2-request.json is like http://paste.openstack.org/show/497122/ | 19:05 |
| samueldmq | this reproduces the behavior of at test that failed | 19:06 |
| lbragstad | samueldmq was that with devstack? | 19:06 |
| *** mkrcmari__ has quit IRC | 19:06 | |
| *** roxanagh_ has quit IRC | 19:06 | |
| *** daemontool_ has quit IRC | 19:06 | |
| samueldmq | lbragstad: yes, but I changed the passwords to be admin everywhere | 19:07 |
| samueldmq | lbragstad: you just need to change what's in tokenv2-request.json | 19:07 |
| samueldmq | to match your user's creds | 19:07 |
| lbragstad | ok | 19:07 |
| lbragstad | samueldmq what's jq? | 19:09 |
| samueldmq | ah, it's used to parse json :) | 19:10 |
| samueldmq | just sudo apt-get install jq | 19:10 |
| *** roxanagh_ has joined #openstack-keystone | 19:11 | |
| lbragstad | samueldmq can you get it to happen consistently with that script? | 19:11 |
| samueldmq | yep | 19:11 |
| lbragstad | samueldmq http://cdn.pasteraw.com/mvrlv5r6nofj68l2uw82idrfnn4mbtd | 19:12 |
| samueldmq | lbragstad: that's consistent, isn't it ? | 19:12 |
| *** dan_nguyen has quit IRC | 19:12 | |
| lbragstad | samueldmq but that's the expected behavior right? | 19:13 |
| lbragstad | get a token, request the roles, revoked the token, request roles and get a 401 ? | 19:13 |
| lbragstad | that's what *should* happen | 19:13 |
| samueldmq | lbragstad: http://paste.openstack.org/show/497123/ | 19:13 |
| samueldmq | lbragstad: exactly, that's what should happen all the time :) | 19:14 |
| lbragstad | oh - I thought you meant you were able to recreate what tempest was seeing | 19:15 |
| lbragstad | where the last 401 is actually a valid set of roles | 19:15 |
| *** stingaci has quit IRC | 19:15 | |
| samueldmq | lbragstad: no, not that easy :p | 19:15 |
| samueldmq | lbragstad: just wanted you to run that and see the logs | 19:15 |
| lbragstad | unfortunately! | 19:15 |
| samueldmq | lbragstad: to see if we're logging all the info we will need | 19:16 |
| samueldmq | to debug | 19:16 |
| samueldmq | :) | 19:16 |
| *** pnavarro has quit IRC | 19:17 | |
| *** yarkot has joined #openstack-keystone | 19:17 | |
| lbragstad | samueldmq here | 19:18 |
| lbragstad | is a snippet from the keystone logs http://cdn.pasteraw.com/8uxeevbgpw8kgqo8ud2epdzcc8ibcbm | 19:18 |
| lbragstad | samueldmq with your patch | 19:18 |
| *** phalmos_ has quit IRC | 19:19 | |
| samueldmq | lbragstad: have you applied the latest patchst ? | 19:20 |
| samueldmq | lbragstad: it's weird I don't see any logging from the sql driver | 19:20 |
| lbragstad | yeah - i think so | 19:20 |
| samueldmq | 'Persisting revocation event: ...' | 19:20 |
| *** roxanagh_ has quit IRC | 19:21 | |
| lbragstad | 2016-05-13 19:16:33.805 2957 DEBUG keystone.revoke.backends.sql [req-a0060f29-3b25-4259-9841-1a47ba975b65 89b214449d25408eb24a3ef9cc59a0e4 c649855d3c85494ab5fea2bec4cf64f4 - default default] Persisting revocation event: {'issued_before' | 19:21 |
| lbragstad | : '2016-05-13T19:16:33.805141Z', 'audit_id': u'YXmiQxZwS4qYIz4eZGyM1Q'} revoke /opt/stack/keystone/keystone/revoke/backends/sql.py:99 | 19:21 |
| samueldmq | lbragstad: ok, let me post that patch several times :B | 19:24 |
| samueldmq | and let's wait, and discover what's going on | 19:24 |
| samueldmq | o/ | 19:25 |
| lbragstad | samueldmq sounds good | 19:25 |
| openstackgerrit | Merged openstack/keystone-specs: Add note about service provider fields https://review.openstack.org/303471 | 19:25 |
| *** ayoung has joined #openstack-keystone | 19:30 | |
| *** ChanServ sets mode: +v ayoung | 19:30 | |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316248 | 19:33 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316250 | 19:34 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316251 | 19:35 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316253 | 19:35 |
| *** stingaci has joined #openstack-keystone | 19:36 | |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316254 | 19:36 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316255 | 19:36 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316256 | 19:37 |
| samueldmq | there we go! | 19:37 |
| lbragstad | samueldmq hopefully we get something | 19:43 |
| *** tonytan4ever has quit IRC | 19:43 | |
| samueldmq | lbragstad: in one of those for tempest, I can see at least one failed in gate-tempest-dsvm-neutron-full | 19:44 |
| samueldmq | lbragstad: but I don't have access to the logs yet :( | 19:44 |
| samueldmq | lbragstad: look 316227 in zuul.o.o | 19:44 |
| lbragstad | https://review.openstack.org/#/c/316227/ | 19:44 |
| patchbot | lbragstad: patch 316227 - tempest - IGNORE: Log X-Auth-Token | 19:44 |
| samueldmq | lbragstad: it doesn't show in the patch yet, you need to go to zuul.openstack.org and use 316227 as a filter | 19:45 |
| lbragstad | samueldmq so - you think this might somehow be related to how tempest clients get tokens? | 19:46 |
| *** neophy has joined #openstack-keystone | 19:47 | |
| samueldmq | lbragstad: maybe, we will see :) | 19:47 |
| samueldmq | lbragstad: if not, our keystone changes will help on understanding what we're doing wrong | 19:48 |
| openstackgerrit | Merged openstack/keystone-specs: keystone-manage doctor https://review.openstack.org/310309 | 19:55 |
| *** roxanagh_ has joined #openstack-keystone | 19:58 | |
| samueldmq | lbragstad: I have something from tempest run | 20:00 |
| lbragstad | samueldmq ? | 20:00 |
| samueldmq | lbragstad: and looks like it isn't their fault | 20:00 |
| lbragstad | do you have the logs? | 20:00 |
| samueldmq | lbragstad: http://logs.openstack.org/27/316227/1/check/gate-tempest-dsvm-neutron-full/fef7c51/console.html#_2016-05-13_19_40_18_996 | 20:00 |
| *** dan_nguyen has joined #openstack-keystone | 20:00 | |
| *** tonytan4ever has joined #openstack-keystone | 20:01 | |
| samueldmq | lbragstad: look for gAAAAABXNiVwu_S6NLq3H2ZiJkh7P17eymY_rDnyQtxDU6bUrRBlauTukXg32kHfcUQnmXM7CbgZKl7bnD3-4OgH_S9uZ4GzjycM4cE6wlx3GrIc0tdrRricGrGh8wwMys6G-pk6J7i3R9JTQsndB8miFrGa69rlAVzjdhiJewK03U67PmV_tJ4 | 20:01 |
| samueldmq | oops sorry didn't need to be the full token | 20:01 |
| samueldmq | lbragstad: specially here http://logs.openstack.org/27/316227/1/check/gate-tempest-dsvm-neutron-full/fef7c51/console.html#_2016-05-13_19_40_19_039 | 20:01 |
| samueldmq | lbragstad: when it does the DELETE token request | 20:02 |
| samueldmq | lbragstad: then it gets 204 from keystone | 20:02 |
| samueldmq | lbragstad: and just after tried to delete a token from a user in a given project (with the revoked token passed in request headers) | 20:02 |
| samueldmq | lbragstad: and it suceeds | 20:02 |
| samueldmq | with 204 | 20:02 |
| *** dmk0202 has joined #openstack-keystone | 20:03 | |
| lbragstad | so that was this guy - specifically | 20:07 |
| lbragstad | https://github.com/openstack/tempest/blob/master/tempest/api/identity/admin/v2/test_roles_negative.py#L183-L196 | 20:07 |
| samueldmq | lbragstad: yes, L192 revokes the token that is used in L193 | 20:08 |
| samueldmq | lbragstad: and is being accepted by the server | 20:08 |
| *** med_ has quit IRC | 20:09 | |
| *** roxanagh_ has quit IRC | 20:09 | |
| samueldmq | lbragstad: now we need to wait for keystone patches to have their jobs finished | 20:10 |
| samueldmq | lbragstad: and hopefully catch something as well | 20:10 |
| lbragstad | ok - so we know tempest is using the right token | 20:11 |
| samueldmq | lbragstad: yep | 20:11 |
| lbragstad | which makes sense because if something was wrong there it would probably be a lot more frequent | 20:11 |
| lbragstad | s/it/the failures/ | 20:11 |
| samueldmq | lbragstad: yep, that makes sense, but we needed to make this sure :) | 20:12 |
| *** fangxu has joined #openstack-keystone | 20:12 | |
| samueldmq | then move a step forward in the investigation | 20:12 |
| *** raildo is now known as raildo-afk | 20:13 | |
| *** spandhe has quit IRC | 20:14 | |
| *** roxanagh_ has joined #openstack-keystone | 20:17 | |
| *** edmondsw has quit IRC | 20:18 | |
| *** dmk0202 has quit IRC | 20:22 | |
| *** mvk_ has quit IRC | 20:22 | |
| *** timcline has quit IRC | 20:23 | |
| *** timcline has joined #openstack-keystone | 20:25 | |
| *** wrightspace has joined #openstack-keystone | 20:27 | |
| *** roxanagh_ has quit IRC | 20:27 | |
| *** roxanagh_ has joined #openstack-keystone | 20:34 | |
| *** sheel has quit IRC | 20:35 | |
| *** ametts has quit IRC | 20:36 | |
| lbragstad | samueldmq it's not looking good for the home team | 20:41 |
| lbragstad | most of the keystone patches with revocation logging are passing | 20:41 |
| *** roxanagh_ has quit IRC | 20:42 | |
| *** roxanagh_ has joined #openstack-keystone | 20:45 | |
| lbragstad | samueldmq ooo! https://review.openstack.org/#/c/316253/ | 20:45 |
| patchbot | lbragstad: patch 316253 - keystone - DO NOT MERGE: LOG revocation mechanism | 20:45 |
| samueldmq | lbragstad: yes! | 20:45 |
| samueldmq | lbragstad: let's look into it, let me find the logs | 20:46 |
| lbragstad | I don't think I'll ever be this happy about a gate failure again | 20:46 |
| lupine | YOU CAN'T TELL ME WHAT TO DO | 20:46 |
| * lupine merges | 20:46 | |
| *** daemontool has joined #openstack-keystone | 20:46 | |
| samueldmq | lbragstad: hahahahah same here o/ | 20:47 |
| lbragstad | samueldmq i can't get to the logs yet | 20:47 |
| *** gordc has quit IRC | 20:47 | |
| samueldmq | lbragstad: http://logs.openstack.org/53/316253/1/check/gate-tempest-dsvm-postgres-full/c9a6d63/console.html | 20:48 |
| samueldmq | lbragstad: we can't get to them through zuul anymore | 20:48 |
| samueldmq | lbragstad: you need to go to http://logs.openstack.org and the pattern is (2 last digits of change)/(full change) | 20:49 |
| samueldmq | eg. http://logs.openstack.org/53/316253 | 20:49 |
| samueldmq | :) | 20:49 |
| *** wrightspace has quit IRC | 20:49 | |
| lbragstad | samueldmq https://github.com/openstack/tempest/blob/master/tempest/api/identity/admin/v3/test_tokens.py#L47 | 20:49 |
| lbragstad | that is what's failing this time | 20:50 |
| samueldmq | lbragstad: nice, test is very clear about what it is dong | 20:50 |
| samueldmq | doing | 20:50 |
| lbragstad | yeah | 20:50 |
| *** neophy has quit IRC | 20:50 | |
| samueldmq | lbragstad: I can see openstack request id req-6cbc3296-612b-4963-84fa-db2240e13ff8 | 20:51 |
| samueldmq | lbragstad: in the main log, let's look for something in the keystone logs | 20:51 |
| samueldmq | lbragstad: http://logs.openstack.org/53/316253/1/check/gate-tempest-dsvm-postgres-full/c9a6d63/logs/apache/keystone.txt.gz | 20:52 |
| *** daemontool has quit IRC | 20:52 | |
| samueldmq | lbragstad: log is too big, still loading | 20:54 |
| lbragstad | samueldmq use wget and parse it locally | 20:54 |
| samueldmq | lbragstad: 'adding revocation event' logged TOOOO MANY times | 20:54 |
| samueldmq | lbragstad: hey man you're smart :D | 20:54 |
| lbragstad | samueldmq i've been riding the struggle bus all week | 20:55 |
| *** doug-fis_ has joined #openstack-keystone | 20:55 | |
| *** doug-fis_ has quit IRC | 20:55 | |
| samueldmq | lbragstad: ahahha | 20:55 |
| samueldmq | lbragstad: 50mb and downloading | 20:55 |
| samueldmq | wow | 20:56 |
| samueldmq | 120mb | 20:56 |
| *** mhickey has quit IRC | 20:57 | |
| *** doug-fish has quit IRC | 20:58 | |
| samueldmq | lbragstad: there shouldn't be TOO many revocation events | 20:59 |
| samueldmq | lbragstad: mybe something is wrong | 20:59 |
| *** slberger1 has joined #openstack-keystone | 21:02 | |
| *** neophy has joined #openstack-keystone | 21:02 | |
| *** slberger has quit IRC | 21:03 | |
| lbragstad | samueldmq so here https://github.com/openstack/tempest/blob/master/tempest/api/identity/admin/v3/test_tokens.py#L41 | 21:05 |
| lbragstad | that's validating the user's fresh token | 21:05 |
| lbragstad | and I think that is logged by your revocation stuff at line 310074 in the logs | 21:05 |
| samueldmq | lbragstad: why do you think it is in that line ? | 21:06 |
| lbragstad | that token's issued_at time 'issued_at': datetime.datetime(2016, 5, 13, 20, 22, 57) | 21:06 |
| lbragstad | samueldmq just walking through the test | 21:06 |
| samueldmq | lbragstad: I am still trying to find myself in all those 'adding revocatio nevent' | 21:06 |
| *** raddaoui has quit IRC | 21:07 | |
| *** iurygregory has quit IRC | 21:09 | |
| lbragstad | samueldmq at line 310525 the token in the test is logged in the revocation backend. | 21:11 |
| lbragstad | Persisting revocation event: {'issued_before': '2016-05-13T20:22:57.765732Z', 'audit_id': u'1cQdglCCS4qZhEY359h4cw'} | 21:11 |
| samueldmq | lbragstad: L310538 it does the get token | 21:13 |
| samueldmq | lbragstad: that shouldn't be allowed | 21:13 |
| lbragstad | the token was issued at 'issued_at': datetime.datetime(2016, 5, 13, 20, 22, 57) | 21:15 |
| *** ninag_ has quit IRC | 21:15 | |
| *** ninag has joined #openstack-keystone | 21:16 | |
| samueldmq | lbragstad: agreed | 21:16 |
| lbragstad | samueldmq so line 310541 is the line that should have invalidate that token | 21:17 |
| samueldmq | lbragstad: if you look for datetime.datetime(2016, 5, 13, 20, 22, 57) | 21:18 |
| samueldmq | lbragstad: you will find many "Checking token against revocation tree" | 21:18 |
| lbragstad | yeah | 21:18 |
| samueldmq | lbragstad: but there is NO "Comparing token against revoke map" | 21:18 |
| lbragstad | i find that weird too | 21:19 |
| samueldmq | lbragstad: so it's why it's not being revoked, it isn't actually compared with data to revoke it | 21:19 |
| lbragstad | hmm | 21:19 |
| lbragstad | so how would that be a race condition though? | 21:19 |
| samueldmq | no idea | 21:20 |
| *** ninag has quit IRC | 21:20 | |
| *** tonytan4ever has quit IRC | 21:21 | |
| lbragstad | samueldmq ah | 21:21 |
| lbragstad | samueldmq that's because LOG.debug('Comparing token against revoke map') is only logged when not names | 21:22 |
| samueldmq | lbragstad: but that should be the case at some point ? shouldn't it ? the call to that function is recursive | 21:22 |
| lbragstad | so if something is matched before that check we don't log that bit | 21:22 |
| samueldmq | lbragstad: hmm | 21:22 |
| lbragstad | samueldmq I guess we only log it if we are comparing dates (issued_before, issued_at) | 21:23 |
| samueldmq | lbragstad: I should be logging that outside that if | 21:24 |
| samueldmq | urrrgh | 21:24 |
| samueldmq | lbragstad: but why it didn't get to compare the dates ? | 21:25 |
| *** ninag has joined #openstack-keystone | 21:25 | |
| samueldmq | lbragstad: why did it consider the token valid before reaching that code, | 21:25 |
| lbragstad | it might have matched on the audit_id | 21:25 |
| samueldmq | ? | 21:25 |
| samueldmq | lbragstad: matching the audit_id would make it valid ? | 21:25 |
| lbragstad | samueldmq i'm not entirely sure but we can revoke by audit id | 21:26 |
| samueldmq | regardless the revocation event' 'issued_before' | 21:26 |
| lbragstad | and that's what we have to do when we revoke a fernet token | 21:26 |
| *** markvoelker has quit IRC | 21:26 | |
| samueldmq | lbragstad: but it's exactly the opposite, it's not being revoked at all | 21:26 |
| lbragstad | right | 21:26 |
| samueldmq | lbragstad: without reaching the end of recursion | 21:26 |
| lbragstad | which is strange... because if that's true it's a race condition that doesn't have anything to do with time (that we know of)? | 21:26 |
| *** ninag_ has joined #openstack-keystone | 21:27 | |
| *** stingaci has quit IRC | 21:27 | |
| *** roxanagh_ has quit IRC | 21:27 | |
| *** ninag has quit IRC | 21:30 | |
| *** ninag_ has quit IRC | 21:32 | |
| *** roxanagh_ has joined #openstack-keystone | 21:32 | |
| samueldmq | lbragstad: we need more data | 21:33 |
| samueldmq | lbragstad: we need to see what is happening inside each call to _search | 21:34 |
| lbragstad | ++ | 21:34 |
| samueldmq | lbragstad: so we will be able to understand why it's stopping at some point and thinking the token is valid | 21:34 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316238 | 21:37 |
| samueldmq | lbragstad: ^ | 21:37 |
| samueldmq | lbragstad: let's think about any other info we could need | 21:38 |
| lbragstad | samueldmq that will log tons of stuff but that's fine | 21:38 |
| lbragstad | that's what we need | 21:38 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316248 | 21:40 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone: Avoid datetime rounding issues https://review.openstack.org/315735 | 21:40 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316250 | 21:40 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316251 | 21:41 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316253 | 21:41 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316254 | 21:41 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316255 | 21:42 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316256 | 21:42 |
| samueldmq | lbragstad: okay, let's hope it's gonna fail again | 21:42 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316238 | 21:42 |
| lbragstad | samueldmq ^ | 21:42 |
| * samueldmq nods | 21:43 | |
| * samueldmq goes to update all them again | 21:43 | |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316248 | 21:44 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316250 | 21:45 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316251 | 21:45 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316253 | 21:46 |
| *** ninag has joined #openstack-keystone | 21:46 | |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316254 | 21:46 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316255 | 21:47 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316256 | 21:47 |
| samueldmq | lbragstad: how long did the last one take ? | 21:47 |
| lbragstad | 45 minutes? | 21:48 |
| samueldmq | lbragstad: hmm, looking at 316238 the results took 1:20 h to get posted in the cahnge | 21:49 |
| samueldmq | :/ | 21:49 |
| samueldmq | let's wait and see | 21:49 |
| *** roxanagh_ has quit IRC | 21:50 | |
| *** ninag has quit IRC | 21:50 | |
| *** dmk0202 has joined #openstack-keystone | 21:53 | |
| *** roxanagh_ has joined #openstack-keystone | 21:53 | |
| *** henrynash has joined #openstack-keystone | 21:55 | |
| *** ChanServ sets mode: +v henrynash | 21:55 | |
| *** woodster_ has quit IRC | 21:58 | |
| *** daemontool has joined #openstack-keystone | 21:59 | |
| *** slberger1 has left #openstack-keystone | 22:01 | |
| *** dmk0202 has quit IRC | 22:06 | |
| *** edtubill has quit IRC | 22:15 | |
| *** markvoelker has joined #openstack-keystone | 22:27 | |
| *** sheel has joined #openstack-keystone | 22:29 | |
| *** alex_xu has quit IRC | 22:30 | |
| *** daemontool has quit IRC | 22:31 | |
| *** markvoelker has quit IRC | 22:31 | |
| *** timcline has quit IRC | 22:33 | |
| *** alex_xu has joined #openstack-keystone | 22:33 | |
| *** dan_nguyen has quit IRC | 22:39 | |
| samueldmq | lbragstad: most of them are passing this time :( | 22:39 |
| lbragstad | samueldmq looks like https://review.openstack.org/#/c/316238/ is going to fail | 22:40 |
| patchbot | lbragstad: patch 316238 - keystone - DO NOT MERGE: LOG revocation mechanism | 22:40 |
| samueldmq | lbragstad: it's tking so long in gate-tempest-dsvm-full | 22:40 |
| * samueldmq thinks lbragstad owns a 60 inches monitor, and can actually see all of them in real time | 22:41 | |
| lbragstad | ha - i wish | 22:41 |
| openstackgerrit | Merged openstack/keystonemiddleware: Adding audit middleware specific notification driver conf https://review.openstack.org/279828 | 22:41 |
| *** neophy has quit IRC | 22:41 | |
| openstackgerrit | guang-yee proposed openstack/keystoneauth: Support TOTP auth plugin https://review.openstack.org/281086 | 22:41 |
| notmorgan | samueldmq: but what if i want to merge it!? | 22:42 |
| notmorgan | samueldmq: fwiw, you should stick a -2 on that if you really don't want it to merge and/or WIP it. | 22:42 |
| notmorgan | samueldmq: just as a safety thing... -2 prevents merge for sure. | 22:42 |
| *** agrebennikov has quit IRC | 22:42 | |
| samueldmq | notmorgan: ok, these are going to be my first -2's | 22:43 |
| notmorgan | lol | 22:44 |
| lbragstad | me too | 22:44 |
| notmorgan | need to issue -2s at some point! | 22:44 |
| notmorgan | might as well be froe your own patches. | 22:44 |
| samueldmq | even though I don't expect a core to merge something with DO NOT MERGE!!!! | 22:44 |
| lbragstad | i haven't issued a -2 yet | 22:44 |
| samueldmq | cores are expect to read things before merging | 22:44 |
| samueldmq | hehe | 22:44 |
| jamielennox | read? all the things? | 22:45 |
| jamielennox | ughh | 22:45 |
| notmorgan | samueldmq: i fon't either but it falls into the category of "i'd -2 it if it was my patch" | 22:45 |
| notmorgan | not that you actually need to | 22:45 |
| notmorgan | jamielennox: yah i know right?! i just +2/+A everything | 22:45 |
| jamielennox | notmorgan: i have a badge and everything | 22:45 |
| lbragstad | filgtm | 22:45 |
| notmorgan | jamielennox: ++ | 22:45 |
| samueldmq | jamielennox: at lest the commit messages lol | 22:46 |
| notmorgan | lbragstad: my motto | 22:46 |
| samueldmq | notmorgan: you can't anymore, not these | 22:46 |
| * notmorgan doesn't have the pin anymore :( | 22:46 | |
| notmorgan | lost... | 22:46 |
| lbragstad | samueldmq logs are up http://logs.openstack.org/38/316238/5/check/gate-tempest-dsvm-neutron-full/c49c786/ | 22:46 |
| samueldmq | lbragstad: which change ? 316238 passed :/ | 22:47 |
| lbragstad | 316238,5 | 22:48 |
| lbragstad | failed gate-tempest-dsvm-neutron-full | 22:48 |
| lbragstad | ah damn | 22:49 |
| lbragstad | it failed because of something else | 22:49 |
| *** catintheroof has quit IRC | 22:49 | |
| samueldmq | lbragstad: yes, all of them passed | 22:49 |
| samueldmq | if I wished them all to pass, they'd fail | 22:50 |
| samueldmq | for sure | 22:50 |
| lbragstad | ah | 22:50 |
| samueldmq | lbragstad: I have another approach | 22:50 |
| samueldmq | lbragstad: tempest patches failed 6 out of 8 | 22:50 |
| samueldmq | lbragstad: I will remove the depends-on form keystone patches, and put on tempest ones | 22:50 |
| samueldmq | lbragstad: makes sense? | 22:50 |
| lbragstad | yeah - that works | 22:51 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316238 | 22:51 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316248 | 22:52 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316250 | 22:52 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316251 | 22:52 |
| *** spzala has quit IRC | 22:52 | |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316253 | 22:53 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316254 | 22:53 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316255 | 22:53 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack/keystone: DO NOT MERGE: LOG revocation mechanism https://review.openstack.org/316256 | 22:53 |
| lbragstad | is anyone here really familiar with the _next_level_keys() method? | 22:58 |
| *** woodster_ has joined #openstack-keystone | 22:58 | |
| samueldmq | lbragstad: yeah we will need to understand all that revocation logic | 23:01 |
| samueldmq | in order to be able to debug | 23:01 |
| lbragstad | samueldmq i'm not sure i understand the logic in the _next_level_keys() method... for one, it returns values not keys i think | 23:01 |
| lbragstad | well - it *yields* values not keys | 23:03 |
| *** stingaci has joined #openstack-keystone | 23:04 | |
| *** stingaci has quit IRC | 23:04 | |
| *** stingaci has joined #openstack-keystone | 23:04 | |
| samueldmq | lbragstad: looks like it returns the corresponding value inside token data? | 23:04 |
| samueldmq | lbragstad: "Generate keys based on current field name and token data." | 23:05 |
| lbragstad | this is weird | 23:06 |
| lbragstad | we get into _search() right | 23:07 |
| lbragstad | and we have a list of things to check for | 23:07 |
| samueldmq | lbragstad: yeah, it should be written in a way we could understand :p | 23:07 |
| lbragstad | we call _next_level_keys() | 23:07 |
| lbragstad | which will return a generator that's evoked by the for loop | 23:07 |
| samueldmq | lbragstad: that's right | 23:08 |
| samueldmq | lbragstad: I don't even know how the tree is structured :/ | 23:08 |
| lbragstad | the first yield is just '*' | 23:08 |
| lbragstad | which i'm assuming is a "match all the things" thing | 23:08 |
| lbragstad | because it would be yielded as key (which is actually a value) | 23:09 |
| lbragstad | to me, a key in this sense would be something like user_id | 23:09 |
| lbragstad | and uuid.uuid4().hex would be the value | 23:09 |
| lbragstad | but w/e | 23:09 |
| samueldmq | lbragstad: hmm, I think I understand something now | 23:09 |
| lbragstad | anywho | 23:09 |
| lbragstad | we get into the for loop | 23:09 |
| samueldmq | lbragstad: "Each node is a hashtable of key=value combinations from revocation events." | 23:09 |
| samueldmq | lbragstad: the for loop iterator trhough key=value available from the token | 23:10 |
| *** stingaci_ has joined #openstack-keystone | 23:10 | |
| samueldmq | lbragstad: and see if that exists in the tree (if exists, should be revoked) | 23:10 |
| lbragstad | but _search is recursive | 23:11 |
| samueldmq | token with user=X arrives, then it will look recursively in the tree for user_id=X | 23:12 |
| *** stingac__ has joined #openstack-keystone | 23:12 | |
| *** stingaci has quit IRC | 23:13 | |
| samueldmq | lbragstad: I just don't quite get how the tree is structured | 23:13 |
| lbragstad | hopefully the logging shows it | 23:14 |
| *** stingaci_ has quit IRC | 23:15 | |
| samueldmq | lbragstad: ++ let's just wait for the logs | 23:16 |
| samueldmq | lbragstad: man, I can't wait to find out what the heck is going on | 23:16 |
| lbragstad | yeah | 23:16 |
| lbragstad | this tree is terrible | 23:17 |
| samueldmq | lbragstad: (and I've only been debugging this today VS your whole week lol) | 23:17 |
| lbragstad | revocation is already hard | 23:17 |
| samueldmq | lbragstad: and that tree makes things harder | 23:17 |
| *** roxanagh_ has quit IRC | 23:17 | |
| *** roxanagh_ has joined #openstack-keystone | 23:18 | |
| *** ekarlso has quit IRC | 23:23 | |
| *** dan_nguyen has joined #openstack-keystone | 23:24 | |
| *** BjoernT has quit IRC | 23:27 | |
| *** markvoelker has joined #openstack-keystone | 23:28 | |
| *** roxanagh_ has quit IRC | 23:28 | |
| *** roxanagh_ has joined #openstack-keystone | 23:30 | |
| *** stingac__ has quit IRC | 23:32 | |
| *** markvoelker has quit IRC | 23:32 | |
| *** roxanagh_ has quit IRC | 23:32 | |
| *** ekarlso has joined #openstack-keystone | 23:32 | |
| *** dan_nguyen has quit IRC | 23:36 | |
| samueldmq | lbragstad: http://logs.openstack.org/56/316256/4/check/gate-tempest-dsvm-postgres-full/6956621/console.html#_2016-05-13_23_39_15_618 | 23:44 |
| samueldmq | !!! | 23:44 |
| openstack | samueldmq: Error: "!!" is not a valid command. | 23:44 |
| samueldmq | lol | 23:44 |
| samueldmq | 773 MB log file : http://logs.openstack.org/56/316256/4/check/gate-tempest-dsvm-postgres-full/6956621/logs/apache/keystone.txt.gz | 23:48 |
| samueldmq | not bad | 23:48 |
| *** gyee has quit IRC | 23:49 | |
| *** dan_nguyen has joined #openstack-keystone | 23:53 | |
| lbragstad | samueldmq that's huge.. | 23:55 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
