Thursday, 2016-06-09

*** raddaoui has quit IRC00:07
*** dan_nguyen has quit IRC00:16
*** r-daneel has quit IRC00:19
*** roxanaghe has quit IRC00:22
*** adrian_otto has quit IRC00:26
*** pgbridge has joined #openstack-keystone00:29
*** sdake has quit IRC00:42
notmorganrderose: it wasn't about upgrading without testing/changing, it was about committing to a contract for the driver interface so it was easier to see what was changed.00:44
notmorganrderose: i've had many operators complain about the volume of change in the driver interfaces between releases. this was an effort to avoid that complaint.00:44
notmorganrderose: especially when the operator was chasing closer to master (like some do)00:45
rderosenotmorgan: I see.  But we can show what changed through release notes, documentation...00:45
notmorganrderose: not really sufficient with the volume to change we've done in the past00:46
*** pgbridge has quit IRC00:46
notmorganrderose: it turned into days-weeks of effort every time just to get the interfaces "working" let alone acceptance testing00:46
notmorganrderose: but i don't think many deployers are custom drivering it anymore.00:47
rderosenotmorgan: ah, I see the reasoning then. hopefully, there is not still a need.  thanks for clarifying.00:47
*** rderose has quit IRC00:53
*** browne has quit IRC01:15
openstackgerritMerged openstack/keystone: Updated from global requirements  https://review.openstack.org/32736801:23
*** roxanaghe has joined #openstack-keystone01:23
*** roxanaghe has quit IRC01:28
samueldmqjamielennox: hi01:48
jamielennoxsamueldmq: howdy01:48
samueldmqjamielennox: regarding patch 30903801:48
patchbotsamueldmq: https://review.openstack.org/#/c/309038/ - keystone - Added X-Forwarding-For support.01:48
samueldmqjamielennox: looks like there is already the support for X-FORWARDED-FOR in https://github.com/openstack/oslo.middleware/blob/master/oslo_middleware/http_proxy_to_wsgi.py01:48
jamielennoxsamueldmq: not for _FOR01:49
samueldmqjamielennox: I can see it treating the HTTP_FORWARDED header.. or is that a different thing ?01:49
samueldmqjamielennox: kk they're different things01:49
jamielennoxsamueldmq: umm, sec01:49
samueldmqjamielennox: sure01:50
jamielennoxhttps://en.wikipedia.org/wiki/List_of_HTTP_header_fields#Common_non-standard_request_fields01:50
jamielennoxso it looks like the standards are X-Forwarded-[Host|Proto|For]01:50
jamielennoxbut the whole thing has been replaced by rfc7239 which uses Forwarded:01:51
jamielennoxso the oslo.middleware is looking for Forwarded first and if that's not found falling back to the X-Forwarded headers01:52
samueldmqjamielennox: so X-Forwarded-For treats the client original IP address01:52
jamielennoxyep, which is the initial bug gyee filed01:53
jamielennoxbug 155012701:53
openstackbug 1550127 in OpenStack Identity (keystone) "Wrong IP Address for error message in keystone.log" [Low,In progress] https://launchpad.net/bugs/1550127 - Assigned to Guang Yee (guang-yee)01:53
samueldmqjamielennox: I was confused because of X-Forwarded-Host (which seems to be the load balancer public address01:53
jamielennoxsamueldmq: yea, that's why i found the wikipedia entry as it's the only place i've seen that explains all the different headers01:53
samueldmqjamielennox: nice, it already Proto and Host, just needs someone to implement For01:54
samueldmqjamielennox: ++01:54
jamielennoxsamueldmq: i think the middleware is really good - it's just a shame it defaults to disabled01:54
jamielennoxsamueldmq: i really don't understand that decision, it's middleware if you wanted to disable it you'd remove it from the pipeline01:54
samueldmqjamielennox: yes that's a good thing, I didn't even know about it before last week :(01:54
jamielennoxand for everyone else it's going to make no difference01:55
samueldmqjamielennox: agreed, no change on behavior for those who simply don't use it01:55
lbragstaddolphm http://cdn.pasteraw.com/iuuf0wmct68xr0utm4o8e9kkwf72wd8 should check https://review.openstack.org/#/c/265455/1 out from gerrit - right?02:00
patchbotlbragstad: patch 265455 - keystone - Expose bug with fernet and trusts02:00
lbragstaddolphm or am i just missing something?02:00
*** dan_nguyen has joined #openstack-keystone02:09
*** ayoung has quit IRC02:17
*** lhcheng has joined #openstack-keystone02:19
*** ChanServ sets mode: +v lhcheng02:19
*** alex_xu has quit IRC02:21
*** alex_xu has joined #openstack-keystone02:23
openstackgerritayoung proposed openstack/keystone: IAM Models  https://review.openstack.org/18465102:23
*** TxGVNN has joined #openstack-keystone02:24
*** ayoung has joined #openstack-keystone02:25
*** ChanServ sets mode: +v ayoung02:25
stevemarnotmorgan: dolphm stable patch https://review.openstack.org/#/c/327381/102:26
patchbotstevemar: patch 327381 - keystone (stable/mitaka) - Revert to caching fernet tokens the same way we do...02:26
notmorganoh hai02:26
notmorganwhats up stevemar ?02:26
stevemarnotmorgan: just looking for someone that has +2 on stable :)02:26
jamielennoxunable to take leave02:27
notmorganstevemar: thats a big changeset...02:27
*** dan_nguyen has quit IRC02:27
notmorganjamielennox: lol02:27
stevemarjamielennox: i was out all day!02:27
jamielennoxi was looking at that patch, i'm really unhappy with how many caching edge cases there are throughout there02:28
notmorganstevemar: OMG............. I've almost got a massive amount of encode/decode/encode/decode/encode/decode logic done in zuul...02:28
notmorganstevemar: sooooooo close....02:28
*** woodster_ has quit IRC02:28
jamielennoxi think MEMOIZE is dangerous through there and we might just want to replace it with an actual cache object that they all share02:29
notmorganwoooooooooooo FAILED (id=0, failures=34)02:29
*** ayoung has quit IRC02:29
notmorganjamielennox: 90% of the time someone does @MEMOIZE wrong the first 5 times.02:30
* notmorgan pulls statistics out of thin air02:30
jamielennoxyea, i'd love to see the maths on that02:30
*** lhcheng has quit IRC02:30
jamielennoxi think there are simple cases where memoize is really useful but it gets overused and makes invalidation painful02:30
notmorganjamielennox: unfortunately, the way keystone is built, standard cache objects would also ... just fail.02:31
notmorganjamielennox: for the same exact reasons.02:31
notmorganjamielennox: people don't understand invalidations regardless of how it's executed.02:31
notmorgan"mycache.get(thing)"02:31
jamielennoxnotmorgan: right invalidation a known hard problem02:31
jamielennoxbut having different caches on _validate_token, _validate_v2_token, _validate_v3_token and validate_non_persistent_token could surely be replaced with just one caching layer02:32
notmorganjamielennox: yes. i would love to drop caching in keystone because i don't have energy to keep on top of all the cache addition/changing patches and i think maybe you and steve are the other people in keystone who can debug it consistently02:32
notmorganjamielennox: that is a problem with our stupid (REALLY STUPID) token validator pipeline02:33
jamielennoxnotmorgan: i turn it off more than i should02:33
notmorganwe should validate tokens exactly one way.02:33
jamielennoxi've always wanted to refactor that but it's just a really dangerous spot and the times i've tried always get too complicated to follow02:33
notmorganjamielennox: at this point with (exception of pki/pkiz) we could move everything to fernet if the performance is even close (with this patch it's closer)02:34
notmorganjamielennox: i think i'm going to propose a change inc. a migration to store the fernet equiv payload in the token table.02:34
notmorganand then we can do "reconstruct" (ignore pki(z))02:34
notmorganand i am inclined to make pki(z) non-persistent if we're not ditching it.02:35
lbragstadjamielennox i'm still working through the performance CI - should have something by tomorrow02:35
jamielennoxnotmorgan: i'm actually really keen to see with the auth_token middleware being used from keystone if we can do PKI from middleware02:35
notmorgansince we could just decode the pki-data02:35
notmorganjamielennox: yes.02:35
jamielennoxnotmorgan: there's a bunch of stuff i just think we should no longer have to do02:35
notmorganjamielennox: even if not, it would be easy to do now.02:35
jamielennoxbut i made the stupid decision to refactor everything to use request rather than our context object and so am a bit stuck02:35
notmorganwe could make .validate_non-persistent_token -> .validate_reconstructable_token02:35
notmorganjamielennox: and merge a ton of logic back in.02:36
notmorganwe just need to split pack/unpack into proper parts that aren't built into .validate_thing02:36
notmorganthat replace ._persistence.get_token_data02:36
jamielennoxlbragstad: will be good to see. are you wiping the worker between runs or just checking out again?02:36
notmorganand ._persistence.save02:37
lbragstadjamielennox no - everything is green field install02:37
jamielennoxlbragstad: i saw your ansible snippet - i didn't even know we pushed changeset refs to git.openstack.org so i'm no help02:37
* notmorgan mutters about things he'd like to do but he's almost 100% successfully context switched to zuul for the moment02:37
lbragstadjamielennox here is an example of what the current perf runs look like on master02:38
lbragstadhttp://cdn.pasteraw.com/lh3vhhhp42by03vpyi60espctimwoh002:38
jamielennoxlbragstad: other than returning to jenkins are you pushing that data somewhere?02:38
lbragstadjamielennox not yet - what i have locally is kind of a mess02:38
lbragstadjust looking for mvp currently02:39
jamielennoxlbragstad: yep02:39
lbragstadthe last step is to get the patch pulled down from gerrit02:39
lbragstadinstalled and rekick apache02:39
lbragstadthen rerun the benchmark02:39
jamielennoxso i've never tried doing ansible with gerrit changes, but we have come across places where the ansible git module does things in different ways to just running git cmdline02:40
lbragstadinteresting02:40
notmorganjamielennox: oh just wait... Gerrit => Zuul => Ansible is a thing!02:40
jamielennoxlike if you clone with a branch name then try and try and do submodule tasks it seems to have pulled some weird limited set of the repo, not what you'd get with a git clone02:41
jamielennoxlbragstad: so it's worth replacing it with a command: git <do stuff>  and if that works just skip the git module02:42
notmorganstevemar: look at the cache thing02:42
lbragstadjamielennox i might have to resort to that02:42
jamielennoxnotmorgan: i'm definitely waiting02:42
lbragstadjamielennox but dolphm got it working here - https://github.com/dolph/keystone-deploy/blob/master/playbooks/roles/keystone/tasks/main.yaml#L12-L2102:42
lbragstadand we were using that to check out patches up for review and testing them02:42
notmorganjamielennox: zuulv3 will be good. but at least as a POC zuulv25 (with zuulansiblelauncher) is a thing we're getting up and running.02:43
jamielennoxlbragstad: obvious difference, he's using review.openstack.org and you're using git.openstack.org02:43
notmorganjamielennox: use git.openstack.org in all cases if you're doing http transport02:43
notmorganlbragstad: ^02:43
jamielennoxi'm not sure if they're backed to the same thing02:43
notmorganjamielennox: they are02:43
stevemarnotmorgan: what about the cache thing?02:43
notmorganstevemar: my comment02:43
jamielennoxnotmorgan: i thought git was just the published branches behind load balancers, but ok02:44
notmorganjamielennox: git fetch https://git.openstack.org/openstack/keystone refs/changes/81/327381/1 && git checkout FETCH_HEAD [for example]02:44
notmorganall the refs are there for pending patches too02:44
notmorganthat is https://review.openstack.org/#/c/327381/102:44
patchbotnotmorgan: patch 327381 - keystone (stable/mitaka) - Revert to caching fernet tokens the same way we do...02:44
notmorganftr.02:44
notmorganjamielennox: if you click "download" and select "anonymous http" you can use git.openstack.org instead of ssh://<user>@review.openstack.org02:45
notmorgan....02:45
jamielennoxnotmorgan: yea, i saw that02:45
notmorgan:)02:45
notmorganbasically, don't use review.openstack.org unless you *need* to imo02:45
jamielennoxso IMO, if you're constantly redeploying the whole machine i think i'd write the setup in bash and not ansible02:46
lbragstadjamielennox we have some of that saved away because we're using lxd02:50
lbragstadso we built an ubuntu image with basic things that quicken the process a bit02:50
*** sheel has joined #openstack-keystone02:52
jamielennoxlbragstad: yea, ok, makes sense02:52
*** browne has joined #openstack-keystone02:57
*** gyee has quit IRC03:01
* notmorgan looks... omg. down to ~3 py3 failures in zuul...03:02
*** sdake has joined #openstack-keystone03:10
*** richm has quit IRC03:14
openstackgerritMerged openstack/keystonemiddleware: Updated from global requirements  https://review.openstack.org/32736903:15
*** iurygregory_ has quit IRC03:17
*** iurygregory_ has joined #openstack-keystone03:17
*** roxanaghe has joined #openstack-keystone03:24
stevemarnotmorgan: i should send a ML post about making cache and messaging always there in ksm03:24
*** sdake has quit IRC03:26
*** roxanaghe has quit IRC03:29
*** adrian_otto has joined #openstack-keystone03:34
jamielennoxstevemar: i have a patch that makes cache a direct requirement anyway so that one is going to be required anyway03:35
*** iurygregory_ has quit IRC03:35
stevemarjamielennox: is the patch posted?03:41
jamielennoxstevemar: yea, that's an old one: https://review.openstack.org/#/c/268664/03:42
patchbotjamielennox: patch 268664 - keystonemiddleware - Use oslo_cache in auth_token middleware03:42
jamielennoxhmm, didn't know it was failing03:42
jamielennoxbut it looks like all its dependencies have merged now03:42
*** lhcheng has joined #openstack-keystone03:45
*** ChanServ sets mode: +v lhcheng03:45
*** links has joined #openstack-keystone03:46
*** lhcheng_ has joined #openstack-keystone03:49
stevemarjamielennox: well, if we're going to make things required, lets do it early in the cycle03:51
stevemarits a pretty big dependency to bring in03:51
*** lhcheng has quit IRC03:52
jamielennoxstevemar: i'll fix up that review then, i haven't looked at it in a while03:52
openstackgerritSteve Martinelli proposed openstack/keystone: Add docstring to delete_project  https://review.openstack.org/28044604:01
openstackgerritSteve Martinelli proposed openstack/keystone: Add docstring to delete_project  https://review.openstack.org/28044604:08
*** TxGVNN has quit IRC04:14
*** roxanaghe has joined #openstack-keystone04:26
*** roxanaghe has quit IRC04:30
notmorganstevemar: yeah04:30
notmorganstevemar: possibly.04:30
jamielennoxnotmorgan: how would you feel about taking the memorycache into oslo.cache?04:40
notmorganjamielennox: uhm.. the crap on in ksm?04:44
notmorganjamielennox: it's already there.04:44
jamielennoxnotmorgan: oh, really?04:44
notmorganjamielennox: yeah04:45
jamielennoxso i found the problem with my oslo.cache review was i wrote it back when we changed the default to a noop and removed the memory cache04:45
jamielennoxis there a way i can tell oslo.cache/dogpile to default to the crappy memory one if nothing else is defined?04:45
notmorganjamielennox: https://github.com/openstack/oslo.cache/blob/master/oslo_cache/backends/dictionary.py04:46
notmorganjamielennox: you just need to tell the config to use that one by default.04:46
notmorganin the config override.04:46
jamielennoxoh, via like CONF.set_default() or something04:47
jamielennoxhmm04:47
*** jaosorior has joined #openstack-keystone04:47
notmorganuh... something like that04:47
notmorganthere is a way to override the default for config generation too04:47
jamielennoxthere doesn't seem to be a way i can set the default via oslo_cache.configure()04:48
notmorgani have a horrible hack in keystone for it04:48
notmorganoslo_cache needs to get the .set_default matic04:48
notmorganmagic*04:48
jamielennoxnotmorgan: is there a reason oslo.cache is using a static _BACKENDS list instead of entrypoints?04:49
notmorganjamielennox: haven't updated it04:49
notmorganjamielennox: need to get around to fixing that04:49
jamielennoxok, so you'd take that?04:49
notmorganyeah. dogpile *should* suppot entrypoints natively04:49
notmorganbtw04:49
jamielennoxnotmorgan: where would you prefer it to go? i'm more comfortable getting it into oslo.cache04:50
jamielennoxbut i'll leave it if dogpile is the better location04:50
notmorganoslo_cache should just make sure we're consuming appropriate entrypoints via dogpile04:51
notmorganbut we can do it eithe rway04:51
* notmorgan would be fine with it.04:51
jamielennoxi think we would still need to do oslo_config opts via oslo_cache entrypoints?04:51
jamielennoxthat option dict building nonsense04:51
jamielennoxnotmorgan: so it looks like dogpile does support backends via entrypoint04:55
jamielennoxit just chooses to register them manually for plugins in tree04:55
notmorganjamielennox: yeah04:56
*** GB21 has joined #openstack-keystone04:57
notmorganstevemar, jamielennox: this makes me rediculously happy: http://i.imgur.com/qtUqN8y.png?105:05
jamielennoxnotmorgan: congrats05:05
notmorganjamielennox: the patch chain is... kindof insane.05:05
jamielennoxthat would have been painful05:05
jamielennoxpy3 conversion is mostly X didn't work, ok .encode(), nope .decode()05:06
jamielennox:)05:06
notmorganjamielennox: https://review.openstack.org/#/q/status:open+project:openstack-infra/zuul+branch:master+topic:v25+owner:%22Morgan+Fainberg+%253Cmorgan.fainberg%2540gmail.com%253E%2205:06
notmorganjamielennox: and https://review.openstack.org/#/c/327310/05:07
patchbotnotmorgan: patch 327310 - openstack-infra/gear - Do not encode the type (b) in the job name05:07
notmorganjamielennox: and that is the second fix to gear needed for this.05:07
jamielennoxnotmorgan: on the upside the 2 or 3 i've clicked on are mostly obvious +As05:08
jamielennoxfirst oslo.cache bug05:09
jamielennoxhttps://github.com/openstack/oslo.cache/blob/master/oslo_cache/backends/mongo.py#L164 doesn't exist05:09
notmorganjamielennox: it has been a lot of splitting it apart because dumping an 800line "fix py3" just felt wrong05:09
notmorganjamielennox: this was the worst one to deal with: https://review.openstack.org/#/c/327435/05:10
patchbotnotmorgan: patch 327435 - openstack-infra/zuul - Python 3 Fixes: More encode/decode fixes05:10
notmorganjamielennox: lol mongo missing05:11
notmorganahahaha clearly no one is using that.05:11
jamielennoxnotmorgan: oh, i think maybe he moved that with 0.605:12
jamielennoxwhich is ugh05:12
notmorganoh yeah maybe05:12
notmorganactually might be my fault05:13
notmorgani did a ton of the work in 0.6 to roll in dogpile.core05:13
jamielennoxnotmorgan: yea, it's a 0.5->0.6 thing05:14
notmorgansorry :(05:14
jamielennoxnotmorgan: it went from dogpile.cache.util.memorized_property to dogpile.util.memoized_property05:14
notmorganahh05:14
notmorgandoh05:14
*** roxanaghe has joined #openstack-keystone05:15
notmorganif that is the extent we broke people though...05:15
notmorganwow not too shappy05:15
notmorganshabby05:15
jamielennoxnotmorgan: maybe, that whole util dir seems wrong05:15
notmorganand ultimately fixable with a pointer in dogpile.cache05:15
notmorganthe old util dir was bad05:15
*** rcernin|off has joined #openstack-keystone05:15
notmorganthe new one is more "correct"05:15
notmorganthe reason it was in dogpile.cache.util was because dogpile.core owned dogpile and dogpile.core namespaces05:15
notmorganso it had to be isolated05:16
jamielennoxok, so creating dogpile.util was intentional05:16
notmorganjamielennox: yes05:21
*** jed56 has quit IRC05:23
*** lamt_ has quit IRC05:23
*** tpeoples has quit IRC05:24
*** DuncanT has quit IRC05:24
*** sheel has quit IRC05:24
*** jraim has quit IRC05:24
*** andreykurilin__ has quit IRC05:25
*** serverascode has quit IRC05:25
*** ctracey has quit IRC05:25
*** briancurtin has quit IRC05:25
*** boris-42 has quit IRC05:25
*** zhiyan has quit IRC05:25
*** nikhil has quit IRC05:25
*** andrewbogott has quit IRC05:25
*** rcernin|off has quit IRC05:26
*** lamt has joined #openstack-keystone05:27
*** DuncanT has joined #openstack-keystone05:27
*** sheel has joined #openstack-keystone05:29
*** briancurtin has joined #openstack-keystone05:30
*** jed56 has joined #openstack-keystone05:31
jamielennoxnotmorgan: i take it the sqlalchemy gerrit works for you?05:31
*** jraim has joined #openstack-keystone05:32
*** andrewbogott has joined #openstack-keystone05:32
*** tpeoples has joined #openstack-keystone05:32
*** andreykurilin__ has joined #openstack-keystone05:32
*** ctracey has joined #openstack-keystone05:33
*** zhiyan has joined #openstack-keystone05:33
*** serverascode has joined #openstack-keystone05:34
*** boris-42 has joined #openstack-keystone05:34
*** nikhil has joined #openstack-keystone05:35
jamielennoxnotmorgan, zzzeek: https://bitbucket.org/zzzeek/dogpile.cache/pull-requests/51/link-moved-functions-in-dogpilecacheutil/diff05:41
notmorganjamielennox: it does.05:41
notmorganjamielennox: i do not have merge access in it, but it does work.05:42
notmorganjamielennox: you should propose the fix to sql-a gerrit ;)05:42
jamielennoxnotmorgan: it doesn't seem to want to connect at all05:42
notmorganjamielennox: awww.05:42
notmorgansec05:42
jamielennoxi added ssh keys and everything05:42
notmorganjamielennox: hmm works for me.05:42
notmorganare you trying to use git-review?05:42
notmorgancause....05:43
notmorganhm05:43
notmorganit should work05:43
notmorgan*shrug*05:43
notmorgani haven't proposed code to it yet though05:43
jamielennoxnotmorgan: he can figure it out tomorrow05:44
jamielennoxnotmorgan: i don't really want to debug our own gerrit let along someone elses05:44
notmorganhehe05:46
jamielennoxzzzeek: for later, ssh works with ssh -p 29418 jamielennox@gerrit.sqlalchemy.org05:47
jamielennoxzzzeek: i can successfully clone with: git clone ssh://jamielennox@gerrit.sqlalchemy.org:29418/zzzeek/dogpile.cache.git05:47
jamielennoxzzzeek: but i think you need to turn on some form of push acl for registered users to let them propose a patch05:48
*** fawadkhaliq has joined #openstack-keystone05:50
*** fawadkhaliq has quit IRC05:50
*** fawadkhaliq has joined #openstack-keystone05:51
*** itisha has quit IRC06:00
*** GB21 has quit IRC06:03
*** yolanda has joined #openstack-keystone06:05
*** nikhil has quit IRC06:07
*** nikhil has joined #openstack-keystone06:09
*** yolanda has quit IRC06:10
*** yolanda has joined #openstack-keystone06:10
*** rcernin|off has joined #openstack-keystone06:15
*** yolanda has quit IRC06:16
*** yolanda has joined #openstack-keystone06:18
*** GB21 has joined #openstack-keystone06:19
*** rcernin|off is now known as rcernin06:21
*** fawadkhaliq has quit IRC06:24
*** pnavarro has joined #openstack-keystone06:35
*** belmoreira has joined #openstack-keystone06:38
*** welldannit has quit IRC06:41
*** GB21 has quit IRC06:42
*** belmoreira has quit IRC06:48
*** welldannit has joined #openstack-keystone06:50
*** roxanaghe has quit IRC06:53
*** jbell8 has joined #openstack-keystone06:54
*** GB21 has joined #openstack-keystone07:00
*** belmoreira has joined #openstack-keystone07:07
*** pcaruana has joined #openstack-keystone07:09
*** browne has quit IRC07:14
*** fawadkhaliq has joined #openstack-keystone07:25
*** fawadkhaliq has quit IRC07:31
openstackgerritRyosuke Mizuno proposed openstack/keystone: Add validation rules for create token using a JSON schema  https://review.openstack.org/32508607:32
*** hoonetorg has quit IRC07:47
*** fawadkhaliq has joined #openstack-keystone07:47
*** tesseract has joined #openstack-keystone07:56
*** hoonetorg has joined #openstack-keystone07:58
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-keystone08:00
*** openstackgerrit has quit IRC08:03
*** openstackgerrit has joined #openstack-keystone08:03
*** adrian_otto has quit IRC08:11
*** permalac has joined #openstack-keystone08:21
*** alex_xu has quit IRC08:22
openstackgerritMerged openstack/oslo.policy: Add helper scripts for generating policy info  https://review.openstack.org/32124308:22
*** alex_xu has joined #openstack-keystone08:25
*** daemontool has joined #openstack-keystone08:25
*** jaosorior has quit IRC08:28
*** jaosorior has joined #openstack-keystone08:28
*** henrynash_ has joined #openstack-keystone08:40
*** ChanServ sets mode: +v henrynash_08:40
*** EinstCrazy has joined #openstack-keystone08:42
*** TxGVNN has joined #openstack-keystone08:44
*** rdo has quit IRC08:51
*** rdo has joined #openstack-keystone08:53
*** jbell8 has quit IRC09:17
*** jbell8 has joined #openstack-keystone09:18
*** lhcheng_ has quit IRC09:18
*** jbell8 has quit IRC09:22
*** jbell8 has joined #openstack-keystone09:22
openstackgerritSamuel de Medeiros Queiroz proposed openstack/keystone: Add docstring to delete_project  https://review.openstack.org/28044609:23
samueldmqmorning keystone09:23
samueldmqhenrynash_: ^09:23
samueldmqhenrynash_: just fixed the typo09:24
henrynash_dome09:24
henrynash_oops, thx, +2’d09:24
*** dmk0202 has joined #openstack-keystone09:28
*** rmizuno has quit IRC09:30
*** shewless has quit IRC09:30
*** nisha_ has joined #openstack-keystone09:43
*** mvk_ has quit IRC09:46
*** nisha_ has quit IRC09:50
*** nisha_ has joined #openstack-keystone09:56
*** henrynash_ has quit IRC10:02
*** fawadkhaliq has quit IRC10:03
*** fawadkhaliq has joined #openstack-keystone10:03
*** rk4n has joined #openstack-keystone10:10
*** rk4n has quit IRC10:16
*** mvk_ has joined #openstack-keystone10:17
*** _amrith_ is now known as amrith10:20
*** rk4n has joined #openstack-keystone10:26
*** rk4n has quit IRC10:26
*** rk4n has joined #openstack-keystone10:26
*** belmoreira has quit IRC10:31
*** nisha__ has joined #openstack-keystone10:32
*** nisha_ has quit IRC10:32
*** nisha__ is now known as nisha_10:42
*** TxGVNN has quit IRC10:50
*** josecastroleon has quit IRC10:51
*** josecastroleon has joined #openstack-keystone10:57
*** nisha__ has joined #openstack-keystone11:01
*** nisha_ has quit IRC11:01
*** fawadk has joined #openstack-keystone11:20
*** fawadkhaliq has quit IRC11:21
*** fawadkhaliq has joined #openstack-keystone11:21
*** shoutm has joined #openstack-keystone11:23
*** fawadk has quit IRC11:25
*** nisha__ is now known as nisha_11:30
*** GB21 has quit IRC11:37
*** gordc has joined #openstack-keystone11:45
*** pauloewerton has joined #openstack-keystone11:45
*** sdake has joined #openstack-keystone11:56
*** GB21 has joined #openstack-keystone11:57
*** EinstCrazy has quit IRC12:04
*** markvoelker has joined #openstack-keystone12:04
*** agireud has quit IRC12:08
*** agireud has joined #openstack-keystone12:09
*** fawadk has joined #openstack-keystone12:14
*** GB21 has quit IRC12:16
*** fawadkhaliq has quit IRC12:18
*** belmoreira has joined #openstack-keystone12:20
*** EinstCrazy has joined #openstack-keystone12:24
*** fawadk has quit IRC12:31
stevemarsamueldmq: thanks for following up12:36
*** TxGVNN has joined #openstack-keystone12:39
*** sdake has quit IRC12:48
*** julim has joined #openstack-keystone12:51
openstackgerritMerged openstack/keystone: Add docstring to delete_project  https://review.openstack.org/28044612:52
openstackgerritMerged openstack/keystone: Honor ldap_filter on filtered group list  https://review.openstack.org/32593912:53
samueldmqstevemar: yw12:55
*** links has quit IRC12:57
*** nisha__ has joined #openstack-keystone13:04
*** nisha_ has quit IRC13:05
*** ayoung has joined #openstack-keystone13:11
*** ChanServ sets mode: +v ayoung13:11
*** amrith is now known as _amrith_13:12
*** links has joined #openstack-keystone13:17
*** edmondsw has joined #openstack-keystone13:22
*** links has quit IRC13:23
*** nisha__ is now known as nisha_13:23
bknudsonDoes anyone know what data is gathered when osprofiler is enabled? Has anybody used it?13:29
*** nisha_ has quit IRC13:29
bknudsonI'll have have to try it out13:30
dolphmi know we deploy it to production because it's in every single default upstream pipeline *grumble*13:36
*** henrynash_ has joined #openstack-keystone13:37
*** ChanServ sets mode: +v henrynash_13:37
amakarovColleagues, please pay attention to cache invalidation issue: https://bugs.launchpad.net/oslo.cache/+bug/159077913:38
openstackLaunchpad bug 1590779 in oslo.cache "Cache region invalidation works for local CacheRegion object only" [Undecided,New]13:38
*** sdake has joined #openstack-keystone13:39
*** jaosorior has quit IRC13:42
*** nisha_ has joined #openstack-keystone13:42
*** nisha_ has quit IRC13:43
*** nisha_ has joined #openstack-keystone13:43
*** itisha has joined #openstack-keystone13:44
knikollao/13:46
*** fawadkhaliq has joined #openstack-keystone13:46
zzzeekjamielennox: the system right now is, people just send me PRs because that's what they do anyway, I pull them into gerrit, then I add them to the "contributors" group so they can work on them.   I havent solved the problem of allowing anyone to register and push code that runs on my own CI environment yet13:46
zzzeekjamielennox: I've added you to "Contributors" which has push access13:47
*** ametts has joined #openstack-keystone13:49
openstackgerritMatthew Edmonds proposed openstack/keystone: fix ldap delete_user group member cleanup  https://review.openstack.org/32735813:52
openstackgerritRon De Rose proposed openstack/keystone: Refactor shadow users  https://review.openstack.org/32359613:55
*** shoutm has quit IRC13:57
*** richm has joined #openstack-keystone13:59
*** ddieterly has joined #openstack-keystone14:01
*** henrynash_ has quit IRC14:04
*** _amrith_ is now known as amrith14:06
*** rodrigods has quit IRC14:06
*** rodrigods has joined #openstack-keystone14:07
*** henrynash_ has joined #openstack-keystone14:07
*** henrynash_ has quit IRC14:07
*** adrian_otto has joined #openstack-keystone14:08
*** fawadkhaliq has quit IRC14:08
openstackgerritRon De Rose proposed openstack/keystone: Shadow LDAP and custom driver users  https://review.openstack.org/32360214:08
*** GB21 has joined #openstack-keystone14:11
openstackgerritRodrigo Duarte proposed openstack/keystone: Fix ValidationError exception name in docstring  https://review.openstack.org/32769614:11
rodrigodshenrynash, samueldmq ^ a remaining nit from https://review.openstack.org/#/c/280446/14:12
patchbotrodrigods: patch 280446 - keystone - Add docstring to delete_project (MERGED)14:12
samueldmqrodrigods: done14:14
rodrigodssamueldmq, thx14:14
*** agireud has quit IRC14:19
*** josdotso has joined #openstack-keystone14:20
dstanekdolphm: i seriously dislike that14:20
*** agireud has joined #openstack-keystone14:20
bknudsonLooks like osprofiler is disabled by default : http://git.openstack.org/cgit/openstack/keystone/tree/etc/keystone.conf.sample#n167214:21
*** GB21 has quit IRC14:21
*** adrian_otto has quit IRC14:21
dolphmdstanek: the osprofile thing?14:21
dstanekbknudson: it's disabled, but in the pipeline14:21
dstanekdolphm: yep14:22
dolphmbknudson: our default configuration is supposed to be production oriented14:22
josdotsoOpenID Connect question:  If CLI auth with python-openstack client requires the end-user to download an openrc.sh or similar which contains an OIDC client id and secret, does that design not compromise OIDC's protection of man in the middle attacks? http://salesforce.stackexchange.com/questions/14009/whats-the-benefit-of-the-client-secret-in-oauth214:22
*** pushkaru has joined #openstack-keystone14:22
bknudsonwe've been asked by our performance people to set this up on our production systems.14:23
dolphmbknudson: you should provide your performance people with their own lab14:23
josdotsoI guess rather than MiM, it's rogue lookalike APIs14:24
edmondswnot sure why the bot didn't announce it, but I just cherry-picked this back to mitaka: https://review.openstack.org/#/c/327703/14:26
patchbotedmondsw: patch 327703 - keystone (stable/mitaka) - Honor ldap_filter on filtered group list14:26
dstanekjosdotso: what MiM prevention would be compromised?14:27
dolphmjosdotso: doesn't that depend on where and how you get the secret? how does that differ than any other side band transfer of secrets?14:27
*** gagehugo has joined #openstack-keystone14:28
josdotsoI was incorrect.  I believe the concern is as follows: "14:29
josdotso14:29
josdotsoWeb apps use client secrets because they represent huge attack vectors. Let us say that someone poisons a DNS entry and sets up a rogue app "lookalike", the juxtapose might not be noticed for months, with this intermediary sucking up tons of data. Client secrets are supposed to mitigate this attack vector. For single user clients, compromise has to come one device at a time, which is horribly inefficient in comparison. While true that they14:29
josdotsoare marginally less secure, they're still required to use TLS (avoids man-in-the-middle) and request-body posting (avoids logs)." http://salesforce.stackexchange.com/questions/14009/whats-the-benefit-of-the-client-secret-in-oauth2#comment20111_1401314:29
josdotsoIt's my understanding that this is why client id and secret are typically behind a website, not downloaded14:30
dolphmjosdotso: "Web apps use client secrets because they represent huge attack vectors." wait, typo? why would you use something *because* it's an attack vector?14:30
dstanekdolphm: the 'web app' is the attack vector in that example - the post talks about DNS poisoning, etc14:31
dstanekjosdotso: so what's your concern?14:31
josdotsoSure thing... (cont.)14:31
dolphmooh, definitely misread14:31
dstanekjosdotso: client ID isn't included in all oauth flows because it's easy to get and spoof14:32
dstanekimagine a desktop twitter client that includes a client ID. you could very easily that that and use it in your own client14:32
*** jorge_munoz has joined #openstack-keystone14:33
dstanekmy understanding of client ID in the oauth2 send is that it's nothing to do with the user - it's to make sure only authorized clients can access an api14:33
josdotsoMy concern is that client ID and secret are obtained from user's host, where client ID and secret must match the client ID and secret configured in Keystone, no?  So there is only the one client id and one client secret configured at IDP for the whole openstack cloud it would seem.  If bad actor / actual user knows the client id and secret, can they not spoof Keystone itself?14:33
josdotsostill learning on this topic, so i appreciate the help14:34
josdotsodstanek: agree14:35
dstanekjosdotso: they can only spoof keystone if they spoof the SSL certs or the client doesn't validate them14:35
rodrigodsdstanek, ++14:35
rodrigodsjosdotso, besides that, all cryptography techniques assumes the keys are "secure"14:36
dstanekjosdotso: i could spoof keystone right now by just setting up my own one. the trick is to get you to use it instead of your own14:36
rodrigodsso... they assume the user host is not compromised14:36
josdotsoagreed on the spoofing and validation.  client computer would likely store client-id and secret in plain text.14:36
josdotsoTrue true.  Ok cool.14:37
josdotsoSo sharing the single client-id and client-secret with every user is okay14:37
dstanekthe client id protects the server against unauthorized clients (not sure openstack cares about it) - it doens't protect the client from unauthorized servers14:37
josdotsoYes, that's becoming clear now.  Hmm...14:37
josdotsoSo SSL is what's making it a sound idea. Works for me.14:38
josdotso(validation)14:38
josdotsoThanks!14:39
*** woodburn has joined #openstack-keystone14:39
*** woodburn1 has quit IRC14:39
*** pushkaru has quit IRC14:41
*** pcaruana has quit IRC14:41
*** pushkaru has joined #openstack-keystone14:42
*** timcline has joined #openstack-keystone14:42
*** raddaoui has joined #openstack-keystone14:44
*** rderose has joined #openstack-keystone14:44
*** rdo has quit IRC14:45
josdotsoSo yeah.  Looks like the only thing of concern would be phishing sites, but that's no less possible using non-OIDC, so no complaints from me: http://homakov.blogspot.com/2012/08/oauth2-one-accesstoken-to-rule-them-all.html?showComment=1413191400799#c469232042291616413314:48
*** slberger has joined #openstack-keystone14:53
*** josdotso has quit IRC14:53
samueldmqrderose: hi14:54
rderosesamueldmq: hi14:54
samueldmqrderose: you've separated the driver option deprecation from that change right?14:54
samueldmqrderose: have you submitted it to gerrit yet ?14:55
rderosesamueldmq: I think so, let me see14:55
*** ddieterly is now known as ddieterly[away]14:57
rderosesamueldmq: give me a few, in a meeting14:57
bretonwhere is notmorgan14:59
*** fawadkhaliq has joined #openstack-keystone15:00
*** nisha_ has quit IRC15:01
*** woodburn has quit IRC15:01
*** woodburn1 has joined #openstack-keystone15:01
*** ddieterly[away] is now known as ddieterly15:01
samueldmqrderose: sure15:02
samueldmqrderose: take your time15:02
*** rdo has joined #openstack-keystone15:02
*** nisha_ has joined #openstack-keystone15:03
*** henrynash_ has joined #openstack-keystone15:05
*** ChanServ sets mode: +v henrynash_15:05
*** EinstCrazy has quit IRC15:05
*** rk4n has quit IRC15:06
*** rk4n has joined #openstack-keystone15:07
*** KevinE has joined #openstack-keystone15:07
*** andrewbogott has quit IRC15:08
*** andrewbogott has joined #openstack-keystone15:08
*** julim has quit IRC15:09
*** rk4n has quit IRC15:15
*** rk4n has joined #openstack-keystone15:16
*** rk4n has quit IRC15:23
*** dan_nguyen has joined #openstack-keystone15:28
*** belmoreira has quit IRC15:29
*** browne has joined #openstack-keystone15:30
*** tesseract has quit IRC15:30
*** adrian_otto has joined #openstack-keystone15:32
*** rcernin has quit IRC15:32
*** GB21 has joined #openstack-keystone15:33
*** woodster_ has joined #openstack-keystone15:34
*** woodburn1 has quit IRC15:34
openstackgerritRon De Rose proposed openstack/keystone: Refactor shadow users  https://review.openstack.org/32359615:35
rderosesamueldmq: Based on dolphm and dstanek comments, I'm removing the deprecation part from the refactor15:35
*** woodburn has joined #openstack-keystone15:36
samueldmqrderose: sure15:36
samueldmqrderose: would you mind to put lazy='subquery' in a separate patch too ?15:36
samueldmqrderose: that's changing the behavior somehow... not the behavior of the API provided itself, but ...15:37
samueldmqrderose: it doesn't hurt, should be an easy approval, just to get a dedicated commit message and explanation of the change15:37
rderosesamueldmq: true15:38
dolphmsamueldmq: that's a whitespace change, isn't it?15:38
rderosedolphm: changing lazy loading to eager loading15:38
rderosedolphm: so not a whitespace change15:39
*** agireud has quit IRC15:40
samueldmqdolphm: what rderose said ^15:40
*** dmk0202 has quit IRC15:41
*** pgbridge has joined #openstack-keystone15:41
rderosesamueldmq dolphm dstanek: will make this 2 commits15:41
*** agireud has joined #openstack-keystone15:42
*** rk4n has joined #openstack-keystone15:43
samueldmqrderose: ++15:44
*** KevinE_ has joined #openstack-keystone15:45
*** KevinE has quit IRC15:46
*** henrynash_ has quit IRC15:46
*** ddieterly is now known as ddieterly[away]15:47
*** aloga has quit IRC15:48
*** aloga has joined #openstack-keystone15:49
*** TxGVNN has quit IRC15:52
*** ddieterly[away] is now known as ddieterly16:10
*** GB21 has quit IRC16:12
notmorganbreton: i was sleeping16:18
*** gyee has joined #openstack-keystone16:19
*** ChanServ sets mode: +v gyee16:19
bretonnotmorgan: good. I have a question about this thing: https://github.com/openstack/keystone/blob/stable/mitaka/keystone/common/cache/core.py#L7116:20
bretonnotmorgan: how did you make sure that it works?16:21
notmorganbreton: direct testing, with a hack that lets it work underneath16:21
bretonit seems that it got broken16:21
bretonbut i cannot figure out why16:22
notmorganbreton: *shrug*16:22
notmorganbreton: i would rather just revert it.16:22
bretonsetter is never called for example.16:22
bretonlet me check the getters now...16:23
notmorganit works it may not be applied everywhere16:23
notmorganit also may have been broken in a recent dogpile release16:24
*** sdake has quit IRC16:25
bretonnotmorgan: for me it doesn't work for COMPUTED_CATALOG_REGION16:25
*** sheel has quit IRC16:25
* notmorgan shrugs16:26
notmorganhonestly, i am kindof over caring about the caching stuff.16:26
notmorgani'm rapidly feeling like we should just rip it all out16:26
notmorganand delete it.16:26
openstackgerritRon De Rose proposed openstack/keystone: Refactor shadow users  https://review.openstack.org/32359616:32
notmorganbreton: i also missed applying it to one of the cache regions.  our manager architecture is awful to cache and even if we didn't use @memoize, we'd run into the same issues with people failing to understand invalidations16:32
openstackgerritRon De Rose proposed openstack/keystone: Shadow LDAP and custom driver users  https://review.openstack.org/32360216:35
openstackgerritRon De Rose proposed openstack/keystone: Shadow LDAP and custom driver users  https://review.openstack.org/32360216:35
*** edtubill has joined #openstack-keystone16:38
*** sdake has joined #openstack-keystone16:38
*** daemontool has quit IRC16:41
*** adrian_otto has quit IRC16:43
*** d0ugal has quit IRC16:43
*** d0ugal has joined #openstack-keystone16:44
*** sdake_ has joined #openstack-keystone16:45
*** sdake has quit IRC16:45
*** tonytan4ever has joined #openstack-keystone16:48
*** ddieterly is now known as ddieterly[away]16:49
openstackgerritRon De Rose proposed openstack/keystone: Change LocalUser sql model to eager loading  https://review.openstack.org/32781716:49
openstackgerritRon De Rose proposed openstack/keystone: Change LocalUser sql model to eager loading  https://review.openstack.org/32781716:51
*** ddieterly[away] is now known as ddieterly16:52
*** sdake_ has quit IRC16:52
bknudsonwhat do you think about having fernet use user name / project name rather than IDs?16:55
lbragstadbknudson that would require packing domain id/name in the payload too then16:55
bknudsonso that I can pass a token to another data center that has users with another ID16:55
bknudsonyes, the domain name for user / project would also be in the token16:56
openstackgerritRon De Rose proposed openstack/keystone: Change LocalUser sql model to eager loading  https://review.openstack.org/32781716:56
lbragstadbknudson so - you have a user named jsmith in two different datacenters but they aren't the same user?16:56
bknudsonthey are, but they were created in each datacenter so they've got a different id16:57
notmorganbknudson: usernames are mutable16:58
notmorganbknudson: so i am against it16:58
lbragstadso the user_id returned in the authentication response from each datacenter is going to change?16:58
bknudsonyes16:58
lbragstadi authenticate against one and it says my user_id is 1234 and i authenticate in another datacenter and it says my user id is 456716:59
notmorganif usernames become immutable, i'd support it, but as long as the name can change, i'd say it is a terrible idea16:59
*** mvk_ has quit IRC16:59
*** roxanaghe has joined #openstack-keystone17:00
*** nisha_ has quit IRC17:01
*** nisha_ has joined #openstack-keystone17:02
bknudsonnotmorgan: are you worried because a user might spoof another user? if the name doesn't exist anymore the token becomes invalid.17:03
*** slberger has quit IRC17:04
bknudsonthe token would still be valid for the cache period17:08
notmorganbknudson: because we should not be relying on mutable values for authz. even the cache period is wrong to rely on.17:09
notmorganfor that17:09
*** slberger has joined #openstack-keystone17:09
notmorgani'm all for fixing usability (though this whole cross dc thing without replication still makes me think someone is doing somerhing horribly wrong)17:10
*** shewless has joined #openstack-keystone17:10
shewlessdstanek: hi. I'm back for more federation punishment!17:11
notmorgan"I want to take an authz from an authoritative source and use it against a different source that didn't issue it and that isn't in sync with the issuing installation"17:11
notmorganthis is screaming in the back of my head a CVE waiting to happen17:12
dstanekshewless: how goes it?17:12
notmorganusing federated auth, so the token for install (remote) is authoritative from that installation is the accepted model for security17:12
bretonyep, setters don't work in _RegionInvalidator.17:13
shewlessdstanek: getting there... sort of.. I'm attempting to get keystone to be an SP to my enterprise IDP. On the IDP end I'm seeing this error: "The request specified an Assertion Consumer Service URL 'http://foo.com:5000/Shibboleth.sso/SAML2/POST' that is not configured on the relying party 'https://foo.com/shibboleth'17:13
*** julim has joined #openstack-keystone17:13
notmorganyou can't take a oauth session or oidc session and just use it without re-verifying/re-authing17:13
shewlessWhat I don't understand is where this is even coming from: http://foo.com:5000/Shibboleth.sso/SAML2/POST17:14
notmorganso, i'm going to just go out on a limb and say that this request is broken from a security model.17:14
*** lhcheng has joined #openstack-keystone17:14
*** ChanServ sets mode: +v lhcheng17:14
dstaneknotmorgan: lbragstad: bknudson: if sync is not an option then why not federation?17:14
notmorgandstanek: basically that is my argument17:15
bknudsondstanek: who's going to fix all the users to support federation?17:15
dstanekshewless: that is in your metadata17:15
dstanekbknudson: what do you mean by fix the users?17:16
bknudsonall the applications that are working today that don't support the federation flow?17:16
bknudsontoday the applications work by passing their existing tokens between data centers and there's no extra work that they're doing17:17
bknudsonto swap tokens17:17
dstanekbknudson: can the client not handle doing that in a federated world?17:18
bknudsonI wish I knew what all the applications are but I don't. We just provide the service that they use.17:19
notmorganbknudson: so... sounds to me like they should be syncing the backend so it's one system that the tokens were authoritatively issued for17:19
bknudsonMaybe in the future we can push clients to do this.17:19
notmorganbknudson: and if that deployment choice doesn't work, fix your app for federation17:19
shewlessdstanek: how is that metadata generated?17:19
bknudsonright, then we need to figure out how to sync the backend and this is going to require mucking with the db17:20
shewlessdstanek: like I know how to get it from the URL but how are the values determined?17:20
notmorganbknudson: i don't get it, how db sync hard?17:20
dstanekshewless: this is the metadata that you get from the SP's /Shibboleth.sso/Metadata. mod_shib generates it based on your configuration17:20
notmorganbknudson: passing a bearer token between two unconnected auth systems is insane.17:21
notmorganand expecting the authz to work17:21
dstanekbknudson: what is the usecase? multiple regions, different clouds, or something else?17:22
shewlessdstanek: mob_shib generates it from /etc/keystone.conf or /etc/apache2/ or somewhere in /etc/shibboleth?17:22
notmorgandstanek: different deployments owned by one company, unconnected, with user account data created in each separately. is my understranding17:22
notmorgandstanek: and they want to just take a token and use it from one on the other17:22
dstanekshewless: /etc/shibboleth i would imagine. shib knows nothing about keystone17:23
notmorgandstanek: without re-auth/federation/etc.17:23
dstanekisn't this why oauth was invented?17:24
notmorgandstanek: this is why <SSO auth of any kind> was made17:25
shewlessdstanek: if I look at my metadata its: https://foo.com/shibboleth/SAML2/POST and not https://foo.com:5000/shibboleth/SAML2/POST. do you know where the 5000 comes from?17:25
*** TxGVNN has joined #openstack-keystone17:25
notmorganand i really really am against keystone implementing it's own SSO thing with bearer tokens17:25
dstanekshewless: that must be in the saml request going out. do you have that port in any of you configs?17:26
*** jtp92 has joined #openstack-keystone17:26
shewlessdstanek: not intentionally17:27
*** rderose has quit IRC17:27
*** scarlisle has joined #openstack-keystone17:27
*** ddieterly is now known as ddieterly[away]17:28
shewlessdstanek: It's part of my OPENSTACK_KEYSTONE_URL in local_settings.py17:28
shewlessdstanek: do you know if it should be?17:28
jtp92I need help with integrating OpenStack and Active Directory, I can't seem to find any decent documentation17:28
openstackgerritMerged openstack/keystone: Fix ValidationError exception name in docstring  https://review.openstack.org/32769617:29
dstanekjtp92: as an ldap backend?17:29
scarlisleHello! If my Keystone public and internal endpoints are using v3, but my admin endpoint is v2, will that affect multi-domain? If so, how?17:29
shewlessjtp92: I found this to be useful: http://docs.openstack.org/developer/keystone/configuration.html17:29
scarlislethis is liberty, sorry for not clarifying17:30
dstanekshewless: the keystone url should have the port17:30
*** josecastroleon has quit IRC17:31
*** lmiccini has quit IRC17:31
shewlessthat's the only reference to 5000 in local_settings.py, /etc/keystone, and /etc/shibboleth :(17:31
*** permalac has quit IRC17:31
*** sdake has joined #openstack-keystone17:31
dstanekshewless: have you tried to upload the metadata again after confirming that the port isn't in it?17:31
shewlessdstanek: I can do that. I did have a look at the metadata I sent and didn't see it but it's worth a shot17:32
bknudsondstanek: yes, it's distributing keystone across multiple regions where the database is not shared due to latency.17:32
shewlessdstanek: does it matter that this part of the config in in the VirtualHost *.5000> section: WSGIScriptAliasMatch ^(/v3/OS-FEDERATION/identity_providers/.*?/protocols/.*?/auth)$ /var/www/keystone/main/$117:34
shewlessdstanek: it was in your example too17:34
*** sdake_ has joined #openstack-keystone17:35
*** aginwala has joined #openstack-keystone17:35
*** rderose has joined #openstack-keystone17:36
*** sdake has quit IRC17:37
*** lmiccini has joined #openstack-keystone17:38
dstanekshewless: yes, my example works17:40
dstanekright now you seem to be dealing with a situation where you're tell the IdP one thing and the metadata is saying something else17:41
shewlessdstanek: yeah I just don't know where that 5000 is coming from.  I'll try to reupload the metadata to see if that helps17:42
dstanekshewless: what IdP are you using?17:46
shewlessdstanek: it's adfs 2.017:48
shewlessdstanek: not by my choice!17:48
shewlessdstanek: I'm told it talks "SAML 2" though17:48
*** vgridnev has joined #openstack-keystone17:50
*** tonytan4ever has quit IRC17:52
*** browne has quit IRC17:52
dstanekshewless: i've been trying to figure out how to test adfs17:53
shewlessdstanek: I can tell you that the idp metadata is usually here: /FederationMetadata/2007-06/FederationMetadata.xml and that my IT guy doesn't really understand federation that well.. otherwise I can at least try some thing to get this setup working..17:55
*** nisha__ has joined #openstack-keystone17:56
*** chris_hultin has quit IRC17:57
*** chris_hultin has joined #openstack-keystone17:57
*** tpeoples has quit IRC17:57
*** serverascode has quit IRC17:58
*** nisha_ has quit IRC17:58
*** lhcheng has quit IRC18:00
*** lhcheng has joined #openstack-keystone18:00
*** ChanServ sets mode: +v lhcheng18:00
*** serverascode has joined #openstack-keystone18:00
*** timburke has quit IRC18:00
*** tpeoples has joined #openstack-keystone18:00
*** aginwala has quit IRC18:01
*** aginwala has joined #openstack-keystone18:02
dstanekshewless: since nothing ever 'just works' we're all in the same boat18:02
*** itisha has quit IRC18:02
*** yolanda has quit IRC18:02
*** timburke has joined #openstack-keystone18:02
*** itisha has joined #openstack-keystone18:03
*** mtreinish has quit IRC18:03
*** mtreinish has joined #openstack-keystone18:03
*** roxanaghe has quit IRC18:04
jtp92dstanek: Yes18:06
jtp92shewless: Thanks, i'll take a look!18:06
dstanekjtp92: yeah, the docs talk a lot about that. i'm sure there's some gaps though, so if you find any don't be afraid to create bugs18:07
*** roxanaghe has joined #openstack-keystone18:09
*** mtreinish has quit IRC18:10
*** rk4n has quit IRC18:12
*** mtreinish has joined #openstack-keystone18:13
*** pnavarro has quit IRC18:13
KevinE_can someone help me with a python question?18:14
* notmorgan tosses dstanek at KevinE_ :P18:14
notmorganKevinE_: ask away, we might be able to help18:15
notmorganwe might not18:15
* dstanek is terrified18:15
notmorganbut asking the question is likely easier to net an answer than asking if you can ask the question18:15
*** mgagne has quit IRC18:16
bknudsonnotmorgan: watch it'll be some odd difference between py2 and py318:16
bknudsonwhat happens if I do 11 / 2 ?18:16
*** clayton has quit IRC18:17
*** mgagne has joined #openstack-keystone18:17
openstackgerritBoris Bobrov proposed openstack/keystone: Revert "Add caching to role assignments"  https://review.openstack.org/32784818:17
amakarovnotmorgan, hi! I'm about the cache bug you and breton discussed above18:17
*** clayton has joined #openstack-keystone18:17
amakarovnotmorgan, he pointed me to your code, and I'm trying to fix the issue in oslo_cache18:18
dstanekbknudson: from __future__ import division18:18
amakarovnotmorgan, what is the better place to do that as we were able to figure out how to fix yours?18:18
amakarovIn my opinion fix to keystone is a hack, but fix in oslo.cache is terrifying :)18:19
notmorganbknudson: i'm... no. NO18:20
notmorganamakarov: fix it in dogpile.cache so hard/soft invalidate is propagated to the backend vs being a property on the region object18:20
notmorganamakarov: don't fix it in oslo_cache OR in keystone18:20
amakarovzzzeek, are you around?18:21
notmorganamakarov: btw, i am also a maintainer on dogpile.cache18:21
*** rderose has quit IRC18:21
amakarovnotmorgan, cool. then you know the workflow, right?18:21
shewlessdstanek: it appears my "entityID" isn't accessible for some reason.. "https://foo.com/shibboleth" - so I need to figure that out18:22
notmorganamakarov: gerrit.sqlalchemy.org18:22
notmorganamakarov: :)18:22
KevinE_notmorgan: dstanek: there's always some sass in this room :p here: http://pastebin.com/nPkTLnqw I commented way out the little steps to explain my issue including the error I'm getting. It should be easy :)18:22
notmorganamakarov: the source code is on bitbucket (for you to clone)18:22
dstanekKevinE_: what's the question?18:22
amakarovnotmorgan, why sqlalchemy? 0_o18:22
amakarovthat's surprising18:23
KevinE_dstanek: I'm getting error TypeError: _generate_random_name() takes exactly 1 argument (2 given) and I don't know what 2 argumements I'm "giving"18:23
notmorganamakarov: because zzzeek is the sqlalchemy maintainer18:23
*** dan_nguyen has quit IRC18:23
notmorganamakarov: so dogpile goes with him there. since he wrote dogpile as well18:23
notmorgans/maintainer/author18:23
dstanekKevinE_: Python passes the instance as self automatically and you are not expected another param, but passing one in18:24
*** tqtran has joined #openstack-keystone18:24
dstanekKevinE_: vm_name_prefix18:24
*** fawadkhaliq has quit IRC18:24
KevinE_dstanek: I thought so, but when I remove that and just make it name=self.scenario_instance._generate_random_name(), I get error TypeError: unbound method generate_random_name() must be called with VMPerf instance as first argument (got nothing instead)18:25
openstackgerritBoris Bobrov proposed openstack/keystone: Revert "Add caching to role assignments"  https://review.openstack.org/32784818:26
*** ddieterly[away] has quit IRC18:28
notmorganbreton: sorry -2, unless you address why this cannot be fixed18:28
notmorganbreton: a wholesale revert needs more justification18:29
notmorganbreton: and i'm fine with the revert, just make sure it isn't (as it sounds now, and i know you don't mean it this way) "oh error, lets revert"18:30
*** ddieterly has joined #openstack-keystone18:30
notmorganbreton: also what version of dogpile.cache are you seeing this issue with18:30
bretonnotmorgan: cache invalidation never worked for regions in catalog and assignments. It still doesn't work in stable/mitaka. Fixing it in stable/mitaka means adding a lot of calls to memcache, which i would rather not do in stable branch.18:31
bretonnotmorgan: it never worked because https://github.com/openstack/keystone/blob/stable/mitaka/keystone/common/cache/core.py#L123 does not set setter and deleter to _hard_invalidated18:31
notmorgani'm going to -2 ripping out caching at this level18:32
notmorganbreton: it clearly did work at some point18:32
notmorganbreton: so what has changed.18:32
dstanekKevinE_: is self.scenario_instance an actual instance?18:32
notmorganbreton: also, create a test that shows it doesn't work. and what we can do to fix it.18:32
bretonnotmorgan: setattr(region, '_hard_invalidated', invalidator.hard_invalidated) does not set setters and getters.18:33
*** rk4n has joined #openstack-keystone18:33
*** ayoung has quit IRC18:33
dstanekKevinE_: or are you saying you removed self too?18:33
*** rk4n has joined #openstack-keystone18:33
notmorganbreton: so.. fix it so it does ;)18:33
bretonnotmorgan: do you think the fix where it does should go to mitaka?18:33
notmorganbreton: i actually know how to fix this, it's just something i wanted to avoid doing.18:34
notmorganbreton: yes. this can be backported.18:34
notmorganbreton: it's private interfaces.18:34
shewlessdstanek: I set my entityID to "https://foo.com/shibboleth" but that page is not accessible.  Do you know how I determine what my entityID is? Where is it configured?18:34
notmorganbreton: the other option is to convert @memoize to using the .get/set methods on the dogpile region18:34
notmorganalso backportable18:34
notmorganbreton: cache fixes are almost always backportable.18:35
bretonnotmorgan: my biggest concern is that it adds a huge amount of calls to memcache. And this huge amount was not tested during the cycle.18:35
notmorganbreton: not a concern really18:35
notmorganbreton: the added load to calculating everything else is/will be much worse18:36
dstanekshewless: does it actually need to talk to that url? by convention i use http://keystone_host/shibboleth18:36
bretonnotmorgan: ok then, i'll propose a fix in 20.18:36
notmorganbreton: :) wfm18:36
notmorganbreton: don't abandon the revert18:36
notmorganleave that as a patchset separate18:36
KevinE_dstanek: Um I'm not sure how to answer that question but I'd guess yes, just because the description of that method says "Function to create an instance" lol18:37
notmorganso if we can't fix it we can just push that through18:37
notmorganbut i'd rather a real fix because i know the caching *is* in-fact important18:37
*** nisha__ has quit IRC18:37
KevinE_dstanek: I just don't know what the error message is telling me at all with your proposed removal of that var18:37
*** sdake_ has quit IRC18:38
shewlessdstanek: it looks like it uses that "relying party"18:39
dstanekKevinE_: you may have to debug it to see what it is. that error message sounds like the one you get from "Class.method()"18:39
shewlessdstanek: and I think if you try and hit "http://keystone_host/shibboleth" it'll accept your connection.. mine doesn't for some reason18:40
dstanekshewless: right. it uses that to look up the meatadata. i don't think it queries it directly18:40
*** TxGVNN has quit IRC18:40
zzzeekAmakarov: am at lunch.   If not a quick q maybe get me in a couple hours ...18:40
notmorganzzzeek: mostly its just a "make _hard/_soft invalidate" work across processes.18:40
notmorganzzzeek: enjoy lunch.18:41
dstanekshewless: in my keysotne that returns a 404. it's not a real resource18:41
amakarovzzzeek, I've discussed the issue with notmorgan - I'm preparing a patch for dogpile.cache18:41
notmorganzzzeek: i'll keep my eyes on it once it's on your gerrit.18:41
shewlessdstanek: oh18:42
notmorganbreton: the real fix might be making a descriptor object instead of a @property18:42
shewlessdstanek: maybe that's not the problem then?'18:42
bretonnotmorgan: that's what i do18:42
zzzeekAmakarov: tricky to generalize unless you have regions communicate thru some magic value in the cache .    That would be an extension point.   Not sure if there's a recipe for something like that18:43
notmorganbreton: cool. also the alternative is to subclass region and make keystone use that with _hard/_soft invalidate fixed.18:43
notmorganzzzeek: i think the easiest thing is to make _hard/_soft invalidate functions that can be passed in like keygen18:43
notmorganamakarov: ^18:43
notmorganzzzeek: with the default action being the one we have today18:44
amakarovzzzeek, having issues sending CR to your gerrit :)18:44
zzzeekNotmorgan: I've always felt that the total key space needs to be specified.   That is, never assume some other system isn't also using your cache.  Therefore you have to log all the keys.   There's a recipe that does something like this in the docs18:44
dstanekshewless: the message you are getting is what i got when the metadata i registered didn't match the things in the SAML doc18:45
notmorganzzzeek: correct.18:45
notmorganzzzeek: i am just saying we should allow the hard/soft region invalidates to be passed in as functions, so you can say "use the driver to check the last invalidatate" or some such.18:45
notmorganzzzeek: the same as we do for things like should_cache_fn. anyway go lunch18:46
notmorganzzzeek: we can chat after that18:46
shewlessdstanek: okay. I checked the metadata I sent and it definitely does not include the port 5000.  I think that maybe my system is using that port for some reason.  Maybe adfs is more picky if it comes in on a different port then, say, a shibboleth idp?18:46
zzzeekOf course .   I'm all for pushing responsibility outside :)18:46
dstanekshewless: i think a shib IdP is just as picky. try to find out what the SAML looks like. maybe debug level in mod_shib?18:47
amakarovnotmorgan, is there some step-by-step guide how to set up git-review to work with gerrit.sqlalchemy.org?18:47
*** ayoung has joined #openstack-keystone18:47
*** ChanServ sets mode: +v ayoung18:47
notmorganamakarov: it shouod work if the .git-review file is right18:47
amakarovnotmorgan, and it is not in a fresh clone :)18:48
notmorganhttps://bitbucket.org/zzzeek/dogpile.cache/src/c6fc55cd04f3a9e7e1301c21fa54c2e17d918c8b/.gitreview?fileviewer=file-view-default18:48
notmorganlooks like it needs a port18:49
notmorganunless gerrit is listening on 22 for that host18:49
notmorganhttps://github.com/openstack/keystone/blob/master/.gitreview [for comparison]18:49
shewlessdstanek: would the SAML show up in keystone.log?18:50
KevinE_dstanek: I think it was just because I had @classmethod above the method ??18:50
shewlessdstanek: likely /var/log/shibboleth/ I guess18:50
amakarovnotmorgan, yes, it tries to get to 29418 port and .git-review doesn't help18:54
notmorgan.gitreview18:54
notmorganamakarov: and did you setup your account there?18:54
notmorganamakarov: you need to login (github sso), and add ssh-key etc18:54
amakarovI've used openid and added an rsa key18:54
notmorgani just pushed this change https://gerrit.sqlalchemy.org/#/c/106/18:54
dstanekshewless: no. keystone doesn't do anything with the saml in the usecase. it's between mod_shib and the IdP18:54
notmorganand it worked18:54
dstanekKevinE_: sure a classmethod means that instead of an instance for the first arg you will get the class18:55
dstanekKevinE_: that is almost certainly not what you want unless you are creating alternative constructors18:55
KevinE_dstanek: so I think that was just leftover from the older code I'm modifying :)18:56
*** gordc has quit IRC18:57
dstanekKevinE_: i'm not sure what the original code was and what your changes work, but i'm guessing that the method was not supposed to be a classmethod since it expects self and used it to call an instance method. it's also broken because _generate_random_name doesn't accept any arguments18:58
*** sdake has joined #openstack-keystone18:59
shewlessdstanek: I think I found the saml: <samlp:AuthnRequest xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol" AssertionConsumerServiceURL="http://mycloud.foo.com:5000/Shibboleth.sso/SAML2/POST" Destination="https://saml.foo.com/adfs/ls/" ID="fsdgf16d6a2fdc5db35" IssueInstant="2016-06-09T18:59:48Z" ProtocolBinding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Version="2.0"><saml:Issuer xmlns:saml="urn:oasis:names:tc:SAML:2.19:01
KevinE_dstanek: http://pastebin.com/XzjHZi7E Right before the ~~ you can see the original method. Andrey told me that due to the new updates to generate_random_name method, I would need to change that method to call self.generate_random_name from scenario instead of utils.generate_random_name. He said I should pass nothing to the latests generate_random_name method.19:02
shewlessdstanek: the "AssertionConsumerServiceURL" has 5000 in it. I think that's the problem.. I don't think this is coming from the metadata.. do you?19:02
openstackgerritBoris Bobrov proposed openstack/keystone: Fix cache invalidation  https://review.openstack.org/32788519:02
KevinE_dstanek: basically I just took his advice and changed it to the super short version you saw, and I suppose you're telling me that I did it in an incorrect way. Do you know what I was supposed to do?19:02
openstackgerritBoris Bobrov proposed openstack/keystone: Fix cache invalidation  https://review.openstack.org/32788519:03
amakarovnotmorgan, it denies me: Please make sure you have the correct access rights and the repository exists.19:03
amakarovnotmorgan, fatal: Upload denied for project 'zzzeek/dogpile.cache'19:03
*** rderose has joined #openstack-keystone19:04
*** tonytan4ever has joined #openstack-keystone19:04
bretonnotmorgan: ^19:05
*** lhcheng has quit IRC19:06
bretoni am actually surprised that it works...19:06
notmorganbreton: nah i'm not19:07
*** darosale has joined #openstack-keystone19:07
bretonnotmorgan: i am because it didn't work in my sandbox and i had to do things with .__class__19:08
amakarovnotmorgan, only launchpad sso works there or github will go?19:08
notmorganbreton: inline comments on that19:08
amakarovnotmorgan, oh, you've told about github19:08
notmorganamakarov: bitbucket.19:09
notmorganamakarov: for the code, github for the SSO login19:09
notmorganon gerrit.sqla19:09
dstanekKevinE_: what are you trying to do?19:10
amakarovnotmorgan, well, I've used github sso to log it, cloned a project from there, manually added gerrit remote and get "Upload denied"19:10
amakarovwhile I'm able to ssh there (without session of course)19:11
dstanekshewless: so i think that's generated in mod_shib. after you removed the :5000 from the metadata did you restart shibd and apache?19:11
amakarovso my key works19:11
openstackgerritBoris Bobrov proposed openstack/keystone: Fix cache invalidation  https://review.openstack.org/32788519:11
notmorganbreton: so we are going to need a subclass i think for this to work.19:11
*** aginwala has quit IRC19:11
notmorganbreton: local test is showing wrong behavior still.19:11
notmorgandescriptors are part of the class object not instantiated object19:12
*** harlowja has quit IRC19:12
dstaneknotmorgan: descriptors...tasty19:12
notmorgandstanek: yeah19:13
bretonnotmorgan: yes, that's why i was surprised19:13
notmorganbreton: you might be able to just set the hard_invalidate, etc on the region class itself before we do the first instantiation19:13
KevinE_dstanek: I have scenario files I'm trying to merge up. In testing, I'm tracing errors around because it's set in a pretty old version of rally. I had an issue where a custom plugin was trying to generate a name using said old version, and I guess my I can't update it to work with the new code19:14
notmorganyou also may be able to just apply it to the region class itself directly even post instantiation19:14
notmorganbreton: yep19:15
openstackgerritBoris Bobrov proposed openstack/keystone: Fix cache invalidation  https://review.openstack.org/32788519:15
bretonnotmorgan: ^19:15
notmorganbreton: just setattr it on the region object itself19:15
notmorganeven post instantiation, all childern will get it19:15
dstanekKevinE_: i don't know much about rally, but i can say that you can't have class method call an instance method like that since it doesn't have access to the instance19:15
*** aginwala has joined #openstack-keystone19:15
notmorganbreton: /me just tested this19:15
bretonnotmorgan: yep, i've changed that to do things with classes19:15
*** aginwala has quit IRC19:16
*** aginwala has joined #openstack-keystone19:16
* breton testing on the env19:16
shewlessdstanek: I've restarted apache and shibd many times.. :) but I never had :5000 in my metadata.. that's what's confusing me.. it wasn't there and it's not there19:16
notmorganbreton: nope, you don't want to do it that way19:17
notmorganyou want to do the apply one time19:17
notmorganand do it before we instantiate a region19:17
*** ctracey has quit IRC19:17
dstanekshewless: this issue seems not that it was in the metadata, but that it's being used in th authn request19:17
notmorganbasically do the apply_patch thing once during import of common.cache19:17
bretongot it19:18
notmorganbreton: and just do it to the dogpile.cache region object directly, delete the unbound _hard/soft invalidated and apply these descriptors19:18
*** itlinux has joined #openstack-keystone19:18
shewlessdstanek: right. I couldn't find anything in any configs that reference 500019:18
notmorganbreton: ooooor just monkey patch the unbound properties directy before instantiation19:18
notmorganbreton: and don't bother with descriptors (either works) -- also... lets make sure we get a test of this.19:19
*** sdake has quit IRC19:19
KevinE_dstanek: I know and I always feel bad for asking in this room, however I do get helpful information and I guess that's the real point of it all. I've found myself to a new totally Rally error so I'll just assume it's progress :)19:19
*** ctracey has joined #openstack-keystone19:20
shewlessdstanek: right. I couldn't find anything in any configs that reference port 5000. Accept that local_settings.py, glance, heat, etc for auth19:20
dstanekKevinE_: i don't mind answering non-keystone questions :-) in this case i can tell you why your Python code doesn't work, but not what it should be19:21
notmorganbreton: also you might need to reference obj not "self" in the descriptor19:21
*** lhcheng has joined #openstack-keystone19:22
*** ChanServ sets mode: +v lhcheng19:22
bretonnotmorgan: got it19:22
amakarovzzzeek, do I need some special permissions to upload change request to zzzeek/dogpile.cache?19:22
*** rderose has quit IRC19:22
openstackgerritMerged openstack/keystone: Refactor shadow users  https://review.openstack.org/32359619:22
notmorganbreton: lots of comments to this effect too help make sure it's on the review as to why it's being done a specific way19:24
*** roxanaghe has quit IRC19:25
*** rderose has joined #openstack-keystone19:27
shewlessdstanek: are you sure your request doesn't also add a 5000 but shibboleth is okay with that? I've tried everything I can think of.. I don't know what's putting the :5000 there19:28
bretonnotmorgan: won't other regions, like the one @MEMOIZE is using, be affected if i monkey-patch before instantiating things?19:30
*** aginwala has quit IRC19:30
notmorganbreton: we don't actually care :)19:30
bretonnotmorgan: why not?19:30
notmorganbreton: we really want all hard/soft invalidates in keystone to back to the cache19:30
notmorganthe setattr was just a cheap way of doing that w/o getting into descriptors/metaprogramming19:30
notmorganbreton: and we applied it only where we wanted it. in this case we apply it everywhere19:31
zzzeekAMakarov: yes19:31
notmorganand it's just fine to do so19:31
amakarovnotmorgan, are you sure you are not granted some special permissions to push CR's ?19:31
openstackgerritMerged openstack/keystone: Shadow LDAP and custom driver users  https://review.openstack.org/32360219:31
zzzeekamakarov: send a PR and i can pull it into gerrit and give you perms19:31
notmorganamakarov: ah. ok so zzzeek has that setup like that.19:32
amakarovzzzeek, PR?19:32
notmorganamakarov: on bitbucket19:32
notmorganamakarov: send the change as pull-request19:32
dstanekshewless: are you able to query the IdP and get back the metadata it's using?19:32
dstanekshewless: i can give it a try and see19:32
zzzeekamakarov: notmorgan should have perms also19:33
shewlessdstanek: I have it's generic "idp" metadata..19:33
shewlessdstanek: but do you mean the metadata it's using for me?19:33
shewlessdstanek: I've seen the "assertion url" stuff on the idp end and none of it has port 500019:34
notmorganzzzeek: i have perms to push to dogpile.cache :)19:34
*** rderose has quit IRC19:34
* notmorgan proved that with a recent patch.19:35
dstanekshewless: my metadata uses :5000 for all URLs19:35
shewlessdstanek: interesting... like actually in your metadata or just in your SAML?19:36
dstanekin metadata19:36
shewlessdstanek: okay that might be an easier problem... I suppose I could "hack" the metadata but I'd prefer to figure out how that part of the metadata is generated.19:37
dstanekshewless: the shibboleth2.xml i posted yesterday uses 500 in the urls19:42
shewlessdstanek: I lost my history from yestreday would you be able to repost the link?19:43
bknudsonshewless: irc logs are here: http://eavesdrop.openstack.org/irclogs/19:44
*** rderose has joined #openstack-keystone19:44
amakarovzzzeek, done19:45
shewlessbknudson: thanks.. cool!19:46
shewlessdstanek: I had the apache configuration you posted but I can't see the shibboleth2.xml paste anywhere from yesterday19:46
dstanekshewless: jas i'll post again19:48
*** harlowja has joined #openstack-keystone19:49
*** amakarov is now known as amakarov_away19:50
dstanekshewless: http://paste.openstack.org/show/509301/19:51
shewlessdstanek: thanks.. I'll give that a try19:52
*** pnavarro has joined #openstack-keystone19:53
*** rderose_ has joined #openstack-keystone19:54
*** rderose has quit IRC19:54
*** roxanaghe has joined #openstack-keystone19:55
dolphmnonameentername: any progress on that transient?19:56
*** jtp92 has quit IRC19:57
nonameenternameYeah, I believe I fixed the issue.  I'm running it on my machine right now19:57
dstanekshewless: let me know if that works19:58
*** tonytan4ever has quit IRC19:58
openstackgerritRon De Rose proposed openstack/keystone: Add password table columns to meet PCI-DSS change password requirements  https://review.openstack.org/31428419:59
*** amrith is now known as _amrith_20:05
*** gordc has joined #openstack-keystone20:06
openstackgerritBoris Bobrov proposed openstack/keystone: Fix cache invalidation  https://review.openstack.org/32788520:07
nonameenternamedolphm: is there a bug for that change?20:08
bretonnotmorgan: ^ it doesn't work though20:08
bretonnotmorgan: > an alternative is to directly monkeypatch the hard/soft invalidated properties/methods/whatever with new code without being a descriptor20:08
bretonhow's that?20:08
bretonooh, so just set dogpile.cache.CacheRegion._soft_invalidated?20:09
notmorganbreton: yes20:09
notmorganbreton: that is basically what you're going to do patching the region object at import time20:10
*** dan_nguyen has joined #openstack-keystone20:10
bretonyep20:10
shewlessdstanek: after I udpate the entityID to have port 5000 when I download the metadata I can see the change in the entityID but not in the "md:AssertaionConsumerService" line in the metadata20:10
shewlessdstanek: do all of your md:AsserationConsumerService Binding" in your metadata have :5000 in them?20:11
dstanekshewless: yes, all URLs have the port in my configuration20:11
bretonnotmorgan: so the patch ^ does the logic, and i see in the logs a lot of checks of _RegionExpiration.hard.None. However, the cache doesn't work properly.20:11
bretoni am figuring out why20:11
shewlessdstanek: the <MetadataProvider is for the IDP right?20:12
notmorganbreton: ++ but you see how the MRO falls through to the class object20:12
*** ddieterly is now known as ddieterly[away]20:12
dstanekshewless: in my case yes20:12
notmorganbreton: you're def. on the path i'd be using to fix this fwiw.20:13
notmorganbreton: so looking good, and this is 100% backportable.20:13
bretonwhat's MRO?20:13
notmorganMethod Resolution Order20:13
bretonoh ok.20:13
notmorganhow python determins how to call what20:13
notmorganbreton: i could go into a lot of detail that would be generally boring on IRC on how MRO is handled.20:14
notmorganbreton: but :)20:14
bretoni know, thanks ^)20:14
notmorganin short... if it's not in __dict__, not on the parent (you need to delattr the thing from the region still), it falls through to the class_obj20:15
notmorgani think.20:15
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS change password requirements - Password table changes  https://review.openstack.org/31428420:15
notmorgananyway i need to chase a different bug. thanks for working on this.20:15
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS change password requirements: SQL changes  https://review.openstack.org/31428420:16
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Change password requirements - SQL changes  https://review.openstack.org/31428420:16
*** dan_nguyen has quit IRC20:19
*** adrian_otto has joined #openstack-keystone20:20
shewlessdstanek: I cannot for the life of me get the md: lines to have :5000 in them. Is there some other xml file in your /etc/shibboleth/ that would be doing that for you?20:21
shewlessdstanek: or maybe it inherits from the entityID?20:21
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Password strength requirements  https://review.openstack.org/32058620:22
dstanekshewless: what do you mean?20:23
shewlessdstanek: when I generate my metadata, only the entityID has port 5000.. the md:assertion stuff at the end does not say port 500020:23
dstanekshewless: i just have those urls in my shibboleth2.xml file20:23
shewlessdstanek: in my metadata I have this: <md:AssertionConsumerService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://mycloud.foo.com/Shibboleth.sso/SAML2/POST" index="1"/>20:24
shewlessdstanek: I don't see any such line in your shibboleth2.xml file20:24
shewlessdstanek: but it's missing the :500020:24
dstanekshewless: metadata != shibboleth2.xml20:25
openstackgerritwerner mendizabal proposed openstack/keystone: Fix TOTP transient test failure  https://review.openstack.org/32792220:25
dstanekshewless: 162.242.221.169:5000/Shibboleth.sso/Metadata20:25
*** adrian_otto1 has joined #openstack-keystone20:25
nonameenternamedolphm: https://review.openstack.org/#/c/327922/20:26
patchbotnonameentername: patch 327922 - keystone - Fix TOTP transient test failure20:26
dolphmnonameentername: awesome, looking20:26
shewlessdstanek: but that is your idp isn't it?20:26
dstanekno, i'm using testshib as an idp20:27
shewlessdstanek: you have 3 lines that say "MetadataProvider" .. I didn't udnerstand that part20:27
dstaneki've been experimenting with dynamic federations so there is some lingering cruft in there20:28
*** aginwala has joined #openstack-keystone20:28
*** adrian_otto1 has quit IRC20:28
*** adrian_otto has quit IRC20:28
dolphmnonameentername: is it the delta between _generate_totp_passcode() and v3_create_token() that causes the issue? or what two methods20:29
*** adrian_otto has joined #openstack-keystone20:30
dstanekshewless: for now just try to upload the corrected metadata and see if it works20:31
nonameenternamedolphm: yes, there is a change that totp gets created and by the time it gets to the server it is no longer valid.20:31
shewlessdstanek: yes good idea.. just waiting for my IT guy to do it..20:31
nonameentername*chance20:31
dstanekshewless: :-( that's a bummer20:32
shewlessdstanek: yup. luckily this adfs server is pretty much the only thing I don't have control over.. so it should be better after this20:32
*** jdennis has quit IRC20:33
*** jdennis has joined #openstack-keystone20:34
shewlessdstanek: btw none of the instructions say port 5000 as far as I can tell: http://docs.openstack.org/developer/keystone/federation/shibboleth.html20:34
shewlessdstanek: probably because that's where keystone is listening I guess20:34
dolphmnonameentername: left a comment - you might have missed one?20:35
*** ddieterly[away] is now known as ddieterly20:35
*** rcernin has joined #openstack-keystone20:36
nonameenternamedolphm: that call expects a failure.  I didn't think it was necessary.20:36
dolphmnonameentername: ah, gotcha20:37
dolphmnonameentername: how easy is the race condition to reproduce? i tried running that test suite about 10 times and it passed every time20:37
bretonnotmorgan: i found why it doesn't work. On get() the key doesn't get hashed and on set() it does.20:37
dolphmi.e. how frequently does it fail20:37
nonameenternamedolphm: I had to set a break point to see it fail20:38
nonameenternameI couldn't reproduce otherwise20:38
bretonnotmorgan: oooh, i know why.20:40
dolphmnonameentername: while we wait for jenkins, i'm going to run those two tests in a loop (without your patch) and see if i can get a failure20:40
*** adrian_otto1 has joined #openstack-keystone20:41
openstackgerritRon De Rose proposed openstack/keystone: PCI-DSS Password strength requirements  https://review.openstack.org/32058620:43
*** adrian_otto has quit IRC20:44
dolphmnonameentername: failed on attempt #12720:48
dolphmnonameentername: with a 40120:48
nonameenternamedolphm: can you check to see if it fails with the patch20:49
shewlessdstanek: didn't work.. adfs is still expecting the "assertion" stuff without port 500020:49
notmorganbreton: i knew you could do it!20:49
shewlessdstanek: for now I have downloaded the metadata and manually added the port 5000 stuff. I will get that uploaded to the idp and see if that works.20:50
shewlessdstanek: BTW these guys had to do the same thing (manually edit to port 5000...) http://www.ibm.com/developerworks/cloud/library/cl-keystone-tfim/20:50
*** neophy has joined #openstack-keystone20:52
*** ametts has quit IRC20:52
*** aginwala has quit IRC20:56
*** jbell8 has quit IRC20:56
openstackgerritBoris Bobrov proposed openstack/keystone: Fix cache invalidation  https://review.openstack.org/32788520:57
*** adrian_otto1 has left #openstack-keystone20:58
*** pauloewerton has quit IRC20:58
bretonmordred: could you please test ^ on your env? Don't forget to restart memcached before testing.20:58
bretonmordred: oops, not you21:00
bretonnotmorgan: ^ you21:00
*** aginwala has joined #openstack-keystone21:00
*** aginwala has quit IRC21:01
*** aginwala has joined #openstack-keystone21:07
notmorganbreton: it will be a bit, i don't have a canned env to work from atm.21:09
notmorganbreton: and trying to watch a zuul bug down.21:09
*** vgridnev has quit IRC21:11
*** rcernin has quit IRC21:15
*** lhcheng has quit IRC21:25
*** gagehugo has quit IRC21:25
*** pushkaru has quit IRC21:28
*** pushkaru has joined #openstack-keystone21:29
*** edmondsw has quit IRC21:29
*** _amrith_ is now known as amrith21:31
*** ayoung has quit IRC21:34
*** ayoung has joined #openstack-keystone21:35
*** ChanServ sets mode: +v ayoung21:35
bretonnotmorgan: take your time, thank you.21:36
*** lhcheng has joined #openstack-keystone21:38
*** ChanServ sets mode: +v lhcheng21:38
*** aginwala has quit IRC21:38
notmorganbreton: fwiw, that looks good to me pre-testing21:39
bretoni think we could use our functional tests to run it.21:41
notmorganbreton: we could easily test it with the fake caching tests21:42
notmorganbreton: you create a cache region, and inspect the dict afdter setting hard-invalidate21:42
notmorganand confirm ._hard_invalidated is the descriptor object21:43
notmorganthis is purely-unit testable :)21:43
notmorgansince it doesn't matter what the backend for dogpile is, just that the backend gets the value on the appropriate key21:43
*** BjoernT has joined #openstack-keystone21:46
notmorganbreton: does invalidated = region.backend.get(self._get_region_key(region)) need the keymangler wrapper?21:47
notmorganor are we at the point the keymangler has been applied already ...21:47
notmorganon a get?21:47
*** roxanaghe has quit IRC21:49
bretonnotmorgan: is seems that i broke something in the last minutes21:53
bretonor uploaded the wrong patch...21:53
*** gordc has quit IRC21:53
notmorganbreton: hehe happens to the best of us, i think this is a sign a unit test is needed :)21:54
bretonyep, uploaded the wrong patch21:55
*** ddieterly is now known as ddieterly[away]21:58
*** rderose_ has quit IRC22:03
*** pushkaru has quit IRC22:04
*** ayoung has quit IRC22:05
*** pushkaru has joined #openstack-keystone22:05
*** pushkaru has quit IRC22:07
*** pumarani__ has joined #openstack-keystone22:07
*** pumarani__ has quit IRC22:08
*** jamielennox is now known as jamielennox|away22:09
*** KevinE_ has quit IRC22:11
*** roxanaghe has joined #openstack-keystone22:11
*** darosale has quit IRC22:15
*** timcline has quit IRC22:18
*** timcline has joined #openstack-keystone22:19
*** timcline has quit IRC22:23
*** pnavarro has quit IRC22:29
*** ddieterly[away] is now known as ddieterly22:29
*** edtubill has quit IRC22:30
dstanekshewless: any luck?22:33
*** ddieterly is now known as ddieterly[away]22:34
*** neophy has quit IRC22:37
openstackgerritBoris Bobrov proposed openstack/keystone: Fix cache invalidation  https://review.openstack.org/32788522:44
*** edtubill has joined #openstack-keystone22:47
*** BjoernT has quit IRC22:52
*** iurygregory_ has joined #openstack-keystone22:52
lbragstaddstanek bknudson henrynash notmorgan dolphm see last few comments - https://review.openstack.org/#/c/265455/22:57
patchbotlbragstad: patch 265455 - keystone - Expose bug with fernet and trusts22:57
lbragstadvery much the first iteration22:57
lbragstadso improvements can be made, but this is what I have so far22:58
*** ddieterly[away] is now known as ddieterly22:58
*** ddieterly has quit IRC22:59
*** edtubill has quit IRC22:59
*** ddieterly has joined #openstack-keystone23:00
*** ddieterly has quit IRC23:00
*** edtubill has joined #openstack-keystone23:01
bknudsonlbragstad: osic performance bot!23:02
bknudsonlbragstad: this is really going to run the tests twice for every patch set?23:03
bknudsonseems like the master link could point to the tests that were run before.23:04
bknudsonalso, when posting results for "master", should include the commit hash23:04
bknudsonthe performance results might be affected by changes in libraries, too...23:16
*** raddaoui has quit IRC23:27
kfox1111we're seeing quite a bit more overhead in sql vs ldap token creation and validation.23:28
kfox1111on the ldap side.23:28
kfox1111I think that's somewhat to be expected on the creation case.23:28
kfox1111but on validation too?23:28
kfox1111maybe group related?23:29
kfox1111does it reach out to ldap ever for validation?23:29
lbragstadbknudson yeah - all good points23:31
lbragstadthings that we can add it if you want to great github issues for them23:31
lbragstadin*23:31
lbragstadbknudson right now the bot will only schedule changes that a comment left saying 'check performance'23:32
lbragstadi have to hook up the last couple bits end to end though23:32
*** jamielennox|away is now known as jamielennox23:37
openstackgerritRon De Rose proposed openstack/keystone: Drop EPHEMERAL user type  https://review.openstack.org/29663923:44
*** chlong has quit IRC23:45
jamielennoxbknudson: hey if you're here can you do a quick look at an oslo patch for me? https://review.openstack.org/#/c/326782/23:52
patchbotjamielennox: patch 326782 - oslo.middleware - Expose sample config opts for http-proxy-to-wsgi23:52
jamielennoxsuper simple but i'm pushing the middleware out to services so i want to know it's not going to change underneath me23:53
*** roxanaghe has quit IRC23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!