Tuesday, 2018-03-13

kmalloceandersson: not that i am aware00:06
*** itlinux has quit IRC00:14
*** oikiki has joined #openstack-keystone00:30
*** oikiki has quit IRC00:44
*** zhurong has joined #openstack-keystone00:46
*** oikiki has joined #openstack-keystone00:48
*** Dinesh_Bhor has joined #openstack-keystone00:52
*** harlowja has quit IRC01:10
wxylbragstad: kmalloc : I forgot morgan has already done it. ;)01:14
kmallocwxy: not all of it :P01:14
*** germs has joined #openstack-keystone01:15
wxykmalloc: mind me to add the test code for it, if you don't have time?01:16
kmallocplease take it over if you want :)01:16
wxykmalloc: cool~01:16
kmallocif there is anything you dislike about it, please feel free to adjust it.01:17
kmallocit's 100%  demo-ish code01:17
wxykmalloc: OK, i'll test it first01:17
kmallocok, well it might be way more than demo code01:17
kmallocwow, i just reviewed what i wrote and ... uh. it's kinda more featureful than I remember01:17
kmallocalso, needs docs01:18
wxykmalloc: sure.01:18
kmallocand you can drop all the v2 magic ;)01:18
*** germs has quit IRC01:19
kmalloci think it's going to be a much better catalog to deploy with than the DB version in a lot of cases.01:19
kmallocbut, you know, I'm biased towards not-api-driven for more "Static" environments01:20
*** nicolasbock has quit IRC01:38
*** namnh has joined #openstack-keystone01:39
*** nicolasbock has joined #openstack-keystone01:40
*** zhurong has quit IRC01:45
*** gongysh has joined #openstack-keystone01:45
*** zhongjun has joined #openstack-keystone01:54
*** r-daneel has quit IRC02:03
*** rcernin has quit IRC02:04
*** r-daneel has joined #openstack-keystone02:11
*** annp has joined #openstack-keystone02:13
*** zhurong has joined #openstack-keystone02:26
*** itlinux has joined #openstack-keystone02:30
*** nicolasbock has quit IRC02:32
*** oikiki has quit IRC02:50
*** oikiki_ has joined #openstack-keystone02:51
*** oikiki_ has quit IRC02:58
*** rcernin has joined #openstack-keystone03:07
*** germs has joined #openstack-keystone03:15
*** germs has quit IRC03:15
*** germs has joined #openstack-keystone03:15
*** edmondsw has joined #openstack-keystone03:18
*** germs has quit IRC03:20
*** edmondsw has quit IRC03:22
*** gongysh has quit IRC03:47
*** harlowja has joined #openstack-keystone03:48
*** links has joined #openstack-keystone03:50
*** dave-mccowan has quit IRC03:56
*** gyee has quit IRC03:59
*** daidv has joined #openstack-keystone04:01
*** gongysh has joined #openstack-keystone04:07
*** gongysh has quit IRC04:08
*** gongysh has joined #openstack-keystone04:20
*** gongysh has quit IRC04:21
*** gongysh has joined #openstack-keystone04:21
*** gongysh has quit IRC04:22
*** daidv has quit IRC04:22
*** gongysh has joined #openstack-keystone04:22
*** daidv has joined #openstack-keystone04:22
*** gongysh has quit IRC04:25
*** zhurong has quit IRC04:53
*** Dinesh_Bhor has quit IRC05:00
*** Dinesh_Bhor has joined #openstack-keystone05:01
*** edmondsw has joined #openstack-keystone05:06
*** edmondsw has quit IRC05:10
*** Dinesh_Bhor has quit IRC05:11
*** germs has joined #openstack-keystone05:16
*** germs has quit IRC05:16
*** germs has joined #openstack-keystone05:16
*** Dinesh_Bhor has joined #openstack-keystone05:19
*** germs has quit IRC05:21
*** jaosorior has quit IRC05:22
*** oikiki has joined #openstack-keystone05:25
*** itlinux has quit IRC05:26
*** karthi has joined #openstack-keystone05:27
*** karthi has quit IRC05:30
*** oikiki has quit IRC05:36
*** oikiki has joined #openstack-keystone05:38
*** zhurong has joined #openstack-keystone05:42
*** karthi has joined #openstack-keystone05:45
*** harlowja has quit IRC05:59
*** gongysh has joined #openstack-keystone06:02
*** daidv has quit IRC06:04
*** david-lyle has quit IRC06:08
*** oikiki has quit IRC06:12
*** Dinesh_Bhor has quit IRC06:21
*** Dinesh_Bhor has joined #openstack-keystone06:23
*** germs has joined #openstack-keystone06:29
*** germs has quit IRC06:29
*** germs has joined #openstack-keystone06:29
*** pcichy has joined #openstack-keystone06:29
*** oikiki has joined #openstack-keystone06:30
*** wxy_ has joined #openstack-keystone06:37
*** d0ugal has joined #openstack-keystone06:48
*** germs has quit IRC06:52
*** edmondsw has joined #openstack-keystone06:54
*** zhurong has quit IRC06:55
*** openstackgerrit has joined #openstack-keystone06:56
openstackgerritOpenStack Proposal Bot proposed openstack/keystoneauth master: Updated from global requirements  https://review.openstack.org/54953606:56
*** edmondsw has quit IRC06:59
*** jaosorior has joined #openstack-keystone07:02
*** karthi has quit IRC07:19
openstackgerritOpenStack Proposal Bot proposed openstack/oslo.policy master: Updated from global requirements  https://review.openstack.org/54955907:22
*** karthi has joined #openstack-keystone07:22
*** rcernin has quit IRC07:24
openstackgerritOpenStack Proposal Bot proposed openstack/python-keystoneclient master: Updated from global requirements  https://review.openstack.org/54956507:26
*** oikiki has quit IRC07:32
*** jdennis has quit IRC07:36
*** martinus__ has joined #openstack-keystone07:39
*** karthi has quit IRC07:40
*** pcaruana has joined #openstack-keystone07:42
*** AlexeyAbashkin has joined #openstack-keystone07:50
*** links has quit IRC07:52
*** links has joined #openstack-keystone07:59
*** tesseract has joined #openstack-keystone08:19
*** tesseract-RH has joined #openstack-keystone08:20
*** tesseract-RH has quit IRC08:20
*** sticker has joined #openstack-keystone08:34
*** karthi has joined #openstack-keystone08:37
*** r-daneel has quit IRC08:41
*** edmondsw has joined #openstack-keystone08:42
*** edmondsw has quit IRC08:46
*** r-daneel has joined #openstack-keystone08:48
openstackgerritwangxiyuan proposed openstack/keystone master: Deprecate the templated catalog  https://review.openstack.org/48271409:25
openstackgerritwangxiyuan proposed openstack/keystone master: [WIP]Add yaml-loaded filesystem catalog backend  https://review.openstack.org/48351409:25
*** mburrows has quit IRC09:30
*** mvk has quit IRC09:32
*** Dinesh_Bhor has quit IRC09:34
*** dmellado has joined #openstack-keystone09:47
*** karthi has quit IRC09:56
*** karthi has joined #openstack-keystone09:57
*** namnh has quit IRC09:58
*** edmondsw has joined #openstack-keystone10:30
*** edmondsw has quit IRC10:35
*** pcaruana has quit IRC10:45
*** mvk has joined #openstack-keystone11:00
*** rcernin has joined #openstack-keystone11:02
*** nicolasbock has joined #openstack-keystone11:06
hugokuoMay I know if the validate token API still available in Queens release?11:22
cmurphyhugokuo: it is not, and that reference is not our official api reference and seems to be very outdated11:25
cmurphyhugokuo: here is our official api reference: https://developer.openstack.org/api-ref/identity/11:25
*** karthi has quit IRC11:33
*** dave-mccowan has joined #openstack-keystone11:42
*** gongysh has quit IRC11:43
*** edmondsw has joined #openstack-keystone11:45
*** rcernin has quit IRC12:00
*** karthi has joined #openstack-keystone12:04
*** raildo has joined #openstack-keystone12:04
*** annp has quit IRC12:07
hugokuocmurphy: thx. Then I have to update our keystone middleware version. The current version seems not support v3 format in the configuration.12:10
cmurphyhugokuo: keystonemiddleware has supported keystone v3 for a very long time, there might be some other issue in your configuration12:13
*** pcichy has quit IRC12:20
*** karthi has quit IRC12:24
*** karthi has joined #openstack-keystone12:26
*** pcichy has joined #openstack-keystone12:29
hugokuoyou are right. We are running keystone-middleware 1.5.0+ which should have v3 support. Let me dig into it.  Thx for advice.12:31
cmurphyno problem12:32
*** zhurong has joined #openstack-keystone12:36
*** odyssey4me has quit IRC12:41
*** odyssey4me has joined #openstack-keystone12:41
*** zhurong has quit IRC12:42
*** zhurong has joined #openstack-keystone12:45
*** jdennis has joined #openstack-keystone12:59
*** HW-Peter has joined #openstack-keystone13:08
*** HW-Peter has quit IRC13:10
*** HW-Peter has joined #openstack-keystone13:11
*** zhurong has quit IRC13:15
*** jaosorior has quit IRC13:19
*** gongysh has joined #openstack-keystone13:28
*** FuslJH95HW has joined #openstack-keystone13:29
*** FuslJH95HW has quit IRC13:31
*** jroll has quit IRC13:41
*** jroll has joined #openstack-keystone13:42
lbragstadjust a reminder that the team meeting is going to be a couple hours earlier today - http://eavesdrop.openstack.org/#Keystone_Team_Meeting13:44
lbragstadall information is updated and an .ics is available ^13:45
cmurphyheh i almost forgot13:47
cmurphybad timing to switch it right after american DST :P13:47
ayounglbragstad, cmurphy so...I was reading the notes from the PTG, specifically those about "OpenStack at the Edge" and I notice that it matches a few use cases my team has addresses recently13:50
ayoungWhat I was wondering was if we could start looking at Keystone to Keystone replication, with the assumption that the Keystone instances are not all at the same level, nor have write access to the same data13:50
ayoungas a starting point, lets assume we say we have 3 sites: Boston, London, Berlin13:51
*** ionel has joined #openstack-keystone13:51
ayoungAnd, we have 3 regions, one per site13:51
ayoungwe also say we have 3 domains, also one per site13:51
ayoungeach site is allowed to add users or role assignments to its domain13:51
ayoungthis will then get synced to the other Keystone instances...lets handwave how for now13:52
ionelGood day! Can anyone help me with TOTP authentication ? The official documentation didn't help.13:52
ayoungbut each will not only be responsible for pushing the data, but tracking the state of that push13:52
ayoungionel, just ask your question.  If someone here knows, they will answer.  If not, well...crickets13:53
cmurphyayoung: what do you mean by "tracking the state of that push"?13:54
ayoungcmurphy, so, say we dump a load of users into Dom-Boston13:54
ayoungand those need to get synced to Dom-Berlin....They got one by one, and we can, at any point, as Dom-Boston is the cmurphy user in Region:Berlin  yet?13:55
ayoungsorr  as -> ask13:55
*** jaosorior has joined #openstack-keystone13:55
ionelayoung: I am looking for basic TOTP + user/pass authentication. I am sure that someone should know.13:56
ayoungoperator queries https://boston/v3/state-of-sync?user=cmurphy or something like that13:56
ayoungionel, don't be so sure.13:56
ayoungionel, the folks that worked on that are not always in here, but post what you did13:57
ayoungand what the state is, and maybe someone can answer.13:57
ayoungmake it easy for us to help you13:57
ayoung'cuz I know Bupkis about our TOTP impl.13:57
ayoungjust read https://docs.openstack.org/keystone/pike/advanced-topics/auth-totp.html13:58
ayoungnow I am an expert13:58
lbragstadcmurphy: agreed...14:00
*** karthi has quit IRC14:01
ayounglbragstad, what time is the meeting?14:02
ionelI have read that. I was about to post the link.14:02
ayoungis it now?14:02
ionelI am stuck at create TOTP credentials14:02
ionelwhare do I put that code ?14:02
ionelthere is not mentioned any file14:03
ayoungionel, the CURL call?14:03
cmurphyionel: that is a shell example, you'd type it into your terminal14:03
ayoungthat can be done via the CLI too, I am pretty sure.  openstack help credential create14:04
ionelI just did14:04
ionel{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}14:05
ayoungionel, the user that makes that call needs a token14:05
lbragstadayoung: 1600 UTC14:05
ionelwhere I setup the token ?14:05
ayoungso admin does it for the first user, or the user auths via password when they do it, or sometjhing like that14:05
ayoungionel,  as admin create a user, set the users password.  Create a keystone rc for that user./  source the keystonerc, make the call.14:06
ayounghttp://adam.younglogic.com/2018/02/keystonerc-for-rdo-cloud/  is an example template14:06
ayoungcmurphy, that curl call does not have a way to authenticate documented:14:07
ayoungcurl -i \14:07
ayoung  -H "Content-Type: application/json" \14:07
ayoung  -d '14:07
cmurphyionel: that example might have been written before we had client support, it looks like that was added: https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/credential.html14:07
cmurphyayoung: you're right it's missing a -H X-Auth-Token14:08
ayoungionel, so, add in the following steps:14:09
ayoungTOKEN=`openstack token issue -f json | jq -r '.id'`14:09
ayoungthen in the curl call:14:10
ayoungcurl -i \14:10
ayoung  -H "Content-Type: application/json" \14:10
ayoung  -d '14:10
ayoung -H X-Auth-Token $TOKEN14:10
ionelI am working on it14:11
ionelThank you :)14:11
cmurphyayoung: okay back to your topic, I think i misread your question at first, you're not talking about Keystone to Keystone in the federation sense are you?14:14
ionelI'll get back to you later. I have to go now. Thank you for all your help.14:17
*** spilla has joined #openstack-keystone14:17
ayoungcmurphy, not really, no14:17
ayoungcmurphy, more in the Its all one big deployment, but too big to upgrade at once sense14:18
ayoungI'd like to start thinking of Regions as things that can vary somewhat independently14:18
ayoungcmurphy, it was use cases like this that pushed me toward PKI for tokens in the first place. If you could replace the symmetric crypto of fernet with asym, you could update the keys for different regions separately, know which key was owned by which region, etc.  But that is not essential.14:21
*** nicolasbock has quit IRC14:22
cmurphylbragstad: ^ maybe another case for unencrypted jwt14:23
*** felipemonteiro has joined #openstack-keystone14:29
*** nicolasbock has joined #openstack-keystone14:32
*** nicolasbock has quit IRC14:41
*** nicolasbock has joined #openstack-keystone14:43
lbragstadsorry - just parsed the scroll back14:49
lbragstadayoung: so you want to gradually upgrade?14:51
lbragstadlike one region at a time?14:51
*** wxy| has joined #openstack-keystone14:54
*** wxy| has quit IRC14:55
*** wxy| has joined #openstack-keystone14:56
*** idlemind has quit IRC14:56
*** gongysh has quit IRC14:57
*** gongysh has joined #openstack-keystone15:01
*** gongysh has quit IRC15:02
ayoungsorry...dog walk intervened15:16
ayounglbragstad, yes, gradual upgrades, one region at a time15:16
ayounglbragstad, assume you have 50 regions.  Bringin the whole cloud down is a big disruption15:17
*** sticker has quit IRC15:17
ayounglbragstad, cmurphy this was what I was reading.  http://markvoelker.github.io/blog/dublin-ptg-edge-sessions/15:19
lbragstadayoung: does the current token provider require that?15:25
lbragstadbringing the whole cloud down?15:25
*** david-lyle has joined #openstack-keystone15:25
ayounglbragstad, If everything is doing Fernet?  I guess it depends on the changes we make from release to release15:26
*** jmlowe has quit IRC15:27
lbragstaddon't you just need to make sure the key repository is the same for each region?15:27
ayoungFernet validation is the same.  But if you are syncing data across multiple Keystone instances via DB replication, and the tables are out of sync, yeah, it would be a disaster15:27
*** jmlowe has joined #openstack-keystone15:27
lbragstadi've actually heard that data replication, especially in EU, is tough15:27
ayoungI'm starting to think we want to give people an option to not replicate at the DB level.  Many people have tried to avoid it over the years.15:28
ayoungHence RFEs like "Add a project with a specific ID"15:28
lbragstadk2k was suppose to solve that15:28
ayoungk2k doesn't sync data, though15:29
ayoungand K2K is useless if the first K is down.15:29
lbragstadso you want users to be able to use any keystone - but you don't want to replicate data?15:32
*** jaosorior has quit IRC15:34
ayounglbragstad, I want to repliact.  But eventually consistent semantics15:34
lbragstadsorry - i thought you just said you wanted to try and give people an option to not replicate at the DB level15:35
ayounglbragstad, right15:36
ayounglbragstad, replicate at the application level.  Deliberately15:36
lbragstadso building replication into keystone's api15:37
ayounglike, Each region gets a domain, and they are only allowed to write data for their own domain,  so they can then push that to the other servers15:37
ayoungand a Keystone to Keystone push mechanism15:37
ayoungIf we did it at the API level, then we would need overrides for things like "write this resource with this explicit ID"15:38
ayoungso, maybe treat the K2KSync as a separate endpoint and API15:38
*** itlinux has joined #openstack-keystone15:45
*** harlowja has joined #openstack-keystone15:46
*** gyee has joined #openstack-keystone15:51
*** links has quit IRC15:55
*** david-lyle has quit IRC15:57
*** dklyle has joined #openstack-keystone15:57
lbragstadping ayoung, breton, cmurphy, dstanek, gagehugo, henrynash, hrybacki, knikolla, lamt, lbragstad, lwanderley, kmalloc, rderose, rodrigods, samueldmq, spilla, aselius, dpar, jdennis, ruan_he, wxy16:00
lbragstadreminder that we'll be having the meeting in #openstack-meeting-alt today16:00
*** harlowja has quit IRC16:06
*** links has joined #openstack-keystone16:08
openstackgerritRussell Tweed proposed openstack/keystone master: Add prerequisite package note to Keystone install guide  https://review.openstack.org/55256816:14
*** links has quit IRC16:30
*** wxy|_ has joined #openstack-keystone16:33
*** wxy| has quit IRC16:34
*** r-daneel has quit IRC16:42
* cmurphy runs home before office hours16:51
openstackgerritJohannes Grassler proposed openstack/keystone-specs master: Add whitelist-extension-for-app-creds  https://review.openstack.org/39633116:53
lbragstadjgr: i'm glad you're going to be giving us a hand this release, happy to have you be a part of the team!16:54
jgrlbragstad: thanks...happy to be along myself :-)16:55
jgrlbragstad: (and happy I finally get to tackle this white list thing...not having this keeps being an issue with all sorts of things)16:56
*** thomasduval has joined #openstack-keystone17:01
*** r-daneel has joined #openstack-keystone17:01
hrybackiI'm amazing my pee isn't citric acid after having eaten so many oranges this past week and a half17:02
hrybackiamazed*. Pee may be normal but my brain isn't -_-17:02
lbragstad#startmeeting keystone-office-hours17:06
openstackMeeting started Tue Mar 13 17:06:57 2018 UTC and is due to finish in 60 minutes.  The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot.17:06
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.17:06
*** openstack changes topic to " (Meeting topic: keystone-office-hours)"17:06
*** ChanServ changes topic to "Queens release schedule: https://releases.openstack.org/queens/schedule.html | Meeting agenda: https://etherpad.openstack.org/p/keystone-weekly-meeting | Bugs that need triaging: http://bit.ly/2iJuN1h | Trello: https://trello.com/b/wmyzbFq5/keystone-rocky-roadmap"17:06
openstackThe meeting name has been set to 'keystone_office_hours'17:07
lbragstadwe do have a bluejeans session going17:07
lbragstadfolks can join at any time17:07
cmurphybluejeans is where?17:07
lbragstad#link https://bluejeans.com/855901362317:08
lbragstadcmurphy: did you already make it home?17:08
*** d0ugal has quit IRC17:09
hrybackigagehugo: wxy kmalloc knikolla ^^17:09
*** pcichy has quit IRC17:18
*** wxy|_ has quit IRC17:19
*** pcichy has joined #openstack-keystone17:21
*** felipemonteiro_ has joined #openstack-keystone17:29
*** felipemonteiro has quit IRC17:31
*** AlexeyAbashkin has quit IRC17:34
aninglbragstad, cmurphy, knikolla: for the deadlock issue in 014 of upgrade, I'm experimenting a solution. Basiaclly in 014_contract_add_domain_id_to_user_table.py, I check if the uniqueconstraints and foreignkeyconstraints have created or not, and only create them if they don't exist yet. This way, the keystone-manage db_sync contract can be re-run and proceed  to the next transaction.17:36
aningsince deadlock is rare, so a re-run should succeed.17:37
aningThe code (in debugging mode) is posted here: http://paste.openstack.org/show/700152/17:38
*** r-daneel has quit IRC17:38
aningCould you please have a look and see if there is any problem in there, and your concerns about the solution overall ?17:39
*** mvk has quit IRC17:43
lbragstadcmurphy: kmalloc http://paste.openstack.org/show/700177/17:48
*** spilla has quit IRC17:51
*** spilla has joined #openstack-keystone17:55
*** felipemonteiro_ has quit IRC18:05
*** oikiki has joined #openstack-keystone18:08
openstackgerritErik Olof Gunnar Andersson proposed openstack/keystone master: Fixing multi-region support in templated v3 catalog  https://review.openstack.org/48236418:10
*** r-daneel has joined #openstack-keystone18:16
*** thomasduval has quit IRC18:17
*** r-daneel_ has joined #openstack-keystone18:25
*** jaosorior has joined #openstack-keystone18:25
*** r-daneel has quit IRC18:26
*** r-daneel_ is now known as r-daneel18:26
*** mvk has joined #openstack-keystone18:28
cmurphyaning: if it doesn't consistently fail then i think retrying makes sense18:29
cmurphyaning: i'm confused that just re-running the contract phase doesn't just work though18:29
cmurphyaning: if you haven't already can you open a bug for it?18:30
*** harlowja has joined #openstack-keystone18:30
*** harlowja has quit IRC18:30
aningNo. If I got a deadloack the first run, when re-run contract, it will fail and complain UniqueConstraint relation "ixu_user_id_domain_id" already exists18:32
aningsince the contract is not a atomic transaction.18:33
aningit fails (deadlock in this case) at certain point, when re-run it will try to do some of the transactions already done in the previous run.18:34
aningcmurphy: yes, I'll open a bug for it.18:37
*** d0ugal has joined #openstack-keystone18:38
*** felipemonteiro has joined #openstack-keystone18:51
cmurphyaning: i see, so then yes adding some idempotency in that migration makes sense to me18:52
aningideally the whole contract, or at least each of the step (014 for example), should be in a atomic transaction.18:54
aningor like I'm experimenting, just check before doing any transactions to ensure it doesn't repeat what have been done before.18:55
*** felipemonteiro_ has joined #openstack-keystone18:55
*** r-daneel_ has joined #openstack-keystone18:58
lbragstadstepping away to grab tea quick - brb18:58
*** r-daneel has quit IRC18:58
*** r-daneel_ is now known as r-daneel18:58
*** AlexeyAbashkin has joined #openstack-keystone18:58
*** felipemonteiro has quit IRC18:59
*** itlinux has quit IRC18:59
*** AlexeyAbashkin has quit IRC19:03
*** d0ugal has quit IRC19:04
*** itlinux has joined #openstack-keystone19:04
*** tesseract has quit IRC19:11
*** abhi89 has joined #openstack-keystone19:14
abhi89Hi All.. I see that in queens release, we have changed 'auth_uri' to 'www_authenticate_uri' (https://review.openstack.org/#/c/508522/) to avoid the confusion with 'auth_url'.. why haven't we changed 'auth_uri' in places like https://github.com/openstack/cinder/blob/master/cinder/quota_utils.py#L2819:19
abhi89if someone updates to queens, its likely that the cinder quota utils code will fail.. there are other references of 'auth_uri' too apart from this..19:20
cmurphyabhi89: we didn't remove auth_uri, we just deprecated it19:20
cmurphywe do still need to find all those places that reference it but in the meantime it should still work19:21
abhi89cmurphy: yes.. that means if we don't change our [keystone_authtoken] section to have 'www_authenticate_uri', everything will work fine.. but if we do, then this quota_utils code will fail as there won't be any 'auth_uri' parameter..19:22
cmurphyyeah :(19:25
*** d0ugal has joined #openstack-keystone19:26
*** d0ugal has quit IRC19:26
*** d0ugal has joined #openstack-keystone19:26
cmurphyabhi89: did you try it and run into an error? I'm reading through oslo.config and it seems like it might just do the right thing but that might just be wishful thinking19:36
abhi89cmurphy: i didnot try but I think it will not find 'auth_uri'19:43
*** raildo has quit IRC19:58
ayounghttps://adam.younglogic.com/2013/07/a-vision-for-keystone/  I think I still stand by this.  Coming up on 5 years.20:00
ayoungcmurphy, what was that abount unencrypted JWT?20:03
cmurphyayoung: http://specs.openstack.org/openstack/keystone-specs/specs/keystone/backlog/json-web-tokens.html20:05
cmurphyayoung: https://review.openstack.org/#/c/54190320:05
cmurphycurrent ideas in there are to encrypt the token same as we do with fernet20:05
cmurphybut that requires sharing the private keys between keystones20:06
cmurphyso we were thinking of just signing and not encrypting20:06
ayoungcmurphy, Out of curiousity, has anyone sized a JWT token with asym key signing?20:06
ayoungwhen I last looked in to it, a fernet sized block signed with asym crypto came out to about 1K20:08
ayoungI think signed without encrypting the body of a JWT makes sense.  But it will leak a little bit of information.  Still, it is no worse than if someone sniffs a token today....I think....I'd have to ruminate on that20:09
cmurphyayoung: no I'm not aware of any of us actually doing a PoC on the tokens to size them, though the jwt reference says "smaller than saml"20:10
cmurphyayoung: it is slightly worse than if someone sniffs a token today20:10
ayoungcmurphy, signing requires a key20:10
ayoungsigning requires encryption, it is just a smaller encrypted document;  only the hash is encrypted.20:10
ayoungdoesn't avoid the need for a key20:11
cmurphyit's slightly worse because if i have an expired encrypted token i can't return that to keystone to get information about it20:11
ayoungfor the same data, JWT is going to be smaller than SAML only (I think) due to the more concise document format20:12
ayoungproblem for PKI tokens was the service catalog20:12
ayoungI was of the opinion that we should stick with the data that is in the Fernet body, and the keystonemiddleware could fetch and cache any data it needed in order to expand the tokens:  project names from IDs, etc.20:13
ayoungit has all the keys distribution problems of PKI if you do, though, so I didn't push on it.20:14
*** openstack changes topic to "Queens release schedule: https://releases.openstack.org/queens/schedule.html | Meeting agenda: https://etherpad.openstack.org/p/keystone-weekly-meeting | Bugs that need triaging: http://bit.ly/2iJuN1h | Trello: https://trello.com/b/wmyzbFq5/keystone-rocky-roadmap"20:15
openstackMeeting ended Tue Mar 13 20:15:55 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)20:15
openstackMinutes:        http://eavesdrop.openstack.org/meetings/keystone_office_hours/2018/keystone_office_hours.2018-03-13-17.06.html20:15
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/keystone_office_hours/2018/keystone_office_hours.2018-03-13-17.06.txt20:15
openstackLog:            http://eavesdrop.openstack.org/meetings/keystone_office_hours/2018/keystone_office_hours.2018-03-13-17.06.log.html20:16
ayoungcmurphy, for example, lets got with the data in most tokens:  UserID, ProjectID, IssueTime:20:17
ayoungI think that is the minimium for a scoped token20:17
*** oikiki has quit IRC20:19
*** oikiki has joined #openstack-keystone20:20
ayoung$ cat token_body.json20:20
ayoung{"user_id":"d627bfb24c3043478a69fd8ade1009e9","project_id":"0b5df1731e67462393dea23b8c24c860","issued_at": "20180313162018"}20:20
ayoungOK, to encryopt that20:27
ayoung openssl aes-256-cbc -in token_body.json -out message.enc20:27
ayoungand base64 gives20:27
ayoungbase64 message.enc20:27
ayoung195  characters20:28
cmurphyfernet token is 184 characters it looks like20:29
ayoungYeah, so a touch longer due to JSON20:29
ayoung base64 file.ssl | wc -c20:31
ayoungthat is from asym20:31
ayounggenerated keys using20:31
ayoungopenssl genrsa -des3 -out private.pem 204820:31
ayoungGenerating RSA private key, 2048 bit long modulus20:32
ayoung  not triple DES20:32
ayoungencrypted using openssl rsautl -encrypt -inkey public.pem -pubin -in token_body.json -out file.ssl20:32
ayoungand then base64 encoding20:32
ayoungthat seems short20:32
ayoungBut, double the length for PKI is probably about right20:33
ayoungsymmetric will be faster, and take fewer cycles.  With asym, you still need to distribute the public keys to anything that needs to validate20:35
ayoungbut with asym non-trusted systems could validate.20:36
kmallocayoung: asym or sym, you're still doing live validation of the data20:37
kmalloci don't really care which encyrption we use.20:37
*** harlowja has joined #openstack-keystone20:37
kmallocbut you're going to have to ask keystone in either case. I care that the token is small, we don't shove more than the payload of fernet (items) in it, and that it validates reasonably fast.20:37
kmallocif that helps the conversation at all20:38
ayoungkmalloc, yep.  One benefit of the Fernet approach is you don't need to hash, just encrypt20:43
ayoungasym helps with the key distribution side of things, and you could build a validation service out of the existing APIs.20:44
*** d0ugal has quit IRC20:45
lbragstadi wonder if it would be worth it to have a config option for signing versus encryption20:46
gagehugolbragstad hrybacki made it home yesterday afternoon20:50
lbragstadgagehugo: nice! how was the vacation?20:50
lbragstadrefreshing i hope20:50
gagehugovery refreshing!20:51
gagehugoit was the opposite of snowpenstack20:52
*** mburrows has joined #openstack-keystone20:52
lbragstadsunny, relaxing, and good wifi then?20:52
gagehugoactually yes20:52
gagehugowas much better wifi than I expected20:53
gagehugoI think it was ~80F every day20:53
gagehugoocean was too cold to swim in still, but got to see some whales20:53
lbragstadgagehugo: are you still on vacation then, recovering?20:54
lbragstador are you officially back now?20:54
gagehugoI'm off today20:55
gagehugoI'm back tomorrow20:55
ayounglbragstad, cmurphy on Whitelist/ApCreds, would you be dead set against adding in a Role ID to the url Pattern?20:57
lbragstadwhat's the purpose?20:57
ayoungAs a way of telling a user: here are the URL patterns you can delegate20:57
ayoungI know it could potentially be more than one role,20:58
ayoungbut that would kindof be the missing link between my spec and the whitelist spec20:58
ayoungso say you want to delegate POST, COMPUTE,  /v2.4/server_create/,20:59
felipemonteiro_ayoung: if you have a moment sometime may you consider removing procedural -2 from https://review.openstack.org/#/c/464678/20:59
felipemonteiro_by no means is it done, still need to update it20:59
ayoungit would tell you "you can do that if you have the _member_ role20:59
kmallocayoung fernet hmacs too20:59
ayoungfelipemonteiro_, it still scares me, but...I'm a trust you.21:00
cmurphyayoung: i don't think we can reasonably manage that at this stage, i don't think we have all the pieces in place to connect roles to urls21:00
lbragstadthat'd be duplicating the source of truth between permissions and role mappings, i think21:00
ayounglbragstad, so we already have a duplicate source of truth here21:00
ayoungthis would be more "guidance"21:01
ayoungcmurphy, I think we do, though21:01
ayoungcmurphy, we can still generate a policy file for nova etc, right?21:01
kmallocayoung: so you are hashing in either case.21:01
ayoungkmalloc, it does?  WHy?21:01
cmurphyayoung: we can generate one but we can't pull the actual one that the operator has configured from nova21:02
ayoungkmalloc, I thought it was just encrypted.  If the doc is short, why hash?21:02
ayoungcmurphy, yeah, but in general, that is not a problem.  Its only a problem if they go super duper customized, and we can walk people through how to manually update it from policy...I think21:02
ayoungactaully, we still can't automatically map from URL to policy, but figuring that out is a one time cost21:03
ayoungcmurphy, what if we made the role_id optional?21:04
ayoungonly enforce it IF it is set?21:04
kmallocayoung: because encryption is not sufficient for authentication of a payload21:04
ayoungkmalloc, for a short payload?  Of course it is21:04
kmallocfernet cannot assume the payload is short.21:05
kmallocWE force a short payload21:05
ayoungkmalloc, ah21:05
kmallocthat doesn't mean the generic use case is short21:05
kmallocand even in the case of short payloads, best practices: hash as well for authentication of message21:05
ayoungpretty sure that all Hash tells is if the unencrypted has been changed, but if you trust the signer, that is not any additional information.21:07
ayoungkmalloc, the short of it is, we need tokens to be short.  I arbitrarily set that size to be 1K21:08
ayoungAlthough, to be honest, we didn't see problems in PKI tokens until we got to 8 K21:09
kmallocexcept PKI tokens were VERY slow even at the smaller sizes21:09
kmallocwell, no take that back21:09
kmallocthey weren't that slow21:10
ayoungkmalloc, yeah, although part of that might have been the fork21:10
kmallocyeah the fork was slow21:10
kmallocbut whatever, lets set that aside21:10
kmallocthe token was fine aside from fork21:10
kmallocthe other issue is the added overhead of transit.21:10
kmallocasking for a 4k object from swift21:10
kmallocif i add 1k every single time, thats a 25% overhead21:10
kmallocadded overhead*21:10
eanderssonIs the gate unhappy?21:11
kmallocthe fernet max length was set to help make swift happier with our tokens21:11
lbragstadeandersson: i did see some announcements yesterday pertaining to the gate21:11
eanderssonah I see21:11
cmurphyayoung: okay, to put it concretely, if the operator creates the pre-approved whitelist with (POST, /v2.1/hypervisors, admin) and user bob with role _member_ tries to create an application credential with whitelist (POST, /v2.1/hypervisors) they'll get a 404?21:11
ayoungthe overhead for a Fernet token seems to be 184 characters long.  What would the acceptable size growth be beyond that?21:11
lbragstadeandersson: i think it was related to some new zuul configs21:11
kmalloc255bytes was pretty much the agreed upon target21:11
eanderssonlooks like doc building is broken21:12
kmallocthough i could probably say we could hit 512b and top out there.21:12
eandersson> /home/zuul/src/git.openstack.org/openstack/keystone/doc/source/api/keystone.contrib.rst:document isn't included in any toctree21:12
cmurphybut if the operator-approved list has (POST, /v2.1/hypervisors, None) and user bob tries to create the application credential for that it will succeed and just be useless21:12
kmallocideally we keep the payload / token as light as possible21:12
ayoungcmurphy, yes, that is what I just proposed.21:12
kmallocit's why we have different formatters, so we don't have to wedge everything in at once21:12
ayoungnot sure I love it, but...21:12
kmallocjust the relevant data for that type (etc)21:12
lbragstadi think we can make that simpler with jwt21:13
cmurphyayoung: okay, i'm not totally opposed but it does seem like it's duplicating different sources of truth21:13
kmallocbut tl;dr on the hash, -- ciphertext attacks are real21:13
ayounglbragstad, I think all of this is still the same with JWT21:13
kmallochashing makes that much less likely to be possible.21:13
lbragstadayoung: jwt builds payloads as key/value pairs21:13
kmallocand smaller payloads are less cryptographically distinct21:13
kmalloc(less entropy)21:13
lbragstadwhere as keytone's fernet token formatter is relying on a order in a tuple21:13
kmalloclbragstad: it might increase token size some, but i think it wont be awful.21:14
kmalloclbragstad: because k/v is more than encoded(v)21:14
lbragstadyeah - i would expect it to21:14
lbragstadand it might not be msgpack'd21:14
ayoungcmurphy, agreedd, but I think we have that no matter what.  If a user does the s (POST, /v2.1/hypervisors, None)  it will pass or fail based on policy.21:14
kmallocthe only reason we're msgpacking is because of limits in json seralization21:15
ayoungall we are doing with the role is adding in guidance to say "don't do that"21:15
lbragstadso what happens when the mapping in keystone is out of date from what an operator specifies for a policy in nova?21:15
kmallocmy answer remains: the token fails21:16
ayounglbragstad, something breaks21:16
kmallocthe answer to make that most configurable and least painful21:16
kmallocallow wildcards.21:16
ayoungOK...which convo do we want to have first21:16
* lbragstad loves context switching21:16
kmalloclbragstad: you're evil21:17
ayounglbragstad, ok, here was my slim JWT format token:21:17
ayoung{"user_id":"d627bfb24c3043478a69fd8ade1009e9","project_id":"0b5df1731e67462393dea23b8c24c860","issued_at": "20180313162018"}21:17
cmurphyplaying with https://jwt.io/ the size can blow up way quite a bit depending on algorithm used, best case is about 25321:17
cmurphyer the debugger on that page21:18
*** d0ugal has joined #openstack-keystone21:18
*** d0ugal has quit IRC21:18
*** d0ugal has joined #openstack-keystone21:18
ayoungthat gave me a 246 character token21:19
ayoungwhat would we be using for Algo etc?21:20
cmurphyunspecified as of yet21:20
ayoung533  with asym?21:21
lbragstadfernet uses a SHA256 HMAC signing key21:22
lbragstadwhich i think is the equivalent of HS25621:22
ayoungSo...do we really hate fernet?21:22
ayoungCuz...I kindof think as defined, it suits our needs21:22
ayoungI'd like for Asym to be an alternative for peopel with key distro issues21:23
lbragstadi don't think we hate it - but i think people see the value in offering a second alternative21:23
ayoungbut I dion't see a benefit to JSON in the body without making it for other people to read21:23
lbragstadand the asymmetric bit is interesting21:23
kmallocfor us, first steps to using JWT is because it is a standard with more eyes on it21:24
kmalloceven if it's opaque like fernet is in our use case21:24
ayoungShould we offer it as an Outreachy internship project?21:24
kmallocwe can explore further token data things usable by other folks when we're on it21:24
kmallocand i'm open to that21:24
lbragstadi know wxy and i are interested in working on this - so is gagehugo21:25
lbragstadwe've gotten a couple requests internally for it21:25
ayoungWant it, or want to work on it?21:25
cmurphyi think it would be a great potential outreachy project as long as we nail down the implementation details beforehand21:26
ayoungSo...no opposition here, so long as we keep Fernet for the meanwhile. Changing token options is painful, and I can't see a huge benefit, so minimial churn is all I really want.21:26
lbragstadayoung: ++21:27
ayoungBut I can accept the "future proof" ourself with JWT argument as a good rationale.21:27
lbragstadi have no intention of changing the default token provider, or removing fernet21:27
cmurphyyeah, we don't hate fernet21:27
* lbragstad pats fernet on the shoulder21:28
*** bhagyashris_ has joined #openstack-keystone21:28
lbragstadi would just hate to be in a position where something happens with the fernet spec (which is somewhat unmaintained despite our efforts to participate in that community)21:28
lbragstadand keystone doesn't have a back up21:28
kmalloci want an option that saves us from "OMG FERNET IS CRITICALLY BROKEN AND WE CANT FIX IT"21:28
lbragstad^ that21:29
kmallocand let folks who want more standard-y things utilize standard things.21:29
kmallocbut mostly the all caps bit21:29
cmurphyalso more opportunity to integrate with non-openstack technologies21:29
ayoungOK...so, back to RBAC and Whitelists21:30
ayoungI still get pings from people interested in RBAC in middleware21:30
ayoungthat struck a chord with people.21:31
*** bhagyashri_s has quit IRC21:31
*** mburrows has quit IRC21:31
ayoungI'd like to be able to move things that way, but I can accept that we have to be incremental21:31
*** mburrows has joined #openstack-keystone21:31
ayoungso what I want to get from the Whitelist is something that moves us that way while still getting the current use cases covered21:31
ayoungSo, saying that the whitelist will still be run through existing RBAC is fine21:32
ayoungWe can still work on splitting the RBAC into scope check vs role check, hopefully based on the data we generate to populate the whitelist21:33
ayoungI suspect that it won't be THAT much.21:34
cmurphyayoung: i think i'm fine with this21:34
ayoungAnd it will be pretty obvious, on an URL by URL basis what role the user really should have21:34
ayoungI think I can get you a set of VERBS + URL Patterns for Nova, Keystone, Glance, Neutron, and Cinder based on the public docs and some HTML scraping21:36
*** AlexeyAbashkin has joined #openstack-keystone21:36
kmallocwe're not going to encode anything like that in keystone by default.21:37
*** d0ugal has quit IRC21:37
kmallocit will have to be on an operator / distribution to provide that information21:37
cmurphyayoung: can we say for now that there's an optional role field in the operator-approved list and leave the details of how to generate the list until the next stage?21:37
ayoungkmalloc, right.  I was thinking more like we produce documents that we can post upstream that can be consumed by the operators as a starting point21:38
kmallocsure, i wont be opposed to that, just as long as we aren't encoding anything in keystone itself by default.21:39
*** AlexeyAbashkin has quit IRC21:41
ayoungkmalloc, yeah, this should be "layered on top of" as opposed to "replacing" to avoid breaking things21:44
openstackgerritRussell Tweed proposed openstack/keystone master: Add prerequisite package note to Keystone install guide  https://review.openstack.org/55256821:52
abhi89cmurphy: so is there anything planned to remove the remaining references of "auth_uri" in R release?21:54
ayoungfelipemonteiro_, ok....so tell me how patrole is going to check RBAC?21:59
*** rcernin has joined #openstack-keystone21:59
cmurphyabhi89: i don't think we have anything explicitly planned, partly i was hoping the other projects would take care of it themselves ;) but you're right it's important and we should follow through on it22:03
cmurphyabhi89: i don't think we can be held responsible for every project though22:07
*** abhi89 has quit IRC22:07
cmurphygoing through http://codesearch.openstack.org/?q=auth_uri&i=nope&files=&repos= a lot of these are deployment projects that should be keeping on top of this themselves22:07
*** abhi89 has joined #openstack-keystone22:07
cmurphywe can at least make sure devstack works when switched and file bugs for the projects that break it22:10
felipemonteiro_ayoung: it calls oslo.policy for expected result, invokes tempest api endpoint that enforces the expected policy, compares expected/actual results for consistency22:12
cmurphylbragstad: hrybacki can/should we pull https://trello.com/c/AXrDT1s0/49-keystonemiddleware-improvements as an epic onto the rocky roadmap? it has some unfinished items, plus followup on www_authenticate_uri impact plus removing all the deprecated crap could be captured there22:13
ayoungfelipemonteiro_, how does it know what URL to call for a policy rule?22:14
felipemonteiro_policy in code documentation for all services that have converted over22:15
felipemonteiro_only missing one is neutron22:15
ayoungfelipemonteiro_, link?22:16
ayoung"policy in code documentation"  gives the URL pattern?22:16
felipemonteiro_nova example: https://github.com/openstack/nova/blob/master/nova/policies/agents.py#L3522:16
felipemonteiro_keystone example: https://github.com/openstack/keystone/blob/master/keystone/common/policies/group.py#L3122:17
ayoungSo we are going to be going with the roles as hard coded into the source.  Wonderful22:17
ayoungfelipemonteiro_, before you do that, I have a bug for you to fix22:17
openstackLaunchpad bug 968696 in OpenStack Identity (keystone) ""admin"-ness not properly scoped" [High,In progress] - Assigned to Adam Young (ayoung)22:17
hrybackicmurphy: will do22:18
felipemonteiro_well the patrole gates just check for the default roles currently established in openstack -- it's designed to check for any role you want though22:18
*** martinus__ has quit IRC22:18
ayoungfelipemonteiro_, and it is outside of the proejcts22:20
ayoungso if they change the RBAC, patrole will complain22:20
ayoungand it will make it hard to fix the Longest standing bug in openstack22:21
*** edmondsw has quit IRC22:21
felipemonteiro_well, there's a bit of ptg background discussion you missed, but the intention is to have it non-voting and, moreover...22:22
ayoungfelipemonteiro_, this is just so bad...it is exactly tjhe opposite of what we need22:22
ayoungfelipemonteiro_, your heart is in the right place22:22
felipemonteiro_even if it is voting (hypothetically) then in-repo zuul.yaml can be changed quickly to relegate back to non-voting22:22
ayoungfelipemonteiro_, please no?22:22
ayoungfix first22:22
ayoungthen patrole22:22
felipemonteiro_the background discussion fyi is that lance and redhat talked about it and asked that i join a session to discuss gating around it in keystne22:23
ayoungfelipemonteiro_, do you understand the bug I linked to?22:23
felipemonteiro_i understand the gist of it, which is that if you have admin in one project you have it universally22:24
ayoungfelipemonteiro_, at lease wait until after lbragstad gets a system scoped rule in there22:24
ayounglbragstad, what is the policy to replace "role:admin",22:24
ayoungwith system scoped roles?22:24
felipemonteiro_well, i'm not trying to jump the gun. i will update the spec and let it play out.22:24
felipemonteiro_it's entirely up to you ayoung (and fellow cores) whether it gets merged :)22:24
ayoung"role:admin and something:system",22:24
ayoungfelipemonteiro_, do you understand the problem of bug 968696 and how patrole will make it harder to fix?22:25
openstackbug 968696 in OpenStack Identity (keystone) ""admin"-ness not properly scoped" [High,In progress] https://launchpad.net/bugs/968696 - Assigned to Adam Young (ayoung)22:25
ayoungfelipemonteiro_, if you don't understand that bug, please go read up on it.  It really should be a CVE, but the world has known about it for ever.22:26
ayoungIts really bad22:27
ayoungand it comes up in just about every conversation around policy22:27
felipemonteiro_well, i agree with you to an extent. but pragmatically speaking, i highly doubt any keystone cores will refuse to fix the issue keystone-side simply because a non-voting gate fails.22:27
ayoungcmurphy, do you know what the "grandfather in the admin " approach is going to be for system roles/22:27
ayoungfelipemonteiro_, until someone makes it non-voting22:28
ayoungfelipemonteiro_, that happend with Tempest to us in the past22:28
ayoungpeople coded tests based on broken behavior, and then we were in a catch 22 to keep us from fixing it22:29
felipemonteiro_i wouldn't advocate for it to even be voting, not for at least another 2 cycles (neutron hasn't gone to policy in code yet).22:29
felipemonteiro_but regardless, i don't really have an opinion here on when it should be added to keystone as a gate. that is for you guys to decide. i just need to update the spec with the discussion feedback from ptg.22:30
*** spilla has quit IRC22:30
cmurphyayoung: you mean for getting things like 'list all servers in all projects' to work? i think both you and kmalloc are advocating leveraging inherited roles for this, we'll just need to document that better22:31
cmurphyayoung: we had some discussion on it on https://trello.com/c/L09dRjq4/73-associating-projects-to-system-scoped-tokens22:31
* kmalloc reads that22:31
kmalloci.. am not sure i'm advocating that.22:31
ayoungcmurphy, not that22:31
ayoungcmurphy, I was talking about how the nova policy for is_admin is currently22:32
ayoungand it needs to be role:admin and token-scope:system22:32
kmallocayoung: i think with system-scope we're planning to work with nova to migrate from role:admin to something system-scope-y22:32
ayoungI had the is_admin_project hack in there22:33
kmallocbut yes something like that first thennnnn moving away from role:admin longer term22:33
ayoungwhich worked because all projects show up is_admin until you enforce a specific one in Keystone config22:33
cmurphyayoung: ah sorry, we'll work with them to change their policy rules https://trello.com/c/XcqlDqWN/43-work-with-nova-to-implement-scopetypes22:33
ayoungso what is the plan.  Please tell me there is a plan.  This delayed the fix a year at this point.22:33
ayoungSo unless we make it OPT IN we are going to break people that upgrade22:34
kmallocwhat cmurphy linked.22:34
kmallocthe plan: start with both functioning22:34
kmalloclong term is as nova becomes more aware of system-scope (for example) migrate to system-scope only. it's a long play22:34
ayoungkmalloc, how do we migrate?22:35
kmallocsupport both22:35
kmallocliek you said, to start22:35
ayoungwe need something that works for both new and old, with an opt in switch22:35
kmallocwith an EOL on the non-system-scope mode22:35
kmallocthat *is* the migration22:35
kmallocsupport both, make sure everything works with system-scope only (openstack-specific), set EOL on non-system-scope, EOL non-system-scope in accordance with timeline22:36
kmallocallow for opt in along the way to system-scope only22:36
ayoungSo that code there is not sufficient22:36
kmallocthe code implemented is getting system-scope working22:37
ayoungIIUC, it will break when a user comes in with an admin role on the wrong project22:37
kmallocit is not meant to be all inclusive as it sits afaict.22:37
kmallocimplementation is a multi-cycle effort22:37
ayoungI think we need to start with is_admin at the base level22:37
kmalloclet alone the rest.22:37
lbragstadfelipemonteiro_: o/22:37
lbragstadi have something for you22:37
kmallocno, we need to have functional system scope that *can* is_admin, do not start with is_admin22:38
kmallocthat is broken22:38
kmallocleverage system-scope to support is_admin and change as needed to get there22:38
kmallocbut starting from "bad design" is still bad design22:38
ayounghere is what I started with https://review.openstack.org/#/c/384148/24/nova/policies/base.py22:38
felipemonteiro_lbragstad: hi22:38
lbragstadfelipemonteiro_: i was trying to update the protection tests in keystone to be a bit more granular and organized - https://review.openstack.org/#/c/551337/322:38
ayoung        'is_admin:True and is_admin_project:True',22:38
ayoung        'Global admin'),22:38
ayoungto start that would be:22:38
lbragstadespecially if we end up with three default roles and multiple scope22:39
lbragstadfelipemonteiro_: curious if you have any feedback on that from a testing perspective22:39
ayoung'is_admin:True and (scope:System or scope:Project'22:39
ayoung'is_admin:True and (scope:System or scope:Project)'22:39
ayoungthat would allows system scoped tokens to be used22:39
kmallocthat seems like a startign place22:40
kmallocthough we neeeeeed to have temptest testing that method as well as a system-scope only mechanism22:41
kmallocand remember only things like hypervisor mucking apis will be system-scope22:41
kmallocnot any "is_admin" thing22:41
ayoungkmalloc, eventually.  But right now, enforcing that will break people using admin tokens scoped to projects to get work down22:41
kmallocright so the answer is what you just described as a starting place22:42
kmallocyou support both22:42
ayoungall APIs should be executable via a system scoped token22:42
felipemonteiro_lbragstad: looking22:43
ayoungbut we need a flag to disable project scoped tokens from certain apis that can be set from config or from keystone22:43
lbragstadi don't think we should support system scoped tokens for project scoped operations in the name of backwards compatibility22:43
lbragstadthat's not to say we shouldn't support that for is_admin_project tokens22:43
ayounglbragstad, we need them anyway22:43
ayoungto clean up resources that are orphaned when the projects are deleted22:44
ayoungor to let an admin go and set things up for other users etc22:44
ayoungI'm less worried about that than the reveres22:44
lbragstadfor the second case a system admin can give themselves authorization on the project to do that22:45
lbragstadand use the recommended flow to set things up22:45
ayoungdeja vu22:45
ayoungthey don't want that22:45
ayoungadmins want to be able to do multi-project operations with a single token22:45
ayoungwe got that message very loudly back in the HMT time frame22:45
ayoungI really am more concerned with removing the project scoped admin role from system operations than anything else22:46
ayoungso...what about this:22:46
ayoungwe make a horribbly complicated but effective rule that checks for a config flage and, if it is not set, or set to false, falls back to the existing behavior22:47
ayoungif it is set, it only lets system scope work on system scoped rules22:47
kmallocsystem-scope (long term vision) should never cross over to any project-scoped actionx22:48
ayoungand we tag the APIs that should be system scoped in nova like you started doing wuith https://review.openstack.org/#/c/525772/22:48
* lbragstad steps away for a minute22:48
ayoungkmalloc, as much as I appreciate that sentiment, that was not what the operators told us22:48
kmallocif you want a "clean up orphaned thing", that should be it's own api -- with a pass of the project_id22:48
ayoungat a MINIMUM we need to be able to delete.22:48
kmallocNOT a "scope to a dead project"22:48
ayoungbecause once you delete a project, you cannot get a token scoped to that22:49
ayoungkmalloc, why do you hate operators?22:49
kmallocso make a nova api that can cleanup everything for a passed in id22:49
kmalloci don't hate operators22:49
kmalloci will -2 things that conflate project scope to system-scope22:49
kmallocsystem-scope can have a "cleanup all resources, even if project is dead, for id X"22:49
kmallocsystem scope cannot have a "make me scoped to project x" mechanism22:49
ayoungare you going to implement this?22:49
kmallocno, you need it, you will ;)22:50
ayoungor are we going to wait another decade?22:50
ayoungI did22:50
ayoungand it sat there22:50
kmallocwell, it's a nova thing22:50
ayoungI'm the pragmatist here22:50
kmallocnot something i have the ability to approve, sadly22:50
ayoungnot nova22:50
ayoungit is keystone's problem22:50
kmallocno, it is not22:50
kmallocif you want to cleanup resources orphaned in service X, you must make service X able to clean it up22:51
kmallocand that is a system-scope action22:51
kmallocallowing keystone to scope tokens to deleted projects is a non-starter22:51
ayoungkmalloc, yeah, but guess what?  Every other cloud service out there does not do that22:51
ayoungAWS, Azure, and Kubernetes all clean up the resources when you delete a project22:51
kmallocexcept they do.22:51
kmallocyou're not scoping in those to a deleted service22:51
kmallocyou're administrativly saying "cleanup resources for X"22:52
ayoungwe are not deleting upon demand22:52
kmallocthere is a difference22:52
kmalloci'm sorry you're not seeing it22:52
kmallocthis is not a keystone problem.22:52
ayoungkmalloc, so, I started at this point, and was talked out of it22:52
kmallockeystone cannot scope to dead projects, it allows for anything to happen for projects that don't exist22:52
ayoungI wanted us to be able to recreate a dead project so we could clean up resources22:52
kmallocit's a massssssssssive security issue22:53
ayoungby issing tokens for it22:53
kmallocand that is a broken model22:53
ayoungKeystone is currently operating on a broken model22:53
ayoungand this is not the worst broken part of it22:53
kmallocnova and neutron and cinder, etc need to be able to do an administrative "project x - delete all owned resources for"22:53
kmallocit is stupid to make it "scope to a project and do normal deletion of each element"22:53
ayoungso...I'll grant you that write operations should be scoped, as much as that will break cloudforms and everything22:53
kmallocif the project is dead22:53
ayoungbut read needs to be done with a token scoped larger than proejct22:54
kmallocyou're now conflating issues22:54
kmallocread != cleanup unless youre doing it the bad way22:54
kmallocfor orphaned resources22:54
ayoungoperators NEED to be able to read the whole tree22:54
ayoungwithout rescoping22:54
kmallocok, lets back up22:54
kmallocyou're context switching massively to different arguments22:55
kmallocyou're asking for cleanup of orphaned resources22:55
kmalloci'm saying that is a nova (or neutron) concern to support22:55
ayoungkmalloc, hold on22:55
kmallocyou're now saying "I want to list everything"22:55
ayoungI am giving two examples of cases where someone needs a token not scope to a proejct to perform an operation22:55
kmallocthat also feels like system-scope. but different. not scoped to a project, even a dead on22:56
ayoungand, I really wanted HMT to be the solutions to both of those22:56
kmallocwe cannot solve this in keystone22:56
kmallocthis is not a keystone problem22:56
kmallocthis is a "we supply mechnisms to provide system-scope actions (non-project-scoped)"22:57
ayoungThere is no where to delegate this problem to22:57
kmallocit is up to the services to define things such as "clean up all resources for X"22:57
kmallocor "list every single instance in the entire world"22:57
kmallocbecause keystone CANNOT supply a mechanism that does that22:57
ayoungand they say "we already do that"22:57
ayoungand then we file bug 968696 a22:57
openstackbug 968696 in OpenStack Identity (keystone) ""admin"-ness not properly scoped" [High,In progress] https://launchpad.net/bugs/968696 - Assigned to Adam Young (ayoung)22:57
kmallocwe can only issue RBAC and Scope or SystemScope22:58
ayoungand then they mark it wishlist22:58
kmalloci'm going to close that bug administatively and say "will not fix as described" :P22:58
kmallocin all seriousness22:58
kmallockeystone cannot solve this22:58
kmallocnova must supply the mechanism to do this22:58
ayoungso don;'t close that22:59
kmallocwe are creating system-scope for things such as "clean up all resources for abandoned project"22:59
kmallocor other very-global things22:59
kmallocthat are outside the scope of normal operations22:59
felipemonteiro_lbragstad: reviewed22:59
kmallocfwiw, i think that bug needs to be closed and new bugs explicitly opened for the various other mechanisms we now are defining23:00
kmallocthat bug doesn't help much anymore. as we're moving to a new mechanism to isolate actions23:00
kmallocand FTR, i wouldn't close the bug directly like i said :P23:00
kmallocso the solution is:23:01
kmalloc1) System Scope becomes a thing, some actions need to be move to system scope23:01
kmalloc2) doing actions such as "Create a VM on project for user" should not be system-scope, it should be handled (possibly) from an admin role inherited from <<keystone.domain.root>>23:01
ayoungkmalloc, as much as I want to agree with you, we can't do that23:02
kmalloc3) some new APIs will be needed for cases such as cleanup orphaned resources, that should be system-scope23:02
ayoungit will break all the tooling out there23:02
kmallocthis is all additive23:02
kmallocthere is nothing breaking23:02
kmallocit is supplying better ways forward.23:02
kmallocand we're not adding more terrible "wedge it into a bad model" mechanisms23:03
ayoung "Create a VM on project for user" should not be system-scope, it should be handled (possibly) from an admin role inherited from <<keystone.domain.root>>"23:03
*** d0ugal has joined #openstack-keystone23:03
kmallocyes, so scope to the project23:03
kmallocwith your inherited role23:03
kmallocand make a VM23:03
ayoungso are we going to issue tokens scoped to a whole tree?23:03
kmallocworks just like anything else23:03
kmallocno, scope to the project23:03
ayoungkmalloc, you are not hearing me23:03
kmallocbecause the role is inherited from keystonje.domain.root, it exists on all projects down the whole tree23:03
ayoungwe proposed that and the operators shouted it down.23:04
ayoungthere were proposals to have tokens scoped to multiple projects, etc23:04
kmallocwhy did they shoot it down23:04
ayoungbecause when you run a cloud, you want to see everything, not just project scoped23:04
kmalloci have no insight, i am getting strictly "we need this" because my customer demands it23:04
kmallocthat isn't helpful23:04
ayoungkmalloc, it was not my customer23:04
kmallocok we're talking past each other23:04
ayoungthis was in convos at the Summit back in the HMT time frame23:05
kmalloci'm walking away before i can't communicate anymore with you today23:05
ayoungask Henry23:05
*** abhi89 has quit IRC23:05
ayoungkmalloc, ok, lets punt on that for a moment23:05
kmalloci have never heard anything from anyone that says "if i need to perform an action for a user i'm opposed to scoping to thier project"23:05
kmallocexcept from henry, and the reason was "because my customer says so"23:05
kmallocthere is nothing saying you can't allow listing of every project in the tree (today or in the future)23:06
ayoungkmalloc, nah, it was in the forum when talking to operators at the summit. I was actually the one that was proposing just what you are proposing23:06
kmallocwhich operator was it?23:06
ayoungso...the oproblem was that the remote projects do not know the tree23:06
kmalloci'm really only ever heard "i don't like that"23:06
kmallocthe non-keystone-services?23:06
ayoungwe can't issue a token to project parent and they will know the relationship between that and the child23:07
kmalloclets back waaaaaaay the heck up23:07
kmallocstop, talking about this23:07
kmallocwhat are we solving23:07
kmallocWhat problem23:07
kmallocdefine the problem as narrowly as possible23:07
ayoungwhat i want to solve is to remove role:admin being the only check done for operations23:07
kmallocbecause we've migrated to yet another use case23:07
kmallocwhat operaton(s)23:08
ayoungmy proposal was to convert those to role:admin scope:system23:08
kmalloceach must be evaluated23:08
kmallocyou can't just blindly convert23:08
kmallocpart of this was very deliberate converting the right apis to system-scope23:08
kmallocsystem-scope is by design narrow, it is meant for actions that are explicitly not project/domain scoped23:08
ayoungwhat you are proposing is splitting them so they are mutually exclusive: either you have project scoped token (however you got it) and you can do project operations, or you have a system scoped token, and you can do system operations23:08
kmallocand a system-op is not "create an instance"23:09
ayoungand, top be honest, I agree with you that is how it should be, in my world view23:09
kmallocit is things like "disable hypervisor X"23:09
kmallocand potentially "create global image in glance Y"23:09
ayoungso, the problem is that changing that is a fundamental assumption that was used to build the management tools23:09
kmallocbreaking apart (to start) things that it stupid to be project based23:09
kmallocthen the management tools need to *also* migrate23:10
ayoungthat predates Keystone23:10
kmallocthat is part of "support both"23:10
kmallocand then "set EOL date of non-system scope"23:10
kmallocthen "EOL non-system scope"23:10
kmallocsome tools will cease to work if they were written for say kilo, and in the V release of openstack23:11
kmalloci hope to god you're not using a kilo tool on "V" release of openstack23:11
kmalloci don't expect it to work23:11
ayoungHMT would make sense as an alternative, but we need to convey more information than we do right now23:11
kmallocand new functions that should be system-scope only wont be "project scoped" at all23:11
kmallocI am advocating leaning on HMT more for the non-system-scope work23:12
ayoungkmalloc, I am totally using a K tool on current openstack.  CloudForms moves slowly23:12
kmallocand that is fine.23:12
ayoungand I don't think that the rest of the world moves much faster23:12
kmallocbut on V, if there is a 2-3 year lead23:12
kmallocthats fine23:12
kmallocthe EOL timeline will be about the timeframe we're talking about23:13
ayoungso...the problem with HMT is that the remote services do not know the tree23:13
ayoungso either we serialize it in the token, or they have to fetch it23:13
kmallocok, now we can talk about that.23:13
ayoungquota is going to have to do that anyway,right?23:13
kmallocand that is a very different issue than what we started with23:13
kmallocor at least how the convo was phrased23:13
kmallocso serialize it in the token response23:14
ayoungI was going to punt on it23:14
kmallocit has a fixed limit on depth23:14
kmalloc(or... it should...)23:14
kmallocnow, do you want the whole tree?23:14
kmallocor just the current branches23:14
kmallocif you want the whole tree...it's more difficult23:14
kmallocbut still ultimately doable23:14
kmallocquota is ... weird.23:14
kmallocand we haven't really moved much on the limit stuff (we're just starting to build the library)23:15
kmallocand it might be "fetch quota"23:15
kmallocwith caching23:15
*** felipemonteiro_ has quit IRC23:16
kmallocayoung: this is why i want to see some mechanism for all services to beable to directly query the authoritative keystone data [like resources, user [not secure, e.g. password hashes], etc]23:16
kmallockeep local caches or via something like consul, a DHT, etc.23:16
kmallocit's on my long long long long list of "try to solve"23:16
kmallocbut i missed the PTG and generally don't get to the planning sessions23:17
ayoungI thought you were going to nix any caching...23:17
kmallocno, caching is going to be required. it just has to be smart enough23:17
kmallocso we can handle mostly-consistent23:17
kmalloccaching of data is fine as long as its done with the expectation that it'll be mostly-consistent, and in absence (disabled) of cache, we admit it will be slower and deal with the direct query for all operations23:19
kmallocwe already do not allow "moving" things in the HMT tree23:20
kmallocthat means the worst that happens is a project is unknown (hey, refresh your cache) or a project is deleted (Stale cache) we can work with.23:20
ayoungkmalloc, OK...follow me here for a moment23:21
ayoungthere are a lot of 3rd part applications writtne to talk to OpenStack23:21
ayoungand they were built using the assumptions that we published to the world23:22
ayounga big one is that you do not need to go and get a new token when do admin operations23:22
ayoungand they wrote those operations to be pretty damn wide spread23:22
kmallocand things evolve.23:22
kmallocand consuming applications learn and grow, especially when its much much much better23:23
ayoungone of the big ones is that, without getting an other token, they can list all servers23:23
kmallocit's what major versions are for (grump semver death)23:23
kmallocthat has never really been true23:23
kmallocit has been true in a large subset of openstack23:23
kmallocbut not globally23:24
ayoungand since these are written in languages other than python, they don't go through our tooling23:24
ayoungso we cannot break these tools23:24
kmallocif everything was as static as you're painting things, nothing would ever move forward23:24
kmallocit seems openstack is held to some higher standard that can never change anything23:24
ayoungkmalloc, everything is as static as I am painting things and that is why nothing has moved forward23:24
ayoungplus, once we had a plan to fix things, everyone that was supporting that plan got fired23:25
kmallocif you provide new mechanisms and EOL timeframes, tools will move23:25
*** gongysh has joined #openstack-keystone23:25
kmallocif you design everything by comittee of every consumer, nothing ever will be done.23:25
*** gongysh has quit IRC23:26
ayoungreally?  I hadn;t noticed23:26
kmallocfrankly, there have been breaking changes to qemu, libvirt, and even linux kernel.23:26
kmallocit's communicated, telegraphed, set timelines23:26
kmallocand changed23:26
kmalloctooling will move23:26
kmallocbe kind, but don't be frozen23:26
ayoungso...I think you are the only person that is willing to die on the hill of "service tokens can't do project level operations"23:26
kmallocask lance to remove my +2/-2 powers and i wont die on that hill23:27
kmallocbut to be frank, yes. i will die on that hill23:27
ayoungkmalloc, I'd rather have you provide a viable route forward for a proposal that you are blockinfg23:27
ayoungbecause I am going to need that +2 in the future23:27
*** gongysh has joined #openstack-keystone23:27
ayoungwe are a disfunctional group here23:27
kmalloci have been trying to, but nothing is viable because $new reason$23:27
ayoungkmalloc, this ain't new.  And this is not the real security hole23:28
ayoungthe real security hole is the adminness thing23:28
kmallocsystem-scope should NOT do project actions.23:28
ayoungso we phase that out over tiome23:28
kmallocit is trying to split the concerns so you aren't trying to conflate things.23:28
kmallocwhich is what is the problem and why we can't solve the admin-ness23:28
kmallocor is a large part of it23:29
kmallocnot the only part23:29
ayoungproject scoped should not do project scoped things in other projects23:29
kmallocand domain scope (bear with me) should live above projects23:29
kmallocand inherited roles allow for scoping down the tree and performing actions on a child.23:29
ayoungok...we can't break things23:29
ayoungwe need to be able to do this in a step by step manner23:29
kmallocand we've covered the steps23:29
ayoungso, one...enable system scoped for some set of operations23:30
kmalloc1) implement system-scope23:30
kmalloc2) make sure both work for current APIs23:30
ayoungso far so good23:30
kmalloc3) make sure system-scope only works (for openstack-specificl tooling, e.g. tempest, ci, etc)23:30
kmalloc(this doesn't change/disable anything, just runs full testing with system-scope only support for designated APIs)23:30
kmalloc4) set a clear EOL on non-system-scope by default (people can always add it back in)23:31
kmalloc5) turn off by default the non-ssystem scope.23:31
kmallocat the right EOL point23:31
kmallocstill can be added back in23:31
kmalloc-- now we have system split from actual scoped actions23:31
kmallocadmin project, vs admin domina23:32
kmallocbut you're not conflating "disable hypervisor" with "admin in project"23:32
ayoungok, now run that past the operators group.  Tell them that you are not going to be able to do all operations with any single token.23:32
kmallocand i bet i can win them over (or a majority)23:32
ayounglet me see if I can find the discussion in the archives23:32
kmallocbut at step 3, is where we start working on better bits of HMT and other project stuff.23:33
kmallocthe deal is things like "add an endpoint" will be no different (except you'd says scope: system)23:33
kmallocthis is not really a lot different than today.23:33
kmallocmost things that will be system-scope specific will be things that should be isolated23:33
ayoungkmalloc, so the people that this effects are the people running clouds that need to clean things up for their customers, either internal or external23:34
ayoungnot endspoint operations, but end user operations23:34
kmallocthe argument and the way you pitch it is that it allows (especially in bigger deployments) isolation of what peoplewho need to work on the control plane, but should not be able to make instances23:34
kmallocyou're still not breaking typical user-or-admin-use-of-openstack operations23:35
ayoungDomain scoped tokens would probably be workable in practice.23:35
ayoungBut the remote services don't know what domain a project is in either23:35
kmallocwe can start leaning on HMT and domains more when we're no longer worried about "this breaks all of openstack"23:35
kmallocin my deployment23:35
kmallocbecause someone turned off my hypervisors23:35
ayoungand we can't automate away the token fetch because the tooling is in languages other than python23:35
*** oikiki has quit IRC23:36
*** oikiki has joined #openstack-keystone23:36
*** AlexeyAbashkin has joined #openstack-keystone23:36
kmallocbecause we now have "manage openstack" and "use openstack" buckets of concern23:36
ayoungand this is enforced at policy time, not token validation23:36
ayoungtoken validation we assume has to be cached23:36
kmallocit is worth fighting this battle, i've had a lot of folks doing a lot of work around this because it's painful and scary23:36
*** edmondsw has joined #openstack-keystone23:36
ayoungwhat if we had a fallback?23:37
kmallocis_admin_project is part of that23:37
kmallocand it's because it's scary23:37
ayoungcases where policy failed but we could say "go ask keystone"23:37
kmallocexcept keystone doesn't know about nova's policay23:37
kmallocor what is acceptable23:37
kmallocyou and i tried that path multiple times.23:37
kmallocso, what is being asked of keystone?23:38
kmalloc(in that fallback)23:38
ayoungwhat if...when an operator wiuth an HMT token tries to do something, and the policy fails because the scope doesn't match, the policy enforcement says "ohm, this is an HMT token?  Let me ask keystone again, with more context"23:38
kmallocwhat is the failure23:38
kmalloc"i don't know the tree, because i've not seen the projct"23:38
ayoungok...to keep it simple23:38
ayounglets say I have a domain scoped toke23:38
ayoungand I use it on nova, which knows nothing about domains23:39
ayoungI try to create a service in project P23:39
ayoungpolic rejects it, but...23:39
kmallocso, if nova allows user to create vms for a scope not their own (afaik that is all custom things not in upstream)23:39
ayoungwait...its a domains scoped token...revalidate, but add in "for proejct P"  a23:39
kmallocthen that is very reasonable to re-ask keystone or at least refresh it's cache23:39
kmallocto know if this makes sense.23:40
ayoungessentially ask "does Dom D contain project P"23:40
kmallocyeah, that's fine. i really don't care how nova does that...23:40
ayoungdomain scoped tokens would be the slow path23:40
kmalloci just don't want to have the concern where we end up with that use for things that shouldn't even be project scoped.23:40
ayoungcould we make that part of middleware, though...23:41
*** AlexeyAbashkin has quit IRC23:41
ayoungok, next thought23:41
ayoungwe send the domain scoped token to nova23:41
kmallocpart of the isolation of concerns is because we don't want to open potential security issues for very administrative things that isn't related to openstack-use23:41
ayoungmiddleware validates it and it passes, but since it is domain scoped...23:41
ayoungdamnit, we don't know the project23:41
*** edmondsw has quit IRC23:41
ayoungwe can't even ask23:41
kmallocthat is an issue with nova knowing what to ask for23:42
kmallocexpanding nova's apis23:42
kmallocto allow for "do action on project X"23:42
kmallocvs "do action on my current scope"23:42
ayoungit needs to be done at the scope check time23:42
kmallocthat wont be middleware.23:42
kmallocnova has to gate if that is allowed, keystone cannot23:42
ayoungthe scoped does not match, either fail (like we do now) or confirm with Keystone23:43
kmalloc**confirm with keystone might be a cache hit23:43
kmallocbut yes.23:43
kmallocwe're on the same page.23:43
ayoungthe way that role assignment inheritance works now, you would not even know the right set of roles23:43
ayoungit would break RBAC in middleware, too23:44
ayoungdman you are a PITA23:44
kmallocwe can expose more information23:44
kmallocand really i'm fine with expanding the information sent as needed.23:44
kmallocyou can check if the role is inherited (with keystone) and if project is child23:45
kmallocif so, you know it's fine.23:45
ayoungok...let me state a working assumption.  HMT tokens are the non-standard path, and can be slower than normal use cases, as they are going to have lots of cache misses23:45
ayoungwhat about "list servers"23:45
kmalloclist servers is "list vms"23:45
ayoungthat either based on the proejct scope or is_admin does it for the whole endpoint (I think)23:46
ayoungyeah, list vms23:46
kmallocjust to be clear what resource type because we have so many overloaded terms23:46
ayounghow would it know what to list?23:46
kmalloci think it's actually ?all_tenants=True (or whatever the qp is)23:46
kmallocand that explicitly checks is_admin23:46
ayoungso instead it would look at the token "is this an HMT token"23:47
ayoungand...if it is, it starts at the top of the tree and works down?23:47
kmallocit probably would start as a "are you role x HMT"23:47
ayoungthat would have to be implemented in every serer23:47
kmallocit might become it's own system-scope API (not list servers, but list_all_project_servers)23:48
kmallocthat is one of the edge cases we would err to not make system-scope to start23:48
ayoungyou've just opened the can of worms23:48
kmalloceach API that is moved is move explicitly23:48
ayoungcuz if we can do it for read, we don't want to give them a new token for write23:48
kmallocand with the blessing of the service23:48
ayoungthis is kindof like inheritance23:49
ayoungfor certain operations, service role implies all projects23:49
kmallocand yes, if it becomes it's own API that brides all domains and all projects23:49
kmallocthen you get a write token for the project you need23:49
ayoungits not a role inference rule, it is a scope inference rule23:49
ayoungkmalloc, are the other cores aligned with you on this>?23:50
ayoungkmalloc, I would like to point out that we've almost monopolized this channel to the point where scrollback is you, me and edmondsw has quit (Ping timeout: 260 seconds)23:54
kmallocayoung: lbragstad most cores in Denver were on board with this23:54
kmalloci don't know how it has changed since then23:55
kmalloci missed dublin23:55
kmalloci wasn't at the forum in... sydney?23:55
kmallocand i am unsure if i'll be in vancouver23:55
kmallocayoung: it's also late(ish) in the day23:55
*** germs has joined #openstack-keystone23:55
kmallocand not many folks are west coast anymore here23:55
kmalloc[i'm kinda in the minority]23:56
ayoungI miss amakarov23:56
*** dklyle has quit IRC23:57
kmallocyeah, i was sad that work died. it needed work/effort around making it functional in the way we needed23:57
kmallocbut it also didn't have much driving as we suffered attrition of devs/cores across openstack23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!