| kmalloc | eandersson: not that i am aware | 00:06 |
|---|---|---|
| *** itlinux has quit IRC | 00:14 | |
| *** oikiki has joined #openstack-keystone | 00:30 | |
| *** oikiki has quit IRC | 00:44 | |
| *** zhurong has joined #openstack-keystone | 00:46 | |
| *** oikiki has joined #openstack-keystone | 00:48 | |
| *** Dinesh_Bhor has joined #openstack-keystone | 00:52 | |
| *** harlowja has quit IRC | 01:10 | |
| wxy | lbragstad: kmalloc : I forgot morgan has already done it. ;) | 01:14 |
| kmalloc | wxy: not all of it :P | 01:14 |
| *** germs has joined #openstack-keystone | 01:15 | |
| wxy | kmalloc: mind me to add the test code for it, if you don't have time? | 01:16 |
| kmalloc | please take it over if you want :) | 01:16 |
| wxy | kmalloc: cool~ | 01:16 |
| kmalloc | https://review.openstack.org/#/c/483514/ | 01:16 |
| kmalloc | if there is anything you dislike about it, please feel free to adjust it. | 01:17 |
| kmalloc | it's 100% demo-ish code | 01:17 |
| wxy | kmalloc: OK, i'll test it first | 01:17 |
| kmalloc | ok, well it might be way more than demo code | 01:17 |
| kmalloc | wow, i just reviewed what i wrote and ... uh. it's kinda more featureful than I remember | 01:17 |
| kmalloc | also, needs docs | 01:18 |
| wxy | kmalloc: sure. | 01:18 |
| kmalloc | and you can drop all the v2 magic ;) | 01:18 |
| *** germs has quit IRC | 01:19 | |
| kmalloc | i think it's going to be a much better catalog to deploy with than the DB version in a lot of cases. | 01:19 |
| kmalloc | but, you know, I'm biased towards not-api-driven for more "Static" environments | 01:20 |
| *** nicolasbock has quit IRC | 01:38 | |
| *** namnh has joined #openstack-keystone | 01:39 | |
| *** nicolasbock has joined #openstack-keystone | 01:40 | |
| *** zhurong has quit IRC | 01:45 | |
| *** gongysh has joined #openstack-keystone | 01:45 | |
| *** zhongjun has joined #openstack-keystone | 01:54 | |
| *** r-daneel has quit IRC | 02:03 | |
| *** rcernin has quit IRC | 02:04 | |
| *** r-daneel has joined #openstack-keystone | 02:11 | |
| *** annp has joined #openstack-keystone | 02:13 | |
| *** zhurong has joined #openstack-keystone | 02:26 | |
| *** itlinux has joined #openstack-keystone | 02:30 | |
| *** nicolasbock has quit IRC | 02:32 | |
| *** oikiki has quit IRC | 02:50 | |
| *** oikiki_ has joined #openstack-keystone | 02:51 | |
| *** oikiki_ has quit IRC | 02:58 | |
| *** rcernin has joined #openstack-keystone | 03:07 | |
| *** germs has joined #openstack-keystone | 03:15 | |
| *** germs has quit IRC | 03:15 | |
| *** germs has joined #openstack-keystone | 03:15 | |
| *** edmondsw has joined #openstack-keystone | 03:18 | |
| *** germs has quit IRC | 03:20 | |
| *** edmondsw has quit IRC | 03:22 | |
| *** gongysh has quit IRC | 03:47 | |
| *** harlowja has joined #openstack-keystone | 03:48 | |
| *** links has joined #openstack-keystone | 03:50 | |
| *** dave-mccowan has quit IRC | 03:56 | |
| *** gyee has quit IRC | 03:59 | |
| *** daidv has joined #openstack-keystone | 04:01 | |
| *** gongysh has joined #openstack-keystone | 04:07 | |
| *** gongysh has quit IRC | 04:08 | |
| *** gongysh has joined #openstack-keystone | 04:20 | |
| *** gongysh has quit IRC | 04:21 | |
| *** gongysh has joined #openstack-keystone | 04:21 | |
| *** gongysh has quit IRC | 04:22 | |
| *** daidv has quit IRC | 04:22 | |
| *** gongysh has joined #openstack-keystone | 04:22 | |
| *** daidv has joined #openstack-keystone | 04:22 | |
| *** gongysh has quit IRC | 04:25 | |
| *** zhurong has quit IRC | 04:53 | |
| *** Dinesh_Bhor has quit IRC | 05:00 | |
| *** Dinesh_Bhor has joined #openstack-keystone | 05:01 | |
| *** edmondsw has joined #openstack-keystone | 05:06 | |
| *** edmondsw has quit IRC | 05:10 | |
| *** Dinesh_Bhor has quit IRC | 05:11 | |
| *** germs has joined #openstack-keystone | 05:16 | |
| *** germs has quit IRC | 05:16 | |
| *** germs has joined #openstack-keystone | 05:16 | |
| *** Dinesh_Bhor has joined #openstack-keystone | 05:19 | |
| *** germs has quit IRC | 05:21 | |
| *** jaosorior has quit IRC | 05:22 | |
| *** oikiki has joined #openstack-keystone | 05:25 | |
| *** itlinux has quit IRC | 05:26 | |
| *** karthi has joined #openstack-keystone | 05:27 | |
| *** karthi has quit IRC | 05:30 | |
| *** oikiki has quit IRC | 05:36 | |
| *** oikiki has joined #openstack-keystone | 05:38 | |
| *** zhurong has joined #openstack-keystone | 05:42 | |
| *** karthi has joined #openstack-keystone | 05:45 | |
| *** harlowja has quit IRC | 05:59 | |
| *** gongysh has joined #openstack-keystone | 06:02 | |
| *** daidv has quit IRC | 06:04 | |
| *** david-lyle has quit IRC | 06:08 | |
| *** oikiki has quit IRC | 06:12 | |
| *** Dinesh_Bhor has quit IRC | 06:21 | |
| *** Dinesh_Bhor has joined #openstack-keystone | 06:23 | |
| *** germs has joined #openstack-keystone | 06:29 | |
| *** germs has quit IRC | 06:29 | |
| *** germs has joined #openstack-keystone | 06:29 | |
| *** pcichy has joined #openstack-keystone | 06:29 | |
| *** oikiki has joined #openstack-keystone | 06:30 | |
| *** wxy_ has joined #openstack-keystone | 06:37 | |
| *** d0ugal has joined #openstack-keystone | 06:48 | |
| *** germs has quit IRC | 06:52 | |
| *** edmondsw has joined #openstack-keystone | 06:54 | |
| *** zhurong has quit IRC | 06:55 | |
| *** openstackgerrit has joined #openstack-keystone | 06:56 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/keystoneauth master: Updated from global requirements https://review.openstack.org/549536 | 06:56 |
| *** edmondsw has quit IRC | 06:59 | |
| *** jaosorior has joined #openstack-keystone | 07:02 | |
| *** karthi has quit IRC | 07:19 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/oslo.policy master: Updated from global requirements https://review.openstack.org/549559 | 07:22 |
| *** karthi has joined #openstack-keystone | 07:22 | |
| *** rcernin has quit IRC | 07:24 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-keystoneclient master: Updated from global requirements https://review.openstack.org/549565 | 07:26 |
| *** oikiki has quit IRC | 07:32 | |
| *** jdennis has quit IRC | 07:36 | |
| *** martinus__ has joined #openstack-keystone | 07:39 | |
| *** karthi has quit IRC | 07:40 | |
| *** pcaruana has joined #openstack-keystone | 07:42 | |
| *** AlexeyAbashkin has joined #openstack-keystone | 07:50 | |
| *** links has quit IRC | 07:52 | |
| *** links has joined #openstack-keystone | 07:59 | |
| *** tesseract has joined #openstack-keystone | 08:19 | |
| *** tesseract-RH has joined #openstack-keystone | 08:20 | |
| *** tesseract-RH has quit IRC | 08:20 | |
| *** sticker has joined #openstack-keystone | 08:34 | |
| *** karthi has joined #openstack-keystone | 08:37 | |
| *** r-daneel has quit IRC | 08:41 | |
| *** edmondsw has joined #openstack-keystone | 08:42 | |
| *** edmondsw has quit IRC | 08:46 | |
| *** r-daneel has joined #openstack-keystone | 08:48 | |
| openstackgerrit | wangxiyuan proposed openstack/keystone master: Deprecate the templated catalog https://review.openstack.org/482714 | 09:25 |
| openstackgerrit | wangxiyuan proposed openstack/keystone master: [WIP]Add yaml-loaded filesystem catalog backend https://review.openstack.org/483514 | 09:25 |
| *** mburrows has quit IRC | 09:30 | |
| *** mvk has quit IRC | 09:32 | |
| *** Dinesh_Bhor has quit IRC | 09:34 | |
| *** dmellado has joined #openstack-keystone | 09:47 | |
| *** karthi has quit IRC | 09:56 | |
| *** karthi has joined #openstack-keystone | 09:57 | |
| *** namnh has quit IRC | 09:58 | |
| *** edmondsw has joined #openstack-keystone | 10:30 | |
| *** edmondsw has quit IRC | 10:35 | |
| *** pcaruana has quit IRC | 10:45 | |
| *** mvk has joined #openstack-keystone | 11:00 | |
| *** rcernin has joined #openstack-keystone | 11:02 | |
| *** nicolasbock has joined #openstack-keystone | 11:06 | |
| hugokuo | morning. | 11:21 |
| hugokuo | May I know if the validate token API still available in Queens release? | 11:22 |
| hugokuo | http://sergslipushenko.github.io/html/api-ref-identity-admin-v2.html#admin-validateToken | 11:22 |
| hugokuo | thx | 11:22 |
| cmurphy | hugokuo: it is not, and that reference is not our official api reference and seems to be very outdated | 11:25 |
| cmurphy | hugokuo: here is our official api reference: https://developer.openstack.org/api-ref/identity/ | 11:25 |
| *** karthi has quit IRC | 11:33 | |
| *** dave-mccowan has joined #openstack-keystone | 11:42 | |
| *** gongysh has quit IRC | 11:43 | |
| *** edmondsw has joined #openstack-keystone | 11:45 | |
| *** rcernin has quit IRC | 12:00 | |
| *** karthi has joined #openstack-keystone | 12:04 | |
| *** raildo has joined #openstack-keystone | 12:04 | |
| *** annp has quit IRC | 12:07 | |
| hugokuo | cmurphy: thx. Then I have to update our keystone middleware version. The current version seems not support v3 format in the configuration. | 12:10 |
| cmurphy | hugokuo: keystonemiddleware has supported keystone v3 for a very long time, there might be some other issue in your configuration | 12:13 |
| *** pcichy has quit IRC | 12:20 | |
| *** karthi has quit IRC | 12:24 | |
| *** karthi has joined #openstack-keystone | 12:26 | |
| *** pcichy has joined #openstack-keystone | 12:29 | |
| hugokuo | you are right. We are running keystone-middleware 1.5.0+ which should have v3 support. Let me dig into it. Thx for advice. | 12:31 |
| cmurphy | no problem | 12:32 |
| *** zhurong has joined #openstack-keystone | 12:36 | |
| *** odyssey4me has quit IRC | 12:41 | |
| *** odyssey4me has joined #openstack-keystone | 12:41 | |
| *** zhurong has quit IRC | 12:42 | |
| *** zhurong has joined #openstack-keystone | 12:45 | |
| *** jdennis has joined #openstack-keystone | 12:59 | |
| *** HW-Peter has joined #openstack-keystone | 13:08 | |
| *** HW-Peter has quit IRC | 13:10 | |
| *** HW-Peter has joined #openstack-keystone | 13:11 | |
| *** zhurong has quit IRC | 13:15 | |
| *** jaosorior has quit IRC | 13:19 | |
| *** gongysh has joined #openstack-keystone | 13:28 | |
| *** FuslJH95HW has joined #openstack-keystone | 13:29 | |
| *** FuslJH95HW has quit IRC | 13:31 | |
| *** jroll has quit IRC | 13:41 | |
| *** jroll has joined #openstack-keystone | 13:42 | |
| lbragstad | just a reminder that the team meeting is going to be a couple hours earlier today - http://eavesdrop.openstack.org/#Keystone_Team_Meeting | 13:44 |
| lbragstad | all information is updated and an .ics is available ^ | 13:45 |
| cmurphy | heh i almost forgot | 13:47 |
| cmurphy | bad timing to switch it right after american DST :P | 13:47 |
| ayoung | lbragstad, cmurphy so...I was reading the notes from the PTG, specifically those about "OpenStack at the Edge" and I notice that it matches a few use cases my team has addresses recently | 13:50 |
| ayoung | What I was wondering was if we could start looking at Keystone to Keystone replication, with the assumption that the Keystone instances are not all at the same level, nor have write access to the same data | 13:50 |
| ayoung | as a starting point, lets assume we say we have 3 sites: Boston, London, Berlin | 13:51 |
| *** ionel has joined #openstack-keystone | 13:51 | |
| ayoung | And, we have 3 regions, one per site | 13:51 |
| ayoung | we also say we have 3 domains, also one per site | 13:51 |
| ayoung | each site is allowed to add users or role assignments to its domain | 13:51 |
| ayoung | this will then get synced to the other Keystone instances...lets handwave how for now | 13:52 |
| ionel | Good day! Can anyone help me with TOTP authentication ? The official documentation didn't help. | 13:52 |
| ayoung | but each will not only be responsible for pushing the data, but tracking the state of that push | 13:52 |
| ayoung | ionel, just ask your question. If someone here knows, they will answer. If not, well...crickets | 13:53 |
| cmurphy | ayoung: what do you mean by "tracking the state of that push"? | 13:54 |
| ayoung | cmurphy, so, say we dump a load of users into Dom-Boston | 13:54 |
| ayoung | and those need to get synced to Dom-Berlin....They got one by one, and we can, at any point, as Dom-Boston is the cmurphy user in Region:Berlin yet? | 13:55 |
| ayoung | sorr as -> ask | 13:55 |
| *** jaosorior has joined #openstack-keystone | 13:55 | |
| ionel | ayoung: I am looking for basic TOTP + user/pass authentication. I am sure that someone should know. | 13:56 |
| ayoung | operator queries https://boston/v3/state-of-sync?user=cmurphy or something like that | 13:56 |
| ayoung | ionel, don't be so sure. | 13:56 |
| ionel | :) | 13:57 |
| ayoung | ionel, the folks that worked on that are not always in here, but post what you did | 13:57 |
| ayoung | and what the state is, and maybe someone can answer. | 13:57 |
| ayoung | make it easy for us to help you | 13:57 |
| ayoung | 'cuz I know Bupkis about our TOTP impl. | 13:57 |
| ayoung | just read https://docs.openstack.org/keystone/pike/advanced-topics/auth-totp.html | 13:58 |
| ayoung | now I am an expert | 13:58 |
| lbragstad | cmurphy: agreed... | 14:00 |
| *** karthi has quit IRC | 14:01 | |
| ayoung | lbragstad, what time is the meeting? | 14:02 |
| ionel | I have read that. I was about to post the link. | 14:02 |
| ayoung | is it now? | 14:02 |
| ionel | I am stuck at create TOTP credentials | 14:02 |
| ionel | whare do I put that code ? | 14:02 |
| ionel | there is not mentioned any file | 14:03 |
| ayoung | ionel, the CURL call? | 14:03 |
| cmurphy | ionel: that is a shell example, you'd type it into your terminal | 14:03 |
| ayoung | that can be done via the CLI too, I am pretty sure. openstack help credential create | 14:04 |
| ionel | I just did | 14:04 |
| ionel | {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}} | 14:05 |
| ayoung | ionel, the user that makes that call needs a token | 14:05 |
| lbragstad | ayoung: 1600 UTC | 14:05 |
| ionel | where I setup the token ? | 14:05 |
| ayoung | so admin does it for the first user, or the user auths via password when they do it, or sometjhing like that | 14:05 |
| ayoung | ionel, as admin create a user, set the users password. Create a keystone rc for that user./ source the keystonerc, make the call. | 14:06 |
| ayoung | http://adam.younglogic.com/2018/02/keystonerc-for-rdo-cloud/ is an example template | 14:06 |
| ayoung | cmurphy, that curl call does not have a way to authenticate documented: | 14:07 |
| ayoung | curl -i \ | 14:07 |
| ayoung | -H "Content-Type: application/json" \ | 14:07 |
| ayoung | -d ' | 14:07 |
| ayoung | { | 14:07 |
| cmurphy | ionel: that example might have been written before we had client support, it looks like that was added: https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/credential.html | 14:07 |
| cmurphy | ayoung: you're right it's missing a -H X-Auth-Token | 14:08 |
| ayoung | ionel, so, add in the following steps: | 14:09 |
| ayoung | TOKEN=`openstack token issue -f json | jq -r '.id'` | 14:09 |
| ayoung | then in the curl call: | 14:10 |
| ayoung | curl -i \ | 14:10 |
| ayoung | -H "Content-Type: application/json" \ | 14:10 |
| ayoung | -d ' | 14:10 |
| ayoung | -H X-Auth-Token $TOKEN | 14:10 |
| ionel | I am working on it | 14:11 |
| ionel | Thank you :) | 14:11 |
| cmurphy | ayoung: okay back to your topic, I think i misread your question at first, you're not talking about Keystone to Keystone in the federation sense are you? | 14:14 |
| ionel | I'll get back to you later. I have to go now. Thank you for all your help. | 14:17 |
| *** spilla has joined #openstack-keystone | 14:17 | |
| ayoung | cmurphy, not really, no | 14:17 |
| ayoung | cmurphy, more in the Its all one big deployment, but too big to upgrade at once sense | 14:18 |
| ayoung | I'd like to start thinking of Regions as things that can vary somewhat independently | 14:18 |
| ayoung | cmurphy, it was use cases like this that pushed me toward PKI for tokens in the first place. If you could replace the symmetric crypto of fernet with asym, you could update the keys for different regions separately, know which key was owned by which region, etc. But that is not essential. | 14:21 |
| *** nicolasbock has quit IRC | 14:22 | |
| cmurphy | right | 14:22 |
| cmurphy | lbragstad: ^ maybe another case for unencrypted jwt | 14:23 |
| *** felipemonteiro has joined #openstack-keystone | 14:29 | |
| *** nicolasbock has joined #openstack-keystone | 14:32 | |
| *** nicolasbock has quit IRC | 14:41 | |
| *** nicolasbock has joined #openstack-keystone | 14:43 | |
| lbragstad | sorry - just parsed the scroll back | 14:49 |
| lbragstad | ayoung: so you want to gradually upgrade? | 14:51 |
| lbragstad | like one region at a time? | 14:51 |
| *** wxy| has joined #openstack-keystone | 14:54 | |
| *** wxy| has quit IRC | 14:55 | |
| *** wxy| has joined #openstack-keystone | 14:56 | |
| *** idlemind has quit IRC | 14:56 | |
| *** gongysh has quit IRC | 14:57 | |
| *** gongysh has joined #openstack-keystone | 15:01 | |
| *** gongysh has quit IRC | 15:02 | |
| ayoung | sorry...dog walk intervened | 15:16 |
| ayoung | lbragstad, yes, gradual upgrades, one region at a time | 15:16 |
| ayoung | lbragstad, assume you have 50 regions. Bringin the whole cloud down is a big disruption | 15:17 |
| *** sticker has quit IRC | 15:17 | |
| ayoung | lbragstad, cmurphy this was what I was reading. http://markvoelker.github.io/blog/dublin-ptg-edge-sessions/ | 15:19 |
| lbragstad | hmm | 15:25 |
| lbragstad | ayoung: does the current token provider require that? | 15:25 |
| lbragstad | bringing the whole cloud down? | 15:25 |
| *** david-lyle has joined #openstack-keystone | 15:25 | |
| ayoung | lbragstad, If everything is doing Fernet? I guess it depends on the changes we make from release to release | 15:26 |
| *** jmlowe has quit IRC | 15:27 | |
| lbragstad | don't you just need to make sure the key repository is the same for each region? | 15:27 |
| ayoung | Fernet validation is the same. But if you are syncing data across multiple Keystone instances via DB replication, and the tables are out of sync, yeah, it would be a disaster | 15:27 |
| *** jmlowe has joined #openstack-keystone | 15:27 | |
| lbragstad | i've actually heard that data replication, especially in EU, is tough | 15:27 |
| ayoung | I'm starting to think we want to give people an option to not replicate at the DB level. Many people have tried to avoid it over the years. | 15:28 |
| ayoung | Hence RFEs like "Add a project with a specific ID" | 15:28 |
| lbragstad | k2k was suppose to solve that | 15:28 |
| ayoung | k2k doesn't sync data, though | 15:29 |
| lbragstad | exactly | 15:29 |
| ayoung | and K2K is useless if the first K is down. | 15:29 |
| lbragstad | so you want users to be able to use any keystone - but you don't want to replicate data? | 15:32 |
| *** jaosorior has quit IRC | 15:34 | |
| ayoung | lbragstad, I want to repliact. But eventually consistent semantics | 15:34 |
| ayoung | replicate | 15:34 |
| lbragstad | sorry - i thought you just said you wanted to try and give people an option to not replicate at the DB level | 15:35 |
| ayoung | lbragstad, right | 15:36 |
| ayoung | lbragstad, replicate at the application level. Deliberately | 15:36 |
| lbragstad | so building replication into keystone's api | 15:37 |
| ayoung | like, Each region gets a domain, and they are only allowed to write data for their own domain, so they can then push that to the other servers | 15:37 |
| ayoung | and a Keystone to Keystone push mechanism | 15:37 |
| ayoung | If we did it at the API level, then we would need overrides for things like "write this resource with this explicit ID" | 15:38 |
| ayoung | so, maybe treat the K2KSync as a separate endpoint and API | 15:38 |
| *** itlinux has joined #openstack-keystone | 15:45 | |
| *** harlowja has joined #openstack-keystone | 15:46 | |
| *** gyee has joined #openstack-keystone | 15:51 | |
| *** links has quit IRC | 15:55 | |
| *** david-lyle has quit IRC | 15:57 | |
| *** dklyle has joined #openstack-keystone | 15:57 | |
| lbragstad | ping ayoung, breton, cmurphy, dstanek, gagehugo, henrynash, hrybacki, knikolla, lamt, lbragstad, lwanderley, kmalloc, rderose, rodrigods, samueldmq, spilla, aselius, dpar, jdennis, ruan_he, wxy | 16:00 |
| lbragstad | reminder that we'll be having the meeting in #openstack-meeting-alt today | 16:00 |
| *** harlowja has quit IRC | 16:06 | |
| *** links has joined #openstack-keystone | 16:08 | |
| openstackgerrit | Russell Tweed proposed openstack/keystone master: Add prerequisite package note to Keystone install guide https://review.openstack.org/552568 | 16:14 |
| *** links has quit IRC | 16:30 | |
| *** wxy|_ has joined #openstack-keystone | 16:33 | |
| *** wxy| has quit IRC | 16:34 | |
| *** r-daneel has quit IRC | 16:42 | |
| * cmurphy runs home before office hours | 16:51 | |
| openstackgerrit | Johannes Grassler proposed openstack/keystone-specs master: Add whitelist-extension-for-app-creds https://review.openstack.org/396331 | 16:53 |
| lbragstad | jgr: i'm glad you're going to be giving us a hand this release, happy to have you be a part of the team! | 16:54 |
| jgr | lbragstad: thanks...happy to be along myself :-) | 16:55 |
| jgr | lbragstad: (and happy I finally get to tackle this white list thing...not having this keeps being an issue with all sorts of things) | 16:56 |
| lbragstad | :) | 17:00 |
| *** thomasduval has joined #openstack-keystone | 17:01 | |
| *** r-daneel has joined #openstack-keystone | 17:01 | |
| hrybacki | I'm amazing my pee isn't citric acid after having eaten so many oranges this past week and a half | 17:02 |
| hrybacki | amazed*. Pee may be normal but my brain isn't -_- | 17:02 |
| lbragstad | #startmeeting keystone-office-hours | 17:06 |
| openstack | Meeting started Tue Mar 13 17:06:57 2018 UTC and is due to finish in 60 minutes. The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot. | 17:06 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 17:06 |
| *** openstack changes topic to " (Meeting topic: keystone-office-hours)" | 17:06 | |
| *** ChanServ changes topic to "Queens release schedule: https://releases.openstack.org/queens/schedule.html | Meeting agenda: https://etherpad.openstack.org/p/keystone-weekly-meeting | Bugs that need triaging: http://bit.ly/2iJuN1h | Trello: https://trello.com/b/wmyzbFq5/keystone-rocky-roadmap" | 17:06 | |
| openstack | The meeting name has been set to 'keystone_office_hours' | 17:07 |
| lbragstad | we do have a bluejeans session going | 17:07 |
| lbragstad | folks can join at any time | 17:07 |
| cmurphy | bluejeans is where? | 17:07 |
| lbragstad | #link https://bluejeans.com/8559013623 | 17:08 |
| cmurphy | ty | 17:08 |
| lbragstad | cmurphy: did you already make it home? | 17:08 |
| *** d0ugal has quit IRC | 17:09 | |
| hrybacki | gagehugo: wxy kmalloc knikolla ^^ | 17:09 |
| *** pcichy has quit IRC | 17:18 | |
| *** wxy|_ has quit IRC | 17:19 | |
| *** pcichy has joined #openstack-keystone | 17:21 | |
| *** felipemonteiro_ has joined #openstack-keystone | 17:29 | |
| *** felipemonteiro has quit IRC | 17:31 | |
| *** AlexeyAbashkin has quit IRC | 17:34 | |
| aning | lbragstad, cmurphy, knikolla: for the deadlock issue in 014 of upgrade, I'm experimenting a solution. Basiaclly in 014_contract_add_domain_id_to_user_table.py, I check if the uniqueconstraints and foreignkeyconstraints have created or not, and only create them if they don't exist yet. This way, the keystone-manage db_sync contract can be re-run and proceed to the next transaction. | 17:36 |
| aning | since deadlock is rare, so a re-run should succeed. | 17:37 |
| aning | The code (in debugging mode) is posted here: http://paste.openstack.org/show/700152/ | 17:38 |
| *** r-daneel has quit IRC | 17:38 | |
| aning | Could you please have a look and see if there is any problem in there, and your concerns about the solution overall ? | 17:39 |
| *** mvk has quit IRC | 17:43 | |
| lbragstad | cmurphy: kmalloc http://paste.openstack.org/show/700177/ | 17:48 |
| *** spilla has quit IRC | 17:51 | |
| *** spilla has joined #openstack-keystone | 17:55 | |
| *** felipemonteiro_ has quit IRC | 18:05 | |
| *** oikiki has joined #openstack-keystone | 18:08 | |
| openstackgerrit | Erik Olof Gunnar Andersson proposed openstack/keystone master: Fixing multi-region support in templated v3 catalog https://review.openstack.org/482364 | 18:10 |
| *** r-daneel has joined #openstack-keystone | 18:16 | |
| *** thomasduval has quit IRC | 18:17 | |
| *** r-daneel_ has joined #openstack-keystone | 18:25 | |
| *** jaosorior has joined #openstack-keystone | 18:25 | |
| *** r-daneel has quit IRC | 18:26 | |
| *** r-daneel_ is now known as r-daneel | 18:26 | |
| *** mvk has joined #openstack-keystone | 18:28 | |
| cmurphy | aning: if it doesn't consistently fail then i think retrying makes sense | 18:29 |
| cmurphy | aning: i'm confused that just re-running the contract phase doesn't just work though | 18:29 |
| cmurphy | aning: if you haven't already can you open a bug for it? | 18:30 |
| *** harlowja has joined #openstack-keystone | 18:30 | |
| *** harlowja has quit IRC | 18:30 | |
| aning | No. If I got a deadloack the first run, when re-run contract, it will fail and complain UniqueConstraint relation "ixu_user_id_domain_id" already exists | 18:32 |
| aning | since the contract is not a atomic transaction. | 18:33 |
| aning | it fails (deadlock in this case) at certain point, when re-run it will try to do some of the transactions already done in the previous run. | 18:34 |
| aning | cmurphy: yes, I'll open a bug for it. | 18:37 |
| *** d0ugal has joined #openstack-keystone | 18:38 | |
| *** felipemonteiro has joined #openstack-keystone | 18:51 | |
| cmurphy | aning: i see, so then yes adding some idempotency in that migration makes sense to me | 18:52 |
| aning | ideally the whole contract, or at least each of the step (014 for example), should be in a atomic transaction. | 18:54 |
| aning | or like I'm experimenting, just check before doing any transactions to ensure it doesn't repeat what have been done before. | 18:55 |
| *** felipemonteiro_ has joined #openstack-keystone | 18:55 | |
| cmurphy | ++ | 18:56 |
| *** r-daneel_ has joined #openstack-keystone | 18:58 | |
| lbragstad | stepping away to grab tea quick - brb | 18:58 |
| *** r-daneel has quit IRC | 18:58 | |
| *** r-daneel_ is now known as r-daneel | 18:58 | |
| *** AlexeyAbashkin has joined #openstack-keystone | 18:58 | |
| *** felipemonteiro has quit IRC | 18:59 | |
| *** itlinux has quit IRC | 18:59 | |
| *** AlexeyAbashkin has quit IRC | 19:03 | |
| *** d0ugal has quit IRC | 19:04 | |
| *** itlinux has joined #openstack-keystone | 19:04 | |
| *** tesseract has quit IRC | 19:11 | |
| *** abhi89 has joined #openstack-keystone | 19:14 | |
| abhi89 | Hi All.. I see that in queens release, we have changed 'auth_uri' to 'www_authenticate_uri' (https://review.openstack.org/#/c/508522/) to avoid the confusion with 'auth_url'.. why haven't we changed 'auth_uri' in places like https://github.com/openstack/cinder/blob/master/cinder/quota_utils.py#L28 | 19:19 |
| abhi89 | if someone updates to queens, its likely that the cinder quota utils code will fail.. there are other references of 'auth_uri' too apart from this.. | 19:20 |
| abhi89 | cmurphy | 19:20 |
| cmurphy | abhi89: we didn't remove auth_uri, we just deprecated it | 19:20 |
| cmurphy | we do still need to find all those places that reference it but in the meantime it should still work | 19:21 |
| abhi89 | cmurphy: yes.. that means if we don't change our [keystone_authtoken] section to have 'www_authenticate_uri', everything will work fine.. but if we do, then this quota_utils code will fail as there won't be any 'auth_uri' parameter.. | 19:22 |
| cmurphy | yeah :( | 19:25 |
| *** d0ugal has joined #openstack-keystone | 19:26 | |
| *** d0ugal has quit IRC | 19:26 | |
| *** d0ugal has joined #openstack-keystone | 19:26 | |
| cmurphy | abhi89: did you try it and run into an error? I'm reading through oslo.config and it seems like it might just do the right thing but that might just be wishful thinking | 19:36 |
| abhi89 | cmurphy: i didnot try but I think it will not find 'auth_uri' | 19:43 |
| *** raildo has quit IRC | 19:58 | |
| ayoung | https://adam.younglogic.com/2013/07/a-vision-for-keystone/ I think I still stand by this. Coming up on 5 years. | 20:00 |
| ayoung | cmurphy, what was that abount unencrypted JWT? | 20:03 |
| cmurphy | ayoung: http://specs.openstack.org/openstack/keystone-specs/specs/keystone/backlog/json-web-tokens.html | 20:05 |
| cmurphy | ayoung: https://review.openstack.org/#/c/541903 | 20:05 |
| cmurphy | current ideas in there are to encrypt the token same as we do with fernet | 20:05 |
| cmurphy | but that requires sharing the private keys between keystones | 20:06 |
| cmurphy | so we were thinking of just signing and not encrypting | 20:06 |
| ayoung | cmurphy, Out of curiousity, has anyone sized a JWT token with asym key signing? | 20:06 |
| ayoung | when I last looked in to it, a fernet sized block signed with asym crypto came out to about 1K | 20:08 |
| ayoung | I think signed without encrypting the body of a JWT makes sense. But it will leak a little bit of information. Still, it is no worse than if someone sniffs a token today....I think....I'd have to ruminate on that | 20:09 |
| cmurphy | ayoung: no I'm not aware of any of us actually doing a PoC on the tokens to size them, though the jwt reference says "smaller than saml" | 20:10 |
| cmurphy | ayoung: it is slightly worse than if someone sniffs a token today | 20:10 |
| ayoung | cmurphy, signing requires a key | 20:10 |
| ayoung | signing requires encryption, it is just a smaller encrypted document; only the hash is encrypted. | 20:10 |
| ayoung | doesn't avoid the need for a key | 20:11 |
| cmurphy | it's slightly worse because if i have an expired encrypted token i can't return that to keystone to get information about it | 20:11 |
| ayoung | for the same data, JWT is going to be smaller than SAML only (I think) due to the more concise document format | 20:12 |
| ayoung | problem for PKI tokens was the service catalog | 20:12 |
| ayoung | I was of the opinion that we should stick with the data that is in the Fernet body, and the keystonemiddleware could fetch and cache any data it needed in order to expand the tokens: project names from IDs, etc. | 20:13 |
| ayoung | it has all the keys distribution problems of PKI if you do, though, so I didn't push on it. | 20:14 |
| lbragstad | #endmeeting | 20:15 |
| *** openstack changes topic to "Queens release schedule: https://releases.openstack.org/queens/schedule.html | Meeting agenda: https://etherpad.openstack.org/p/keystone-weekly-meeting | Bugs that need triaging: http://bit.ly/2iJuN1h | Trello: https://trello.com/b/wmyzbFq5/keystone-rocky-roadmap" | 20:15 | |
| openstack | Meeting ended Tue Mar 13 20:15:55 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 20:15 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/keystone_office_hours/2018/keystone_office_hours.2018-03-13-17.06.html | 20:15 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/keystone_office_hours/2018/keystone_office_hours.2018-03-13-17.06.txt | 20:15 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/keystone_office_hours/2018/keystone_office_hours.2018-03-13-17.06.log.html | 20:16 |
| ayoung | cmurphy, for example, lets got with the data in most tokens: UserID, ProjectID, IssueTime: | 20:17 |
| ayoung | I think that is the minimium for a scoped token | 20:17 |
| *** oikiki has quit IRC | 20:19 | |
| *** oikiki has joined #openstack-keystone | 20:20 | |
| ayoung | $ cat token_body.json | 20:20 |
| ayoung | {"user_id":"d627bfb24c3043478a69fd8ade1009e9","project_id":"0b5df1731e67462393dea23b8c24c860","issued_at": "20180313162018"} | 20:20 |
| ayoung | OK, to encryopt that | 20:27 |
| ayoung | openssl aes-256-cbc -in token_body.json -out message.enc | 20:27 |
| ayoung | and base64 gives | 20:27 |
| ayoung | base64 message.enc | 20:27 |
| ayoung | U2FsdGVkX1+zDzoPrZ+NLQheue+0cc3BuVGX/YsrX1RUU4pn9r5Q5wFyCOaXVCZbYrlPZqE6swr1 | 20:27 |
| ayoung | ZndJsByBdA2dluoRuK/GCbgUhFoIegM6mb+/+XOVezKCQsTTAo8DUCLNSLu1VcAByOILXFsTsg3Q | 20:27 |
| ayoung | GW/ckOtnukwfEwnKg40KLD9bU68OrIikmW89PmQ5 | 20:27 |
| ayoung | 195 characters | 20:28 |
| cmurphy | fernet token is 184 characters it looks like | 20:29 |
| ayoung | Yeah, so a touch longer due to JSON | 20:29 |
| ayoung | base64 file.ssl | wc -c | 20:31 |
| ayoung | 349 | 20:31 |
| ayoung | that is from asym | 20:31 |
| ayoung | generated keys using | 20:31 |
| ayoung | openssl genrsa -des3 -out private.pem 2048 | 20:31 |
| ayoung | Generating RSA private key, 2048 bit long modulus | 20:32 |
| ayoung | not triple DES | 20:32 |
| ayoung | encrypted using openssl rsautl -encrypt -inkey public.pem -pubin -in token_body.json -out file.ssl | 20:32 |
| ayoung | and then base64 encoding | 20:32 |
| ayoung | that seems short | 20:32 |
| ayoung | But, double the length for PKI is probably about right | 20:33 |
| ayoung | symmetric will be faster, and take fewer cycles. With asym, you still need to distribute the public keys to anything that needs to validate | 20:35 |
| ayoung | but with asym non-trusted systems could validate. | 20:36 |
| kmalloc | ayoung: asym or sym, you're still doing live validation of the data | 20:37 |
| kmalloc | i don't really care which encyrption we use. | 20:37 |
| *** harlowja has joined #openstack-keystone | 20:37 | |
| kmalloc | but you're going to have to ask keystone in either case. I care that the token is small, we don't shove more than the payload of fernet (items) in it, and that it validates reasonably fast. | 20:37 |
| kmalloc | if that helps the conversation at all | 20:38 |
| ayoung | kmalloc, yep. One benefit of the Fernet approach is you don't need to hash, just encrypt | 20:43 |
| ayoung | asym helps with the key distribution side of things, and you could build a validation service out of the existing APIs. | 20:44 |
| *** d0ugal has quit IRC | 20:45 | |
| lbragstad | i wonder if it would be worth it to have a config option for signing versus encryption | 20:46 |
| gagehugo | lbragstad hrybacki made it home yesterday afternoon | 20:50 |
| gagehugo | o/ | 20:50 |
| lbragstad | gagehugo: nice! how was the vacation? | 20:50 |
| lbragstad | refreshing i hope | 20:50 |
| gagehugo | very refreshing! | 20:51 |
| gagehugo | it was the opposite of snowpenstack | 20:52 |
| *** mburrows has joined #openstack-keystone | 20:52 | |
| lbragstad | sunny, relaxing, and good wifi then? | 20:52 |
| gagehugo | actually yes | 20:52 |
| lbragstad | :) | 20:52 |
| gagehugo | was much better wifi than I expected | 20:53 |
| gagehugo | I think it was ~80F every day | 20:53 |
| gagehugo | ocean was too cold to swim in still, but got to see some whales | 20:53 |
| lbragstad | awesome | 20:53 |
| lbragstad | gagehugo: are you still on vacation then, recovering? | 20:54 |
| lbragstad | or are you officially back now? | 20:54 |
| gagehugo | I'm off today | 20:55 |
| gagehugo | I'm back tomorrow | 20:55 |
| lbragstad | ack | 20:55 |
| ayoung | lbragstad, cmurphy on Whitelist/ApCreds, would you be dead set against adding in a Role ID to the url Pattern? | 20:57 |
| lbragstad | what's the purpose? | 20:57 |
| ayoung | As a way of telling a user: here are the URL patterns you can delegate | 20:57 |
| ayoung | I know it could potentially be more than one role, | 20:58 |
| ayoung | but that would kindof be the missing link between my spec and the whitelist spec | 20:58 |
| ayoung | so say you want to delegate POST, COMPUTE, /v2.4/server_create/, | 20:59 |
| felipemonteiro_ | ayoung: if you have a moment sometime may you consider removing procedural -2 from https://review.openstack.org/#/c/464678/ | 20:59 |
| felipemonteiro_ | by no means is it done, still need to update it | 20:59 |
| ayoung | it would tell you "you can do that if you have the _member_ role | 20:59 |
| kmalloc | ayoung fernet hmacs too | 20:59 |
| ayoung | felipemonteiro_, it still scares me, but...I'm a trust you. | 21:00 |
| cmurphy | ayoung: i don't think we can reasonably manage that at this stage, i don't think we have all the pieces in place to connect roles to urls | 21:00 |
| lbragstad | that'd be duplicating the source of truth between permissions and role mappings, i think | 21:00 |
| ayoung | lbragstad, so we already have a duplicate source of truth here | 21:00 |
| ayoung | this would be more "guidance" | 21:01 |
| ayoung | cmurphy, I think we do, though | 21:01 |
| ayoung | cmurphy, we can still generate a policy file for nova etc, right? | 21:01 |
| kmalloc | ayoung: so you are hashing in either case. | 21:01 |
| ayoung | kmalloc, it does? WHy? | 21:01 |
| cmurphy | ayoung: we can generate one but we can't pull the actual one that the operator has configured from nova | 21:02 |
| ayoung | kmalloc, I thought it was just encrypted. If the doc is short, why hash? | 21:02 |
| ayoung | cmurphy, yeah, but in general, that is not a problem. Its only a problem if they go super duper customized, and we can walk people through how to manually update it from policy...I think | 21:02 |
| ayoung | actaully, we still can't automatically map from URL to policy, but figuring that out is a one time cost | 21:03 |
| ayoung | cmurphy, what if we made the role_id optional? | 21:04 |
| ayoung | only enforce it IF it is set? | 21:04 |
| kmalloc | ayoung: because encryption is not sufficient for authentication of a payload | 21:04 |
| ayoung | kmalloc, for a short payload? Of course it is | 21:04 |
| kmalloc | fernet cannot assume the payload is short. | 21:05 |
| kmalloc | WE force a short payload | 21:05 |
| ayoung | kmalloc, ah | 21:05 |
| kmalloc | that doesn't mean the generic use case is short | 21:05 |
| kmalloc | and even in the case of short payloads, best practices: hash as well for authentication of message | 21:05 |
| ayoung | pretty sure that all Hash tells is if the unencrypted has been changed, but if you trust the signer, that is not any additional information. | 21:07 |
| ayoung | kmalloc, the short of it is, we need tokens to be short. I arbitrarily set that size to be 1K | 21:08 |
| ayoung | Although, to be honest, we didn't see problems in PKI tokens until we got to 8 K | 21:09 |
| kmalloc | except PKI tokens were VERY slow even at the smaller sizes | 21:09 |
| kmalloc | well, no take that back | 21:09 |
| kmalloc | they weren't that slow | 21:10 |
| ayoung | kmalloc, yeah, although part of that might have been the fork | 21:10 |
| kmalloc | yeah the fork was slow | 21:10 |
| kmalloc | but whatever, lets set that aside | 21:10 |
| kmalloc | the token was fine aside from fork | 21:10 |
| kmalloc | the other issue is the added overhead of transit. | 21:10 |
| kmalloc | asking for a 4k object from swift | 21:10 |
| kmalloc | if i add 1k every single time, thats a 25% overhead | 21:10 |
| kmalloc | added overhead* | 21:10 |
| eandersson | Is the gate unhappy? | 21:11 |
| kmalloc | the fernet max length was set to help make swift happier with our tokens | 21:11 |
| lbragstad | eandersson: i did see some announcements yesterday pertaining to the gate | 21:11 |
| eandersson | ah I see | 21:11 |
| cmurphy | ayoung: okay, to put it concretely, if the operator creates the pre-approved whitelist with (POST, /v2.1/hypervisors, admin) and user bob with role _member_ tries to create an application credential with whitelist (POST, /v2.1/hypervisors) they'll get a 404? | 21:11 |
| ayoung | the overhead for a Fernet token seems to be 184 characters long. What would the acceptable size growth be beyond that? | 21:11 |
| lbragstad | eandersson: i think it was related to some new zuul configs | 21:11 |
| kmalloc | 255bytes was pretty much the agreed upon target | 21:11 |
| eandersson | looks like doc building is broken | 21:12 |
| kmalloc | though i could probably say we could hit 512b and top out there. | 21:12 |
| eandersson | > /home/zuul/src/git.openstack.org/openstack/keystone/doc/source/api/keystone.contrib.rst:document isn't included in any toctree | 21:12 |
| cmurphy | but if the operator-approved list has (POST, /v2.1/hypervisors, None) and user bob tries to create the application credential for that it will succeed and just be useless | 21:12 |
| kmalloc | ideally we keep the payload / token as light as possible | 21:12 |
| ayoung | cmurphy, yes, that is what I just proposed. | 21:12 |
| kmalloc | it's why we have different formatters, so we don't have to wedge everything in at once | 21:12 |
| ayoung | not sure I love it, but... | 21:12 |
| kmalloc | just the relevant data for that type (etc) | 21:12 |
| lbragstad | i think we can make that simpler with jwt | 21:13 |
| cmurphy | ayoung: okay, i'm not totally opposed but it does seem like it's duplicating different sources of truth | 21:13 |
| kmalloc | but tl;dr on the hash, -- ciphertext attacks are real | 21:13 |
| ayoung | lbragstad, I think all of this is still the same with JWT | 21:13 |
| kmalloc | hashing makes that much less likely to be possible. | 21:13 |
| lbragstad | ayoung: jwt builds payloads as key/value pairs | 21:13 |
| kmalloc | and smaller payloads are less cryptographically distinct | 21:13 |
| kmalloc | (less entropy) | 21:13 |
| lbragstad | where as keytone's fernet token formatter is relying on a order in a tuple | 21:13 |
| kmalloc | lbragstad: it might increase token size some, but i think it wont be awful. | 21:14 |
| kmalloc | lbragstad: because k/v is more than encoded(v) | 21:14 |
| lbragstad | yeah - i would expect it to | 21:14 |
| lbragstad | right | 21:14 |
| lbragstad | and it might not be msgpack'd | 21:14 |
| ayoung | cmurphy, agreedd, but I think we have that no matter what. If a user does the s (POST, /v2.1/hypervisors, None) it will pass or fail based on policy. | 21:14 |
| kmalloc | the only reason we're msgpacking is because of limits in json seralization | 21:15 |
| ayoung | all we are doing with the role is adding in guidance to say "don't do that" | 21:15 |
| lbragstad | so what happens when the mapping in keystone is out of date from what an operator specifies for a policy in nova? | 21:15 |
| kmalloc | my answer remains: the token fails | 21:16 |
| ayoung | lbragstad, something breaks | 21:16 |
| kmalloc | the answer to make that most configurable and least painful | 21:16 |
| kmalloc | allow wildcards. | 21:16 |
| ayoung | OK...which convo do we want to have first | 21:16 |
| cmurphy | lol | 21:16 |
| * lbragstad loves context switching | 21:16 | |
| kmalloc | lbragstad: you're evil | 21:17 |
| lbragstad | lol | 21:17 |
| ayoung | lbragstad, ok, here was my slim JWT format token: | 21:17 |
| ayoung | {"user_id":"d627bfb24c3043478a69fd8ade1009e9","project_id":"0b5df1731e67462393dea23b8c24c860","issued_at": "20180313162018"} | 21:17 |
| cmurphy | playing with https://jwt.io/ the size can blow up way quite a bit depending on algorithm used, best case is about 253 | 21:17 |
| cmurphy | er the debugger on that page | 21:18 |
| *** d0ugal has joined #openstack-keystone | 21:18 | |
| *** d0ugal has quit IRC | 21:18 | |
| *** d0ugal has joined #openstack-keystone | 21:18 | |
| cmurphy | unencrypted | 21:18 |
| ayoung | that gave me a 246 character token | 21:19 |
| ayoung | what would we be using for Algo etc? | 21:20 |
| cmurphy | unspecified as of yet | 21:20 |
| lbragstad | https://auth0.com/blog/critical-vulnerabilities-in-json-web-token-libraries/ | 21:21 |
| ayoung | 533 with asym? | 21:21 |
| lbragstad | fernet uses a SHA256 HMAC signing key | 21:22 |
| lbragstad | which i think is the equivalent of HS256 | 21:22 |
| ayoung | So...do we really hate fernet? | 21:22 |
| ayoung | Cuz...I kindof think as defined, it suits our needs | 21:22 |
| lbragstad | sure | 21:22 |
| ayoung | I'd like for Asym to be an alternative for peopel with key distro issues | 21:23 |
| lbragstad | i don't think we hate it - but i think people see the value in offering a second alternative | 21:23 |
| ayoung | but I dion't see a benefit to JSON in the body without making it for other people to read | 21:23 |
| lbragstad | and the asymmetric bit is interesting | 21:23 |
| kmalloc | for us, first steps to using JWT is because it is a standard with more eyes on it | 21:24 |
| lbragstad | +1 | 21:24 |
| kmalloc | even if it's opaque like fernet is in our use case | 21:24 |
| ayoung | Should we offer it as an Outreachy internship project? | 21:24 |
| kmalloc | we can explore further token data things usable by other folks when we're on it | 21:24 |
| kmalloc | and i'm open to that | 21:24 |
| ayoung | https://www.outreachy.org/communities/cfp/openstack/ | 21:24 |
| lbragstad | i know wxy and i are interested in working on this - so is gagehugo | 21:25 |
| lbragstad | we've gotten a couple requests internally for it | 21:25 |
| ayoung | Want it, or want to work on it? | 21:25 |
| cmurphy | i think it would be a great potential outreachy project as long as we nail down the implementation details beforehand | 21:26 |
| ayoung | So...no opposition here, so long as we keep Fernet for the meanwhile. Changing token options is painful, and I can't see a huge benefit, so minimial churn is all I really want. | 21:26 |
| lbragstad | ayoung: ++ | 21:27 |
| ayoung | But I can accept the "future proof" ourself with JWT argument as a good rationale. | 21:27 |
| lbragstad | i have no intention of changing the default token provider, or removing fernet | 21:27 |
| cmurphy | yeah, we don't hate fernet | 21:27 |
| * lbragstad pats fernet on the shoulder | 21:28 | |
| *** bhagyashris_ has joined #openstack-keystone | 21:28 | |
| lbragstad | i would just hate to be in a position where something happens with the fernet spec (which is somewhat unmaintained despite our efforts to participate in that community) | 21:28 |
| lbragstad | and keystone doesn't have a back up | 21:28 |
| kmalloc | i want an option that saves us from "OMG FERNET IS CRITICALLY BROKEN AND WE CANT FIX IT" | 21:28 |
| lbragstad | ^ that | 21:29 |
| kmalloc | and let folks who want more standard-y things utilize standard things. | 21:29 |
| kmalloc | but mostly the all caps bit | 21:29 |
| cmurphy | also more opportunity to integrate with non-openstack technologies | 21:29 |
| ayoung | OK...so, back to RBAC and Whitelists | 21:30 |
| ayoung | I still get pings from people interested in RBAC in middleware | 21:30 |
| ayoung | that struck a chord with people. | 21:31 |
| *** bhagyashri_s has quit IRC | 21:31 | |
| *** mburrows has quit IRC | 21:31 | |
| ayoung | I'd like to be able to move things that way, but I can accept that we have to be incremental | 21:31 |
| *** mburrows has joined #openstack-keystone | 21:31 | |
| ayoung | so what I want to get from the Whitelist is something that moves us that way while still getting the current use cases covered | 21:31 |
| ayoung | So, saying that the whitelist will still be run through existing RBAC is fine | 21:32 |
| ayoung | We can still work on splitting the RBAC into scope check vs role check, hopefully based on the data we generate to populate the whitelist | 21:33 |
| ayoung | I suspect that it won't be THAT much. | 21:34 |
| cmurphy | ayoung: i think i'm fine with this | 21:34 |
| ayoung | And it will be pretty obvious, on an URL by URL basis what role the user really should have | 21:34 |
| ayoung | I think I can get you a set of VERBS + URL Patterns for Nova, Keystone, Glance, Neutron, and Cinder based on the public docs and some HTML scraping | 21:36 |
| *** AlexeyAbashkin has joined #openstack-keystone | 21:36 | |
| kmalloc | we're not going to encode anything like that in keystone by default. | 21:37 |
| *** d0ugal has quit IRC | 21:37 | |
| kmalloc | it will have to be on an operator / distribution to provide that information | 21:37 |
| cmurphy | ayoung: can we say for now that there's an optional role field in the operator-approved list and leave the details of how to generate the list until the next stage? | 21:37 |
| ayoung | Yep | 21:37 |
| cmurphy | cool | 21:38 |
| ayoung | kmalloc, right. I was thinking more like we produce documents that we can post upstream that can be consumed by the operators as a starting point | 21:38 |
| kmalloc | sure, i wont be opposed to that, just as long as we aren't encoding anything in keystone itself by default. | 21:39 |
| *** AlexeyAbashkin has quit IRC | 21:41 | |
| ayoung | kmalloc, yeah, this should be "layered on top of" as opposed to "replacing" to avoid breaking things | 21:44 |
| openstackgerrit | Russell Tweed proposed openstack/keystone master: Add prerequisite package note to Keystone install guide https://review.openstack.org/552568 | 21:52 |
| abhi89 | cmurphy: so is there anything planned to remove the remaining references of "auth_uri" in R release? | 21:54 |
| ayoung | felipemonteiro_, ok....so tell me how patrole is going to check RBAC? | 21:59 |
| *** rcernin has joined #openstack-keystone | 21:59 | |
| cmurphy | abhi89: i don't think we have anything explicitly planned, partly i was hoping the other projects would take care of it themselves ;) but you're right it's important and we should follow through on it | 22:03 |
| cmurphy | abhi89: i don't think we can be held responsible for every project though | 22:07 |
| *** abhi89 has quit IRC | 22:07 | |
| cmurphy | going through http://codesearch.openstack.org/?q=auth_uri&i=nope&files=&repos= a lot of these are deployment projects that should be keeping on top of this themselves | 22:07 |
| *** abhi89 has joined #openstack-keystone | 22:07 | |
| cmurphy | we can at least make sure devstack works when switched and file bugs for the projects that break it | 22:10 |
| felipemonteiro_ | ayoung: it calls oslo.policy for expected result, invokes tempest api endpoint that enforces the expected policy, compares expected/actual results for consistency | 22:12 |
| cmurphy | lbragstad: hrybacki can/should we pull https://trello.com/c/AXrDT1s0/49-keystonemiddleware-improvements as an epic onto the rocky roadmap? it has some unfinished items, plus followup on www_authenticate_uri impact plus removing all the deprecated crap could be captured there | 22:13 |
| ayoung | felipemonteiro_, how does it know what URL to call for a policy rule? | 22:14 |
| felipemonteiro_ | policy in code documentation for all services that have converted over | 22:15 |
| felipemonteiro_ | only missing one is neutron | 22:15 |
| ayoung | felipemonteiro_, link? | 22:16 |
| ayoung | "policy in code documentation" gives the URL pattern? | 22:16 |
| felipemonteiro_ | nova example: https://github.com/openstack/nova/blob/master/nova/policies/agents.py#L35 | 22:16 |
| felipemonteiro_ | keystone example: https://github.com/openstack/keystone/blob/master/keystone/common/policies/group.py#L31 | 22:17 |
| ayoung | So we are going to be going with the roles as hard coded into the source. Wonderful | 22:17 |
| ayoung | felipemonteiro_, before you do that, I have a bug for you to fix | 22:17 |
| ayoung | https://bugs.launchpad.net/keystone/+bug/968696 | 22:17 |
| openstack | Launchpad bug 968696 in OpenStack Identity (keystone) ""admin"-ness not properly scoped" [High,In progress] - Assigned to Adam Young (ayoung) | 22:17 |
| hrybacki | cmurphy: will do | 22:18 |
| ayoung | base.RULE_ADMIN_API | 22:18 |
| felipemonteiro_ | well the patrole gates just check for the default roles currently established in openstack -- it's designed to check for any role you want though | 22:18 |
| *** martinus__ has quit IRC | 22:18 | |
| ayoung | felipemonteiro_, and it is outside of the proejcts | 22:20 |
| ayoung | so if they change the RBAC, patrole will complain | 22:20 |
| ayoung | and it will make it hard to fix the Longest standing bug in openstack | 22:21 |
| *** edmondsw has quit IRC | 22:21 | |
| felipemonteiro_ | well, there's a bit of ptg background discussion you missed, but the intention is to have it non-voting and, moreover... | 22:22 |
| ayoung | felipemonteiro_, this is just so bad...it is exactly tjhe opposite of what we need | 22:22 |
| ayoung | felipemonteiro_, your heart is in the right place | 22:22 |
| felipemonteiro_ | even if it is voting (hypothetically) then in-repo zuul.yaml can be changed quickly to relegate back to non-voting | 22:22 |
| ayoung | felipemonteiro_, please no? | 22:22 |
| ayoung | fix first | 22:22 |
| ayoung | then patrole | 22:22 |
| felipemonteiro_ | the background discussion fyi is that lance and redhat talked about it and asked that i join a session to discuss gating around it in keystne | 22:23 |
| ayoung | felipemonteiro_, do you understand the bug I linked to? | 22:23 |
| felipemonteiro_ | i understand the gist of it, which is that if you have admin in one project you have it universally | 22:24 |
| ayoung | felipemonteiro_, at lease wait until after lbragstad gets a system scoped rule in there | 22:24 |
| ayoung | lbragstad, what is the policy to replace "role:admin", | 22:24 |
| ayoung | with system scoped roles? | 22:24 |
| felipemonteiro_ | well, i'm not trying to jump the gun. i will update the spec and let it play out. | 22:24 |
| felipemonteiro_ | it's entirely up to you ayoung (and fellow cores) whether it gets merged :) | 22:24 |
| ayoung | "role:admin and something:system", | 22:24 |
| ayoung | felipemonteiro_, do you understand the problem of bug 968696 and how patrole will make it harder to fix? | 22:25 |
| openstack | bug 968696 in OpenStack Identity (keystone) ""admin"-ness not properly scoped" [High,In progress] https://launchpad.net/bugs/968696 - Assigned to Adam Young (ayoung) | 22:25 |
| ayoung | felipemonteiro_, if you don't understand that bug, please go read up on it. It really should be a CVE, but the world has known about it for ever. | 22:26 |
| ayoung | Its really bad | 22:27 |
| ayoung | and it comes up in just about every conversation around policy | 22:27 |
| felipemonteiro_ | well, i agree with you to an extent. but pragmatically speaking, i highly doubt any keystone cores will refuse to fix the issue keystone-side simply because a non-voting gate fails. | 22:27 |
| ayoung | cmurphy, do you know what the "grandfather in the admin " approach is going to be for system roles/ | 22:27 |
| ayoung | ? | 22:27 |
| ayoung | felipemonteiro_, until someone makes it non-voting | 22:28 |
| ayoung | felipemonteiro_, that happend with Tempest to us in the past | 22:28 |
| ayoung | people coded tests based on broken behavior, and then we were in a catch 22 to keep us from fixing it | 22:29 |
| felipemonteiro_ | i wouldn't advocate for it to even be voting, not for at least another 2 cycles (neutron hasn't gone to policy in code yet). | 22:29 |
| felipemonteiro_ | but regardless, i don't really have an opinion here on when it should be added to keystone as a gate. that is for you guys to decide. i just need to update the spec with the discussion feedback from ptg. | 22:30 |
| *** spilla has quit IRC | 22:30 | |
| cmurphy | ayoung: you mean for getting things like 'list all servers in all projects' to work? i think both you and kmalloc are advocating leveraging inherited roles for this, we'll just need to document that better | 22:31 |
| cmurphy | ayoung: we had some discussion on it on https://trello.com/c/L09dRjq4/73-associating-projects-to-system-scoped-tokens | 22:31 |
| * kmalloc reads that | 22:31 | |
| kmalloc | i.. am not sure i'm advocating that. | 22:31 |
| ayoung | cmurphy, not that | 22:31 |
| ayoung | cmurphy, I was talking about how the nova policy for is_admin is currently | 22:32 |
| ayoung | role:is_admin | 22:32 |
| ayoung | er | 22:32 |
| ayoung | role:admin | 22:32 |
| ayoung | and it needs to be role:admin and token-scope:system | 22:32 |
| kmalloc | ayoung: i think with system-scope we're planning to work with nova to migrate from role:admin to something system-scope-y | 22:32 |
| ayoung | I had the is_admin_project hack in there | 22:33 |
| ayoung | https://review.openstack.org/#/c/384148/ | 22:33 |
| kmalloc | but yes something like that first thennnnn moving away from role:admin longer term | 22:33 |
| ayoung | which worked because all projects show up is_admin until you enforce a specific one in Keystone config | 22:33 |
| cmurphy | ayoung: ah sorry, we'll work with them to change their policy rules https://trello.com/c/XcqlDqWN/43-work-with-nova-to-implement-scopetypes | 22:33 |
| ayoung | so what is the plan. Please tell me there is a plan. This delayed the fix a year at this point. | 22:33 |
| ayoung | So unless we make it OPT IN we are going to break people that upgrade | 22:34 |
| kmalloc | what cmurphy linked. | 22:34 |
| kmalloc | the plan: start with both functioning | 22:34 |
| kmalloc | long term is as nova becomes more aware of system-scope (for example) migrate to system-scope only. it's a long play | 22:34 |
| ayoung | kmalloc, how do we migrate? | 22:35 |
| kmalloc | support both | 22:35 |
| kmalloc | liek you said, to start | 22:35 |
| ayoung | we need something that works for both new and old, with an opt in switch | 22:35 |
| kmalloc | with an EOL on the non-system-scope mode | 22:35 |
| kmalloc | that *is* the migration | 22:35 |
| cmurphy | yep | 22:36 |
| kmalloc | support both, make sure everything works with system-scope only (openstack-specific), set EOL on non-system-scope, EOL non-system-scope in accordance with timeline | 22:36 |
| kmalloc | allow for opt in along the way to system-scope only | 22:36 |
| ayoung | So that code there is not sufficient | 22:36 |
| kmalloc | the code implemented is getting system-scope working | 22:37 |
| ayoung | IIUC, it will break when a user comes in with an admin role on the wrong project | 22:37 |
| kmalloc | it is not meant to be all inclusive as it sits afaict. | 22:37 |
| kmalloc | implementation is a multi-cycle effort | 22:37 |
| ayoung | I think we need to start with is_admin at the base level | 22:37 |
| kmalloc | let alone the rest. | 22:37 |
| lbragstad | felipemonteiro_: o/ | 22:37 |
| lbragstad | i have something for you | 22:37 |
| kmalloc | no, we need to have functional system scope that *can* is_admin, do not start with is_admin | 22:38 |
| kmalloc | that is broken | 22:38 |
| kmalloc | leverage system-scope to support is_admin and change as needed to get there | 22:38 |
| kmalloc | but starting from "bad design" is still bad design | 22:38 |
| ayoung | here is what I started with https://review.openstack.org/#/c/384148/24/nova/policies/base.py | 22:38 |
| felipemonteiro_ | lbragstad: hi | 22:38 |
| ayoung | so... | 22:38 |
| lbragstad | felipemonteiro_: i was trying to update the protection tests in keystone to be a bit more granular and organized - https://review.openstack.org/#/c/551337/3 | 22:38 |
| ayoung | 'global_admin', | 22:38 |
| ayoung | 'is_admin:True and is_admin_project:True', | 22:38 |
| ayoung | 'Global admin'), | 22:38 |
| ayoung | to start that would be: | 22:38 |
| lbragstad | especially if we end up with three default roles and multiple scope | 22:39 |
| lbragstad | scopes* | 22:39 |
| lbragstad | felipemonteiro_: curious if you have any feedback on that from a testing perspective | 22:39 |
| ayoung | 'is_admin:True and (scope:System or scope:Project' | 22:39 |
| ayoung | 'is_admin:True and (scope:System or scope:Project)' | 22:39 |
| ayoung | that would allows system scoped tokens to be used | 22:39 |
| kmalloc | that seems like a startign place | 22:40 |
| kmalloc | though we neeeeeed to have temptest testing that method as well as a system-scope only mechanism | 22:41 |
| kmalloc | and remember only things like hypervisor mucking apis will be system-scope | 22:41 |
| kmalloc | not any "is_admin" thing | 22:41 |
| ayoung | kmalloc, eventually. But right now, enforcing that will break people using admin tokens scoped to projects to get work down | 22:41 |
| ayoung | done | 22:41 |
| kmalloc | right so the answer is what you just described as a starting place | 22:42 |
| kmalloc | you support both | 22:42 |
| ayoung | all APIs should be executable via a system scoped token | 22:42 |
| lbragstad | .... | 22:42 |
| felipemonteiro_ | lbragstad: looking | 22:43 |
| ayoung | but we need a flag to disable project scoped tokens from certain apis that can be set from config or from keystone | 22:43 |
| lbragstad | i don't think we should support system scoped tokens for project scoped operations in the name of backwards compatibility | 22:43 |
| lbragstad | that's not to say we shouldn't support that for is_admin_project tokens | 22:43 |
| ayoung | lbragstad, we need them anyway | 22:43 |
| ayoung | to clean up resources that are orphaned when the projects are deleted | 22:44 |
| ayoung | or to let an admin go and set things up for other users etc | 22:44 |
| ayoung | I'm less worried about that than the reveres | 22:44 |
| lbragstad | for the second case a system admin can give themselves authorization on the project to do that | 22:45 |
| ayoung | heh | 22:45 |
| lbragstad | and use the recommended flow to set things up | 22:45 |
| ayoung | deja vu | 22:45 |
| ayoung | they don't want that | 22:45 |
| ayoung | admins want to be able to do multi-project operations with a single token | 22:45 |
| ayoung | we got that message very loudly back in the HMT time frame | 22:45 |
| ayoung | I really am more concerned with removing the project scoped admin role from system operations than anything else | 22:46 |
| ayoung | so...what about this: | 22:46 |
| ayoung | we make a horribbly complicated but effective rule that checks for a config flage and, if it is not set, or set to false, falls back to the existing behavior | 22:47 |
| ayoung | if it is set, it only lets system scope work on system scoped rules | 22:47 |
| kmalloc | system-scope (long term vision) should never cross over to any project-scoped actionx | 22:48 |
| kmalloc | s | 22:48 |
| kmalloc | ever | 22:48 |
| ayoung | and we tag the APIs that should be system scoped in nova like you started doing wuith https://review.openstack.org/#/c/525772/ | 22:48 |
| * lbragstad steps away for a minute | 22:48 | |
| ayoung | kmalloc, as much as I appreciate that sentiment, that was not what the operators told us | 22:48 |
| kmalloc | if you want a "clean up orphaned thing", that should be it's own api -- with a pass of the project_id | 22:48 |
| ayoung | at a MINIMUM we need to be able to delete. | 22:48 |
| kmalloc | NOT a "scope to a dead project" | 22:48 |
| kmalloc | ever. | 22:48 |
| kmalloc | no. | 22:48 |
| kmalloc | no | 22:48 |
| ayoung | because once you delete a project, you cannot get a token scoped to that | 22:49 |
| ayoung | kmalloc, why do you hate operators? | 22:49 |
| ayoung | :) | 22:49 |
| kmalloc | so make a nova api that can cleanup everything for a passed in id | 22:49 |
| kmalloc | i don't hate operators | 22:49 |
| kmalloc | i will -2 things that conflate project scope to system-scope | 22:49 |
| kmalloc | system-scope can have a "cleanup all resources, even if project is dead, for id X" | 22:49 |
| kmalloc | system scope cannot have a "make me scoped to project x" mechanism | 22:49 |
| ayoung | are you going to implement this? | 22:49 |
| kmalloc | no, you need it, you will ;) | 22:50 |
| ayoung | or are we going to wait another decade? | 22:50 |
| ayoung | I did | 22:50 |
| ayoung | and it sat there | 22:50 |
| kmalloc | well, it's a nova thing | 22:50 |
| ayoung | I'm the pragmatist here | 22:50 |
| kmalloc | not something i have the ability to approve, sadly | 22:50 |
| ayoung | not nova | 22:50 |
| ayoung | it is keystone's problem | 22:50 |
| kmalloc | no, it is not | 22:50 |
| kmalloc | if you want to cleanup resources orphaned in service X, you must make service X able to clean it up | 22:51 |
| kmalloc | and that is a system-scope action | 22:51 |
| kmalloc | allowing keystone to scope tokens to deleted projects is a non-starter | 22:51 |
| ayoung | kmalloc, yeah, but guess what? Every other cloud service out there does not do that | 22:51 |
| ayoung | AWS, Azure, and Kubernetes all clean up the resources when you delete a project | 22:51 |
| kmalloc | except they do. | 22:51 |
| kmalloc | you're not scoping in those to a deleted service | 22:51 |
| kmalloc | you're administrativly saying "cleanup resources for X" | 22:52 |
| ayoung | we are not deleting upon demand | 22:52 |
| kmalloc | there is a difference | 22:52 |
| kmalloc | i'm sorry you're not seeing it | 22:52 |
| kmalloc | this is not a keystone problem. | 22:52 |
| ayoung | kmalloc, so, I started at this point, and was talked out of it | 22:52 |
| kmalloc | keystone cannot scope to dead projects, it allows for anything to happen for projects that don't exist | 22:52 |
| ayoung | I wanted us to be able to recreate a dead project so we could clean up resources | 22:52 |
| kmalloc | it's a massssssssssive security issue | 22:53 |
| ayoung | by issing tokens for it | 22:53 |
| kmalloc | and that is a broken model | 22:53 |
| ayoung | Keystone is currently operating on a broken model | 22:53 |
| ayoung | and this is not the worst broken part of it | 22:53 |
| kmalloc | nova and neutron and cinder, etc need to be able to do an administrative "project x - delete all owned resources for" | 22:53 |
| kmalloc | it is stupid to make it "scope to a project and do normal deletion of each element" | 22:53 |
| ayoung | so...I'll grant you that write operations should be scoped, as much as that will break cloudforms and everything | 22:53 |
| kmalloc | if the project is dead | 22:53 |
| ayoung | but read needs to be done with a token scoped larger than proejct | 22:54 |
| kmalloc | you're now conflating issues | 22:54 |
| ayoung | no... | 22:54 |
| kmalloc | read != cleanup unless youre doing it the bad way | 22:54 |
| kmalloc | for orphaned resources | 22:54 |
| ayoung | operators NEED to be able to read the whole tree | 22:54 |
| ayoung | without rescoping | 22:54 |
| kmalloc | ok, lets back up | 22:54 |
| kmalloc | you're context switching massively to different arguments | 22:55 |
| kmalloc | you're asking for cleanup of orphaned resources | 22:55 |
| kmalloc | i'm saying that is a nova (or neutron) concern to support | 22:55 |
| ayoung | kmalloc, hold on | 22:55 |
| kmalloc | you're now saying "I want to list everything" | 22:55 |
| ayoung | I am giving two examples of cases where someone needs a token not scope to a proejct to perform an operation | 22:55 |
| kmalloc | that also feels like system-scope. but different. not scoped to a project, even a dead on | 22:56 |
| kmalloc | e | 22:56 |
| ayoung | and, I really wanted HMT to be the solutions to both of those | 22:56 |
| kmalloc | so... | 22:56 |
| kmalloc | we cannot solve this in keystone | 22:56 |
| kmalloc | this is not a keystone problem | 22:56 |
| kmalloc | this is a "we supply mechnisms to provide system-scope actions (non-project-scoped)" | 22:57 |
| ayoung | There is no where to delegate this problem to | 22:57 |
| kmalloc | it is up to the services to define things such as "clean up all resources for X" | 22:57 |
| kmalloc | or "list every single instance in the entire world" | 22:57 |
| ayoung | why? | 22:57 |
| kmalloc | because keystone CANNOT supply a mechanism that does that | 22:57 |
| ayoung | and they say "we already do that" | 22:57 |
| ayoung | and then we file bug 968696 a | 22:57 |
| openstack | bug 968696 in OpenStack Identity (keystone) ""admin"-ness not properly scoped" [High,In progress] https://launchpad.net/bugs/968696 - Assigned to Adam Young (ayoung) | 22:57 |
| kmalloc | we can only issue RBAC and Scope or SystemScope | 22:58 |
| ayoung | and then they mark it wishlist | 22:58 |
| kmalloc | i'm going to close that bug administatively and say "will not fix as described" :P | 22:58 |
| kmalloc | in all seriousness | 22:58 |
| kmalloc | keystone cannot solve this | 22:58 |
| kmalloc | nova must supply the mechanism to do this | 22:58 |
| ayoung | so don;'t close that | 22:59 |
| kmalloc | we are creating system-scope for things such as "clean up all resources for abandoned project" | 22:59 |
| kmalloc | or other very-global things | 22:59 |
| kmalloc | that are outside the scope of normal operations | 22:59 |
| felipemonteiro_ | lbragstad: reviewed | 22:59 |
| kmalloc | fwiw, i think that bug needs to be closed and new bugs explicitly opened for the various other mechanisms we now are defining | 23:00 |
| kmalloc | that bug doesn't help much anymore. as we're moving to a new mechanism to isolate actions | 23:00 |
| kmalloc | and FTR, i wouldn't close the bug directly like i said :P | 23:00 |
| kmalloc | so the solution is: | 23:01 |
| kmalloc | 1) System Scope becomes a thing, some actions need to be move to system scope | 23:01 |
| kmalloc | 2) doing actions such as "Create a VM on project for user" should not be system-scope, it should be handled (possibly) from an admin role inherited from <<keystone.domain.root>> | 23:01 |
| kmalloc | " | 23:01 |
| ayoung | kmalloc, as much as I want to agree with you, we can't do that | 23:02 |
| kmalloc | 3) some new APIs will be needed for cases such as cleanup orphaned resources, that should be system-scope | 23:02 |
| ayoung | it will break all the tooling out there | 23:02 |
| kmalloc | this is all additive | 23:02 |
| kmalloc | there is nothing breaking | 23:02 |
| kmalloc | it is supplying better ways forward. | 23:02 |
| kmalloc | and we're not adding more terrible "wedge it into a bad model" mechanisms | 23:03 |
| ayoung | "Create a VM on project for user" should not be system-scope, it should be handled (possibly) from an admin role inherited from <<keystone.domain.root>>" | 23:03 |
| *** d0ugal has joined #openstack-keystone | 23:03 | |
| kmalloc | yes, so scope to the project | 23:03 |
| kmalloc | with your inherited role | 23:03 |
| kmalloc | and make a VM | 23:03 |
| ayoung | so are we going to issue tokens scoped to a whole tree? | 23:03 |
| kmalloc | works just like anything else | 23:03 |
| kmalloc | no, scope to the project | 23:03 |
| ayoung | kmalloc, you are not hearing me | 23:03 |
| kmalloc | because the role is inherited from keystonje.domain.root, it exists on all projects down the whole tree | 23:03 |
| ayoung | we proposed that and the operators shouted it down. | 23:04 |
| ayoung | there were proposals to have tokens scoped to multiple projects, etc | 23:04 |
| kmalloc | why? | 23:04 |
| kmalloc | why did they shoot it down | 23:04 |
| ayoung | because when you run a cloud, you want to see everything, not just project scoped | 23:04 |
| kmalloc | i have no insight, i am getting strictly "we need this" because my customer demands it | 23:04 |
| kmalloc | that isn't helpful | 23:04 |
| ayoung | kmalloc, it was not my customer | 23:04 |
| kmalloc | ok we're talking past each other | 23:04 |
| ayoung | this was in convos at the Summit back in the HMT time frame | 23:05 |
| kmalloc | i'm walking away before i can't communicate anymore with you today | 23:05 |
| ayoung | ask Henry | 23:05 |
| *** abhi89 has quit IRC | 23:05 | |
| ayoung | kmalloc, ok, lets punt on that for a moment | 23:05 |
| kmalloc | i have never heard anything from anyone that says "if i need to perform an action for a user i'm opposed to scoping to thier project" | 23:05 |
| kmalloc | except from henry, and the reason was "because my customer says so" | 23:05 |
| kmalloc | there is nothing saying you can't allow listing of every project in the tree (today or in the future) | 23:06 |
| ayoung | kmalloc, nah, it was in the forum when talking to operators at the summit. I was actually the one that was proposing just what you are proposing | 23:06 |
| kmalloc | which operator was it? | 23:06 |
| ayoung | so...the oproblem was that the remote projects do not know the tree | 23:06 |
| kmalloc | i'm really only ever heard "i don't like that" | 23:06 |
| kmalloc | the non-keystone-services? | 23:06 |
| ayoung | we can't issue a token to project parent and they will know the relationship between that and the child | 23:07 |
| kmalloc | lets back waaaaaaay the heck up | 23:07 |
| kmalloc | stop, talking about this | 23:07 |
| kmalloc | what are we solving | 23:07 |
| kmalloc | What problem | 23:07 |
| kmalloc | define the problem as narrowly as possible | 23:07 |
| ayoung | what i want to solve is to remove role:admin being the only check done for operations | 23:07 |
| kmalloc | because we've migrated to yet another use case | 23:07 |
| kmalloc | what operaton(s) | 23:08 |
| ayoung | my proposal was to convert those to role:admin scope:system | 23:08 |
| kmalloc | each must be evaluated | 23:08 |
| kmalloc | you can't just blindly convert | 23:08 |
| kmalloc | part of this was very deliberate converting the right apis to system-scope | 23:08 |
| kmalloc | system-scope is by design narrow, it is meant for actions that are explicitly not project/domain scoped | 23:08 |
| ayoung | what you are proposing is splitting them so they are mutually exclusive: either you have project scoped token (however you got it) and you can do project operations, or you have a system scoped token, and you can do system operations | 23:08 |
| kmalloc | right | 23:09 |
| kmalloc | and a system-op is not "create an instance" | 23:09 |
| ayoung | and, top be honest, I agree with you that is how it should be, in my world view | 23:09 |
| kmalloc | it is things like "disable hypervisor X" | 23:09 |
| kmalloc | and potentially "create global image in glance Y" | 23:09 |
| ayoung | so, the problem is that changing that is a fundamental assumption that was used to build the management tools | 23:09 |
| kmalloc | breaking apart (to start) things that it stupid to be project based | 23:09 |
| kmalloc | then the management tools need to *also* migrate | 23:10 |
| ayoung | that predates Keystone | 23:10 |
| kmalloc | that is part of "support both" | 23:10 |
| kmalloc | and then "set EOL date of non-system scope" | 23:10 |
| kmalloc | then "EOL non-system scope" | 23:10 |
| kmalloc | some tools will cease to work if they were written for say kilo, and in the V release of openstack | 23:11 |
| kmalloc | i hope to god you're not using a kilo tool on "V" release of openstack | 23:11 |
| kmalloc | i don't expect it to work | 23:11 |
| ayoung | HMT would make sense as an alternative, but we need to convey more information than we do right now | 23:11 |
| kmalloc | and new functions that should be system-scope only wont be "project scoped" at all | 23:11 |
| kmalloc | I am advocating leaning on HMT more for the non-system-scope work | 23:12 |
| ayoung | kmalloc, I am totally using a K tool on current openstack. CloudForms moves slowly | 23:12 |
| kmalloc | and that is fine. | 23:12 |
| ayoung | and I don't think that the rest of the world moves much faster | 23:12 |
| kmalloc | but on V, if there is a 2-3 year lead | 23:12 |
| kmalloc | thats fine | 23:12 |
| kmalloc | the EOL timeline will be about the timeframe we're talking about | 23:13 |
| ayoung | so...the problem with HMT is that the remote services do not know the tree | 23:13 |
| ayoung | so either we serialize it in the token, or they have to fetch it | 23:13 |
| kmalloc | ok, now we can talk about that. | 23:13 |
| ayoung | quota is going to have to do that anyway,right? | 23:13 |
| kmalloc | and that is a very different issue than what we started with | 23:13 |
| kmalloc | or at least how the convo was phrased | 23:13 |
| kmalloc | so serialize it in the token response | 23:14 |
| ayoung | I was going to punt on it | 23:14 |
| kmalloc | it has a fixed limit on depth | 23:14 |
| kmalloc | (or... it should...) | 23:14 |
| kmalloc | now, do you want the whole tree? | 23:14 |
| kmalloc | or just the current branches | 23:14 |
| kmalloc | if you want the whole tree...it's more difficult | 23:14 |
| kmalloc | but still ultimately doable | 23:14 |
| kmalloc | quota is ... weird. | 23:14 |
| kmalloc | and we haven't really moved much on the limit stuff (we're just starting to build the library) | 23:15 |
| kmalloc | and it might be "fetch quota" | 23:15 |
| kmalloc | with caching | 23:15 |
| *** felipemonteiro_ has quit IRC | 23:16 | |
| kmalloc | ayoung: this is why i want to see some mechanism for all services to beable to directly query the authoritative keystone data [like resources, user [not secure, e.g. password hashes], etc] | 23:16 |
| kmalloc | keep local caches or via something like consul, a DHT, etc. | 23:16 |
| kmalloc | it's on my long long long long list of "try to solve" | 23:16 |
| kmalloc | but i missed the PTG and generally don't get to the planning sessions | 23:17 |
| ayoung | I thought you were going to nix any caching... | 23:17 |
| kmalloc | no, caching is going to be required. it just has to be smart enough | 23:17 |
| kmalloc | so we can handle mostly-consistent | 23:17 |
| kmalloc | caching of data is fine as long as its done with the expectation that it'll be mostly-consistent, and in absence (disabled) of cache, we admit it will be slower and deal with the direct query for all operations | 23:19 |
| kmalloc | we already do not allow "moving" things in the HMT tree | 23:20 |
| kmalloc | that means the worst that happens is a project is unknown (hey, refresh your cache) or a project is deleted (Stale cache) we can work with. | 23:20 |
| ayoung | kmalloc, OK...follow me here for a moment | 23:21 |
| ayoung | there are a lot of 3rd part applications writtne to talk to OpenStack | 23:21 |
| ayoung | and they were built using the assumptions that we published to the world | 23:22 |
| ayoung | a big one is that you do not need to go and get a new token when do admin operations | 23:22 |
| ayoung | and they wrote those operations to be pretty damn wide spread | 23:22 |
| kmalloc | and things evolve. | 23:22 |
| kmalloc | and consuming applications learn and grow, especially when its much much much better | 23:23 |
| ayoung | one of the big ones is that, without getting an other token, they can list all servers | 23:23 |
| kmalloc | it's what major versions are for (grump semver death) | 23:23 |
| kmalloc | that has never really been true | 23:23 |
| kmalloc | it has been true in a large subset of openstack | 23:23 |
| kmalloc | but not globally | 23:24 |
| ayoung | and since these are written in languages other than python, they don't go through our tooling | 23:24 |
| ayoung | so we cannot break these tools | 23:24 |
| kmalloc | if everything was as static as you're painting things, nothing would ever move forward | 23:24 |
| kmalloc | it seems openstack is held to some higher standard that can never change anything | 23:24 |
| ayoung | kmalloc, everything is as static as I am painting things and that is why nothing has moved forward | 23:24 |
| ayoung | plus, once we had a plan to fix things, everyone that was supporting that plan got fired | 23:25 |
| kmalloc | if you provide new mechanisms and EOL timeframes, tools will move | 23:25 |
| *** gongysh has joined #openstack-keystone | 23:25 | |
| kmalloc | if you design everything by comittee of every consumer, nothing ever will be done. | 23:25 |
| *** gongysh has quit IRC | 23:26 | |
| ayoung | really? I hadn;t noticed | 23:26 |
| kmalloc | frankly, there have been breaking changes to qemu, libvirt, and even linux kernel. | 23:26 |
| kmalloc | it's communicated, telegraphed, set timelines | 23:26 |
| kmalloc | and changed | 23:26 |
| kmalloc | tooling will move | 23:26 |
| kmalloc | be kind, but don't be frozen | 23:26 |
| ayoung | so...I think you are the only person that is willing to die on the hill of "service tokens can't do project level operations" | 23:26 |
| kmalloc | ask lance to remove my +2/-2 powers and i wont die on that hill | 23:27 |
| kmalloc | but to be frank, yes. i will die on that hill | 23:27 |
| ayoung | kmalloc, I'd rather have you provide a viable route forward for a proposal that you are blockinfg | 23:27 |
| ayoung | because I am going to need that +2 in the future | 23:27 |
| *** gongysh has joined #openstack-keystone | 23:27 | |
| ayoung | we are a disfunctional group here | 23:27 |
| kmalloc | i have been trying to, but nothing is viable because $new reason$ | 23:27 |
| ayoung | kmalloc, this ain't new. And this is not the real security hole | 23:28 |
| ayoung | the real security hole is the adminness thing | 23:28 |
| kmalloc | system-scope should NOT do project actions. | 23:28 |
| ayoung | so we phase that out over tiome | 23:28 |
| kmalloc | it is trying to split the concerns so you aren't trying to conflate things. | 23:28 |
| ayoung | time | 23:28 |
| kmalloc | which is what is the problem and why we can't solve the admin-ness | 23:28 |
| kmalloc | or is a large part of it | 23:29 |
| kmalloc | not the only part | 23:29 |
| ayoung | project scoped should not do project scoped things in other projects | 23:29 |
| kmalloc | and domain scope (bear with me) should live above projects | 23:29 |
| kmalloc | and inherited roles allow for scoping down the tree and performing actions on a child. | 23:29 |
| ayoung | ok...we can't break things | 23:29 |
| ayoung | we need to be able to do this in a step by step manner | 23:29 |
| kmalloc | and we've covered the steps | 23:29 |
| ayoung | so, one...enable system scoped for some set of operations | 23:30 |
| kmalloc | 1) implement system-scope | 23:30 |
| kmalloc | 2) make sure both work for current APIs | 23:30 |
| ayoung | so far so good | 23:30 |
| kmalloc | 3) make sure system-scope only works (for openstack-specificl tooling, e.g. tempest, ci, etc) | 23:30 |
| kmalloc | (this doesn't change/disable anything, just runs full testing with system-scope only support for designated APIs) | 23:30 |
| ayoung | understood | 23:31 |
| kmalloc | 4) set a clear EOL on non-system-scope by default (people can always add it back in) | 23:31 |
| kmalloc | 5) turn off by default the non-ssystem scope. | 23:31 |
| kmalloc | at the right EOL point | 23:31 |
| kmalloc | still can be added back in | 23:31 |
| kmalloc | -- now we have system split from actual scoped actions | 23:31 |
| kmalloc | admin project, vs admin domina | 23:32 |
| kmalloc | but you're not conflating "disable hypervisor" with "admin in project" | 23:32 |
| ayoung | ok, now run that past the operators group. Tell them that you are not going to be able to do all operations with any single token. | 23:32 |
| kmalloc | and i bet i can win them over (or a majority) | 23:32 |
| ayoung | let me see if I can find the discussion in the archives | 23:32 |
| kmalloc | but at step 3, is where we start working on better bits of HMT and other project stuff. | 23:33 |
| kmalloc | the deal is things like "add an endpoint" will be no different (except you'd says scope: system) | 23:33 |
| kmalloc | this is not really a lot different than today. | 23:33 |
| kmalloc | most things that will be system-scope specific will be things that should be isolated | 23:33 |
| ayoung | kmalloc, so the people that this effects are the people running clouds that need to clean things up for their customers, either internal or external | 23:34 |
| ayoung | not endspoint operations, but end user operations | 23:34 |
| kmalloc | the argument and the way you pitch it is that it allows (especially in bigger deployments) isolation of what peoplewho need to work on the control plane, but should not be able to make instances | 23:34 |
| kmalloc | you're still not breaking typical user-or-admin-use-of-openstack operations | 23:35 |
| ayoung | Domain scoped tokens would probably be workable in practice. | 23:35 |
| ayoung | But the remote services don't know what domain a project is in either | 23:35 |
| kmalloc | we can start leaning on HMT and domains more when we're no longer worried about "this breaks all of openstack" | 23:35 |
| kmalloc | in my deployment | 23:35 |
| kmalloc | because someone turned off my hypervisors | 23:35 |
| ayoung | and we can't automate away the token fetch because the tooling is in languages other than python | 23:35 |
| *** oikiki has quit IRC | 23:36 | |
| ayoung | but... | 23:36 |
| *** oikiki has joined #openstack-keystone | 23:36 | |
| *** AlexeyAbashkin has joined #openstack-keystone | 23:36 | |
| kmalloc | because we now have "manage openstack" and "use openstack" buckets of concern | 23:36 |
| ayoung | and this is enforced at policy time, not token validation | 23:36 |
| ayoung | token validation we assume has to be cached | 23:36 |
| kmalloc | it is worth fighting this battle, i've had a lot of folks doing a lot of work around this because it's painful and scary | 23:36 |
| ayoung | but... | 23:36 |
| *** edmondsw has joined #openstack-keystone | 23:36 | |
| ayoung | what if we had a fallback? | 23:37 |
| kmalloc | is_admin_project is part of that | 23:37 |
| kmalloc | and it's because it's scary | 23:37 |
| ayoung | cases where policy failed but we could say "go ask keystone" | 23:37 |
| kmalloc | except keystone doesn't know about nova's policay | 23:37 |
| kmalloc | policy | 23:37 |
| kmalloc | or what is acceptable | 23:37 |
| kmalloc | you and i tried that path multiple times. | 23:37 |
| kmalloc | so, what is being asked of keystone? | 23:38 |
| kmalloc | (in that fallback) | 23:38 |
| ayoung | what if...when an operator wiuth an HMT token tries to do something, and the policy fails because the scope doesn't match, the policy enforcement says "ohm, this is an HMT token? Let me ask keystone again, with more context" | 23:38 |
| kmalloc | what is the failure | 23:38 |
| kmalloc | "i don't know the tree, because i've not seen the projct" | 23:38 |
| ayoung | ok...to keep it simple | 23:38 |
| ayoung | lets say I have a domain scoped toke | 23:38 |
| ayoung | and I use it on nova, which knows nothing about domains | 23:39 |
| ayoung | I try to create a service in project P | 23:39 |
| ayoung | polic rejects it, but... | 23:39 |
| kmalloc | so, if nova allows user to create vms for a scope not their own (afaik that is all custom things not in upstream) | 23:39 |
| ayoung | wait...its a domains scoped token...revalidate, but add in "for proejct P" a | 23:39 |
| kmalloc | then that is very reasonable to re-ask keystone or at least refresh it's cache | 23:39 |
| kmalloc | to know if this makes sense. | 23:40 |
| ayoung | essentially ask "does Dom D contain project P" | 23:40 |
| kmalloc | yeah, that's fine. i really don't care how nova does that... | 23:40 |
| ayoung | domain scoped tokens would be the slow path | 23:40 |
| kmalloc | i just don't want to have the concern where we end up with that use for things that shouldn't even be project scoped. | 23:40 |
| ayoung | could we make that part of middleware, though... | 23:41 |
| *** AlexeyAbashkin has quit IRC | 23:41 | |
| ayoung | ok, next thought | 23:41 |
| ayoung | we send the domain scoped token to nova | 23:41 |
| kmalloc | part of the isolation of concerns is because we don't want to open potential security issues for very administrative things that isn't related to openstack-use | 23:41 |
| ayoung | middleware validates it and it passes, but since it is domain scoped... | 23:41 |
| ayoung | damnit, we don't know the project | 23:41 |
| *** edmondsw has quit IRC | 23:41 | |
| ayoung | we can't even ask | 23:41 |
| kmalloc | that is an issue with nova knowing what to ask for | 23:42 |
| kmalloc | expanding nova's apis | 23:42 |
| kmalloc | to allow for "do action on project X" | 23:42 |
| kmalloc | vs "do action on my current scope" | 23:42 |
| ayoung | it needs to be done at the scope check time | 23:42 |
| kmalloc | that wont be middleware. | 23:42 |
| kmalloc | right | 23:42 |
| kmalloc | nova has to gate if that is allowed, keystone cannot | 23:42 |
| ayoung | the scoped does not match, either fail (like we do now) or confirm with Keystone | 23:43 |
| kmalloc | **confirm with keystone might be a cache hit | 23:43 |
| kmalloc | but yes. | 23:43 |
| kmalloc | we're on the same page. | 23:43 |
| ayoung | the way that role assignment inheritance works now, you would not even know the right set of roles | 23:43 |
| ayoung | it would break RBAC in middleware, too | 23:44 |
| ayoung | dman you are a PITA | 23:44 |
| kmalloc | we can expose more information | 23:44 |
| kmalloc | and really i'm fine with expanding the information sent as needed. | 23:44 |
| kmalloc | you can check if the role is inherited (with keystone) and if project is child | 23:45 |
| kmalloc | if so, you know it's fine. | 23:45 |
| ayoung | ok...let me state a working assumption. HMT tokens are the non-standard path, and can be slower than normal use cases, as they are going to have lots of cache misses | 23:45 |
| ayoung | what about "list servers" | 23:45 |
| kmalloc | list servers is "list vms" | 23:45 |
| kmalloc | right? | 23:45 |
| ayoung | that either based on the proejct scope or is_admin does it for the whole endpoint (I think) | 23:46 |
| ayoung | yeah, list vms | 23:46 |
| kmalloc | just to be clear what resource type because we have so many overloaded terms | 23:46 |
| kmalloc | ok | 23:46 |
| ayoung | how would it know what to list? | 23:46 |
| kmalloc | i think it's actually ?all_tenants=True (or whatever the qp is) | 23:46 |
| kmalloc | and that explicitly checks is_admin | 23:46 |
| ayoung | so instead it would look at the token "is this an HMT token" | 23:47 |
| ayoung | and...if it is, it starts at the top of the tree and works down? | 23:47 |
| kmalloc | it probably would start as a "are you role x HMT" | 23:47 |
| ayoung | that would have to be implemented in every serer | 23:47 |
| ayoung | server | 23:47 |
| kmalloc | it might become it's own system-scope API (not list servers, but list_all_project_servers) | 23:48 |
| kmalloc | that is one of the edge cases we would err to not make system-scope to start | 23:48 |
| ayoung | you've just opened the can of worms | 23:48 |
| kmalloc | each API that is moved is move explicitly | 23:48 |
| ayoung | cuz if we can do it for read, we don't want to give them a new token for write | 23:48 |
| kmalloc | and with the blessing of the service | 23:48 |
| ayoung | this is kindof like inheritance | 23:49 |
| ayoung | for certain operations, service role implies all projects | 23:49 |
| kmalloc | and yes, if it becomes it's own API that brides all domains and all projects | 23:49 |
| kmalloc | then you get a write token for the project you need | 23:49 |
| ayoung | its not a role inference rule, it is a scope inference rule | 23:49 |
| kmalloc | bridges* | 23:49 |
| ayoung | kmalloc, are the other cores aligned with you on this>? | 23:50 |
| ayoung | kmalloc, I would like to point out that we've almost monopolized this channel to the point where scrollback is you, me and edmondsw has quit (Ping timeout: 260 seconds) | 23:54 |
| kmalloc | ayoung: lbragstad most cores in Denver were on board with this | 23:54 |
| kmalloc | i don't know how it has changed since then | 23:55 |
| kmalloc | i missed dublin | 23:55 |
| kmalloc | i wasn't at the forum in... sydney? | 23:55 |
| kmalloc | and i am unsure if i'll be in vancouver | 23:55 |
| kmalloc | ayoung: it's also late(ish) in the day | 23:55 |
| *** germs has joined #openstack-keystone | 23:55 | |
| kmalloc | and not many folks are west coast anymore here | 23:55 |
| kmalloc | [i'm kinda in the minority] | 23:56 |
| ayoung | https://github.com/openstack/keystone-specs/blob/master/specs/keystone/backlog/materialize-project-hierarchy.rst | 23:56 |
| ayoung | I miss amakarov | 23:56 |
| *** dklyle has quit IRC | 23:57 | |
| kmalloc | yeah, i was sad that work died. it needed work/effort around making it functional in the way we needed | 23:57 |
| kmalloc | but it also didn't have much driving as we suffered attrition of devs/cores across openstack | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!