| bknudson | nice | 00:00 |
|---|---|---|
| morganfainberg | https://www.consul.io/docs/agent/dns.html | 00:00 |
| morganfainberg | bknudson: so.. yes, does exactly what i'd want it to do | 00:00 |
| morganfainberg | a SUB-url thing would potentially need to be an extra record | 00:01 |
| morganfainberg | i don't think SRV can support that | 00:01 |
| morganfainberg | but JSON home could support/allow that type of discovery | 00:01 |
| morganfainberg | bknudson: so my thought is something like consul - keystonemiddleware registers with it so consul knows what services are "active", and instead of asking keystone itself for the catalog(s) or looking in the tokens, the service can pull straight from consul - keystone can then broker to external clients that are using the old interface/in-token interface | 00:03 |
| bknudson | so the other thing we've got going on is people hope to get some kind of security by limiting the service catalog and enforcing that in middleware | 00:03 |
| dstanek | bknudson: yeah, i don't quite get that | 00:04 |
| morganfainberg | bknudson: keystone can still do that type of stuff - we'll need to figure out how to manage that | 00:04 |
| morganfainberg | bknudson: if it's a real ask | 00:04 |
| morganfainberg | bknudson: well RAX does that - if the endpoint isn't in the catalog for that user, they get rejected talking to another endpoint | 00:04 |
| dstanek | morganfainberg: will they still do that with keystoe | 00:05 |
| morganfainberg | bknudson: it's the idea of not allowing CUSTOMER_X from talking to the BETA_ENDPOINTS but maybe an early-adopter customer is allowed to use beta | 00:05 |
| bknudson | which one you works for RAX? | 00:05 |
| morganfainberg | bknudson: dstanek :P | 00:05 |
| morganfainberg | bknudson: but i spent a good deal of time talking their uses over with them. | 00:05 |
| *** zzzeek has joined #openstack-keystone | 00:05 | |
| dstanek | bknudson: unfortunately they don't use keystone :-P | 00:05 |
| morganfainberg | dstanek: they do want the same support in keystone though | 00:05 |
| morganfainberg | dstanek: last i heard. and i've heard that ask from other orgs too | 00:06 |
| morganfainberg | dstanek: ctina's for example | 00:06 |
| morganfainberg | (oh she's not here at the moment) | 00:06 |
| dstanek | so i think rax has a separate service that sits in front of identity that does this type of filtering | 00:07 |
| morganfainberg | dstanek: repose | 00:07 |
| morganfainberg | dstanek: because i alwasy wanted to use java in my python IaaS services | 00:07 |
| bknudson | https://github.com/rackerlabs/repose | 00:07 |
| dstanek | haha, i know repose does the rate limiting and some other stuff - not sure about the catalog pieces | 00:08 |
| morganfainberg | dstanek: it also does token things | 00:08 |
| morganfainberg | dstanek: or os i'm told | 00:08 |
| morganfainberg | dstanek: basically instead of auth_token this thing | 00:08 |
| bknudson | it's scala | 00:08 |
| morganfainberg | even better | 00:09 |
| dstanek | did you just call it crap? | 00:09 |
| gyee | lmao | 00:09 |
| * morganfainberg sure didn't | 00:09 | |
| gyee | go on | 00:09 |
| morganfainberg | anyway | 00:09 |
| * morganfainberg is thinking POC with consul to drive the catalog. | 00:10 | |
| gyee | that's ya ticket to Tokyo | 00:10 |
| *** Rockyg has joined #openstack-keystone | 00:12 | |
| morganfainberg | bknudson: i nominate gyee to make us an awesome flashy keystone webpage /me ducks | 00:12 |
| gyee | hah | 00:13 |
| *** bknudson has left #openstack-keystone | 00:14 | |
| *** bknudson has joined #openstack-keystone | 00:14 | |
| *** ChanServ sets mode: +v bknudson | 00:14 | |
| *** markvoelker has joined #openstack-keystone | 00:24 | |
| *** zzzeek has quit IRC | 00:26 | |
| *** markvoelker has quit IRC | 00:30 | |
| *** pballand has quit IRC | 00:47 | |
| *** lhcheng has quit IRC | 00:48 | |
| *** woodster_ has quit IRC | 00:51 | |
| *** Rockyg has quit IRC | 00:52 | |
| *** liusheng has joined #openstack-keystone | 00:58 | |
| *** htruta has quit IRC | 00:59 | |
| *** tobe has joined #openstack-keystone | 01:05 | |
| *** david-ly_ has joined #openstack-keystone | 01:06 | |
| *** david-lyle has quit IRC | 01:09 | |
| *** spandhe has quit IRC | 01:16 | |
| *** ankita_wagh has quit IRC | 01:18 | |
| *** ncoghlan has joined #openstack-keystone | 01:20 | |
| *** csoukup has joined #openstack-keystone | 01:21 | |
| *** lhcheng has joined #openstack-keystone | 01:27 | |
| *** ChanServ sets mode: +v lhcheng | 01:27 | |
| *** fangzhou has quit IRC | 01:28 | |
| *** jasondotstar has joined #openstack-keystone | 01:33 | |
| *** dramakri has quit IRC | 01:34 | |
| *** davechen has joined #openstack-keystone | 01:40 | |
| *** boris-42 has quit IRC | 01:42 | |
| davechen | henrynash: hi, | 01:42 |
| davechen | henrynash: I think I think updte the commit message for that patch (#195001) | 01:43 |
| davechen | henrynash: Just say that patch will fix when there is no reqest body instead of body is empty, so it will be more clear. | 01:44 |
| davechen | fix the case* | 01:44 |
| *** davechen1 has joined #openstack-keystone | 01:46 | |
| *** davechen has quit IRC | 01:49 | |
| *** ankita_wagh has joined #openstack-keystone | 01:50 | |
| *** davechen has joined #openstack-keystone | 01:52 | |
| *** davechen1 has quit IRC | 01:53 | |
| *** davechen1 has joined #openstack-keystone | 01:55 | |
| *** ankita_wagh has quit IRC | 01:55 | |
| *** ankita_wagh has joined #openstack-keystone | 01:56 | |
| *** davechen has quit IRC | 01:57 | |
| *** _cjones_ has quit IRC | 01:58 | |
| *** davechen1 is now known as davechen | 01:59 | |
| *** jasondotstar has quit IRC | 01:59 | |
| *** ROT26 has quit IRC | 02:04 | |
| *** crc32 has quit IRC | 02:04 | |
| *** wolsen has quit IRC | 02:05 | |
| *** wolsen has joined #openstack-keystone | 02:05 | |
| *** jasondotstar has joined #openstack-keystone | 02:10 | |
| *** spandhe has joined #openstack-keystone | 02:12 | |
| *** ROT26 has joined #openstack-keystone | 02:13 | |
| *** markvoelker has joined #openstack-keystone | 02:14 | |
| *** dims has joined #openstack-keystone | 02:14 | |
| *** dims_ has quit IRC | 02:15 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/oslo.policy: Add tox target to find missing requirements https://review.openstack.org/195842 | 02:16 |
| *** markvoelker has quit IRC | 02:18 | |
| *** dims has quit IRC | 02:20 | |
| *** dims has joined #openstack-keystone | 02:22 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/oslo.policy: Add six and oslo.utils to requirements https://review.openstack.org/195846 | 02:23 |
| *** jdennis has quit IRC | 02:27 | |
| *** d0ugal has quit IRC | 02:29 | |
| *** d0ugal has joined #openstack-keystone | 02:29 | |
| *** d0ugal is now known as Guest5335 | 02:30 | |
| *** csoukup has quit IRC | 02:33 | |
| *** nkinder has quit IRC | 02:37 | |
| *** r-daneel has quit IRC | 02:39 | |
| *** ankita_wagh has quit IRC | 02:39 | |
| *** jasondotstar has quit IRC | 02:42 | |
| *** jdennis has joined #openstack-keystone | 02:44 | |
| *** stevemar has joined #openstack-keystone | 02:51 | |
| *** gyee has quit IRC | 02:58 | |
| *** dims has quit IRC | 03:01 | |
| *** woodster_ has joined #openstack-keystone | 03:16 | |
| *** spandhe has quit IRC | 03:19 | |
| *** boris-42 has joined #openstack-keystone | 03:22 | |
| *** mestery has joined #openstack-keystone | 03:23 | |
| *** jdennis has quit IRC | 03:25 | |
| *** liusheng has quit IRC | 03:26 | |
| *** ROT26 has quit IRC | 03:28 | |
| *** sudorandom has quit IRC | 03:47 | |
| *** stevemar has quit IRC | 03:48 | |
| *** stevemar has joined #openstack-keystone | 03:48 | |
| *** stevemar has quit IRC | 03:51 | |
| *** david-ly_ is now known as david-lyle | 03:51 | |
| *** sudorandom has joined #openstack-keystone | 03:53 | |
| *** dramakri has joined #openstack-keystone | 04:02 | |
| *** markvoelker has joined #openstack-keystone | 04:02 | |
| *** vilobhmm has joined #openstack-keystone | 04:04 | |
| *** markvoelker has quit IRC | 04:07 | |
| *** liusheng has joined #openstack-keystone | 04:10 | |
| *** rushiagr_away is now known as rushiagr | 04:13 | |
| *** ankita_wagh has joined #openstack-keystone | 04:16 | |
| *** ncoghlan has quit IRC | 04:32 | |
| *** vilobhmm has quit IRC | 04:36 | |
| *** tobe has quit IRC | 04:40 | |
| *** stevemar has joined #openstack-keystone | 04:50 | |
| *** stevemar has quit IRC | 04:53 | |
| *** arunkant_ has joined #openstack-keystone | 04:54 | |
| *** pballand has joined #openstack-keystone | 04:57 | |
| *** arunkant__ has quit IRC | 04:58 | |
| *** arunkant has joined #openstack-keystone | 04:58 | |
| *** arunkant_ has quit IRC | 05:01 | |
| *** pballand has quit IRC | 05:01 | |
| *** rm_work|away is now known as rm_work | 05:01 | |
| *** pballand has joined #openstack-keystone | 05:02 | |
| *** dramakri has quit IRC | 05:02 | |
| *** ncoghlan has joined #openstack-keystone | 05:06 | |
| *** ncoghlan has quit IRC | 05:13 | |
| *** tobe has joined #openstack-keystone | 05:19 | |
| *** woodster_ has quit IRC | 05:21 | |
| *** toddnni has quit IRC | 05:28 | |
| *** ajayaa has joined #openstack-keystone | 05:31 | |
| openstackgerrit | Steve Martinelli proposed openstack/keystone: switch to oslo.cache https://review.openstack.org/195873 | 05:33 |
| *** richm has quit IRC | 05:34 | |
| *** pballand has quit IRC | 05:35 | |
| ajayaa | Hi guys. Does the python-keystoneclient work with domain scoped tokens? | 05:36 |
| *** toddnni has joined #openstack-keystone | 05:36 | |
| ajayaa | When I pass a domain scoped token to the client I get EndpointNotFound exception. | 05:36 |
| ajayaa | Any idea? | 05:36 |
| ajayaa | jamielennox|away ^^ | 05:36 |
| *** rushiagr is now known as rushiagr_away | 05:43 | |
| *** ankita_wagh has quit IRC | 05:44 | |
| davechen | ajayaa: why not try OSC? | 05:44 |
| davechen | I think Steve may also has the answer, but seems he is not online. | 05:45 |
| ajayaa | davechen, I am trying to use python sdk to write some tests. | 05:45 |
| ajayaa | OSC also uses python-keystoneclient, I think. | 05:45 |
| davechen | ajayaa: yep, but OSC has the clear usage about the V3 APIs. | 05:46 |
| davechen | just thought that domain should go to V3 | 05:47 |
| ajayaa | When you say OSC, are you talking about the cli? | 05:47 |
| davechen | yep. | 05:47 |
| ajayaa | davechen, I am writing some functional tests using python testtools and keystoneclient.v3 client. | 05:48 |
| davechen | you are not lucky, cores is not available at this time. :) | 05:48 |
| ajayaa | So cli won't be of any use to me. | 05:48 |
| ajayaa | Usually jamielennox would be here. | 05:48 |
| ajayaa | I will come back later | 05:48 |
| davechen | see you. good luck. | 05:49 |
| ajayaa | davechen, :) | 05:49 |
| *** markvoelker has joined #openstack-keystone | 05:51 | |
| *** boris-42 has quit IRC | 05:52 | |
| *** links has joined #openstack-keystone | 05:55 | |
| *** markvoelker has quit IRC | 05:56 | |
| *** pnavarro has joined #openstack-keystone | 05:58 | |
| *** ankita_wagh has joined #openstack-keystone | 05:59 | |
| *** ajayaa has quit IRC | 06:04 | |
| *** tobe has quit IRC | 06:05 | |
| *** mestery has quit IRC | 06:08 | |
| *** rushiagr_away is now known as rushiagr | 06:15 | |
| *** ajayaa has joined #openstack-keystone | 06:21 | |
| *** boris-42 has joined #openstack-keystone | 06:26 | |
| *** ajayaa has quit IRC | 06:27 | |
| *** rm_work is now known as rm_work|away | 06:27 | |
| *** arunkant_ has joined #openstack-keystone | 06:33 | |
| *** Guest5335 is now known as d0ugal | 06:34 | |
| *** d0ugal has quit IRC | 06:34 | |
| *** d0ugal has joined #openstack-keystone | 06:34 | |
| *** browne has quit IRC | 06:37 | |
| *** arunkant has quit IRC | 06:37 | |
| *** arunkant__ has joined #openstack-keystone | 06:38 | |
| *** tobe has joined #openstack-keystone | 06:40 | |
| *** arunkant_ has quit IRC | 06:41 | |
| *** ajayaa has joined #openstack-keystone | 06:42 | |
| *** belmoreira has joined #openstack-keystone | 06:43 | |
| *** arunkant_ has joined #openstack-keystone | 06:48 | |
| *** arunkant__ has quit IRC | 06:51 | |
| *** lufix has joined #openstack-keystone | 06:58 | |
| *** spandhe has joined #openstack-keystone | 07:15 | |
| *** lsmola has joined #openstack-keystone | 07:21 | |
| *** liusheng has quit IRC | 07:26 | |
| *** liusheng has joined #openstack-keystone | 07:27 | |
| openstackgerrit | Dave Chen proposed openstack/keystone: Show friendly message when request body is not provided https://review.openstack.org/195903 | 07:36 |
| *** rushiagr is now known as rushiagr_away | 07:36 | |
| *** rlt_ has joined #openstack-keystone | 07:36 | |
| openstackgerrit | Dave Chen proposed openstack/keystone: Show friendly message when request body is not provided https://review.openstack.org/195001 | 07:37 |
| openstackgerrit | Dave Chen proposed openstack/keystone: Show friendly message when request body is not provided https://review.openstack.org/195903 | 07:39 |
| openstackgerrit | Dave Chen proposed openstack/keystone: Show friendly message when request body is not provided https://review.openstack.org/195429 | 07:40 |
| openstackgerrit | Dave Chen proposed openstack/keystone: Show friendly message when request body is not provided https://review.openstack.org/195429 | 07:40 |
| liusheng | dstanek: ping | 07:47 |
| liusheng | dstanek: Hi, could you please take a look at https://review.openstack.org/186987 and the bug description ? thanks | 07:49 |
| *** rm_work|away is now known as rm_work | 07:51 | |
| *** spandhe has quit IRC | 07:56 | |
| openstackgerrit | henry-nash proposed openstack/keystone-specs: Enable listing of role assignments in a project hierarchy https://review.openstack.org/187045 | 08:07 |
| *** rm_work is now known as rm_work|away | 08:07 | |
| tobasco | can/should keystone admin api on port 5000 be exposed to internet? | 08:11 |
| tobasco | i.e should i allow users to manipulate users using the api or only through horizon | 08:12 |
| *** rm_work|away is now known as rm_work | 08:12 | |
| *** dguerri` is now known as dguerri | 08:20 | |
| tobasco | keystone tokens gets generated with a expires date of -1 hour what the current time is, date on all nodes (keystone, controller) says correct time | 08:28 |
| *** lhcheng has quit IRC | 08:44 | |
| *** lsmola has quit IRC | 08:53 | |
| *** fhubik has joined #openstack-keystone | 08:56 | |
| *** fhubik is now known as fhubik_afk | 08:56 | |
| *** davechen has left #openstack-keystone | 08:57 | |
| *** ankita_wagh has quit IRC | 08:58 | |
| *** BAKfr has quit IRC | 09:01 | |
| *** tobe has quit IRC | 09:02 | |
| *** BAKfr has joined #openstack-keystone | 09:03 | |
| *** husanu1 has joined #openstack-keystone | 09:09 | |
| *** bradjones has quit IRC | 09:15 | |
| *** bradjones has joined #openstack-keystone | 09:16 | |
| *** bradjones has quit IRC | 09:16 | |
| *** bradjones has joined #openstack-keystone | 09:16 | |
| *** husanu1 has quit IRC | 09:18 | |
| *** fhubik_afk is now known as fhubik | 09:20 | |
| *** boris-42 has quit IRC | 09:22 | |
| *** husanu2 has joined #openstack-keystone | 09:23 | |
| *** husanu2 has quit IRC | 09:28 | |
| *** markvoelker has joined #openstack-keystone | 09:29 | |
| *** markvoelker has quit IRC | 09:33 | |
| *** BAKfr has quit IRC | 09:34 | |
| *** BAKfr has joined #openstack-keystone | 09:39 | |
| *** danboid has joined #openstack-keystone | 09:47 | |
| danboid | Are there any scripts or tools to help inspect keystone logs? | 09:48 |
| openstackgerrit | Marek Denis proposed openstack/keystone: OS-FEDERATION no longer extension in docs https://review.openstack.org/192671 | 09:49 |
| danboid | By inspect I mean parse a keystone log and show you how many auths occured in a given time, sum the content lengths within that period and stuff like that | 09:52 |
| dstanek | danboid: not that i have seen, but i'm sure operators have tooling to do that | 09:53 |
| danboid | dstanek: operators? | 09:53 |
| dstanek | danboid: people running clouds | 09:53 |
| *** stevemar has joined #openstack-keystone | 09:53 | |
| danboid | Oh, sys admins! :) | 09:54 |
| danboid | I wouldn't know what one of them is :) | 09:54 |
| dstanek | sorta - operators i think is used because it is a broader topic that just system administration | 09:54 |
| *** stevemar has quit IRC | 09:57 | |
| *** dims has joined #openstack-keystone | 09:59 | |
| *** Kennan has joined #openstack-keystone | 09:59 | |
| Kennan | hi bknudson: pr ayoung: there? | 10:00 |
| Kennan | I have question about [keystone_authtoken] | 10:00 |
| Kennan | I found many projects used that | 10:00 |
| Kennan | But right now, it seems some projects have | 10:00 |
| Kennan | username and password | 10:01 |
| Kennan | other projects have | 10:01 |
| Kennan | admin_username and admin_password | 10:01 |
| Kennan | could you two guys or other cores give explanation ? | 10:01 |
| Kennan | what's that for ? | 10:01 |
| Kennan | what's difference ? | 10:01 |
| *** arunkant__ has joined #openstack-keystone | 10:03 | |
| hugokuo | ls | 10:03 |
| *** richm has joined #openstack-keystone | 10:04 | |
| *** arunkant_ has quit IRC | 10:07 | |
| *** e0ne has joined #openstack-keystone | 10:15 | |
| *** dims has quit IRC | 10:18 | |
| *** liusheng has quit IRC | 10:18 | |
| *** henrynash has quit IRC | 10:19 | |
| *** lufix has quit IRC | 10:27 | |
| dstanek | Kennan: who uses just username/password? i thought it have had to be admin_user/admin_password | 10:30 |
| Kennan | hi dstanek: did you try devstack these days? | 10:30 |
| Kennan | I found many projects configure username and password | 10:31 |
| dstanek | Kennan: sure | 10:31 |
| dstanek | which one? | 10:31 |
| Kennan | glance heat | 10:31 |
| Kennan | etc. | 10:31 |
| Kennan | also check redhat install guide http://docs.openstack.org/kilo/install-guide/install/yum/content/glance-install.html | 10:31 |
| Kennan | it also username and password | 10:32 |
| Kennan | not have admin_username | 10:32 |
| Kennan | and admin_password | 10:32 |
| Kennan | for ubuntu it is same | 10:32 |
| dstanek | odd...i don't see that used in the code anywhere; i would not expect that to work; maybe it's from an older version | 10:34 |
| dstanek | yeah, i don't even see that in the kilo version of keystoneclient.middleware.auth_token | 10:35 |
| dstanek | and our docs do say to use admin_user | 10:36 |
| *** links has quit IRC | 10:36 | |
| Kennan | strange: dstanek I found devstack now used username and password in glance, heat etc. Some still user admin_username and admin_password | 10:36 |
| dstanek | Kennan: heat for me is using admin_user/admin_password | 10:37 |
| Kennan | dstanek: check this configure_auth_token_middleware | 10:38 |
| Kennan | http://docs.openstack.org/developer/devstack/lib/keystone.html | 10:38 |
| Kennan | iniset $conf_file $section username $admin_user | 10:38 |
| Kennan | iniset $conf_file $section password $SERVICE_PASSWORD | 10:38 |
| Kennan | it did that | 10:38 |
| Kennan | many project call configure_auth_token_middleware | 10:38 |
| dstanek | maybe i'm running a very out of date version, but all of my configs are using admin_* | 10:39 |
| dstanek | if that's the case, then i have no idea | 10:40 |
| dstanek | for me heat is doing this: iniset $HEAT_CONF keystone_authtoken admin_user heat | 10:42 |
| Kennan | yes, dstanek: so I feel strange about it, Maybe keystone guys know devstack configure like that. I looking help for that :) | 10:42 |
| dstanek | Kennan: have you looked at your configs to see which ones have just username? | 10:43 |
| Kennan | nvoa glance | 10:43 |
| Kennan | neutron | 10:44 |
| Kennan | also cinder did like that | 10:44 |
| dstanek | i think maybe i have to rebuild because all of my configs are fine | 10:44 |
| Kennan | so I really think it cause conflict and strange for username and admin_username | 10:45 |
| Kennan | password and admin_password | 10:45 |
| Kennan | may need more keystone guys input about that | 10:46 |
| Kennan | why need configrue that ways? | 10:46 |
| dstanek | the majority won't be online for a few hours | 10:46 |
| Kennan | ok dstanek: | 10:49 |
| *** dims has joined #openstack-keystone | 10:59 | |
| *** e0ne is now known as e0ne_ | 11:08 | |
| *** links has joined #openstack-keystone | 11:17 | |
| *** markvoelker has joined #openstack-keystone | 11:17 | |
| *** e0ne_ has quit IRC | 11:18 | |
| *** links has quit IRC | 11:19 | |
| *** markvoelker has quit IRC | 11:22 | |
| *** jdennis has joined #openstack-keystone | 11:23 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/oslo.policy: Add six and oslo.utils to requirements https://review.openstack.org/195846 | 11:30 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/oslo.policy: Add tox target to find missing requirements https://review.openstack.org/195842 | 11:30 |
| *** ajayaa has quit IRC | 11:32 | |
| *** e0ne has joined #openstack-keystone | 11:39 | |
| *** aix has quit IRC | 11:41 | |
| *** bradjones has quit IRC | 11:42 | |
| *** bradjones has joined #openstack-keystone | 11:45 | |
| *** bradjones has quit IRC | 11:45 | |
| *** bradjones has joined #openstack-keystone | 11:45 | |
| dstanek | marekd: you around? | 11:46 |
| dstanek | rodrigods: how about you? | 11:48 |
| *** ajayaa has joined #openstack-keystone | 11:51 | |
| *** markvoelker has joined #openstack-keystone | 12:00 | |
| marekd | dstanek: yes | 12:01 |
| marekd | dstanek: what;s up? | 12:02 |
| dstanek | marekd: do you have a few for some k2k questions? | 12:02 |
| marekd | dstanek: sure | 12:02 |
| dstanek | some guys at rax and trying to set it up and running into some issues i haven't seen before....jas | 12:02 |
| marekd | dstanek: what are those issues? | 12:03 |
| *** hughsaunders has joined #openstack-keystone | 12:03 | |
| dstanek | i'm getting them in here | 12:03 |
| marekd | dstanek: roger | 12:03 |
| *** Ctina_ has joined #openstack-keystone | 12:04 | |
| *** odyssey4me has joined #openstack-keystone | 12:05 | |
| dstanek | hughsaunders: what is the last issue you just ran into? | 12:05 |
| hughsaunders | hey all, I'm trying to get a test environment setup for k2k federation. So far its at the point where I can request a SAML assertion from IDP keystone and post it to SP keystone. SP keystone rediercts and sets a cookie. However I can't use that cookie to request a token. At that stage I get "shib_check_user found no per-request structure" | 12:06 |
| odyssey4me | dstanek so hughsaunders is trying to get k2k idp/sp right, whereas I'm trying to get a keystone sp with the www.testshib.org idp right | 12:06 |
| hughsaunders | That sounds like it might be an issue with the mapping object I had to create on the SP keystone | 12:07 |
| odyssey4me | I am wondering whether anyone has a kilo reference set of configs we can work from, whether for k2k or for keystone to another idp? | 12:08 |
| *** aix has joined #openstack-keystone | 12:08 | |
| *** dims is now known as dimsum__ | 12:09 | |
| *** fhubik is now known as fhubik_afk | 12:09 | |
| *** dguerri is now known as dguerri` | 12:10 | |
| marekd | hughsaunders: odyssey4me i can share you my mapping i sed for my testbed... | 12:10 |
| marekd | hughsaunders: i can also try to take a look at you mapping if it's not confidential and stuff. | 12:11 |
| marekd | s/you/sour/ | 12:11 |
| hughsaunders | marekd: that would be useful, thanks | 12:11 |
| hughsaunders | marekd: current mapping is on like 76 of this: https://etherpad.openstack.org/p/osad-keystone-idp-sp it was mostly taken from http://blog.rodrigods.com/it-is-time-to-play-with-keystone-to-keystone-federation-in-kilo/ just modified to add my test user | 12:12 |
| *** dims_ has joined #openstack-keystone | 12:13 | |
| hughsaunders | *line | 12:13 |
| marekd | hughsaunders: ugh, rodrigo made wrong mapping rules and ppl use them :( | 12:14 |
| marekd | let me try with fixing it... | 12:14 |
| odyssey4me | o_O | 12:14 |
| odyssey4me | hughsaunders to clarify, are you using the master (liberty) branch for your deployment - or the head of stable/kilo? | 12:15 |
| marekd | ah no, sorry i think yours look fine... | 12:15 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/oslo.policy: Add six and oslo.utils to requirements https://review.openstack.org/195846 | 12:15 |
| *** dimsum__ has quit IRC | 12:16 | |
| odyssey4me | marekd the blogs and docs are unfortunately a bit inconsistent - some appear to be stuck in the juno timeframe, while others are not - this makes our lives a little complicated :( | 12:17 |
| hughsaunders | odyssey4me: keystone_git_install_branch: 7f76f23bccd772df35da63584685584576289255 # HEAD of "master" as of 08.06.2015 | 12:17 |
| marekd | odyssey4me: rodrigods made two blog posts, one with juno (broken in fact) and the second with kilo version (works) | 12:18 |
| marekd | odyssey4me: ok, one question is group id (set to fedgroup.id) somehow a real group id? | 12:18 |
| marekd | or maybe it's replaced with uuid later on ? | 12:18 |
| hughsaunders | marekd: do you know of docs for the federation parts of the keystone api? I couldn't find them in http://developer.openstack.org/api-ref-identity-v3.html or the v3 extensions page | 12:19 |
| marekd | odyssey4me: also, which problems are you actually running into? If I were you I'd first make sure keystone-sp works (with testshib for instance) and later proceed to keystone-idp (much simpler part tbh) | 12:20 |
| marekd | hughsaunders: http://specs.openstack.org/openstack/keystone-specs/api/v3/identity-api-v3-os-federation-ext.html maybe this? | 12:20 |
| *** rmstar has joined #openstack-keystone | 12:20 | |
| *** packet has joined #openstack-keystone | 12:20 | |
| odyssey4me | marekd so hughsaunders is trying to test k2k idp/sp, whereas I'm trying to test a keystone sp against testshib so that I hopefully get at least the sp right | 12:21 |
| marekd | odyssey4me: allright. | 12:21 |
| marekd | odyssey4me: so, what are you stuck with? | 12:21 |
| odyssey4me | marekd unfortunately my redirect seems to stick in keystone and the keystone service doesn't see to redirect the auth to the testshib idp for some reason | 12:21 |
| hughsaunders | marekd: thanks, I guess the docs team will move that into docs when they have time. | 12:22 |
| marekd | odyssey4me: you are using browser, right? | 12:23 |
| odyssey4me | marekd so what information can I give you that will help dig into the solution? alternatively if you like I can give you access to my test server to try to identify where things are going wrong | 12:23 |
| odyssey4me | marekd yes, I am - and I've configured Horizon according to http://docs.openstack.org/developer/keystone/extensions/websso.html | 12:23 |
| marekd | odyssey4me: did you upload all metadata to testshib? | 12:25 |
| odyssey4me | marekd, yep - I did | 12:25 |
| marekd | odyssey4me: allright, if i can access your machine that would be cool. | 12:26 |
| odyssey4me | I've also confirmed that the idp metadata is being downloaded to the keystone server | 12:26 |
| marekd | i can try to debug it quickly. | 12:26 |
| odyssey4me | thanks marekd - I really appreciate it... the shibboleth /shibd configuration has really being doing my head in | 12:27 |
| marekd | no worries. | 12:27 |
| marekd | i know it's a piece of shhhh | 12:27 |
| hughsaunders | ibboleth... | 12:28 |
| *** dguerri` is now known as dguerri | 12:28 | |
| *** aix has quit IRC | 12:28 | |
| *** fhubik_afk is now known as fhubik | 12:30 | |
| *** edmondsw has joined #openstack-keystone | 12:32 | |
| hughsaunders | marekd: with the mapping rules, can I specify a group in the IdP keystone and map all members of that group into a group on the SP keystone? At the moment I have individual users listed, but thats not ideal | 12:36 |
| *** raildo has joined #openstack-keystone | 12:37 | |
| *** fhubik has quit IRC | 12:38 | |
| *** iurygregory has joined #openstack-keystone | 12:39 | |
| hughsaunders | hmm, actually that federation spec gives some more exmaples of mappings | 12:40 |
| *** fhubik has joined #openstack-keystone | 12:41 | |
| *** htruta has joined #openstack-keystone | 12:44 | |
| *** tellesnobrega has joined #openstack-keystone | 12:45 | |
| marekd | hughsaunders: sorry, was helping odyssey4me not... | 12:50 |
| marekd | so right now you can map only to a group | 12:50 |
| *** bknudson has quit IRC | 12:50 | |
| marekd | or the existing user (user that exists in the backend, but not the core of a federted workflow) | 12:51 |
| marekd | you need some link between your identity and roles/role assignments. | 12:51 |
| *** mestery has joined #openstack-keystone | 12:52 | |
| *** packet has quit IRC | 12:53 | |
| *** htruta has quit IRC | 12:53 | |
| hughsaunders | marekd: np, thanks for helping odyssey4me :) | 12:53 |
| *** henrynash has joined #openstack-keystone | 12:55 | |
| *** ChanServ sets mode: +v henrynash | 12:55 | |
| *** mestery has quit IRC | 12:57 | |
| *** tellesnobrega has quit IRC | 13:04 | |
| *** dimsum__ has joined #openstack-keystone | 13:07 | |
| *** tellesnobrega has joined #openstack-keystone | 13:08 | |
| *** dims_ has quit IRC | 13:10 | |
| *** danboid has quit IRC | 13:14 | |
| *** henrynash has quit IRC | 13:18 | |
| *** bknudson has joined #openstack-keystone | 13:19 | |
| *** ChanServ sets mode: +v bknudson | 13:19 | |
| *** ihrachyshka has joined #openstack-keystone | 13:23 | |
| ihrachyshka | bknudson, hey. re that bug about keystone crashing in grenade. afaik it should not reload code unless it's main script is modified | 13:23 |
| bknudson | ihrachyshka: I can do some experiments... maybe it's documented somewhere how it's supposed to work | 13:24 |
| ihrachyshka | at least that's what I get from reading mod_wsgi docs on daemon mode | 13:24 |
| ihrachyshka | bknudson, it is: https://code.google.com/p/modwsgi/wiki/ReloadingSourceCode | 13:24 |
| ihrachyshka | we run in daemon mode | 13:25 |
| ihrachyshka | btw it also crashes on imports from keystone.* namespace, so it's not about oslo libraries being updated. | 13:26 |
| ihrachyshka | UNLESS the error does not really show the culprit but some bogus error | 13:26 |
| *** pdar has left #openstack-keystone | 13:26 | |
| ihrachyshka | like e.g. something crashes with ImportError on some oslo library, then it's unwinds stack and shows ImportError for a module that triggered the oslo import, but without details about actual crash | 13:27 |
| *** ajayaa has quit IRC | 13:29 | |
| bknudson | crashing on imports from keystone.* would be interesting since I don't see why grenade would be modifying those files | 13:29 |
| bknudson | do you have an example log? | 13:29 |
| *** stevemar has joined #openstack-keystone | 13:31 | |
| ihrachyshka | bknudson, a sec... | 13:31 |
| bknudson | I've seen the ArgsAlreadyParsed , but that's not on import | 13:33 |
| *** stevemar has quit IRC | 13:33 | |
| *** dguerri is now known as dguerri` | 13:33 | |
| ihrachyshka | bknudson, http://logs.openstack.org/77/195277/16/check/check-grenade-dsvm-neutron/99916dc/logs/apache/keystone.txt.gz | 13:33 |
| ihrachyshka | bknudson, that ArgsAlreadyParsed always came after Import error | 13:33 |
| bknudson | ImportError: cannot import name schema | 13:33 |
| bknudson | you get that exception when the file doesn't exist? | 13:34 |
| bknudson | I'll try it | 13:34 |
| ihrachyshka | bknudson, yeah, you would get it. Not sure if that's the only case for that kind of failure though. | 13:35 |
| ihrachyshka | mod_wsgi can do all kinds of tricks caching python modules | 13:35 |
| bknudson | you're not suggesting that keystone is purposefully deleting it's own files? | 13:35 |
| openstackgerrit | Dave Chen proposed openstack/keystone: Show friendly message when request body is not provided https://review.openstack.org/195429 | 13:36 |
| openstackgerrit | Dave Chen proposed openstack/keystone: Show friendly message when request body is not provided https://review.openstack.org/195001 | 13:36 |
| ihrachyshka | iiuc it does cache modules to avoid performance hit for python env bootstrap on each hit, and you make mod_wsgi reload everything by touching main wsgi script | 13:36 |
| bknudson | ihrachyshka: I'm going to try it out with devstack... delete some files and stuff | 13:37 |
| ihrachyshka | bknudson, I don't know! :) it may be grenade, or that error may not be the real problem but just some fossil from earlier failure | 13:37 |
| *** arunkant_ has joined #openstack-keystone | 13:38 | |
| ihrachyshka | we can try to play with wsgi settings, like running more processes and less threads maybe. not sure, I was really far from all that until we started to be hit by this in neutron. | 13:38 |
| ihrachyshka | the fact that it's neutron only is suspicious though, and it does not help to get attention from other devs either | 13:38 |
| ihrachyshka | since every other gate works :) | 13:38 |
| ihrachyshka | and no one cares about neutron being blocked | 13:38 |
| bknudson | we've become numb to neutron failures | 13:38 |
| ihrachyshka | well, it's definitely not neutron's fault I would say. it may be some interesting bug that is triggered by neutron upgrade that does more python module updates and such, or some timing thing due to more work to do during grenade. but I don't see how neutron is involved per se, outside of its integration into grenade process. | 13:40 |
| bknudson | I don't think neutron is deleting keystone files. | 13:40 |
| *** browne has joined #openstack-keystone | 13:41 | |
| gordc | bknudson: just curious but do you guys recall why you gate on check-swift-dsvm-functional | 13:41 |
| ihrachyshka | btw neutron is upgraded after keystone, so its python module version bumps go after keystone is up | 13:41 |
| *** arunkant__ has quit IRC | 13:41 | |
| ihrachyshka | that said, the only version bump I see is around oslo.middleware - it's downgraded, then upgraded back | 13:41 |
| ihrachyshka | and that flip-flopping seems to be due to poor pip dep solver | 13:42 |
| ihrachyshka | again, not sure it's relevant, just some weird sound from trenches | 13:42 |
| *** woodster_ has joined #openstack-keystone | 13:44 | |
| openstackgerrit | Rodrigo Duarte proposed openstack/keystone-specs: API changes for Reseller https://review.openstack.org/153007 | 13:44 |
| bknudson | ihrachyshka: I was able to recreate "2015-06-26 08:44:27.200423 ImportError: cannot import name schema" | 13:44 |
| bknudson | I deleted the file | 13:44 |
| rodrigods | bknudson ^removed the WIP from the API spec changes for Reseller and is_domain in token response | 13:45 |
| bknudson | and then I did some requests... several worked. | 13:45 |
| ihrachyshka | bknudson, meh, I don't think it's actually deleted though. Though maybe grenade does some weird things with git checkout?.. | 13:45 |
| ihrachyshka | bknudson, several worked, but not all? | 13:45 |
| *** raildo has quit IRC | 13:45 | |
| *** iurygregory has quit IRC | 13:45 | |
| bknudson | ihrachyshka: actually, I tried some and they were still working | 13:46 |
| ihrachyshka | bknudson, yeah, because, as per mod_wsgi docs, it does not reload anything until wsgi script is touched | 13:46 |
| bknudson | and one just worked... that's weird | 13:46 |
| *** iurygregory has joined #openstack-keystone | 13:46 | |
| *** e0ne is now known as e0ne_ | 13:46 | |
| bknudson | I'll try touching the wsgi script | 13:46 |
| *** raildo has joined #openstack-keystone | 13:47 | |
| bknudson | I thought it must be creating some instances dynamically | 13:47 |
| *** boris-42 has joined #openstack-keystone | 13:47 | |
| ihrachyshka | bknudson, now try to touch the wsgi script and see how quick it starts to fail :) and then back. I wonder whether workers are somehow cached there with wrong env and then are hit with requests once all is good on file system, but not at all in that cached env | 13:47 |
| *** dguerri` is now known as dguerri | 13:48 | |
| *** dguerri is now known as dguerri` | 13:48 | |
| *** htruta has joined #openstack-keystone | 13:49 | |
| *** dguerri` is now known as dguerri | 13:50 | |
| bknudson | it seems to be failing all the time now. | 13:50 |
| ihrachyshka | bknudson, after touch? | 13:51 |
| bknudson | ihrachyshka: y, I touched the files. | 13:51 |
| *** lufix has joined #openstack-keystone | 13:52 | |
| bknudson | when I put that file back it place it starts working again | 13:52 |
| ihrachyshka | without touching? | 13:52 |
| bknudson | I didn't have to touch the script to get it working again | 13:52 |
| bknudson | you can see from the keystone log that it's starting several times | 13:53 |
| bknudson | http://logs.openstack.org/77/195277/16/check/check-grenade-dsvm-neutron/99916dc/logs/apache/keystone.txt.gz -- it logs the config options when it starts up | 13:53 |
| ihrachyshka | bknudson, I suspect it's multiple processes? | 13:54 |
| bknudson | for some reason at 2015-06-26 11:18:17.586639 it starts up all the time, whereas before that it's not starting -- http://logs.openstack.org/77/195277/16/check/check-grenade-dsvm-neutron/99916dc/logs/apache/keystone.txt.gz#_2015-06-26_11_18_17_586639 | 13:54 |
| ihrachyshka | bknudson, devstack uses WSGIDaemonProcess keystone-public processes=5 threads=1 user=%USER% display-name=%{GROUP} %VIRTUALENV% | 13:54 |
| bknudson | ihrachyshka: that's what I've got in my config file | 13:55 |
| bknudson | https://code.google.com/p/modwsgi/wiki/ReloadingSourceCode#Restarting_Apache_Processes -- mentions MaxRequestsPerChild ... | 13:55 |
| ihrachyshka | I would expect apache to start those 5 processes and allow them to sit there waiting for requests | 13:55 |
| *** e0ne_ is now known as e0ne | 13:56 | |
| ihrachyshka | bknudson, not sure you want to reload it on every request | 13:56 |
| bknudson | it's very slow | 13:57 |
| *** ayoung has quit IRC | 14:00 | |
| openstackgerrit | Rodrigo Duarte proposed openstack/keystone-specs: Add is_domain to tokens for projects acting as a domain https://review.openstack.org/193543 | 14:02 |
| openstackgerrit | Rodrigo Duarte proposed openstack/keystone-specs: Add is_domain to tokens for projects acting as a domain https://review.openstack.org/193543 | 14:03 |
| *** timsim has left #openstack-keystone | 14:06 | |
| *** browne has quit IRC | 14:10 | |
| *** ayoung has joined #openstack-keystone | 14:12 | |
| *** ChanServ sets mode: +v ayoung | 14:12 | |
| *** e0ne is now known as e0ne_ | 14:14 | |
| *** browne has joined #openstack-keystone | 14:16 | |
| ayoung | Ah, Stevemar has alovely blog post on openidc... | 14:16 |
| bknudson | ihrachyshka: I made the logging a little clearer here and it starts up 10 processes on the first 10 requests. | 14:16 |
| *** e0ne_ is now known as e0ne | 14:16 | |
| bknudson | I can't figure out how to get it to start the processes when the httpd starts rather than on-demand | 14:17 |
| ihrachyshka | bknudson, so it's on demand? and after pool is full, they are reused, right? | 14:17 |
| bknudson | ihrachyshka: yes. I'll see if it reloads ever once the 10 have started | 14:18 |
| ihrachyshka | bknudson, http://stackoverflow.com/questions/1702562/speeding-up-the-first-page-load-in-django | 14:18 |
| *** ajayaa has joined #openstack-keystone | 14:19 | |
| bknudson | we'd need an import script... I tried to make one once. | 14:19 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:19 | |
| bknudson | ihrachyshka: https://review.openstack.org/#/c/71642/ | 14:20 |
| openstackgerrit | Rodrigo Duarte proposed openstack/keystone-specs: Add is_domain to tokens for projects acting as a domain https://review.openstack.org/193543 | 14:20 |
| ihrachyshka | bknudson, please share your findings in that bug, we need to have it documented. I will need to leave now, but please look into the issue if you have cycles. we really depend on it being solved in timely manner. we'll owe you :) | 14:20 |
| dstanek | bknudson: ihrachyshka: the problem isn't that it's reloading it's that there seems to be some extra state lingering right? | 14:21 |
| bknudson | dstanek: when it reloads it finds there's some files missing | 14:21 |
| ihrachyshka | dstanek, well... I think workers failing to reply to requests is still an issue | 14:21 |
| ihrachyshka | even if there is another problem of keystone failiing to recover after a worker failure | 14:21 |
| dstanek | bknudson: really? what is missing? | 14:22 |
| bknudson | dstanek: apparently keystone.assignment.schema | 14:23 |
| bknudson | http://logs.openstack.org/77/195277/16/check/check-grenade-dsvm-neutron/99916dc/logs/apache/keystone.txt.gz#_2015-06-26_11_22_06_803004 | 14:23 |
| bknudson | although we've seen other missing files in oslo, etc. | 14:23 |
| dstanek | bknudson: are they actually missing or is the python path borked? | 14:24 |
| bknudson | from oslo_utils import excutils | 14:24 |
| *** henrynash has joined #openstack-keystone | 14:24 | |
| *** ChanServ sets mode: +v henrynash | 14:24 | |
| bknudson | ?? I don't see anything in the log saying the python path is borked | 14:24 |
| ihrachyshka | well, I would be really surprised if files are gone indeed. they may have some newer versions though, making mod_wsgi caching mechanism producing interesting crashes | 14:25 |
| rodrigods | henrynash, we have added the API spec changes to the is_domain spec | 14:25 |
| ihrachyshka | but I'm probably talking nonsense | 14:26 |
| rodrigods | henrynash, let's hope it is approved today | 14:26 |
| henrynash | rodigods: I’ll check it out | 14:26 |
| *** r-daneel has joined #openstack-keystone | 14:26 | |
| bknudson | I don't think we're going to be able to fix this in keystone... it'll have to be grenade | 14:27 |
| bknudson | why are packages getting reinstalled? | 14:27 |
| bknudson | maybe keystone installed the wrong version or neutron is | 14:27 |
| bknudson | do the logs show reinstalling packages? | 14:28 |
| dstanek | bknudson: why i was looking earlier in the week thing were definitely getting reinstalled | 14:28 |
| bknudson | dstanek: on purpose? I don't see why that would be required if the right version was installed to begin with | 14:29 |
| bknudson | http://logs.openstack.org/66/185066/3/check/check-grenade-dsvm-neutron/45d8663/logs/grenade.sh.txt.gz#_2015-06-18_09_08_25_942 ? | 14:30 |
| ihrachyshka | bknudson, which packages do you mean? | 14:31 |
| bknudson | all packages | 14:31 |
| ihrachyshka | bknudson, well, this is done because the overall process looks like: install kilo, start services; check they are running; then stop services; upgrade code to master (it includes calling to pip install and bumping versions for all unsatisfied deps), then start services; and run tempest. | 14:32 |
| ihrachyshka | it's upgrade test, so it upgrades packages. | 14:32 |
| bknudson | that shouldn't be a problem if keystone is restarted after packages are updated. | 14:33 |
| ihrachyshka | bknudson, if you mean python-openstackclient installation triggering keystoneclient version bump, then no, it's not the culprit: I sent a patch that moved its installation to before keystone upgrade, and it didn't help | 14:34 |
| ihrachyshka | I would agree though that that particular openstackclient installation should not be done in global python path | 14:34 |
| ihrachyshka | venv would suffice and isolate services from unneeded version bumps | 14:35 |
| *** samueldmq has joined #openstack-keystone | 14:35 | |
| bknudson | so here's the call to upgrade_project keystone: http://logs.openstack.org/66/185066/3/check/check-grenade-dsvm-neutron/45d8663/logs/grenade.sh.txt.gz#_2015-06-18_09_05_42_311 | 14:36 |
| bknudson | here's the keystone files: http://git.openstack.org/cgit/openstack-dev/grenade/tree/projects/10_keystone | 14:37 |
| dstanek | that error log shows it fails because of a glance call. is that actually glance failing to talk to keystone? | 14:37 |
| ihrachyshka | dstanek, it's keystone crashing on import error and apache returning 501 | 14:39 |
| *** ihrachyshka is now known as ihrachyshka|away | 14:40 | |
| henrynash | rodigods: see comments I added | 14:43 |
| dstanek | bknudson: i can't tell from this log...is keystone being restarted after the upgrade? | 14:44 |
| bknudson | dstanek: I'm trying to figure that out, too | 14:45 |
| bknudson | if it's running in apache you need to restart apache | 14:45 |
| henrynash | rodigods: oh…now I see what you did - you merged it into mine? | 14:45 |
| *** pnavarro is now known as pnavarro|off | 14:45 | |
| henrynash | rodigods: not sure taht makes sense | 14:45 |
| dstanek | bknudson: right. which makes me wonder about this http://git.openstack.org/cgit/openstack-dev/grenade/tree/projects/10_keystone/shutdown.sh | 14:45 |
| bknudson | http://logs.openstack.org/66/185066/3/check/check-grenade-dsvm-neutron/45d8663/logs/grenade.sh.txt.gz#_2015-06-18_09_05_09_279 | 14:45 |
| bknudson | looks like it reloads apache2 ? | 14:45 |
| henrynash | rodigods: me spec doesn’t talk about adding is_domain to the project entity etc. | 14:47 |
| bknudson | dstanek: Stop apache2 at 090509, Upgrade keystone at 090542, Restart apache2 at 090543 | 14:48 |
| bknudson | that all seems right | 14:48 |
| bknudson | http://logs.openstack.org/66/185066/3/check/check-grenade-dsvm-neutron/45d8663/logs/grenade.sh.txt.gz#_2015-06-18_09_05_50_645 | 14:49 |
| bknudson | then it goes on to upgrade ceilometer, etc. | 14:49 |
| rodrigods | henrynash, yep, but I can remove from your change | 14:49 |
| henrynash | rodigods: great….I made commens on your patch | 14:50 |
| *** vilobhmm has joined #openstack-keystone | 14:50 | |
| henrynash | rodigods: technically, your patch would be first to add is_domain to the project entity and calrify the rules of scoping…but NOT mention the token change….then mine would follow yours adding in the token change | 14:51 |
| rodrigods | henrynash, yes... we discussed this here | 14:52 |
| rodrigods | we just want to speed up the approval | 14:52 |
| dstanek | bknudson: but serveral more upgrades happen between the keystone restart and the keystone failure | 14:52 |
| henrynash | rodigods: so I’m Ok leaving it in teh current order….but just can’t squash the API changes into mine if mine is first…or I have to rewrite my spec! | 14:53 |
| bknudson | dstanek: y, upgrading other servers... if those are messing with already-installed packages that might be a problem. | 14:53 |
| bknudson | packages that keystone is using | 14:53 |
| rodrigods | henrynash, ++ | 14:54 |
| *** ROT26 has joined #openstack-keystone | 14:54 | |
| e0ne | morganfainberg: hello. fyi, i created spec for oslo.service about non-eventlet WSGI app https://review.openstack.org/#/c/196088/ | 14:55 |
| *** stevemar has joined #openstack-keystone | 14:56 | |
| *** fhubik has quit IRC | 14:56 | |
| *** belmoreira has quit IRC | 14:57 | |
| *** htruta has quit IRC | 14:58 | |
| openstackgerrit | Rodrigo Duarte proposed openstack/keystone-specs: Add is_domain to tokens for projects acting as a domain https://review.openstack.org/193543 | 14:59 |
| openstackgerrit | Rodrigo Duarte proposed openstack/keystone-specs: API changes for Reseller https://review.openstack.org/153007 | 14:59 |
| rodrigods | henrynash, fixed the nits and replied your comment regarding is_domain | 14:59 |
| rodrigods | henrynash, check if you agree with it | 14:59 |
| henrynash | rodigods: looking | 14:59 |
| bknudson | dstanek: so in the logs we see | 15:00 |
| bknudson | Collecting oslo.middleware!=2.0.0,>=1.2.0 (from keystone==2015.2.0.dev1) | 15:00 |
| bknudson | Collecting oslo.middleware<1.1.0,>=1.0.0 (from oslo.messaging>=1.8.0->glance==2015.2.0.dev91) | 15:00 |
| dstanek | yay for compatibility! | 15:00 |
| bknudson | so it downgrades oslo.middleware | 15:00 |
| bknudson | that can't be right | 15:01 |
| bknudson | I'm not saying that's the problem but maybe there are more of these. | 15:01 |
| openstackgerrit | Rodrigo Duarte proposed openstack/keystone-specs: API changes for Reseller https://review.openstack.org/153007 | 15:01 |
| *** lufix has quit IRC | 15:02 | |
| dstanek | e0ne: is that just documenting what we already do in keystone? | 15:02 |
| morganfainberg | bknudson: the error is in loading oslo_utils.excutils, i'm wondering if we're hitting one of those weird namespace errors on oslo.utils vs oslo_utils | 15:02 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/oslo.policy: Add tox target to find missing requirements https://review.openstack.org/195842 | 15:03 |
| e0ne | dstanek: it's propose to make it based on oslo.service | 15:03 |
| bknudson | we see other errors, too | 15:03 |
| morganfainberg | bknudson: the oslo_config one is a redherring | 15:03 |
| morganfainberg | bknudson: that is because we had an error earlier. | 15:03 |
| morganfainberg | and the options are already registered | 15:03 |
| henrynash | rodigdos: agreed | 15:04 |
| rodrigods | henrynash, great! | 15:04 |
| *** ajayaa has quit IRC | 15:05 | |
| bknudson | morganfainberg: here's another one: http://logs.openstack.org/77/195277/16/check/check-grenade-dsvm-neutron/99916dc/logs/apache/keystone.txt.gz#_2015-06-26_11_22_06_803004 | 15:05 |
| bknudson | http://logs.openstack.org/77/195277/16/check/check-grenade-dsvm-neutron/99916dc/logs/apache/keystone.txt.gz#_2015-06-26_11_21_59_792406 -- ImportError: No module named keystoneclient.common | 15:05 |
| *** mestery has joined #openstack-keystone | 15:06 | |
| morganfainberg | bknudson: i think we need to get infra to hold one of these instances for us | 15:06 |
| morganfainberg | bknudson: and poke at it directly | 15:06 |
| dstanek | e0ne: oh, i see. i didn't realize that a framework is needed here. can you add details about what the framework is for to the review? | 15:06 |
| e0ne | dstanek: thanks, i'll do | 15:07 |
| e0ne | dstanek: i'm pretty sure that we'll have some the same code in all projects so i'll try to move it to oslo | 15:07 |
| dstanek | e0ne: imo, we should wait and see a few impls before picking one as a framework | 15:08 |
| e0ne | dstanek: fair enouph. let's see what we'll we have in cinder | 15:08 |
| *** henrynash has quit IRC | 15:09 | |
| e0ne | dstanek: according to your comment. i don't want to implement new web framework. i'm going to to it based on webob or another one which is used in projects | 15:10 |
| e0ne | dstanek: i need to add more clear explanation into the spec. thanks for review | 15:10 |
| dstanek | e0ne: i wasn't talking about a framework; i was talking about the one you are proposing | 15:11 |
| e0ne | dstanek: i understood. thank you | 15:11 |
| dstanek | e0ne: the interesting thing is that i'm working on a flask port for keystone that will not use webob | 15:11 |
| e0ne | :) | 15:12 |
| morganfainberg | e0ne; we are moving to flask | 15:12 |
| e0ne | imo, it's great idea! | 15:13 |
| dstanek | e0ne: i think in that spec you should layout the responsibilities of the framework in general terms | 15:13 |
| openstackgerrit | Monty Taylor proposed openstack/keystoneauth: Remove opestack-common.conf https://review.openstack.org/196098 | 15:13 |
| *** dimsum__ has quit IRC | 15:13 | |
| dstanek | for a wsgi app you just need to export an application object | 15:13 |
| dstanek | bknudson: once my devstack node builds i'll create a grenade node too - why do these things take so long :-( | 15:14 |
| bknudson | looks like a lot of requirements are way out of date... | 15:14 |
| bknudson | in several packages | 15:14 |
| bknudson | e.g., glance is failing jenkins | 15:14 |
| e0ne | dstanek: agree. | 15:16 |
| e0ne | dstanek: i would like to implement such feature in cinder and look what code will be the same or the similar to the keystone's | 15:16 |
| e0ne | dstanek: and yes, it's not needed in case of flask:) | 15:17 |
| *** mestery has quit IRC | 15:17 | |
| dstanek | e0ne: i'm curious what you're thinking because keystone already support wsgi and eventlet deployment | 15:18 |
| *** samueldmq has quit IRC | 15:21 | |
| morganfainberg | bknudson: trying to grab the failing test on a real node | 15:22 |
| morganfainberg | dstanek, ^cc | 15:23 |
| dstanek | morganfainberg: awesome | 15:23 |
| *** e0ne is now known as e0ne_ | 15:25 | |
| *** e0ne_ is now known as e0ne | 15:25 | |
| bknudson | could we get https://review.openstack.org/#/c/193741/ merged sometime so that we can get keystone requirements updated? | 15:27 |
| morganfainberg | bknudson: +2 | 15:28 |
| e0ne | dstanek: afair, only keystone supports these two deployments modes | 15:28 |
| dstanek | bknudson: looks like that's rolling now | 15:29 |
| bknudson | I think glance has the same bug which is why their requirements is so out of date | 15:30 |
| *** radez is now known as radez_g0n3 | 15:30 | |
| dstanek | sigmavirus24: ^ | 15:30 |
| * sigmavirus24 looks | 15:30 | |
| *** radez_g0n3 is now known as radez | 15:30 | |
| *** tellesnobrega has quit IRC | 15:34 | |
| *** tellesnobrega has joined #openstack-keystone | 15:38 | |
| *** stevemar has quit IRC | 15:40 | |
| *** stevemar has joined #openstack-keystone | 15:41 | |
| *** zzzeek has joined #openstack-keystone | 15:42 | |
| *** ajayaa has joined #openstack-keystone | 15:48 | |
| *** stevemar has quit IRC | 15:50 | |
| *** stevemar has joined #openstack-keystone | 15:50 | |
| *** stevemar has quit IRC | 15:50 | |
| *** stevemar has joined #openstack-keystone | 15:51 | |
| *** pballand has joined #openstack-keystone | 15:54 | |
| *** htruta has joined #openstack-keystone | 15:56 | |
| *** vilobhmm has quit IRC | 15:58 | |
| *** janonymous_ has joined #openstack-keystone | 16:00 | |
| janonymous_ | https://review.openstack.org/#/c/193866/ | 16:01 |
| *** njewell has joined #openstack-keystone | 16:04 | |
| *** anhhuynx has joined #openstack-keystone | 16:04 | |
| *** Jason10258 has joined #openstack-keystone | 16:04 | |
| morganfainberg | crap | 16:05 |
| morganfainberg | node passed when i was looking at it | 16:05 |
| morganfainberg | there is some weird order bug | 16:05 |
| morganfainberg | bknudson, dstanek, ^ | 16:05 |
| openstackgerrit | Merged openstack/keystoneauth: Remove keystoneclient lingering files. https://review.openstack.org/195710 | 16:08 |
| *** rushiagr_away is now known as rushiagr | 16:09 | |
| *** e0ne is now known as e0ne_ | 16:14 | |
| *** gabriel-bezerra has quit IRC | 16:15 | |
| *** e0ne_ has quit IRC | 16:15 | |
| mfisch | dolphm: idea from ducttape_, if someone hands me an invalid fernet token does it try all the keys on the box? if so that would spike load * number of keys you hold | 16:15 |
| morganfainberg | mfisch: you do try each key - the idea is you shouldn't keep keys around outside of the (rotate_window + token_TTL) | 16:16 |
| *** Lactem has joined #openstack-keystone | 16:16 | |
| morganfainberg | mfisch: generally i wouldn't expect too many keys - unless you're doing an insane rotation | 16:17 |
| mfisch | you guys may want to publish some guidance on that | 16:17 |
| morganfainberg | mfisch: asusming 86400s TTL, and a 1x a day rotation, you wont have more than 2-3 keys at a time | 16:17 |
| mfisch | the rotation tool does not handle "remove these 5 really really old keys" | 16:17 |
| mfisch | not to my knowledge anyway | 16:17 |
| mfisch | we have 3 keys | 16:17 |
| Lactem | http://docs.openstack.org/developer/python-keystoneclient/using-api.html The example at the very bottom of that page (how to make an endpoint) uses Client. Does anyone know how to get Client in a test case such as in one of these methods: https://github.com/openstack/keystone/blob/master/keystone/tests/unit/test_v3_catalog.py#L410-L724 ? | 16:17 |
| morganfainberg | mfisch: 3 keys is about what i'd expect on normal installs | 16:17 |
| morganfainberg | mfisch: and a max keynum of ~4 or so. | 16:18 |
| mfisch | an invalid token before was not 3x cpu load, so its something for people to keep in mind | 16:18 |
| mfisch | I think today we got hit with that from swift | 16:18 |
| bknudson | didn't the server need to go through every token to see if it was handed an invalid one? | 16:18 |
| morganfainberg | mfisch: 3x CPU in fernet is also much lower than 3x "get from DB" or 3x "PKI decode" | 16:18 |
| mfisch | it wasn't 3x before for invalid | 16:19 |
| mfisch | select * from token where id='x'; (0 rows selected) | 16:19 |
| morganfainberg | mfisch: invalid as in revoked? | 16:19 |
| bknudson | that requires a table scan | 16:19 |
| morganfainberg | mfisch: or invalid as in expirred? | 16:19 |
| mfisch | as in old yeah | 16:19 |
| mfisch | expired | 16:19 |
| bknudson | or a walk through the index if there's an index on the token id | 16:19 |
| mfisch | actually this is all wrong | 16:19 |
| morganfainberg | bknudson: it was a walk through the index | 16:20 |
| mfisch | if an old token comes in, the first key decrypts it, and then keystone says "this is old", | 16:20 |
| Jason10258 | Do any of you know where the CA key files/configs are? | 16:20 |
| morganfainberg | since there was an index | 16:20 |
| *** fangzhou has joined #openstack-keystone | 16:20 | |
| Lactem | Woah it's Jason10258. I'm a huge fan. | 16:20 |
| bknudson | it must have been a hash of the token then since the token is 8k | 16:20 |
| mfisch | I assume it doesnt have to fallback to other keys | 16:20 |
| morganfainberg | mfisch: yes that has to be done. it's a 3x decrypt vs a walk the index to DB | 16:20 |
| morganfainberg | mfisch: if it succeeds it will stop looking at subsequent keys | 16:20 |
| mfisch | morganfainberg: but an expired token will decrypt with the primary key still | 16:20 |
| morganfainberg | mfisch: correct. it's 3x cost on a rotated key (well 2x in your case) | 16:21 |
| Jason10258 | Do any of you know where the CA key files/configs are? | 16:21 |
| morganfainberg | mfisch: if you get a really old token - that is where the cpu cost if higher | 16:22 |
| morganfainberg | mfisch: not just any expired | 16:22 |
| Jason10258 | or do they just not exist anymore? | 16:22 |
| mfisch | I'm speaking of a regular old expired token | 16:22 |
| morganfainberg | mfisch: then it's decrypted with the current key | 16:22 |
| mfisch | I assume it's: decrypt, look at valid time, reject | 16:22 |
| mfisch | ok | 16:22 |
| morganfainberg | mfisch: no fallback | 16:22 |
| mfisch | would not make sense otherwise | 16:22 |
| morganfainberg | mfisch: fallback only happens if the key is mismatched and it needs to try the alternate keys | 16:23 |
| morganfainberg | and i *think* it does the HMAC for that check | 16:23 |
| *** ducttape_ has joined #openstack-keystone | 16:23 | |
| morganfainberg | instead of a full HMAC + decrypt | 16:23 |
| morganfainberg | so it checks HMAC sig and that is how it knows if it should even try and decrypt | 16:23 |
| morganfainberg | which iirc is lower cost. | 16:24 |
| *** Akshay00 has joined #openstack-keystone | 16:24 | |
| mfisch | thanks morganfainberg | 16:24 |
| rodrigods | morganfainberg, can you take a look in https://review.openstack.org/#/c/193543/ ? I believe we addressed bknudson comment to remove the WIP from the follow up patch | 16:28 |
| *** tellesnobrega has quit IRC | 16:28 | |
| openstackgerrit | Merged openstack/keystone: Don't try to drop FK constraints for sqlite https://review.openstack.org/193741 | 16:30 |
| *** gyee has joined #openstack-keystone | 16:31 | |
| *** ChanServ sets mode: +v gyee | 16:31 | |
| *** fangzhou_ has joined #openstack-keystone | 16:32 | |
| Jason10258 | Do any of you know where the CA key files/configs are? | 16:32 |
| Jason10258 | or do they just not exist anymore? | 16:32 |
| *** fangzhou has quit IRC | 16:33 | |
| *** fangzhou_ is now known as fangzhou | 16:33 | |
| Jason10258 | I am an intern at Intel working on a bug (https://bugs.launchpad.net/keystone/+bug/1287414) and I am having troubles finding the semi-existant CA keys | 16:34 |
| openstack | Launchpad bug 1287414 in Keystone "Keystone should not require CA key" [Low,Triaged] - Assigned to Jason O'Brien (jason10258) | 16:34 |
| morganfainberg | Jason10258: i think those are only generated if you use pkisetup | 16:35 |
| morganfainberg | Jason10258: which is only needed when you are doing pki tokens | 16:35 |
| *** tellesnobrega has joined #openstack-keystone | 16:35 | |
| Jason10258 | ok thank you . Ill try that out | 16:35 |
| *** Ephur has joined #openstack-keystone | 16:36 | |
| *** gabriel-bezerra has joined #openstack-keystone | 16:38 | |
| *** _cjones_ has joined #openstack-keystone | 16:39 | |
| *** Lactem has quit IRC | 16:39 | |
| *** roxanaghe has joined #openstack-keystone | 16:43 | |
| *** e0ne has joined #openstack-keystone | 16:43 | |
| *** dguerri is now known as dguerri` | 16:44 | |
| *** e0ne has quit IRC | 16:45 | |
| *** henrynash has joined #openstack-keystone | 16:46 | |
| *** ChanServ sets mode: +v henrynash | 16:46 | |
| morganfainberg | ayoung: is jamielennox|away on vacation? | 16:48 |
| mfisch | morganfainberg: can I change the crypt_strength without screwing up in-flight tokens? | 16:48 |
| morganfainberg | mfisch: uhmm........ | 16:49 |
| mfisch | I was going to do an experiment on something | 16:49 |
| morganfainberg | mfisch: what crypt strength you changing? | 16:49 |
| mfisch | the one in keystone.conf, isn't it used for the tokens? | 16:50 |
| mfisch | # The value passed as the keyword "rounds" to passlib's | 16:50 |
| mfisch | # encrypt method. (integer value) | 16:50 |
| mfisch | #crypt_strength=40000 | 16:50 |
| morganfainberg | nope | 16:50 |
| morganfainberg | that is for password rounds | 16:50 |
| morganfainberg | https://github.com/openstack/keystone/blob/master/keystone/common/config.py#L83-L84 | 16:51 |
| mfisch | ah I thought dstanek said otherwise, thanks | 16:51 |
| morganfainberg | yeah fernet uses HMAC256(create_time, AES128(payload)) i think | 16:51 |
| morganfainberg | and is fixed | 16:51 |
| mfisch | ty | 16:51 |
| morganfainberg | fixed in the cryptography library we use | 16:51 |
| morganfainberg | not something we specify | 16:51 |
| morganfainberg | mfisch: so you can totally change the crypt strength and not affect tokens ¬_¬ | 16:52 |
| mfisch | ubuntu package has the wrong default in the config file | 16:53 |
| *** dimsum__ has joined #openstack-keystone | 16:53 | |
| morganfainberg | we just updated that default recently | 16:53 |
| mfisch | that would explain | 16:53 |
| mfisch | looks like 40k to 10k? | 16:53 |
| morganfainberg | yep | 16:53 |
| morganfainberg | https://github.com/openstack/keystone/commit/67e0ba5ee2108731050e26f7b4dd6c8d3dab118d | 16:53 |
| *** spandhe has joined #openstack-keystone | 16:54 | |
| *** browne has quit IRC | 16:59 | |
| *** pballand has quit IRC | 17:00 | |
| morganfainberg | henrynash, rodrigods, ayoung, do we have a blog post about setting up an LDAP backed domain somewhere | 17:01 |
| *** HT_sergio has joined #openstack-keystone | 17:01 | |
| morganfainberg | dstanek, ^cc | 17:01 |
| morganfainberg | i'm sure we had one somewhere... | 17:01 |
| morganfainberg | mfisch, ^ cc | 17:01 |
| henrynash | morganfainberg: I think samueldmq had something in terms | 17:01 |
| *** pballand has joined #openstack-keystone | 17:02 | |
| henrynash | ayoung too, me thinks | 17:02 |
| morganfainberg | ah here http://adam.younglogic.com/2015/02/adding-an-ldap-backed-domain-to-a-packstack-install/ | 17:02 |
| mfisch | not a domain sorry | 17:02 |
| *** atiwari has joined #openstack-keystone | 17:02 | |
| *** atiwari has quit IRC | 17:03 | |
| *** tellesnobrega has quit IRC | 17:03 | |
| *** atiwari has joined #openstack-keystone | 17:03 | |
| openstackgerrit | Merged openstack/keystone: Switch to oslo.service https://review.openstack.org/193732 | 17:06 |
| morganfainberg | bknudson, ^ woooo | 17:06 |
| openstackgerrit | Merged openstack/keystone: Update sample configuration file https://review.openstack.org/193879 | 17:06 |
| bknudson | morganfainberg: 1k fewer lines of code to worry about | 17:06 |
| morganfainberg | bknudson: exactly! | 17:06 |
| *** anhhuynx has quit IRC | 17:08 | |
| *** lhcheng has joined #openstack-keystone | 17:08 | |
| *** ChanServ sets mode: +v lhcheng | 17:08 | |
| openstackgerrit | henry-nash proposed openstack/keystone-specs: Enable retrieval of default values of domain config options https://review.openstack.org/185650 | 17:08 |
| *** ajayaa has quit IRC | 17:08 | |
| *** lhcheng_ has joined #openstack-keystone | 17:09 | |
| *** ROT26 has quit IRC | 17:09 | |
| *** shaleh has joined #openstack-keystone | 17:10 | |
| henrynash | bknudson: so I think we got the API sorted out in the follow-on patch to: https://review.openstack.org/#/c/193543/ (it’s no longer WIP and aligns with this now)….if you agree, maybe you can mark this one... | 17:10 |
| bknudson | ok. I didn't look at the WIP patch before. It's on my list | 17:11 |
| Akshay00 | henrynash:I am working on https://bugs.launchpad.net/keystone/+bug/1287414 with Jason10258 And I noticed you did the initial commit which added ca_key as a command line option. Trying to figure out how best to remove it and if that is the best plan. As a note I am also a high school intern at Intel :) | 17:11 |
| openstack | Launchpad bug 1287414 in Keystone "Keystone should not require CA key" [Low,Triaged] - Assigned to Jason O'Brien (jason10258) | 17:11 |
| *** lhcheng has quit IRC | 17:12 | |
| henrynash | Akshay00: I did? you sure it was not ayoung? | 17:13 |
| Akshay00 | henrynash: I saw this commit 1ed2046eaa91fa36926d66a5fe1e88ccd65373bb | 17:14 |
| ayoung | henrynash, all your fault. | 17:14 |
| ayoung | Not really | 17:15 |
| henrynash | ayoung: :-) | 17:15 |
| ayoung | Akshay00, so, I think that is just for the case I am trying to destroy anyway | 17:15 |
| ayoung | I want to get rid of the whole openssl approach to cert generation | 17:15 |
| ayoung | one of many things I submtitted to under duress. I was young, I needed the money. | 17:15 |
| gyee | he's telling the truth | 17:16 |
| ayoung | Akshay00, so, the reason that is there is to have a place to hold the key when generating the self signing. You do need that if you do pki_setup or ssl_setup, as you need to sign the non-ca cert | 17:16 |
| gyee | ayoung, you want to put certmonger in there or you just want to kill off the whole thing? | 17:17 |
| ayoung | http://2.bp.blogspot.com/-X4jXPyzIt10/Ur3Am5WsSZI/AAAAAAAABcQ/rRu-6QzCSys/s640/Jake+no+glasses.png | 17:17 |
| ayoung | gyee, I want certmonger, and make it the abstraction for selfsign, for people that want it, but also the way to talk to a real CA | 17:18 |
| gyee | lets do this | 17:18 |
| ayoung | gyee, HP has its own CA server, right? Does it have a published protocol for posting CSRS? | 17:18 |
| gyee | no need to fix that bug then | 17:18 |
| gyee | ayoung, we are using ephemeral CA at the moment | 17:19 |
| *** rlt_ has quit IRC | 17:20 | |
| Akshay00 | henrynash: Do you suggest that I try to find a different bug for me to work on, if you have a plan to remove it | 17:20 |
| Akshay00 | ?? | 17:20 |
| henrynash | Akshay00: was that for me or ayoung? | 17:21 |
| ayoung | henrynash, I am not a manager. I resent being treated like one. | 17:21 |
| Akshay00 | henrynash:that was for you | 17:21 |
| ayoung | Everyone is asking me what they should work on. | 17:21 |
| *** ihrachyshka|away is now known as ihrachyshka | 17:22 | |
| raildo | ayoung, delegate | 17:22 |
| *** e0ne has joined #openstack-keystone | 17:22 | |
| amakarov | bknudson, hi! I see some of your patches don't have neither bug nor bp/spec reference: can you please educate me, are there any principles when bp is needed and when it is not? | 17:23 |
| henrynash | Akshay00: feel free to pick a bug from the backlog…. | 17:23 |
| *** ducttape_ has left #openstack-keystone | 17:23 | |
| bknudson | amakarov: if it's not changing the behavior of the server then no bp or spec is needed | 17:23 |
| bknudson | amakarov: if somebody thinks it would be good for those to have a bp or a spec I'll add one. | 17:23 |
| Akshay00 | So should I switch bugs then? | 17:23 |
| Akshay00 | henrynash:? | 17:24 |
| amakarov | bknudson, thank you - things are clearer now for me ) | 17:24 |
| *** samueldmq has joined #openstack-keystone | 17:25 | |
| bknudson | not putting a bug or blueprint on it means that it's harder to track and prioritize... but I'm fine with waiting a while for these changes to get in. | 17:25 |
| *** EmilienM is now known as EmilienM|brb | 17:25 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 17:27 | |
| *** ajayaa has joined #openstack-keystone | 17:28 | |
| *** pballand has quit IRC | 17:30 | |
| *** pballand has joined #openstack-keystone | 17:31 | |
| *** ankita_wagh has joined #openstack-keystone | 17:35 | |
| *** Akshay00 has quit IRC | 17:36 | |
| *** Akshay00 has joined #openstack-keystone | 17:36 | |
| *** browne has joined #openstack-keystone | 17:37 | |
| *** henrynash has quit IRC | 17:39 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 17:41 | |
| *** tqttran_afk has joined #openstack-keystone | 17:41 | |
| *** Lactem has joined #openstack-keystone | 17:42 | |
| Lactem | dolphm: You there? | 17:42 |
| Lactem | I wrote something. https://gist.github.com/Lactem/5a43296b8975da24db51 Would that suffice? (You said to make a test that does what https://paste.ee/p/pdyrS does.) | 17:44 |
| *** fangzhou has quit IRC | 17:47 | |
| Lactem | Can any of the pros take a look and see if they think that would pass for the bug https://bugs.launchpad.net/keystone/+bug/1098564 or not? @morganfainberg @Akshay00 @Jason10258 | 17:51 |
| openstack | Launchpad bug 1098564 in Keystone "Cannot delete a service or endpoint" [Low,Incomplete] - Assigned to Theodore Ilie (theoilie-ti) | 17:51 |
| stevemar | dimsum__: morganfainberg https://review.openstack.org/#/c/195865/1 | 17:54 |
| *** e0ne is now known as e0ne_ | 17:55 | |
| *** anhhuynx has joined #openstack-keystone | 17:56 | |
| *** Lactem has quit IRC | 17:57 | |
| anhhuynx | I found out that when you do >keystone ec2-credentials-list on the CLI it would only list the admin's credentials, and you have to add the option --user-id <user-id> in order to get that user's credentials | 17:58 |
| anhhuynx | is that intended, or is that a bug? | 17:58 |
| *** pballand has quit IRC | 17:58 | |
| anhhuynx | or am I doing something wrong? | 17:58 |
| *** tqttran_afk is now known as tqtran | 18:00 | |
| *** Lactem has joined #openstack-keystone | 18:01 | |
| *** ankita_wagh has quit IRC | 18:01 | |
| *** ankita_w_ has joined #openstack-keystone | 18:01 | |
| *** pballand has joined #openstack-keystone | 18:02 | |
| *** EmilienM|brb is now known as EmilienM | 18:04 | |
| *** rushiagr is now known as rushiagr_away | 18:04 | |
| *** hogepodge has quit IRC | 18:06 | |
| *** samueldmq has quit IRC | 18:06 | |
| *** samueldmq has joined #openstack-keystone | 18:06 | |
| *** lhcheng_ has quit IRC | 18:07 | |
| *** arunkant__ has joined #openstack-keystone | 18:07 | |
| *** ankita_w_ has quit IRC | 18:07 | |
| *** arunkant_ has quit IRC | 18:08 | |
| *** ankita_wagh has joined #openstack-keystone | 18:08 | |
| stevemar | bknudson: got a minute? | 18:09 |
| *** e0ne_ is now known as e0ne | 18:10 | |
| breton | bknudson: are you going to test https://review.openstack.org/195766 somehow? I'll write a functests for it if you haven't yet. | 18:11 |
| openstackgerrit | Steve Martinelli proposed openstack/keystone: switch to oslo.cache https://review.openstack.org/195873 | 18:11 |
| openstackgerrit | Fernando Diaz proposed openstack/keystone: Adding Documentation for Mapping Combinations https://review.openstack.org/192850 | 18:12 |
| *** arunkant__ has quit IRC | 18:13 | |
| *** diazjf has joined #openstack-keystone | 18:13 | |
| bknudson | breton: I haven't written tests for it | 18:14 |
| *** rushiagr_away is now known as rushiagr | 18:17 | |
| bknudson | breton: I proposed the change to devstack | 18:17 |
| *** sjcherry has joined #openstack-keystone | 18:18 | |
| sjcherry | Latest keystone in git won't start. | 18:19 |
| sjcherry | 2015-06-26 14:16:15.750 19309 CRITICAL keystone [-] TypeError: Service <keystone.common.environment.eventlet_server.Server object at 0x32efc10> must an instance of <class 'oslo_service.service.ServiceBase'>! | 18:19 |
| stevemar | sjcherry: using eventlet? | 18:19 |
| stevemar | bknudson: uh oh | 18:19 |
| sjcherry | Cinder had the same problem last week after the switch to oslo.service. | 18:19 |
| dstanek | sjcherry: that's odd | 18:19 |
| stevemar | sjcherry: yeah, we just switched to oslo.service now | 18:19 |
| dstanek | did we switch to oslo_service? | 18:19 |
| stevemar | dstanek: a few minutes ago | 18:19 |
| dstanek | haha, ok | 18:20 |
| breton | bknudson: what change? | 18:20 |
| breton | bknudson: nevermind, see it now | 18:21 |
| sjcherry | The fix for cinder was in https://review.openstack.org/#/c/195369 | 18:21 |
| sjcherry | I expect each service will see the same issue one by one. | 18:24 |
| gyee | ayoung, why do we need epsilon for k2k federation? its that a requirement for mod_mellon? | 18:28 |
| *** david8hu has joined #openstack-keystone | 18:28 | |
| ayoung | gyee, other way around,. ipsilon uses mod_mellon. Don't know that we need it for K2K | 18:29 |
| ayoung | I think that Isilon could be useful for K2K. I have not put that much brainpower in to the problem | 18:30 |
| *** dramakri has joined #openstack-keystone | 18:30 | |
| gyee | ayoung, I remember nkinder mentioned mod_mellon is waiting on something from epsilon in order to support ecp | 18:30 |
| gyee | maybe I am hearing him wrong | 18:30 |
| *** Lactem has quit IRC | 18:31 | |
| ayoung | gyee, other way round | 18:31 |
| ayoung | mod_mellon has some not-yet-merged upstream commits we are waiting for | 18:31 |
| ayoung | ipsilon, not epsilon...different greek letter | 18:31 |
| *** arif-ali has quit IRC | 18:31 | |
| gyee | ayoung, ah, my bad | 18:32 |
| openstackgerrit | Steve Martinelli proposed openstack/keystone: Use oslo.service ServiceBase when loading from eventlet https://review.openstack.org/196175 | 18:32 |
| stevemar | sjcherry: review ^ ? | 18:32 |
| *** Akshay00 has quit IRC | 18:32 | |
| *** arif-ali has joined #openstack-keystone | 18:36 | |
| *** arif-ali has quit IRC | 18:37 | |
| *** arif-ali has joined #openstack-keystone | 18:38 | |
| bknudson | stevemar: what happened? did it break? | 18:38 |
| bknudson | I thought we told everybody to run keystone under httpd? | 18:39 |
| *** rushiagr is now known as rushiagr_away | 18:39 | |
| gyee | what break? eventlet? | 18:39 |
| *** njewell has quit IRC | 18:39 | |
| *** Jason10258 has quit IRC | 18:40 | |
| bknudson | When I run with devstack keystone master works just fine. | 18:41 |
| *** hogepodge has joined #openstack-keystone | 18:41 | |
| dstanek | bknudson: me too. i wonder if it's an oslo.service version problem... | 18:42 |
| *** e0ne has quit IRC | 18:42 | |
| bknudson | oslo.service==0.1.0 | 18:43 |
| bknudson | which is the latest on pypi | 18:43 |
| dstanek | bknudson: i'm wondering if others don't have the latest | 18:44 |
| bknudson | oslo.service>=0.1.0 is in global-requirements | 18:44 |
| sjcherry | Cinder is the only other project that I've seen change to oslo.service. | 18:44 |
| dstanek | or maybe they are running the bleeding edge: http://git.openstack.org/cgit/openstack/oslo.service/commit/?id=59ed2ec386c90fc6fdb5e7353a62f729e071f795 | 18:44 |
| bknudson | that change isn't backwards-compatible. | 18:45 |
| dstanek | bknudson: nope | 18:45 |
| dstanek | sjcherry: i suspect once olso_service is released with that commit then others will likely need to fix | 18:47 |
| bknudson | keystone-all fails with master oslo.service. | 18:47 |
| sjcherry | dstanek, Yep. That's why I poked in here as soon as I saw that error in my logs. | 18:47 |
| *** samueldmq2 has joined #openstack-keystone | 18:48 | |
| *** samueldmq2 has quit IRC | 18:48 | |
| sjcherry | I saw it earlier in Cinder but that one the cinder-api service wouldn't even start. | 18:48 |
| bknudson | proposed a revert: https://review.openstack.org/#/c/196183/ | 18:49 |
| sjcherry | oslo.service 0.1.0 has the same check but only in the lauch() function. | 18:49 |
| bknudson | I guess as long as we can get ahead of it it's fine | 18:50 |
| bknudson | this is why I hate ABCMeta. | 18:51 |
| morganfainberg | bknudson: because it doesn't enforce useful things? Because #python | 18:51 |
| gyee | hah | 18:51 |
| *** samueldmq has quit IRC | 18:52 | |
| bknudson | it enforces a little bit, but not much | 18:52 |
| bknudson | so what's the point | 18:52 |
| bknudson | fine if you control all the code but doesn't make sense in a library | 18:52 |
| morganfainberg | We should rewrite keystone in rust | 18:52 |
| *** arunkant has joined #openstack-keystone | 18:52 | |
| dstanek | it enforces just enough to be painful | 18:52 |
| morganfainberg | Or golang | 18:52 |
| bknudson | and the way it's used here it's just checking isinstance in a couple places... wtf | 18:53 |
| bknudson | C++ | 18:53 |
| stevemar | bknudson: you just dont get it, it's too hip and cool | 18:53 |
| morganfainberg | Nah. The ldap driver would be too much to maintain in c++ (did it in a past life) | 18:53 |
| bknudson | considering keystone-all worked fine before the change proves it was unnecessary | 18:53 |
| morganfainberg | Let's rewrite the ldap integration from scratch | 18:53 |
| gyee | ++ | 18:54 |
| morganfainberg | In a new language. | 18:54 |
| morganfainberg | Cause why not m | 18:54 |
| bknudson | let's not write the ldap integration, use apache support | 18:54 |
| openstackgerrit | Steve Martinelli proposed openstack/keystone: switch to oslo.cache https://review.openstack.org/195873 | 18:54 |
| *** e0ne has joined #openstack-keystone | 18:54 | |
| gyee | bknudson, been there, done that | 18:54 |
| openstackgerrit | Boris Bobrov proposed openstack/keystone: functional tests for keystone on subpaths https://review.openstack.org/196186 | 18:54 |
| morganfainberg | Aaaaaaaaannnnnyyyway | 18:54 |
| morganfainberg | So what is broken with oslo_service? | 18:54 |
| morganfainberg | Just Oslo service bad in a recent commit? | 18:54 |
| stevemar | morganfainberg: only with master branch of oslo_service | 18:54 |
| bknudson | morganfainberg: it's not broken... yet | 18:54 |
| morganfainberg | Ah ok | 18:55 |
| gyee | thought we are getting rid of eventlet so what's the point? | 18:55 |
| bknudson | it removes 1k lines of code from oslo-incubator | 18:55 |
| breton | in fact, I still don't like that we move away from eventlet | 18:56 |
| breton | it was easy to use it for debugging | 18:56 |
| gyee | I use print | 18:56 |
| breton | do import pdb; pdb.set_trace() and launch from terminal keystone-all | 18:56 |
| stevemar | breton: use rpdb | 18:56 |
| *** rushiagr_away is now known as rushiagr | 18:56 | |
| morganfainberg | gyee: 1k fewer lines to deal with until we get liberty eol | 18:56 |
| breton | stevemar: what about gdb? I am not sure I can use it with apache | 18:57 |
| stevemar | never tried that one | 18:57 |
| gyee | gdb? that works with python code? | 18:57 |
| sigmavirus24 | gyee: it can | 18:57 |
| gyee | nice! I didn't know that | 18:57 |
| morganfainberg | breton: sorry, but eventlet was a bad option for a pure wsgi app in the first place. Mod_wsgi gunicorn uwsgi all better options and have been around a long time | 18:58 |
| kfox1111 | morganfainberg: I just read up on how aws is doing things. they extended their metadata server to hand back their equiv of keystone tokens, just like in my origonal proposal. :/ | 18:58 |
| sigmavirus24 | it's better for debugging the interpreter or C extensions, but it works gyee | 18:58 |
| gyee | sigmavirus24, thanks! I'll give it a try | 18:58 |
| breton | stevemar: I had to use it to debug https://bugs.launchpad.net/keystone/+bug/1420788 | 18:58 |
| openstack | Launchpad bug 1420788 in Keystone juno "Logging blocks on race condition under eventlet" [Medium,Fix released] - Assigned to Boris Bobrov (bbobrov) | 18:58 |
| sigmavirus24 | gyee: have fun ;) | 18:58 |
| kfox1111 | see: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html section "Retrieving Security Credentials from Instance Metadata" | 18:58 |
| morganfainberg | breton: another case why eventlet is a bad idea.. Doesn't occur in Apache/managed wsgi | 18:59 |
| bknudson | we need eventlet so that we can debug -- eventlet problems | 18:59 |
| morganfainberg | :P | 18:59 |
| morganfainberg | bknudson: right? ;) | 18:59 |
| gyee | keep us employed | 18:59 |
| morganfainberg | kfox1111: then do it that way, but realize anyone who uses config drive will not get your neat feature. | 18:59 |
| breton | it was terrible for running in production, but pretty good for running on localhost | 19:00 |
| breton | django does that, for example | 19:00 |
| kfox1111 | yeah. I'm afraid with config drive only allowing static data, the dynamic rekeying stuff that the metadata route enables is out of the question anyway. :/ | 19:00 |
| morganfainberg | breton: django doesn't use eventlet. It uses webrick? We could do similar. | 19:00 |
| stevemar | dimsum__: having some issues with oslo.cache config options | 19:00 |
| stevemar | dimsum__: did it work for you? | 19:00 |
| morganfainberg | breton: but that is the same method as the managed wsgi, just a single process | 19:01 |
| dstanek | morganfainberg: flask will take care of this going forward | 19:01 |
| breton | dstanek: when will flask happen? | 19:01 |
| dstanek | or at least can | 19:01 |
| morganfainberg | dstanek: yeah I know. It's internal single process manager that works just like a wsgi manager like mod_wsgi | 19:01 |
| morganfainberg | breton: this cycle | 19:02 |
| stevemar | bknudson: got a minute to look at https://review.openstack.org/#/c/195873/ | 19:02 |
| kfox1111 | with static, if you mothball your vm for a while, then start it back up, if you ever have to rotate your keys then your vm's dead anyway. :/ | 19:02 |
| dstanek | breton: I want to have first pass done by Monday | 19:02 |
| bknudson | stevemar: I'm actually fine with https://review.openstack.org/#/c/196175/ | 19:02 |
| breton | ok, that's cool | 19:02 |
| stevemar | i'll resurrect it then bknudson | 19:02 |
| morganfainberg | kfox1111: so, you have a bigger political battle to fight I think. Make metadata service not sucky and available everywhere. | 19:03 |
| *** Ctina__ has joined #openstack-keystone | 19:03 | |
| bknudson | I guess that's the way we're supposed to do it. | 19:03 |
| dstanek | breton: I was shooting for this week, but to much life happened | 19:03 |
| kfox1111 | morganfainberg: Yeah. :/ | 19:03 |
| kfox1111 | I personally don't think its sucky. I do understand others don't think the same. | 19:03 |
| morganfainberg | kfox1111: haven | 19:03 |
| bknudson | stevemar: did you check if there's any unused reqs after removing all the code in https://review.openstack.org/#/c/195873/ ? | 19:03 |
| kfox1111 | yeah. :/ | 19:04 |
| morganfainberg | Having run it.. It needs love | 19:04 |
| breton | dstanek: I'll be happy to review it when it happens | 19:04 |
| morganfainberg | It causes issues at scale. | 19:04 |
| morganfainberg | Or it did at least 2 releases ago | 19:04 |
| stevemar | bknudson: not yet, should do that soon though, there seems to be an issue with registering the config options | 19:04 |
| kfox1111 | yeah. the implementation could use work. but I don't think thats reason to throw the baby out with the bath water. | 19:04 |
| kfox1111 | amazon's md server scales, so its at least theoratically possible. | 19:04 |
| morganfainberg | "rm -rf eventlet_code_in_keystone" | 19:05 |
| stevemar | bknudson: and looking at this: https://github.com/openstack/oslo.cache/blob/master/oslo_cache/_opts.py#L124-L130 it should be registering automagically | 19:05 |
| morganfainberg | That would also solve our issues :P | 19:05 |
| kfox1111 | yay. byby eventlet! :) | 19:05 |
| bknudson | stevemar: automatically? | 19:05 |
| bknudson | if by automatic you mean when you call the function | 19:06 |
| *** Ctina_ has quit IRC | 19:06 | |
| kfox1111 | so are you ok doing something where some instance user management service (md server or otherwise) manages the certs and gives vm's keystone tokens when needed? or is that still -1? | 19:06 |
| kfox1111 | just going to go back and basically rewrite the whole blody spec and want to lay out all the options again. :/ | 19:06 |
| stevemar | hmm i guess i need to call cache.configure(CONF) | 19:07 |
| *** Ctina__ has quit IRC | 19:07 | |
| gyee | kfox1111, with that spec, we are potentially creating thousands accounts in keystone, one per instance right? | 19:09 |
| *** harlowja has quit IRC | 19:09 | |
| kfox1111 | possibly. though the account is managed by ca, so it doesn't actually exist in keystone. | 19:09 |
| kfox1111 | kind of a federated thing. | 19:09 |
| gyee | kfox1111, I see, so we just map a cert to a group in Keystone for example | 19:10 |
| *** pballand has quit IRC | 19:10 | |
| kfox1111 | yup. | 19:10 |
| gyee | that'll work | 19:10 |
| stevemar | bknudson: calling https://github.com/openstack/oslo.cache/blob/master/oslo_cache/core.py#L316-L317 at keystone/server/common.py doesn't seem to make things happy | 19:11 |
| *** ankita_wagh has quit IRC | 19:11 | |
| *** pballand has joined #openstack-keystone | 19:12 | |
| *** ankita_wagh has joined #openstack-keystone | 19:13 | |
| *** amakarov is now known as amakarov_away | 19:13 | |
| *** janonymous_ has quit IRC | 19:14 | |
| bknudson | stevemar: I think it goes in http://git.openstack.org/cgit/openstack/keystone/tree/keystone/common/config.py#n1121 | 19:14 |
| *** r-daneel has quit IRC | 19:17 | |
| *** ankita_wagh has quit IRC | 19:17 | |
| stevemar | bknudson: that seems to make the tests happy, not genconfig though | 19:18 |
| bknudson | genconfig uses the entrypoint | 19:18 |
| bknudson | doesn't it? | 19:18 |
| *** rushiagr is now known as rushiagr_away | 19:19 | |
| *** ankita_wagh has joined #openstack-keystone | 19:19 | |
| *** markvoelker has quit IRC | 19:20 | |
| *** woodster_ has quit IRC | 19:21 | |
| *** harlowja has joined #openstack-keystone | 19:22 | |
| *** markvoelker has joined #openstack-keystone | 19:26 | |
| *** Jason10258 has joined #openstack-keystone | 19:31 | |
| *** Jason10258 has quit IRC | 19:31 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 19:31 | |
| morganfainberg | bknudson: it needs to be added to the config file thing too. | 19:31 |
| morganfainberg | For genconfig iirc. | 19:31 |
| morganfainberg | Since keystone is treated as an actual lib too. | 19:32 |
| openstackgerrit | Steve Martinelli proposed openstack/keystone: switch to oslo.cache https://review.openstack.org/195873 | 19:32 |
| bknudson | http://git.openstack.org/cgit/openstack/keystone/tree/config-generator/keystone.conf | 19:32 |
| *** markvoelker has quit IRC | 19:32 | |
| morganfainberg | https://github.com/openstack/keystone/blob/master/config-generator/keystone.conf | 19:32 |
| morganfainberg | Yeah. | 19:32 |
| bknudson | jinx | 19:32 |
| *** markvoelker has joined #openstack-keystone | 19:32 | |
| stevemar | i added it there >.< | 19:32 |
| bknudson | are other projects using oslo.cache? | 19:34 |
| raildo | anyone can review this spec? :D https://review.openstack.org/#/c/193543/10 | 19:34 |
| kfox1111 | morganfainberg: so are you ok doing something where some instance user management service (md server or otherwise) manages the certs and gives vm's keystone tokens when needed? or is that still -1? | 19:35 |
| *** Lactem has joined #openstack-keystone | 19:35 | |
| morganfainberg | kfox1111: that's fine if it really is the only option. | 19:36 |
| kfox1111 | ok. | 19:36 |
| morganfainberg | kfox1111: but I'm eating lunch so -- no hard decisions right now ;) | 19:36 |
| kfox1111 | hehe. ok. | 19:36 |
| kfox1111 | fair enough. :) | 19:36 |
| *** e0ne has quit IRC | 19:36 | |
| *** jay__ has joined #openstack-keystone | 19:36 | |
| kfox1111 | since everyone seems to want to see all the options, just trying to figure out options to lay out. | 19:36 |
| *** atiwari has quit IRC | 19:37 | |
| *** dramakri has quit IRC | 19:38 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 19:38 | |
| *** markvoelker_ has joined #openstack-keystone | 19:38 | |
| *** markvoelker has quit IRC | 19:40 | |
| openstackgerrit | Steve Martinelli proposed openstack/keystone: switch to oslo.cache https://review.openstack.org/195873 | 19:42 |
| *** markvoelker has joined #openstack-keystone | 19:44 | |
| *** markvoelker_ has quit IRC | 19:45 | |
| *** sjcherry has quit IRC | 19:45 | |
| morganfainberg | stevemar: have bandwidth for like 3 easy reviews? | 19:47 |
| *** Lactem has quit IRC | 19:48 | |
| morganfainberg | stevemar: https://review.openstack.org/#/c/193543/10 and the two cleanup ones in keystoneauth (preceding the namespace change) | 19:48 |
| *** fangzhou has joined #openstack-keystone | 19:48 | |
| stevemar | morganfainberg: ay ay, captn | 19:48 |
| stevemar | link me the two from keystoneauth | 19:48 |
| morganfainberg | Starts here... https://review.openstack.org/#/c/195712/ you don't need to do the namespace one | 19:50 |
| morganfainberg | If you aren't up for t. | 19:50 |
| morganfainberg | Looks like one in ksa already merged. | 19:50 |
| morganfainberg | So just that one. | 19:50 |
| *** markvoelker has quit IRC | 19:54 | |
| *** mestery has joined #openstack-keystone | 19:54 | |
| *** HT_sergio has quit IRC | 19:54 | |
| *** markvoelker has joined #openstack-keystone | 19:58 | |
| *** pballand has quit IRC | 19:59 | |
| *** pballand has joined #openstack-keystone | 20:02 | |
| *** e0ne has joined #openstack-keystone | 20:03 | |
| *** pnavarro|off has quit IRC | 20:07 | |
| *** markvoelker_ has joined #openstack-keystone | 20:09 | |
| *** markvoelker has quit IRC | 20:11 | |
| *** markvoelker_ has quit IRC | 20:12 | |
| *** jay__ has quit IRC | 20:14 | |
| *** browne has quit IRC | 20:15 | |
| *** dramakri has joined #openstack-keystone | 20:17 | |
| *** tqtran has quit IRC | 20:17 | |
| morganfainberg | dstanek: do you have a WIP patch for flask or not quite that far yet? | 20:18 |
| *** pnavarro|off has joined #openstack-keystone | 20:19 | |
| dstanek | morganfainberg: i don't, but i can get one together pretty quickly i think | 20:23 |
| dstanek | i just have to unbreak a few things that i broke | 20:23 |
| *** e0ne has quit IRC | 20:24 | |
| *** ankita_wagh has quit IRC | 20:25 | |
| *** ankita_wagh has joined #openstack-keystone | 20:27 | |
| *** pballand has quit IRC | 20:30 | |
| *** pballand has joined #openstack-keystone | 20:32 | |
| openstackgerrit | Morgan Fainberg proposed openstack/keystone: Updated from global requirements https://review.openstack.org/190405 | 20:33 |
| stevemar | morganfainberg: why the move to keystoneauth1 | 20:37 |
| morganfainberg | stevemar: so we can have major versions co-installed on the system | 20:38 |
| stevemar | dimsum__: i think the double role/assignment options in oslo.cache are breaking us | 20:38 |
| morganfainberg | basically the idea is that any change that could break an interface at all justifies a new major version | 20:38 |
| morganfainberg | this library has to be insanely stable | 20:38 |
| morganfainberg | it's going to be used by SDK | 20:38 |
| stevemar | fair enough | 20:38 |
| morganfainberg | and client libs | 20:38 |
| morganfainberg | the next step is ditching oslo.config for it | 20:39 |
| *** e0ne has joined #openstack-keystone | 20:39 | |
| morganfainberg | and we're like 90% ready for a release | 20:39 |
| morganfainberg | or 95% | 20:39 |
| morganfainberg | (stable release that is) | 20:39 |
| raildo | btw, thanks stevemar for the reviews in the hmt stuff on openstack client :) | 20:42 |
| stevemar | raildo: np at all, part of the job! | 20:42 |
| *** e0ne has quit IRC | 20:44 | |
| *** dramakri has quit IRC | 20:46 | |
| stevemar | morganfainberg: do you have the ability to release a new oslo.cache? | 20:49 |
| *** ajayaa has quit IRC | 20:49 | |
| morganfainberg | Nope | 20:50 |
| morganfainberg | Have to ask dhellman or one of the other release managers. | 20:50 |
| *** markvoelker has joined #openstack-keystone | 20:51 | |
| dimsum__ | stevemar: i can | 20:52 |
| dimsum__ | will be able to do it in a couple of hours. ok? | 20:52 |
| dimsum__ | unless what you need is in already in master | 20:53 |
| kfox1111 | morganfainberg: just to explore some more options... what about regular federation. I know nothing about saml, but could the metadata server be made an identity provider and then it just redirects to keystone with the appropriate bits? | 20:54 |
| morganfainberg | kfox1111: you're likely to be unhappy with that - the metadata service is already doing too muhc. you probably want a microservice then | 20:54 |
| morganfainberg | kfox1111: that can focus on "IdP" specific code | 20:55 |
| * dimsum__ vanishes for a couple of hours | 20:55 | |
| morganfainberg | kfox1111: it can't "just provide" identity. Federation has a lot of overhead because it has to be trusted data. | 20:55 |
| morganfainberg | kfox1111: the client certs are hooking into that to help aleviate the need to be a true IdP. | 20:56 |
| morganfainberg | hence the recommendation to go that route | 20:56 |
| stevemar | dimsum__: i still need to figure out what the fix is :) | 20:56 |
| kfox1111 | ah. ok. just curious. | 20:57 |
| stevemar | morganfainberg: got a minute? | 20:57 |
| *** mestery has quit IRC | 20:57 | |
| morganfainberg | stevemar: sure | 20:57 |
| stevemar | morganfainberg: https://review.openstack.org/#/c/195872/ | 20:58 |
| morganfainberg | stevemar: expecting a call in like 2 mins, but worst case can come back when done | 20:58 |
| stevemar | oh it'll be longer than that i think | 20:58 |
| morganfainberg | how did you break that? | 20:58 |
| stevemar | https://github.com/openstack/oslo.cache/blob/master/oslo_cache/tests/test_cache.py#L197-L201 | 20:58 |
| stevemar | morganfainberg: ^ | 20:59 |
| morganfainberg | ahahah | 20:59 |
| morganfainberg | yeah | 20:59 |
| morganfainberg | hard-coded "assignment" | 20:59 |
| kfox1111 | I wonder if I should just recommend making a new network service that runs on every hypervisor that attaches through a secondary network interface that provides this service. :/ | 20:59 |
| stevemar | morganfainberg: create a new group at test time? | 20:59 |
| kfox1111 | its frustrating since the metadata servers supposed to just be that. :/ | 20:59 |
| morganfainberg | stevemar: there is a way to register the group dynamically with the config fixture | 20:59 |
| morganfainberg | stevemar: i recommend using a specifically generated group rather than a "keystone" one | 20:59 |
| stevemar | theres also this one https://github.com/openstack/oslo.cache/blob/master/oslo_cache/tests/test_cache.py#L140 | 20:59 |
| morganfainberg | same thing | 21:00 |
| morganfainberg | use the config fixture to register a group like uuid.uuid4().hex | 21:00 |
| morganfainberg | or some such | 21:00 |
| morganfainberg | and reference it | 21:00 |
| stevemar | yeah, i'll do that | 21:00 |
| morganfainberg | :) | 21:00 |
| morganfainberg | kfox1111: maybe. | 21:01 |
| morganfainberg | kfox1111: i can't speak to the "need another agent" | 21:01 |
| kfox1111 | heh. yeah. :/ | 21:01 |
| *** woodster_ has joined #openstack-keystone | 21:02 | |
| kfox1111 | I guess that's just one more possible implementation for the alternatives section | 21:04 |
| kfox1111 | (its going to get huge. :/) | 21:04 |
| *** pballand has quit IRC | 21:05 | |
| *** bknudson has quit IRC | 21:06 | |
| *** raildo has quit IRC | 21:06 | |
| *** browne has joined #openstack-keystone | 21:08 | |
| *** htruta has quit IRC | 21:13 | |
| kfox1111 | morganfainberg: I have an idea.... | 21:18 |
| kfox1111 | I did this exact thing in barbican, but maybe its better to do it simply in keystone... | 21:18 |
| kfox1111 | we give Nova metadata server its own self signed cert. we load that cert's ca into keystone and give it a domain. | 21:19 |
| *** markvoelker has quit IRC | 21:19 | |
| kfox1111 | we extend the nova metadata server to create a signed message with the cert saying "the bearer of this message is nova instance id=xxxxx-xxxx-xxx-xxxxx" | 21:20 |
| kfox1111 | the instance passes that signed message to keystone as autentication credentials, kesytone gives back a token. | 21:21 |
| *** iurygregory has quit IRC | 21:21 | |
| kfox1111 | no barbican needed in that workflow, or lots of certs, or anything. | 21:21 |
| kfox1111 | we'd only need to add a simple authentication plugin to keystone to support that workflow. | 21:22 |
| gyee | kfox1111, you can do that today already, via federation | 21:22 |
| kfox1111 | gyee: morganfainberg just said it was hard to do that with federation as it exists today. are we missing something? | 21:23 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 21:23 | |
| *** harlowja has quit IRC | 21:23 | |
| *** openstack has joined #openstack-keystone | 21:24 | |
| gyee | have the path protected by mod_ssl | 21:24 |
| kfox1111 | morganfainberg: what do you think? | 21:25 |
| gyee | kfox1111, seem my comment at the bottom http://adam.younglogic.com/2015/03/key-fed-lookup-redux/ | 21:26 |
| gyee | I got it working with PAM, its not that hard to create one for SSL | 21:26 |
| kfox1111 | gyee: have a look at: https://review.openstack.org/#/c/159573/ | 21:27 |
| gyee | looking | 21:28 |
| kfox1111 | specifically vendordata_barbican.py and token.py | 21:28 |
| *** markvoelker has joined #openstack-keystone | 21:28 | |
| kfox1111 | what would go into the keystone plugin would basically be whats in context.py | 21:29 |
| kfox1111 | note, "token" there doen't meen keystone token. just a signed message. | 21:30 |
| *** markvoelker has quit IRC | 21:30 | |
| *** stevemar has quit IRC | 21:30 | |
| *** Raildo has joined #openstack-keystone | 21:31 | |
| *** stevemar has joined #openstack-keystone | 21:32 | |
| gyee | kfox1111, you are assuming PKI/Z token | 21:32 |
| morganfainberg | gyee: "federation" is different than the mod_ssl, he was talking true IdP | 21:33 |
| kfox1111 | no, the code is just reused from thje PKI/Z module. :) | 21:34 |
| *** Raildo is now known as raildo | 21:34 | |
| kfox1111 | its reusing its code to sign/verify the signed message. | 21:34 |
| morganfainberg | gyee: client certs as federation would be ok | 21:34 |
| morganfainberg | kfox1111: FYI s/mime (CMS) adds about a minimum of 1KB overhead | 21:34 |
| gyee | morganfainberg, right, I thought that's what he wants | 21:34 |
| morganfainberg | gyee: yeah that was what i was recommending vs. being a true "IdP" because it wouldn't be a true IdP | 21:35 |
| morganfainberg | certs are a bit stubby attribute wise (without going crazy non-standard) | 21:35 |
| kfox1111 | morganfainberg: I'm just thinking, maybe a way to extend some kind of trusted relationship between Nova and Keystone such that, if nova says, give me a token for user=instanceid, its given one. | 21:35 |
| *** harlowja has joined #openstack-keystone | 21:35 | |
| morganfainberg | kfox1111: that is how the client cert stuff is meant to work - but it requires a group (IdP) and an assignment on said group | 21:36 |
| kfox1111 | certs are a way towards that, but seems like it may be unnessisary? | 21:36 |
| gyee | kfox1111, security folks won't let you get away with impersonation | 21:36 |
| morganfainberg | kfox1111: we still need a way to authoritatively validate the reuqests | 21:37 |
| morganfainberg | gyee: it isn't impersonation, he's asking for an ephemeral uers | 21:37 |
| morganfainberg | user* | 21:37 |
| gyee | that would be federation then | 21:37 |
| morganfainberg | like how federation works, but a bit more "vm = user-id" | 21:37 |
| morganfainberg | vm-id* | 21:37 |
| morganfainberg | ugh | 21:37 |
| morganfainberg | vm-id == user_id | 21:37 |
| kfox1111 | authoratatively can be done via nova's existing keystone user? | 21:37 |
| morganfainberg | kfox1111: not really. Keystone doesn't support the idea that $service can create a token for $arbitrary_user_info | 21:38 |
| *** henrynash has joined #openstack-keystone | 21:38 | |
| *** ChanServ sets mode: +v henrynash | 21:38 | |
| morganfainberg | kfox1111: we require a cryptographically verified set of data that has an explicit trust. | 21:38 |
| kfox1111 | today. would that be out of the question though? it may be by far the easiest thing to implement? | 21:38 |
| morganfainberg | kfox1111: i'd say we're dangerously close to the cert thing now :P | 21:39 |
| kfox1111 | quite possibly. | 21:39 |
| morganfainberg | and it means keystone has much less code to write to support you | 21:39 |
| kfox1111 | I'm just thinking the cert is long lasting. probably longer then is strictly nessisary. | 21:39 |
| gyee | if you are exchanging cert for a token, zero code to write :) | 21:39 |
| kfox1111 | then its gota be stored in a db. | 21:39 |
| gyee | for keystone | 21:39 |
| morganfainberg | if you're asking for an API to let $service issue an ephemeral user + token - you're now asking for custom APIs and i'll be blunt - it is unlikely to get in during liberty | 21:40 |
| kfox1111 | if a signed message can be used instead, but have all the same properties, that might be generated on the fly and be easier to mange. | 21:40 |
| morganfainberg | kfox1111: and i'd need to do some serious thinking about how to isolate the security concerns | 21:40 |
| kfox1111 | understood. | 21:40 |
| morganfainberg | kfox1111: there *is* the ephemeral PKI work | 21:40 |
| kfox1111 | just exploring options. since others keep throwing up roadblocks. | 21:40 |
| morganfainberg | but i don't know where that is today | 21:40 |
| gyee | why do you need a user token? to fetch the secret from barbican right? | 21:41 |
| morganfainberg | i'm not opposed to some extra trust mechanism with clear delegation to an ephemeral user via a transport | 21:41 |
| morganfainberg | kfox1111: what i don't want is to rush it in liberty and open security holes | 21:41 |
| morganfainberg | kfox1111: if you need this in liberty (sounds like the timeline) i'd say certs are the easiest | 21:41 |
| kfox1111 | gyee: thats one example, or to let a guest agent talk to zaqar, or allow an app to fetch stuff from swift, or nagios to discover instances to monitor, etc. | 21:41 |
| gyee | but that can be done via group mapping | 21:42 |
| morganfainberg | if you are talking about a service that can run with just user-id (ACLs are in the service) then cert is free | 21:42 |
| kfox1111 | morgainfainberg: agreed. I think its probably dead for liberty at this point anywah. nova's killed it by waiting so long to review. :/ | 21:42 |
| morganfainberg | if you need roles in keystone, it's a little more work | 21:42 |
| morganfainberg | and we need to make sure we're not causing you a roadblock by requiring the role to be on a roup | 21:42 |
| morganfainberg | group* | 21:42 |
| morganfainberg | s/in keystone/from keystone | 21:42 |
| gyee | just create a project with the correct roles assigned | 21:43 |
| kfox1111 | if tokens are handed out by the metadata server, then the way it gets the token from keystone is an implementation detail that can be resolved later. | 21:43 |
| openstackgerrit | Merged openstack/keystoneauth: Remove catalog/translation targets from tox.ini https://review.openstack.org/195712 | 21:43 |
| *** anhhuynx has quit IRC | 21:43 | |
| kfox1111 | we can start with something really stupid like user/pw and sql, and replace it with ca's or federation or whatever and no one will be the wiser. | 21:43 |
| morganfainberg | gyee: the point is making sure we're not forcing them into a model that wont work - just requires mapping it out | 21:43 |
| openstackgerrit | Merged openstack/keystoneauth: Move to the keystoneauth1 namespace https://review.openstack.org/191003 | 21:43 |
| openstackgerrit | Merged openstack/keystoneauth: Remove opestack-common.conf https://review.openstack.org/196098 | 21:43 |
| kfox1111 | thats why I really like the idea of the md server handing out tokens. keystone tokens are always going to be a thing. | 21:44 |
| kfox1111 | kesystone username/pw, certs, whatever might not. | 21:44 |
| morganfainberg | kfox1111: no they wont. i'm working to make them also disappear | 21:44 |
| kfox1111 | its nicely abstracted. | 21:44 |
| morganfainberg | kfox1111: bearer tokens are awful | 21:44 |
| kfox1111 | hmm... | 21:44 |
| morganfainberg | kfox1111: but the impact to your system will be minimal - it'll just be a new way to auth | 21:44 |
| morganfainberg | ;) | 21:44 |
| morganfainberg | so don't worry about the bearer tokens dieing | 21:44 |
| kfox1111 | k. | 21:45 |
| morganfainberg | it really wont be hard to move to a new system once they are ready to go | 21:45 |
| morganfainberg | just don't expect it to *always* be bearer tokens | 21:45 |
| morganfainberg | as in try and look at the token to figure out things ;) | 21:45 |
| kfox1111 | yeah. the contents of th token are opaque. | 21:45 |
| morganfainberg | assume the authorization is opaque | 21:45 |
| kfox1111 | exactly. :) | 21:45 |
| kfox1111 | thats what I like about the idea. we assume it goes back to keystone and keystone's the only one to make any meaning out of it. | 21:45 |
| kfox1111 | then we can replace the mechanism in the md server with anything we want over the years and the api doesn't have to change. | 21:46 |
| kfox1111 | if somehow quantum encryptiion becomes a thing, :) nova->keystone can use that to get a token. :) | 21:46 |
| *** bknudson has joined #openstack-keystone | 21:47 | |
| *** ChanServ sets mode: +v bknudson | 21:47 | |
| gyee | with cloud, quantum is possible | 21:47 |
| gyee | just wait and see :) | 21:47 |
| kfox1111 | hehe. | 21:47 |
| kfox1111 | so... to really simplify the spec for now, can I just say the initial implementation will create username/pw's in keystone in its own domain? I can always write a migration script that goes through the db and converts them to barbican CA certs or some trusted handshaked federation thingy in the future when we decide what that should look like? | 21:49 |
| *** pballand has joined #openstack-keystone | 21:50 | |
| kfox1111 | It should be possible to be done in place without a downtime? | 21:50 |
| gyee | one user per instance? | 21:50 |
| *** htruta has joined #openstack-keystone | 21:50 | |
| stevemar | morganfainberg: https://review.openstack.org/#/c/195872/ cc dimsum__ | 21:51 |
| kfox1111 | one user per instance that has requested the instance user feature. | 21:51 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/keystone: Updated from global requirements https://review.openstack.org/190405 | 21:51 |
| morganfainberg | kfox1111: i think you're writing yourself into a corner by updating things in keystone | 21:51 |
| bknudson | stevemar: oslo.cache doesn't need role and assignment drivers? | 21:51 |
| gyee | we are looking at potentially thousands of users | 21:51 |
| morganfainberg | kfox1111: creating users/groups | 21:51 |
| morganfainberg | kfox1111: i am really against nova being able to do that | 21:51 |
| *** ihrachyshka has quit IRC | 21:52 | |
| bknudson | heat already does it | 21:52 |
| morganfainberg | kfox1111: because then every otherservice wants to do it. | 21:52 |
| morganfainberg | bknudson: heat is a bit special | 21:52 |
| kfox1111 | yes, but hte whole point of instance users is to prevent other services needing to do that. :/ | 21:52 |
| bknudson | morganfainberg: "special" ? | 21:52 |
| gyee | like special olympic? | 21:52 |
| bknudson | he he | 21:52 |
| * kfox1111 chuckles | 21:52 | |
| morganfainberg | bknudson: i have un-pc words to use for some of heats specialness | 21:52 |
| morganfainberg | kfox1111: then we should make it more generic - does neutron need this for it's ports somehow? | 21:53 |
| kfox1111 | heat creates users since it needs instances to talk back to it. if nova instance users were there, it could just switch to them I think. | 21:53 |
| stevemar | bknudson: it certainly does not | 21:53 |
| kfox1111 | no, but octavia needs it for getting user's barbican ssl certs to load balancers. | 21:53 |
| morganfainberg | kfox1111: i aslo worry about adding 1000s of users to the SQL DB backend in keystone | 21:53 |
| morganfainberg | legitimately the SQL backend is terribad | 21:53 |
| kfox1111 | sahara could use it to esablish a connection between a guest agent running in a sahara instance zaquar and its controllers. | 21:54 |
| morganfainberg | and we've opted to spend zero time making it a lot better, assuming it's minimal usage | 21:54 |
| morganfainberg | kfox1111: making this work like an emphemeral/federated user is a lot less overhead. | 21:54 |
| morganfainberg | imo | 21:54 |
| stevemar | bknudson: grrr, still not registering the config options when i generate a new config file | 21:55 |
| kfox1111 | morganfainberg: I agree, the scaling aspect sucks. I'm just trying to split the need of getting an API going and abstracted, from an implementation that can be continuously imporved over time. | 21:55 |
| morganfainberg | kfox1111: i also think you're going to see most deployers tell you to fly a kite on allowing nova to create users in keystone | 21:55 |
| kfox1111 | agreed. if we can figure out how to make it work quickly. | 21:55 |
| morganfainberg | kfox1111: so you're writing a feature that the majority of people wont want to enable | 21:55 |
| kfox1111 | will they be more receptive to be having to install barbican? | 21:55 |
| bknudson | stevemar: the config generator is silent on failures | 21:55 |
| morganfainberg | kfox1111: i'd say yes. | 21:55 |
| stevemar | bknudson: i am looking at git diff | 21:56 |
| kfox1111 | I want to agree. at the moment, I'd disagree simply because there are no rpms yet. :/ | 21:56 |
| kfox1111 | and heat already does instance users. :/ | 21:56 |
| *** rwsu has quit IRC | 21:56 | |
| *** pnavarro|off has quit IRC | 21:57 | |
| morganfainberg | kfox1111: i mean i can't stop you from proposing this to nova and having them agree it is the right way | 21:57 |
| morganfainberg | i can tell you none of the public clouds will ever turn this on | 21:57 |
| morganfainberg | or at least RAX and HP wont [afaik] | 21:57 |
| kfox1111 | is rax even running keystone yet? | 21:57 |
| kfox1111 | I thought they were doing their own thing in a lot of ways. :/ | 21:58 |
| morganfainberg | no, but they are moving to it soon(tm) [required for defcore compliance] | 21:58 |
| gyee | jython last I heard | 21:58 |
| morganfainberg | among other reasons | 21:58 |
| * hogepodge looks around | 21:58 | |
| morganfainberg | but RAX users and such are managed outside of keystone | 21:58 |
| kfox1111 | what I'm interested in most is starting to grow the community of heat developers able to create cloud scaled apps that can be contributed. | 21:58 |
| morganfainberg | same with HP's users | 21:58 |
| morganfainberg | keystone does not manage users for either cloud | 21:59 |
| morganfainberg | keysotne consumes the users. | 21:59 |
| kfox1111 | if it takes a while for the public clouds to adopt it, thats probably ok, because it takes them so long to adaopt any openstack release anyway. :/ | 21:59 |
| morganfainberg | and brokers the access to the clouds | 21:59 |
| kfox1111 | if we can fix the issue before they adopt, they won't even notice. | 21:59 |
| morganfainberg | kfox1111: the user DBs are read-only as far as keystone is concerned | 21:59 |
| gyee | kfox1111, in HP cloud, everything user is connected to a billing account | 21:59 |
| kfox1111 | how are they supporting heat? | 21:59 |
| kfox1111 | (or are they?) | 21:59 |
| gyee | so Nova can't just create users | 21:59 |
| morganfainberg | and i've publically stated (and stand this line) that keystone's user-management API will never be defcore required | 22:00 |
| gyee | in fact, user creation is part of onboarding workflow | 22:00 |
| morganfainberg | because many many many deployments do not want to allow keystone write access to the user-store | 22:00 |
| kfox1111 | thats one of the reasons I wanted instances users to be different hten just letting users have a domain and create instance users themselves. | 22:00 |
| kfox1111 | that kind of policy prohibits it. | 22:00 |
| hogepodge | morganfainberg: it's one I agree with | 22:00 |
| hogepodge | morganfainberg: based on my own past usage | 22:01 |
| hogepodge | amongst other things | 22:01 |
| morganfainberg | hogepodge: you and I are much on the same page for what should/shouldn't be required for a cloud to be "interoperable" | 22:01 |
| gyee | agree 10000% | 22:01 |
| morganfainberg | kfox1111: so i think we need to make the users look like ephemeral users. | 22:01 |
| gyee | IdP is outside of defcore | 22:01 |
| hogepodge | Give me a machine running an image with a network and storage? One can dream. ;-) | 22:01 |
| kfox1111 | morganfainberg: I totally agree with you. | 22:02 |
| bknudson | that's only because they don't want to take advantage of per-domain backends | 22:02 |
| morganfainberg | kfox1111: the short path is CA -> client cert | 22:02 |
| morganfainberg | bknudson: even with per-domain backends, keystone cannot write to the per-domain backend | 22:02 |
| morganfainberg | bknudson: those are read-only | 22:02 |
| bknudson | you can have both read-write and read-only backends | 22:02 |
| morganfainberg | afaik we can't write to a per-domain backend | 22:03 |
| bknudson | oops, domains | 22:03 |
| morganfainberg | the code doesn't support it | 22:03 |
| morganfainberg | only supports writes to default | 22:03 |
| morganfainberg | (not defualt domain, default backend) | 22:03 |
| bknudson | we really only support having sql as the primary domain | 22:03 |
| bknudson | which allows adding domains and r/w users of course | 22:03 |
| morganfainberg | bknudson: we unfortunately broke deployments by asserting that | 22:03 |
| morganfainberg | bknudson: we need to undo that | 22:03 |
| morganfainberg | :( | 22:04 |
| gyee | bknudson, how about lets spend a weekend hack up SCIM with Flask? | 22:04 |
| morganfainberg | honestly, i think we need to take a hard look at SCIM for this | 22:04 |
| bknudson | there's already SCIM servers | 22:04 |
| bknudson | according to SCIM.com | 22:04 |
| morganfainberg | it might be what is needed | 22:04 |
| gyee | in django | 22:04 |
| morganfainberg | nova can utilize SCIM to handle the identity bits per-vm. | 22:04 |
| breton | > even with per-domain backends, keystone cannot write to the per-domain backend | 22:04 |
| breton | really? | 22:05 |
| morganfainberg | breton: as far as i know, yes. it was a design choice | 22:05 |
| kfox1111 | so, I guess we're at, barbican ca, keystone x590 federation, nova metadata uses cert, hands out tokens to the vms? | 22:05 |
| bknudson | a read-only sql backend makes no sense | 22:05 |
| breton | I thought I created users in ldap configured per-domain | 22:05 |
| morganfainberg | breton: it might have been changed, but oh dear god i don't think we test those paths well | 22:05 |
| bknudson | we don't test per-domain backends | 22:06 |
| bknudson | or ldap | 22:06 |
| morganfainberg | kfox1111: *or* | 22:06 |
| morganfainberg | kfox1111: another way to communicate (preferably via an apache module) the user information in a secure way | 22:06 |
| morganfainberg | that we can hook into the federated system | 22:06 |
| *** roxanaghe has quit IRC | 22:06 | |
| morganfainberg | kfox1111: x509 is a known quantity and we know the work needed to get there | 22:07 |
| *** pballand has quit IRC | 22:07 | |
| gyee | x509 cert for token should be zero addition work for keystone | 22:07 |
| morganfainberg | kfox1111: in fact, almost (if not all) the work will be done as a side effect of initiatives we're pushing to make service users better. thats the reason why i keep comin back to it | 22:07 |
| *** stevemar has quit IRC | 22:07 | |
| breton | bknudson: I see several tests with domain_specific_drivers_enabled=True in tests/unit/test_backend_ldap.py, don't they work? | 22:08 |
| *** stevemar has joined #openstack-keystone | 22:08 | |
| morganfainberg | breton: we have some tests but it's not really tested in a devstack | 22:08 |
| *** edmondsw has quit IRC | 22:08 | |
| bknudson | breton: you can have a writable ldap domain-specific driver. | 22:08 |
| morganfainberg | unit != functional/truely working | 22:08 |
| kfox1111 | morganfainberg: I'm not sure I see how thats different. | 22:08 |
| *** edmondsw has joined #openstack-keystone | 22:08 | |
| bknudson | why would a real deployment ever use read-write ldap backend? | 22:08 |
| *** edmondsw has quit IRC | 22:08 | |
| morganfainberg | bknudson: i don't know. i stopped trying to understand that | 22:09 |
| bknudson | if I was using ldap I'd have better tools for adding users already | 22:09 |
| morganfainberg | bknudson: but people do it and usually the answer is very hand-wavey | 22:09 |
| * breton tryed out of curiosity | 22:09 | |
| bknudson | ldapadd vs openstack user add | 22:09 |
| breton | *tried | 22:09 |
| gyee | bknudson, sure ldap -f add_users.ldif | 22:09 |
| morganfainberg | bknudson: 100% agree with you | 22:09 |
| morganfainberg | or freeIPA web interface (it's remarkably good) | 22:09 |
| kfox1111 | I guess, what would be truely fantastic would be a mechanism like: | 22:09 |
| kfox1111 | nova has a ca... each hypervisor gets a cert. | 22:10 |
| kfox1111 | each hypervisor runs a web server attached to the instances it maintains. | 22:11 |
| kfox1111 | any request to the web server signs a message to keystone saying "i need a token from instance X" | 22:11 |
| *** pballand has joined #openstack-keystone | 22:11 | |
| kfox1111 | keystone verifies the CA then goes back to nova api and asks is instance x running on that hypervisor. if it is, it gives back a token for intance X. | 22:12 |
| morganfainberg | kfox1111: so stop here | 22:12 |
| morganfainberg | before you get too much further | 22:12 |
| morganfainberg | i have a very important question | 22:12 |
| *** stevemar has quit IRC | 22:12 | |
| morganfainberg | what authorization does the token for instance X need? | 22:12 |
| kfox1111 | nothing for most of the use cases. just enough of keystone integration for a user to add acls to barbican or zaqar to enable specific resources. | 22:13 |
| morganfainberg | i really don't know what that means. | 22:13 |
| morganfainberg | do you need roles? | 22:13 |
| kfox1111 | no roles. | 22:13 |
| morganfainberg | or is an unscoped token sufficient? | 22:13 |
| morganfainberg | does it need to be part of a project? | 22:13 |
| kfox1111 | unscoped token is sufficient so long as they can get the service catalog entries to point to barbican/zaqar. | 22:14 |
| morganfainberg | nope | 22:14 |
| kfox1111 | not part of a project. | 22:14 |
| morganfainberg | needs a project then | 22:14 |
| morganfainberg | unscoped doesn't have a catalog today | 22:14 |
| bknudson | we could just allow service users to generate unscoped tokens | 22:14 |
| bknudson | with whatever user you want | 22:14 |
| kfox1111 | can be a dummy project. doesn't really need it but if that make it work, whatever. | 22:14 |
| morganfainberg | sure, i'm trying to get the full scope of what you want | 22:14 |
| kfox1111 | k. | 22:15 |
| morganfainberg | also keystone going back to nova - no. | 22:15 |
| kfox1111 | no roles, no groups, one domain (that managed by nova) and some way of getting a keystone user id to hang acl's off of. and some day trusts. | 22:15 |
| * morganfainberg thinks for a minute | 22:15 | |
| bknudson | trusts makes it more difficult | 22:15 |
| morganfainberg | yeah | 22:15 |
| morganfainberg | kfox1111: ok so lets stop with what the mechanism to get the token is | 22:16 |
| kfox1111 | k. | 22:16 |
| morganfainberg | what specifically would be done with the token | 22:16 |
| morganfainberg | real-world usecase | 22:16 |
| morganfainberg | not theoretical | 22:16 |
| kfox1111 | k. | 22:16 |
| morganfainberg | if possible | 22:16 |
| *** gordc is now known as gordc_ | 22:16 | |
| kfox1111 | vm is provisioning itself. It gets a token somehow, and a barbican endpoint somehow. | 22:16 |
| kfox1111 | it calls the get secret rest api of the barbican endpoint, fetches a secret, and provisions it self. | 22:17 |
| kfox1111 | secret could be mysql password, ssl wildcard certificate, etc. | 22:17 |
| morganfainberg | sure | 22:17 |
| kfox1111 | another use case is, the vm starts up, gets a token somehow, | 22:17 |
| kfox1111 | starts up the heat guest agent, | 22:18 |
| kfox1111 | connects to zaqar queue heatinstance-xxxx-xxxx-xx and starts listening for request to come in to run software-deployments. | 22:18 |
| kfox1111 | almost every use case I can think of that I need to do for the next year falls into one of those two catagories. | 22:19 |
| kfox1111 | there can be others, but those can wait. | 22:19 |
| *** htruta has quit IRC | 22:19 | |
| kfox1111 | sound ok? | 22:20 |
| morganfainberg | i mean i kindof hear a CMS use-case mixed with a heat-stack use-case | 22:20 |
| morganfainberg | am i hearing that wrong? | 22:20 |
| kfox1111 | cms use case for the former. using heat to stand up apps. | 22:21 |
| morganfainberg | yah | 22:21 |
| kfox1111 | the latter is the guest agent use case. controller -> vm management control. | 22:21 |
| morganfainberg | sure. but that tends to flow from CMS -> config -> things | 22:21 |
| kfox1111 | heat is usually actually used to provision those too these days. | 22:21 |
| morganfainberg | on top of IaaS things | 22:21 |
| kfox1111 | yeah. | 22:21 |
| kfox1111 | kind of the saas stuff. trove, sahara, etc. | 22:22 |
| kfox1111 | lbaas. | 22:22 |
| *** dramakri has joined #openstack-keystone | 22:22 | |
| *** lhcheng has joined #openstack-keystone | 22:22 | |
| *** ChanServ sets mode: +v lhcheng | 22:22 | |
| kfox1111 | anything that has a controller provisioning vm's on the bahalf of the user really. | 22:24 |
| kfox1111 | it can be used for other things, like I mentioned before, those two use cases are the things I really care about currently. | 22:24 |
| morganfainberg | hmmmmm | 22:24 |
| kfox1111 | because each guest agent is implementing things differently and most are doing it suboptomally since its hard to do right. | 22:25 |
| kfox1111 | sahara does ssh from controller to guest agent. (hurts with SDN) | 22:25 |
| kfox1111 | trove is doing it with rabbit (not multitenant aware. security risk) | 22:25 |
| kfox1111 | heat's doing it with their own users. (bleh :) | 22:26 |
| kfox1111 | lbaas implemented their own thing (was lementing not having instance users in the mailing list the other day) | 22:26 |
| kfox1111 | no clue whta magnum's doing. afraid to find out. | 22:26 |
| kfox1111 | they all should just be using zaqar with instance users. | 22:26 |
| morganfainberg | magnum is abstracting away the containers as $container_things, after the vm is spun up | 22:27 |
| kfox1111 | zaqar works nicely with SDN since its going the right way through the nat. | 22:27 |
| morganfainberg | so it knows it's bay and can interact with it directly | 22:27 |
| *** lhcheng_ has joined #openstack-keystone | 22:27 | |
| morganfainberg | magnum is not a huge issue on this front | 22:27 |
| openstackgerrit | Brant Knudson proposed openstack/keystone: Remove setUp for RevokeTests https://review.openstack.org/179259 | 22:27 |
| morganfainberg | docker/docker_swarm is not intended to be bound to keystone | 22:27 |
| morganfainberg | magnum's interfaces might be. | 22:28 |
| kfox1111 | the controller never has to poke commands at the vm? | 22:28 |
| morganfainberg | indirectly | 22:28 |
| kfox1111 | like "drain the vm of containers, its going down"? | 22:28 |
| morganfainberg | docker and kubernetes can't be under keystone authZ today | 22:28 |
| morganfainberg | so it does it in the docker/kubernetes methods | 22:28 |
| kfox1111 | does it use a guest agent at all on the vm's? | 22:28 |
| morganfainberg | fairly certian it's like a nova-cpu agent | 22:29 |
| kfox1111 | nova-compute uses rabbit as a back channel for control. | 22:29 |
| morganfainberg | i'd have to go look | 22:29 |
| morganfainberg | but my guess is it uses it's own transport that is not tied to keystone | 22:29 |
| morganfainberg | whatever that transport is | 22:29 |
| kfox1111 | yeah. but my guess is its iether, rabbit, ssh, or http polling. :/ | 22:30 |
| morganfainberg | at least that was what it was doing before | 22:30 |
| *** lhcheng has quit IRC | 22:30 | |
| morganfainberg | likely ssh or rabbit | 22:30 |
| kfox1111 | each has their problems. :/ | 22:30 |
| morganfainberg | don't think it's http polling | 22:30 |
| morganfainberg | at least i'd hope not | 22:30 |
| morganfainberg | or zmq *shrug* | 22:30 |
| kfox1111 | odly, http polling plays best with sdn of the 3 while still providing security. :/ | 22:30 |
| *** diazjf has left #openstack-keystone | 22:31 | |
| kfox1111 | thats why folks are pushing zaqar hard as the guest agent commmunication solution. guest agents really need a multitenant safe, client initiated scalable message queue. :/ | 22:31 |
| kfox1111 | it only works with keyston credentials though, hence the instance user need. | 22:31 |
| morganfainberg | who is pushing that? zaqar folks or lots of folks? | 22:32 |
| * morganfainberg is out of the loop | 22:32 | |
| kfox1111 | zaqar and some of us ops that keep tripping over everyproject badly reimplementing the wheel. | 22:32 |
| morganfainberg | this isn't meant to be a snarky response, just a legitimate "who is pushing zaqar as the solution" | 22:32 |
| kfox1111 | sahara kills us today. every instance has to have a floting ip so sahara can talk to it. :/ | 22:32 |
| kfox1111 | every time I try a new openstack service, its repeating history. :/ | 22:33 |
| morganfainberg | so.. i think there is somewhat of a disconnect here | 22:33 |
| morganfainberg | it almost feels like we're trying to solve PaaS issues with IaaS tools | 22:33 |
| morganfainberg | it's blended somewhere inbetween | 22:34 |
| kfox1111 | SaaS, but go on. | 22:34 |
| morganfainberg | which makes it hard | 22:34 |
| morganfainberg | well magnum == PaaS | 22:34 |
| morganfainberg | sahara is somewher between SaaS and PaaS | 22:34 |
| morganfainberg | closer to SaaS | 22:34 |
| kfox1111 | actually, magnum's in a very weird place. its SaaS, in that its deploying Software as a Service (magnum) | 22:34 |
| kfox1111 | which itself is a PaaS. :) | 22:34 |
| morganfainberg | kfox1111: lets just call that PaaS then :P | 22:35 |
| * kfox1111 chuckes. | 22:35 | |
| morganfainberg | net result - it's a platform as a service | 22:35 |
| kfox1111 | k. | 22:35 |
| kfox1111 | bleh. ment its deploying Kubernetes. | 22:35 |
| morganfainberg | because you could say IaaS is SaaS just deploying IaaS things (net result) | 22:35 |
| kfox1111 | Kubernetes is platform as a service, magnum is deploying the software Kubernetes. | 22:36 |
| morganfainberg | linux on kvm is *really* just software | 22:36 |
| morganfainberg | :P | 22:36 |
| kfox1111 | anyway... | 22:36 |
| morganfainberg | it's semantics | 22:36 |
| kfox1111 | true. | 22:36 |
| kfox1111 | fair enough. everything's about definitions. I'll use yours for now. | 22:37 |
| morganfainberg | what i'm trying to dig into is how much of this is really tied to the user that is provisioning a vm | 22:37 |
| kfox1111 | in my little world, long term most users wont be provisioning vm's they will be using prebuilt templates to do that. | 22:37 |
| kfox1111 | Ie, go to the app catalog, choose my sweet cloud app, hit run. | 22:38 |
| morganfainberg | and how linking it to the low level of what is used to make nova, glanc,e swift, cinder helps. [not saying this is wrong by any means][ | 22:38 |
| kfox1111 | its some autoscaled, multitiered web app. | 22:38 |
| kfox1111 | yeah. to me, those components really are building blocks to the end resut a user cares about. they are gears the user doesn't care about. | 22:39 |
| morganfainberg | kfox1111: right | 22:39 |
| kfox1111 | the problem is getting them so they work together smoothly enough for a cloud developer can write a generic enough template that they can make available to users. | 22:39 |
| kfox1111 | IE, add their app to the app store. | 22:39 |
| kfox1111 | right now, most "apps" are very specific to one cloud. the one it was developed for. | 22:40 |
| kfox1111 | so they can't contribute it back. :/ | 22:40 |
| kfox1111 | Comparable to Linux back its infancy. | 22:40 |
| *** tqtran has joined #openstack-keystone | 22:40 | |
| kfox1111 | only a developer could write software for it, and it only ran on their own box. | 22:40 |
| kfox1111 | I want to get to a place where OpenStack is like Android. You pull up the store, search through a wealth of apps, and hit launch, then use it. | 22:41 |
| kfox1111 | We're close, but some there are a few major pain points still in openstack for that. :/ | 22:43 |
| kfox1111 | secrets to vm's is one of the biggest. | 22:43 |
| kfox1111 | (User managable dns subdomains + ssl certs is another) | 22:44 |
| kfox1111 | anyway. way off in left field again. | 22:45 |
| openstackgerrit | Brant Knudson proposed openstack/keystone: Document policy target for operation https://review.openstack.org/168521 | 22:45 |
| morganfainberg | so i think i'm back to ephemeral PKI is really the best solution here | 22:45 |
| kfox1111 | k. | 22:46 |
| morganfainberg | SSL certs with CAs, while heavy-weight-ish | 22:46 |
| morganfainberg | really are the easiest way to convey "this VM is allowed to do X" | 22:46 |
| morganfainberg | and we can encode the vm's uuid (or whatever) as part of the DN | 22:46 |
| kfox1111 | yeah. k. | 22:46 |
| morganfainberg | ephemeral PKi work is happening, but i don't know the state of it | 22:47 |
| morganfainberg | it might be too new | 22:47 |
| *** Akshay00 has joined #openstack-keystone | 22:47 | |
| kfox1111 | well, the "it can do x" is really "use project dummy"? | 22:47 |
| morganfainberg | but still a CA + certs are a good way to go. | 22:47 |
| gyee | we are deploying anchor | 22:47 |
| kfox1111 | since thats required today to get a scoped token? | 22:47 |
| morganfainberg | kfox1111: we can map it to a special project if needed. | 22:47 |
| *** packet has joined #openstack-keystone | 22:47 | |
| kfox1111 | k. | 22:47 |
| morganfainberg | kfox1111: or all to a group with a special role+project | 22:47 |
| kfox1111 | yeah. | 22:48 |
| gyee | https://github.com/stackforge/anchor | 22:48 |
| morganfainberg | if it *has* to be keystone tokens (though this sounds more and more like it could also not be keystone tokens) | 22:48 |
| morganfainberg | keystone can be configured to handle this. | 22:48 |
| morganfainberg | longer term we could make it the default configuration if there is really a demand | 22:48 |
| morganfainberg | gyee: ++ that was what i was thinking, but i couldn't remember where it was | 22:49 |
| kfox1111 | ok. so. nova has a db entry for cert. it gets a signed one from barbican. stores it. when a vm requests a token, the md servuer uses the cert to request a token and hand it back? | 22:49 |
| *** lhcheng_ has quit IRC | 22:49 | |
| morganfainberg | kfox1111: it could also use certmonger or any other mechanism for getting the cert | 22:49 |
| morganfainberg | and if barbican is used, nova can own the cert and request it on demand for environments that don't want keys (etc) in the nova-db | 22:50 |
| *** Lactem has joined #openstack-keystone | 22:50 | |
| kfox1111 | morganfainberg: maybe. I identified another set of constraints. let me run those by you... | 22:50 |
| Lactem | Has Dolph been on? | 22:50 |
| morganfainberg | Lactem: i've seen him on twitter | 22:50 |
| morganfainberg | but not much else today | 22:50 |
| Lactem | Okay. | 22:50 |
| Lactem | I needed his opinion on something he was guiding me with. | 22:50 |
| kfox1111 | vms can be stopped, suspended or shelved. so they may not be able to self refresh a cert before they expire. but are still desired to work after rehydrating or restarting. | 22:51 |
| morganfainberg | Lactem: it's also ~5pm his time | 22:51 |
| Lactem | He's been on around this time before, though. | 22:51 |
| morganfainberg | Lactem: it is also a friday ;) | 22:51 |
| Lactem | But yeah it's getting late. | 22:51 |
| kfox1111 | the other thing is vm's can be snapshotted. so it would be very good if instance user certs aren't snapshotted along with the vm. | 22:51 |
| Lactem | Very true. | 22:51 |
| *** Akshay00 has quit IRC | 22:51 | |
| morganfainberg | kfox1111: i think SpamapS' recommendation of a microservice is sounding more and more correct rather than wedging this into metadata | 22:52 |
| morganfainberg | and it would solve the configdrive issue | 22:52 |
| kfox1111 | except his suggestion requires a one time fetch. | 22:52 |
| Lactem | morganfainberg: Would you mind taking a quick look if I can't reach Dolph today? I just want to know if this code is path-worthy or not. | 22:52 |
| kfox1111 | I'm not sure how that would work with the suspended case. | 22:52 |
| kfox1111 | same with snapshotting. | 22:52 |
| morganfainberg | Lactem: sure. | 22:52 |
| Lactem | I wrote something. https://gist.github.com/Lactem/5a43296b8975da24db51 Would that suffice? (You said to make a test that does what https://paste.ee/p/pdyrS does.) | 22:52 |
| Lactem | ^ That's the message I sent to him. | 22:52 |
| morganfainberg | whoa. uhm. hold on a sec | 22:53 |
| kfox1111 | I'm ok with the other service, so long as it doesnt' complicate things. but I think you really need a cert or something in order to talk to that microservice, and then whats the point? | 22:53 |
| Lactem | Okay. | 22:53 |
| Lactem | It's about bug https://bugs.launchpad.net/keystone/+bug/1098564 by the way. | 22:53 |
| openstack | Launchpad bug 1098564 in Keystone "Cannot delete a service or endpoint" [Low,Incomplete] - Assigned to Theodore Ilie (theoilie-ti) | 22:53 |
| morganfainberg | oh not cool irccloud... auto-embeding gists and such | 22:53 |
| morganfainberg | ugh | 22:53 |
| morganfainberg | Lactem: my settings made it hard to follow the convo w/ kfox1111 | 22:54 |
| morganfainberg | Lactem: let me finish this up and then i'll loolk | 22:54 |
| morganfainberg | kfox1111: you know i was wrong | 22:55 |
| morganfainberg | kfox1111: this sounds like you want puppet/chef-as-a-service built into the cloud | 22:55 |
| Lactem | Alright. No rush. | 22:55 |
| kfox1111 | morganfainberg: I've wanted that. :) | 22:56 |
| morganfainberg | kfox1111: i think we're crossing things up in ways that are problematic | 22:56 |
| kfox1111 | none of configuration management systems handle ephemeral vms well though. | 22:56 |
| kfox1111 | ansible's the closest I've seen to dealing with it right. :/ | 22:56 |
| kfox1111 | most config management systems assume they are driving the deployment. they don't handle heat autoscaling well. :/ | 22:57 |
| kfox1111 | possilby. whatcha thinking? | 22:57 |
| morganfainberg | i'm thinking the focus becomes make CMS handle ephemeral vms | 22:57 |
| morganfainberg | and drive it with say ansible | 22:57 |
| morganfainberg | play to the strengths of the technology | 22:58 |
| kfox1111 | k. lets start at the root.... heat autoscaling. | 22:58 |
| kfox1111 | you have a template, created on demand by heat when load gets high enough, that needs to be instantiated. | 22:58 |
| morganfainberg | this whole spec is starting to feel like we need to invent something new and wedge a square peg into a round hole | 22:58 |
| morganfainberg | kfox1111: and playbooks do a reasonable job of things like this. | 22:58 |
| morganfainberg | reasonable - not fantastic | 22:59 |
| morganfainberg | but we can improve reasonable to something better | 22:59 |
| kfox1111 | sure. | 22:59 |
| kfox1111 | heat suports using playbooks within the instance. | 22:59 |
| kfox1111 | but the instance needs to get secrets from somewhere. | 22:59 |
| kfox1111 | heat autoscale launches a new heat stack for the instance. | 23:00 |
| morganfainberg | but these secrets aren't OpenStack secrets | 23:00 |
| kfox1111 | has a resource for a nova vm. | 23:00 |
| morganfainberg | these are app secrets | 23:00 |
| kfox1111 | correct. | 23:00 |
| morganfainberg | so what is the store for a user secret? | 23:00 |
| kfox1111 | but its automated within openstack, so it has to be pulled in somehow based on only knowlege openstack has. | 23:00 |
| morganfainberg | barbican? | 23:00 |
| kfox1111 | barbican. | 23:01 |
| morganfainberg | and heat has authority to act on user's behalf | 23:01 |
| morganfainberg | and contact barbican | 23:01 |
| morganfainberg | this could all be driven from the heat side. | 23:01 |
| kfox1111 | yes, but there's no way to get a secret into the vm securely? | 23:01 |
| kfox1111 | heat has a trust for the user, so it can use that... | 23:01 |
| morganfainberg | how does ansible get secrets to anything securely atm? | 23:02 |
| *** pballand has quit IRC | 23:02 | |
| morganfainberg | how does chef? or puppet do it? | 23:02 |
| kfox1111 | but you don't want to push the user's trust into the vm? | 23:02 |
| kfox1111 | the management node, or whatever you want to call it. the thing running ansible, | 23:02 |
| kfox1111 | scp's files from that node to the other nodes it manages. | 23:02 |
| morganfainberg | so could heat push this data down? | 23:02 |
| morganfainberg | you've already trusted heat with a lot of powers | 23:03 |
| kfox1111 | its the controller -> vm contact thing again. | 23:03 |
| kfox1111 | hmmm.... | 23:03 |
| Lactem | Interesting conversation. | 23:03 |
| morganfainberg | heat is already driving everything here | 23:03 |
| *** pballand has joined #openstack-keystone | 23:03 | |
| kfox1111 | so extend the heat api, to make barbican secrets available via heat api, then pass a keystone heat user into the vm so it can call back and get it? | 23:04 |
| morganfainberg | not an unreasonable approach if heat already does things | 23:04 |
| morganfainberg | that way as heats model evolves and gets better, you already benefit | 23:04 |
| morganfainberg | you're not re-inventing everything | 23:04 |
| kfox1111 | it doesn't solve the guest agent issue, but its a way to solve the secrets from heat use case. | 23:05 |
| morganfainberg | the guest agent issue is a go back to the projects and make them stop doing things badly | 23:05 |
| morganfainberg | if every one of them is different, i'm going to go out on a limb and say it's not going to get better with $magic keystone users | 23:06 |
| morganfainberg | it's just going to be a custom thing for every one of them | 23:06 |
| kfox1111 | its only different so far because each of them, when they ask the others how did you solve it, has said "by our selves" :/ | 23:06 |
| morganfainberg | kfox1111: we can wor on socialization issues like that | 23:07 |
| kfox1111 | octavia said to me "if we would have known instance users were coming" we would hae waited and just used that. | 23:07 |
| kfox1111 | or contributed. | 23:07 |
| morganfainberg | we don't need to reinvent a mix between CMS and PaaS | 23:07 |
| morganfainberg | we already have a lot of those | 23:07 |
| kfox1111 | instances are going to want to talk to openstack services one day. | 23:07 |
| kfox1111 | keystone's the gatekeeper to that. | 23:08 |
| kfox1111 | jenkins and nagios are some other examples. | 23:08 |
| morganfainberg | and i'm likely going to tell you the same thing then. they don't get a special ability to make users | 23:08 |
| bknudson | the instance might run and instance of keystone | 23:08 |
| morganfainberg | they can either (case of jenkins?) get a service user with the right scope of access (or use the owner's account/serice account) | 23:08 |
| kfox1111 | true. they just need a way to get a keystone token. which is what the spec's totally about. | 23:08 |
| kfox1111 | I don't really care how it happens, just that they are available when needed. :) | 23:09 |
| *** stevemar has joined #openstack-keystone | 23:09 | |
| kfox1111 | which is why the api reflected "instance asks metadata server for token". the rest is all totally implementation detail. | 23:09 |
| morganfainberg | kfox1111: i really don't think you're going to have a huge demand for VMs to talk to OpenStack services directly - most of them are one-off/ansible-push type deals and then the subordinate services don't need to talk to nova, glance, etc | 23:10 |
| kfox1111 | your right though, there's nothing really stopping nova from accepting the user's token as part of the instance create, and then using that to hand out tokens to the vm. | 23:10 |
| kfox1111 | morganfainberg: amazon implements both chef as aservice, and instance users. | 23:11 |
| morganfainberg | kfox1111: i'm trying to not wedge this into keystone-must-be-in-the-middle, or exclude that. | 23:11 |
| kfox1111 | they felt it important enough to implement due to customer demand. | 23:11 |
| morganfainberg | kfox1111: i can get you instance users. | 23:11 |
| morganfainberg | kfox1111: i can. but it's going to be x509 certs and some enhancements around that | 23:11 |
| openstackgerrit | Brant Knudson proposed openstack/keystone: Updated from global requirements https://review.openstack.org/190405 | 23:11 |
| kfox1111 | thats fine. I can go with that. no problem. | 23:11 |
| bknudson | maybe we can get this right. | 23:11 |
| bknudson | might be better off just waiting until a reqs update. | 23:12 |
| morganfainberg | but it sounds to me like the first step is going to be push the data down | 23:12 |
| morganfainberg | and that hits the immidiate need | 23:12 |
| morganfainberg | heat drives, uses ansible or whatever else | 23:12 |
| morganfainberg | and we can get the x509 user stuff in place in liberty | 23:12 |
| *** stevemar has quit IRC | 23:13 | |
| morganfainberg | at that point we can evaluate the demand for instance users | 23:13 |
| kfox1111 | so lets double check thjat workflow. because we keep hitting "secret to vm to download secrets". | 23:13 |
| morganfainberg | and what needs to be done to make it better. we also will have improved some of how keystone is working (paid down tech debt) that makes some of this stuff easier | 23:13 |
| bknudson | the x509 is essentially attribute mapping | 23:13 |
| morganfainberg | bknudson: correct | 23:13 |
| bknudson | you could do that in several ways | 23:13 |
| bknudson | e.g., saml | 23:13 |
| kfox1111 | how does heat get a secret to the vm in order for it to talk back to heat relyably to get the secrets from it? | 23:13 |
| bknudson | (so why pick x.509 and not something else?) | 23:14 |
| morganfainberg | bknudson: less overhead issuing a cert than building a SAML2 IdP | 23:14 |
| morganfainberg | bknudson: not needing shibboleth/mellon to deconstruct things | 23:15 |
| morganfainberg | bknudson: and we're already doing the work for tokenless service users | 23:15 |
| bknudson | come up with your own format. | 23:15 |
| bknudson | sign a JSON document | 23:15 |
| *** zzzeek has quit IRC | 23:15 | |
| kfox1111 | bknudson: I have a prototype of that working in barbican. :) | 23:15 |
| kfox1111 | even reuses keystone's pki/z code. :) | 23:16 |
| morganfainberg | bknudson: if you want to do that and go to bat with the corporate security folks i'm all for pointing them all at you ;) | 23:16 |
| morganfainberg | when they come and ask me about it | 23:16 |
| bknudson | corporate * folks are all about saying no | 23:17 |
| morganfainberg | bknudson: i am also trying to not NIH a new standard way of security communicating these attributes | 23:17 |
| morganfainberg | bknudson: same reason i pushed so hard to make k2k use SAML | 23:17 |
| morganfainberg | it's politically an easier sell to corp * types | 23:17 |
| morganfainberg | and eases adoption | 23:17 |
| bknudson | should have pushed k2k to use x.509 | 23:17 |
| bknudson | ... wondering why pushing x.509 here and saml elsewhere | 23:17 |
| *** henrynash has quit IRC | 23:17 | |
| morganfainberg | because its a lot more work to setup k2k type federation | 23:18 |
| morganfainberg | and the workflows are more complex | 23:18 |
| *** ekarlso has quit IRC | 23:18 | |
| *** ekarlso has joined #openstack-keystone | 23:18 | |
| kfox1111 | so in the heat managing secrets for vm's, how does heat get a secret to the vm so that it can download secrets? Especially in light that they want to support zaqar with their guest agent some day soon? | 23:19 |
| morganfainberg | saml was used in the k2k because it really is an SP/IdP relationship | 23:19 |
| morganfainberg | but sure we could do saml between nova and ekystone for the vm users | 23:19 |
| Lactem | morganfainberg: I'm going to leave soon. Would you mind commenting on the gist? I won't see an IRC message. | 23:19 |
| morganfainberg | Lactem: sure i'll comment @ the gist | 23:19 |
| morganfainberg | Lactem: sorry. | 23:19 |
| Lactem | Thank you. | 23:20 |
| Lactem | Take your time. | 23:20 |
| morganfainberg | Lactem: for the delay | 23:20 |
| Lactem | Happy Friday! | 23:20 |
| Lactem | Wait are you doing it right now? | 23:20 |
| morganfainberg | Lactem: no still in this convo here | 23:20 |
| Lactem | Okay I'll leave then. See you later. | 23:20 |
| *** Lactem has quit IRC | 23:20 | |
| *** gyee has quit IRC | 23:21 | |
| morganfainberg | kfox1111: so lets take a big issue off the table | 23:21 |
| kfox1111 | having an instance user solves both cases for heat. it can use the same instance user to run the guest angent, and pull the secrets from barbican. | 23:21 |
| kfox1111 | ok. | 23:21 |
| morganfainberg | kfox1111: i lost my train of thought...crap | 23:21 |
| morganfainberg | noidea what iw as going to type next | 23:21 |
| morganfainberg | :P | 23:21 |
| kfox1111 | sry. :/ | 23:21 |
| morganfainberg | not your fault | 23:22 |
| morganfainberg | its my brain | 23:22 |
| kfox1111 | we were disussing making heat deal with secrets directly. something about that? | 23:23 |
| morganfainberg | kfox1111: yeah no idea it's lost | 23:23 |
| kfox1111 | k. | 23:23 |
| morganfainberg | kfox1111: so explain/link me the amazon flow | 23:23 |
| morganfainberg | for the intance user | 23:23 |
| kfox1111 | ok. so they have IAM. | 23:24 |
| morganfainberg | i know iam, assume i've worked with aws before ;) | 23:24 |
| morganfainberg | just not with the instance users | 23:24 |
| *** pballand has quit IRC | 23:24 | |
| morganfainberg | (i did in a past job) | 23:24 |
| kfox1111 | a user can create a document saying something like I want to delegate acces to these specific roles/resources. | 23:24 |
| kfox1111 | they register that document with keystone basically. | 23:25 |
| kfox1111 | then: | 23:25 |
| kfox1111 | aws ec2 run-instances --iam-instance-profile role_name | 23:25 |
| *** lhcheng has joined #openstack-keystone | 23:25 | |
| *** ChanServ sets mode: +v lhcheng | 23:25 | |
| kfox1111 | where profile_name there is that document handle. | 23:25 |
| kfox1111 | the vm gets launched. | 23:26 |
| kfox1111 | the vm can then: | 23:26 |
| kfox1111 | curl http://169.254.169.254/latest/meta-data/iam/security-credentials/role_name | 23:26 |
| morganfainberg | what prevents $instance from getting $other_instance_cred | 23:26 |
| kfox1111 | they get back basically a scoped token for the role. | 23:26 |
| morganfainberg | is this magic inside aws somewhere? | 23:26 |
| kfox1111 | the metadata server handles authentication between it and the vm. | 23:26 |
| *** _cjones_ has quit IRC | 23:27 | |
| kfox1111 | thats part of the magic of the md server. | 23:27 |
| kfox1111 | same with nova's. | 23:27 |
| *** dramakri has quit IRC | 23:27 | |
| morganfainberg | ok so this is still boiling down to "make md server acceptible and used everywherE" | 23:27 |
| morganfainberg | if that happens you have an easier case to work between nova and keystone | 23:27 |
| morganfainberg | but since MD is not everywhere and/or is broken in a lot of places | 23:27 |
| * kfox1111 nods. | 23:27 | |
| *** dramakri has joined #openstack-keystone | 23:27 | |
| kfox1111 | if I can get 80% of the deployments working I'd be happy with that. :/ | 23:28 |
| kfox1111 | there's only so much I can do. | 23:28 |
| morganfainberg | bug hogepodge and figure out how to make it a defcore thing :P | 23:28 |
| kfox1111 | the problem I have with using config drive is its static. | 23:28 |
| * morganfainberg didn't say that too loudly i hope | 23:28 | |
| kfox1111 | no way to refresh a certificate or something there. | 23:28 |
| morganfainberg | kfox1111: so assume you need the degenerate case of static | 23:28 |
| kfox1111 | thats on my list. ;) | 23:28 |
| morganfainberg | and you can't refresh | 23:28 |
| morganfainberg | or you need something + config drive | 23:29 |
| kfox1111 | I cant think of a way to make the config drive case work with a vm that's mothballed for a month or a year. | 23:29 |
| morganfainberg | or you need md everywhere | 23:29 |
| kfox1111 | eventually a cert passed through it will expire. | 23:29 |
| morganfainberg | if someone waits that long | 23:29 |
| kfox1111 | and it wont have a valid cert to request a new one. :/ | 23:29 |
| morganfainberg | they can spin a new vm | 23:29 |
| kfox1111 | as much as I like cattle, | 23:29 |
| kfox1111 | pets are way to common. :/ | 23:30 |
| morganfainberg | you can't solve everyone's cases | 23:30 |
| morganfainberg | solve for 80% of the real world | 23:30 |
| morganfainberg | who is going to mothball for a year+ | 23:30 |
| kfox1111 | yeah. so which one's more common, pets, or people who don't trust the md server? | 23:30 |
| morganfainberg | people who don't trust the md server | 23:30 |
| morganfainberg | in the context of what you're working on | 23:30 |
| kfox1111 | are there metrics fro that? truely curious. | 23:30 |
| morganfainberg | you're working on a premise of cattle | 23:30 |
| morganfainberg | therefore pets are the outlier | 23:31 |
| kfox1111 | true. but I think instances that dont trust the md server are more rare. | 23:31 |
| morganfainberg | the bigger issue is you're fighting some places use CD and someplaces use MD but it timesout | 23:31 |
| morganfainberg | and some places have hacked up MD to be hack-y | 23:31 |
| kfox1111 | yeah. | 23:31 |
| *** pballand has joined #openstack-keystone | 23:31 | |
| morganfainberg | so don't solve the pet case here | 23:31 |
| kfox1111 | thats why I almsot think its better to come up with a whole new md server just for this. :/ | 23:32 |
| morganfainberg | if you re-write MD to be smarter/better/faster/stronger/daft punk/something else | 23:32 |
| morganfainberg | and focus on making it so that config drive *isnt* a better option | 23:32 |
| morganfainberg | this becomes easier. | 23:33 |
| morganfainberg | if you wedge this functionality into the MD server today - you get something that is equally half baked | 23:33 |
| kfox1111 | but, | 23:33 |
| morganfainberg | i think you need to do both things and the case is instance users | 23:33 |
| morganfainberg | i'm telling you, from experience i wouldn;t run metadata server in my cloud | 23:33 |
| morganfainberg | period | 23:33 |
| kfox1111 | if their users need the catalog and it isnt supported in their cloud, ops myight switch / invest in making md better. | 23:33 |
| morganfainberg | if i was deployin openstack, i'd deploy config drive | 23:34 |
| morganfainberg | today | 23:34 |
| kfox1111 | truely curous, why? | 23:34 |
| morganfainberg | metadata's implementation is pretty bad | 23:34 |
| morganfainberg | in a scaled environment (minor scale) it timesout | 23:34 |
| kfox1111 | I have seen that in the past, but not in the past year. | 23:35 |
| kfox1111 | though I don't hit it proably as hard as you. | 23:35 |
| morganfainberg | past job, hit it pretty darn hard | 23:35 |
| morganfainberg | ~40-100 hypervisors | 23:35 |
| morganfainberg | not excessive scale | 23:35 |
| morganfainberg | config drive also has less moving parts | 23:35 |
| kfox1111 | I've got about that too. | 23:35 |
| kfox1111 | true. the neutron metadata proxy's a kick in the pants. :) | 23:36 |
| kfox1111 | but the one advantage is its dynamic. config drive's static. :/ | 23:36 |
| morganfainberg | i'd rather have metadata from a user/consumer perspective | 23:36 |
| morganfainberg | i take static and bombproof | 23:36 |
| morganfainberg | to dynamic and crashy | 23:36 |
| morganfainberg | or dyanmic and not-as-stable | 23:36 |
| morganfainberg | less customer calls | 23:36 |
| morganfainberg | less overhead | 23:37 |
| kfox1111 | fair. but like most things http, should it just retry a few times? :) | 23:37 |
| kfox1111 | true. | 23:37 |
| kfox1111 | so... is there another option we're missing? | 23:37 |
| kfox1111 | config drive like but not? | 23:37 |
| morganfainberg | door #3 | 23:37 |
| morganfainberg | config drive does the stuff metadata does poorly | 23:37 |
| morganfainberg | metadata does the stuff config drive does poorly | 23:38 |
| morganfainberg | or door #4 | 23:38 |
| kfox1111 | an intance user cert config drive? | 23:38 |
| morganfainberg | microservice that is like metadata but only does instance-users | 23:38 |
| kfox1111 | when it needs refereshing, nova unplugs rebuilds, replugs? | 23:38 |
| morganfainberg | i can't answer the right approach | 23:38 |
| morganfainberg | but if you like the amazon model, get MD to be the required lot. | 23:39 |
| kfox1111 | nova compute generates the cert, registeres it with barbican. | 23:39 |
| kfox1111 | amazon just happend to implement it in roughly the same way I thought of solving it, independently. | 23:39 |
| morganfainberg | we can figure out the best way to source per-vm-users in keystone, but the easiest wayt to do that is x509 per instance | 23:40 |
| kfox1111 | That has some apeal to me to continue dowh that path since it works at scale for them. | 23:40 |
| morganfainberg | and quickest-to-success route is that | 23:40 |
| kfox1111 | yeah. I'll write the spec up with that. | 23:40 |
| morganfainberg | but it has some cons, and it doesn't work 100% like you'd expect because keystone wasn't designed with this in mind | 23:40 |
| kfox1111 | I'm basically redoing everything else. :/ | 23:40 |
| morganfainberg | anything that is more APIs - it's going to be further out than liberty (meaning N release is likely when you can consume it) | 23:41 |
| morganfainberg | on the keystone side that is | 23:41 |
| morganfainberg | or O release if it's really compelx | 23:41 |
| kfox1111 | yeah. | 23:41 |
| kfox1111 | understood. thats why I only care about coming up with a very simple api from the users perspective, | 23:42 |
| kfox1111 | and let us find a good solution long term to the rest. | 23:42 |
| bknudson | the big-O release is going to be all about scalability | 23:42 |
| morganfainberg | x509 or other IdP like secure mechanism that hooks into what we have today | 23:42 |
| bknudson | computer science joke | 23:42 |
| morganfainberg | is a good bet | 23:42 |
| kfox1111 | curl 169.254.169.254/openstack/latetst/token and the rest is deatails we can always change later. | 23:42 |
| kfox1111 | bknudson: :) | 23:42 |
| morganfainberg | bknudson: the O(n) release? | 23:43 |
| morganfainberg | bknudson: ;) | 23:43 |
| kfox1111 | initial implementation with ca should be easy and scale pretty well. | 23:43 |
| kfox1111 | passing the cert to the user does handle the config drive case, and your right, maybe the non pet case is good enough. | 23:44 |
| morganfainberg | kfox1111: don't try and solve the pet case right away | 23:44 |
| morganfainberg | kfox1111: your premise is starting from cattle | 23:44 |
| kfox1111 | but it ties our hands to ca x509 /keystone forever. | 23:44 |
| kfox1111 | because that's then part of the API. | 23:44 |
| kfox1111 | do you believe that will be good enough forever? | 23:45 |
| morganfainberg | kfox1111: maybe the API should be "authentication credentials and metadata about how to auth" | 23:45 |
| morganfainberg | in the spec | 23:45 |
| morganfainberg | so you can make it typed | 23:45 |
| morganfainberg | { 'instance_user_authn': { 'type': 'x509', ....} | 23:46 |
| *** ankita_wagh has quit IRC | 23:46 | |
| kfox1111 | hmm... yeah. | 23:46 |
| kfox1111 | but the keystone endpoint given would have to take that document no matter what was in it. | 23:46 |
| morganfainberg | give yourself enough flexibility, but be opinionated about the goals | 23:46 |
| kfox1111 | well... | 23:46 |
| morganfainberg | kfox1111: also let it specify the authentication endpoint ;) | 23:46 |
| kfox1111 | the keystone url handed back can be scoped to that type of auth... | 23:47 |
| kfox1111 | yeah. | 23:47 |
| morganfainberg | by opinionated i mean: we are providing you with the type of authentication, the url, and x,y,z things" | 23:47 |
| kfox1111 | ok. yeah. | 23:47 |
| morganfainberg | don't make it "you can turn 40,000 knobs to make it better/different" | 23:47 |
| kfox1111 | it does require the vm to be smarter, but probably ok... | 23:47 |
| morganfainberg | sure. | 23:48 |
| morganfainberg | you can present the same exact data via MD if you want | 23:48 |
| * kfox1111 grumbles about still maintaining centos 6 images.... | 23:48 | |
| morganfainberg | the difference is authn: "keystone token" | 23:48 |
| morganfainberg | is the type | 23:48 |
| morganfainberg | or something | 23:48 |
| morganfainberg | so you have options here. | 23:49 |
| kfox1111 | with the prevouls spec, we were just going to do a json document with url and such, and a binary blob for cert, or whatever. | 23:49 |
| morganfainberg | sure | 23:49 |
| kfox1111 | I think that would probably still wrok. | 23:49 |
| morganfainberg | just add a little more syntax to allow future iteration w/o API contract break | 23:49 |
| morganfainberg | don't make it too many knobs | 23:49 |
| morganfainberg | just future proof yourself | 23:49 |
| kfox1111 | yeah. put the type in the json doc, and then if the client doesn't support it, it just throws an error. | 23:49 |
| morganfainberg | yep | 23:50 |
| kfox1111 | if it does, it can select the right mechanism, and then fetch a token. | 23:50 |
| morganfainberg | correct | 23:50 |
| kfox1111 | It still feels odd to push the abstraction all the way out to the vm :/ | 23:50 |
| morganfainberg | maybe the metadata form is "ask metadata at url X for token" | 23:50 |
| kfox1111 | means you have to write that tricky logic for many operating systems. :/ | 23:50 |
| morganfainberg | so it's <metadata addr>/openstack/instance_user/token | 23:50 |
| morganfainberg | and it's type is "metadata_proxy" | 23:51 |
| morganfainberg | or something | 23:51 |
| kfox1111 | right. | 23:51 |
| *** tqtran has quit IRC | 23:51 | |
| kfox1111 | then that part should probably be made plugable in the nova metadata server... | 23:51 |
| kfox1111 | which they will love... :/ | 23:51 |
| morganfainberg | start with the basic fixed form | 23:52 |
| morganfainberg | really | 23:52 |
| morganfainberg | just let the API syntax not lock you in | 23:52 |
| kfox1111 | fair engouh. don't bother them with that. we can always add it later if need be. | 23:52 |
| kfox1111 | right. | 23:52 |
| openstackgerrit | Nathan Jewell proposed openstack/keystone: Saves output of run_tests.sh to .log file https://review.openstack.org/196285 | 23:52 |
| morganfainberg | it is much easier to add pluggability than to change an API | 23:53 |
| morganfainberg | contract | 23:53 |
| kfox1111 | true. | 23:53 |
| kfox1111 | in fact..... | 23:53 |
| morganfainberg | and you may never need to make it pluggable | 23:53 |
| kfox1111 | if some day they agree to make everything just be the metadata server, | 23:53 |
| kfox1111 | then the binary blob can just be a token, and the auth_type can be "direct" | 23:53 |
| morganfainberg | and as a transition you could even do a configdrive -> type metadata request | 23:54 |
| kfox1111 | yeah..... really liking that idea. | 23:54 |
| morganfainberg | now. | 23:54 |
| *** dramakri has quit IRC | 23:54 | |
| morganfainberg | clearly define the scope (this is x509 now) | 23:54 |
| morganfainberg | or some such | 23:54 |
| kfox1111 | right. | 23:55 |
| morganfainberg | don't leave the door open for it to be everybody's special auth mech | 23:55 |
| *** packet has quit IRC | 23:55 | |
| morganfainberg | you can always add more type definitions | 23:55 |
| morganfainberg | but each auth type should be specifically and clearly defined | 23:55 |
| morganfainberg | so maybe you have x509 and direct to start | 23:55 |
| kfox1111 | right. only one today. x509 cert. | 23:55 |
| morganfainberg | or whatever you want to call it | 23:55 |
| morganfainberg | like i said, be opinionated to the right degree | 23:56 |
| morganfainberg | :) | 23:56 |
| kfox1111 | I'd rather not complicate it further. too much potential for bike shedding. :/ | 23:56 |
| morganfainberg | less to bikeshed on that way | 23:56 |
| kfox1111 | yup. | 23:56 |
| morganfainberg | but you can clearly define future types | 23:57 |
| morganfainberg | and the initial step works for both configdrive and metadata | 23:57 |
| kfox1111 | yeah. | 23:57 |
| morganfainberg | *and* doesn't try to solve pet cases to start - to be honest, most pets will use traditional CMS | 23:57 |
| kfox1111 | true. | 23:58 |
| morganfainberg | they aren't going to autoscale a pet | 23:58 |
| kfox1111 | true too. | 23:58 |
| kfox1111 | though I do run in to some of our production systems where certs expire because fetch-crl or whatever doesn't run and refresh things. | 23:58 |
| *** raildo has quit IRC | 23:58 | |
| kfox1111 | so I am a bit sensitive to things expiring and giving me a bad day. :/ | 23:59 |
| kfox1111 | pets tend to have that problem. :/ | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
