Monday, 2019-01-28

*** dklyle has joined #openstack-keystone00:16
*** dklyle has quit IRC00:31
*** whoami-rajat has joined #openstack-keystone00:49
*** ileixe has joined #openstack-keystone00:54
*** jistr has quit IRC01:00
*** jistr has joined #openstack-keystone01:01
*** Dinesh_Bhor has joined #openstack-keystone01:36
*** markvoelker has joined #openstack-keystone01:39
*** markvoelker has quit IRC02:12
*** tkajinam_ has joined #openstack-keystone02:19
*** tkajinam has quit IRC02:21
*** ileixe has left #openstack-keystone02:54
*** whoami-rajat has quit IRC03:09
*** markvoelker has joined #openstack-keystone03:09
*** markvoelker has quit IRC03:42
*** Dinesh_Bhor has quit IRC04:15
*** Dinesh_Bhor has joined #openstack-keystone04:16
*** spsurya has joined #openstack-keystone04:28
*** lifeless has quit IRC04:31
*** markvoelker has joined #openstack-keystone04:39
*** lifeless has joined #openstack-keystone04:51
*** imacdonn has quit IRC04:59
*** imacdonn has joined #openstack-keystone04:59
*** tkajinam_ is now known as tkajinam05:03
*** markvoelker has quit IRC05:11
*** tkajinam has quit IRC05:31
*** shyamb has joined #openstack-keystone05:34
*** tkajinam has joined #openstack-keystone06:13
*** vishakha has joined #openstack-keystone06:18
*** imacdonn has quit IRC06:34
*** shyamb has quit IRC06:43
*** shyamb has joined #openstack-keystone06:50
*** markvoelker has joined #openstack-keystone07:09
*** shyamb has quit IRC07:21
*** shyamb has joined #openstack-keystone07:27
*** bnemec has quit IRC07:40
*** markvoelker has quit IRC07:42
*** awalende has joined #openstack-keystone08:00
*** pcaruana has joined #openstack-keystone08:01
*** shyamb has quit IRC08:14
*** tkajinam has quit IRC08:15
*** bnemec has joined #openstack-keystone08:35
*** markvoelker has joined #openstack-keystone08:39
*** shyamb has joined #openstack-keystone09:03
*** jaosorior has joined #openstack-keystone09:10
*** markvoelker has quit IRC09:12
*** xek has joined #openstack-keystone09:15
*** whoami-rajat has joined #openstack-keystone09:18
*** markvoelker has joined #openstack-keystone10:10
*** shyamb has quit IRC10:13
*** shyamb has joined #openstack-keystone10:19
*** shyamb has quit IRC10:32
*** markvoelker has quit IRC10:43
*** mvkr has joined #openstack-keystone11:11
*** szaher has quit IRC11:26
*** szaher has joined #openstack-keystone11:29
*** shyamb has joined #openstack-keystone11:31
*** Dinesh_Bhor has quit IRC11:32
*** markvoelker has joined #openstack-keystone11:40
*** bnemec has quit IRC11:40
*** bnemec has joined #openstack-keystone11:42
*** whoami-rajat has quit IRC11:45
*** ileixe has joined #openstack-keystone12:02
ileixeHi guys, I got a simple question about fernet token. I will be appreciated a lot for anyone who can answer me.12:03
ileixeThe problem is keystonemiddleware auth_token cache does not update token until the expiration time reached, even though user added new endpoints.12:04
ileixeIs it acting on normal? It seems to UUID token be invalidate itself when token endpoint is changed.12:05
*** ileixe has quit IRC12:08
*** ileixe has joined #openstack-keystone12:09
*** markvoelker has quit IRC12:13
*** pcaruana has quit IRC12:36
*** pcaruana has joined #openstack-keystone12:37
*** shyamb has quit IRC12:38
*** shyamb has joined #openstack-keystone12:38
*** whoami-rajat has joined #openstack-keystone13:26
*** shyamb has quit IRC13:29
*** shyamb has joined #openstack-keystone13:30
*** pcaruana has quit IRC13:32
*** shyamb has quit IRC13:47
*** pcaruana has joined #openstack-keystone13:50
*** ileixe has quit IRC13:53
*** jmlowe has quit IRC14:03
*** dave-mccowan has joined #openstack-keystone14:09
*** erus has joined #openstack-keystone14:17
*** lbragstad has joined #openstack-keystone14:20
*** ChanServ sets mode: +o lbragstad14:20
*** awalende has quit IRC14:35
*** awalende has joined #openstack-keystone14:36
*** itlinux has quit IRC14:37
*** awalende has quit IRC14:40
*** mvkr has quit IRC14:44
openstackgerriterus proposed openstack/keystone master: Add experimental job for CentOS
*** jmlowe has joined #openstack-keystone15:06
openstackgerritLance Bragstad proposed openstack/keystone master: Handle special cases with msgpack and python3
*** mvkr has joined #openstack-keystone15:19
erushi knikolla how are you?15:24
knikollahey erus, i'm good. what about you?15:24
erusi'm really good, but dying :) it's very hot t.t15:25
knikollaerus: it's the opposite here, haha.15:32
erushaha i want snoow15:33
*** mchlumsky has joined #openstack-keystone15:33
erusknikolla today the thermal sensation is 40 grades :(15:34
erusi'm literally dying T.T15:34
knikollait's a nice -3C here.15:35
erusi envy you(?) xD15:35
*** mchlumsky has quit IRC15:36
knikollaerus: there's good and bad days. thursday will be -11C.15:36
*** mchlumsky has joined #openstack-keystone15:37
erusknikolla woow, well tomorrow it will be 37 grades but we'll see what about the thermal sensation and the humidity :(15:38
*** mvkr has quit IRC15:39
knikollaerus: probably lbragstad has the worst in terms of weather, haha.15:48
lbragstadha - it's not too bad15:49
knikollagotta love that -36C on wednesday, am i right?15:49
eruswhere are you lbragstad?15:50
lbragstadNorth Dakota, USA15:50
lbragstadaka... middle. of. no. where.15:50
knikollaMount Bragstad, lol15:50
erusohh right, a lot of snow i guess :)15:51
*** openstackgerrit has quit IRC15:51
lbragstadknikolla does BU use keycloak?15:52
knikollalbragstad: yes. as an idp proxy.15:53
*** mvkr has joined #openstack-keystone15:53
lbragstaddoes BU issue certificates to authenticate to it?15:54
knikollalbragstad: hmmm... let me reiterate the answer. BU uses Shibboleth-IdP. And service providers use SAML to talk to it. For it's own services, BU does certificates and other things, but for external things it's strictly SAML and AFAIK doesn't issue certs. MOC uses Keycloak as an IdP proxy to our services.15:56
knikollalbragstad: i have no internal insight into BU IT. That's a separate department entirely.15:57
lbragstadi was thinking about the x509 stuff we stumbled across last week and was wondering if it would tie into BUs federation story at all15:57
knikollalbragstad: there is a separate service, called CILogon, which issues x509 certs for university logins called
knikollait acts as an IdP proxy to NSF funded SPs15:59
*** dklyle has joined #openstack-keystone16:06
*** itlinux has joined #openstack-keystone16:06
knikollaerus: approved your two patches adding centos support to fed testing. great work!16:43
erusoh thanks knikolla o/16:43
erusi'm with suse now :)16:44
knikollawhat do you mean?16:44
erusi'm working with opensuse i mean for adding support for suse16:45
eruswill the script work with mellon? i mean is it planned?16:46
*** pcaruana has quit IRC16:47
erusgreat :D16:48
*** openstackgerrit has joined #openstack-keystone16:49
openstackgerritVishakha Agarwal proposed openstack/keystone master: Drop ephemeral user_type support
*** erus has quit IRC16:52
*** dims has quit IRC17:08
*** gyee has joined #openstack-keystone17:09
lbragstadgagehugo do you have thoughts on this?
openstackLaunchpad bug 1808305 in python-keystoneclient "discrepancy in response of "check_*" methods" [Undecided,New]17:23
lbragstadi'm inclined to say the tags API might not be a good example for that bug17:23
*** whoami-rajat has quit IRC17:35
*** bnemec has quit IRC17:39
*** erus has joined #openstack-keystone17:41
*** mvkr has quit IRC17:51
*** mvkr has joined #openstack-keystone17:58
*** whoami-rajat has joined #openstack-keystone18:02
*** bnemec has joined #openstack-keystone18:03
*** mvkr has quit IRC18:51
*** ayoung has joined #openstack-keystone18:56
ayoungI need to find a way to get Hexchat to automatically connect with the FreeNode anti-spam check thing.18:56
knikollaayoung: best thing i did was $5/month for irc cloud.18:59
knikollaayoung: btw i was hanging out in arlington yesterday.18:59
ayoungknikolla, I heard there was a troublemaker in town18:59
ayoungknikolla, you back at work yet, or still on Gov't enforced Sabatical19:00
*** dave-mccowan has quit IRC19:00
knikollaayoung: I'm back at work now. It wasn't govt enforced at all. BU filed late. The govt did a great job approving it on time, saving me the trouble.19:01
knikollaThankfully immigration is mostly self funded by application fees.19:01
ayoungknikolla, got a MOC access question for you.  Care if I ask it in here?19:01
knikollasure, go ahead.19:02
ayoungGot a server with a public IP.  Can't ssh in.  My security group is "all open."  Is there something wonky going on with public IPs, or did I just mess things up?>19:03
* kmalloc wakes up sick and now tries to get goin.19:04
knikollaayoung: not aware of anything wonky going on, though sometimes the neutron agent in specific compute nodes likes to play games. File a ticket here with vm name and ip and we'll look into it.19:04
kmallocayoung: uh. i found that i had to explicitly open all ports sometimes the default "all allowed" doesn't work19:04
ayounglet me get back to a functional set of nodes...I tore them down last night...and messed up my playbook....1 sec19:05
kmallocknikolla: i don't like irccloud, but it works and it works well.19:05
kmallocmost of the time.19:05
kmallocayoung: i'm excited. our new place is going to have fruit trees soon (planting them this weekend)19:06
*** jmlowe has quit IRC19:08
*** dave-mccowan has joined #openstack-keystone19:09
*** spsurya has quit IRC19:14
*** dims has joined #openstack-keystone19:15
ayoungkmalloc, very nice.  What are you planting?19:15
gyeekmalloc, make sure you have both male and female trees for pollination :-)19:17
kmallocayoung: ranier cherry, lapins cherry, and dwarf nectarine19:18
kmallocayoung: for now. probably 1 or 2 more cherry trees once we have the rest of the yard/garden in order19:18
kmallocmaybe an apple or pomegranate19:19
ayounggyee, I know you are joking, but there is a kernel of truth.  Certain types of trees need to pollinate with slightly different other breeds to bear fruit19:19
gyeethat's how the people at the nursery described that to me once19:19
kmallocyeah ranier cherry and lapins cross polinate well. nectarine is self-polinating (for now)19:19
kmallocinterestingly, seattle is an 8B zone just shy of being able to support citrus... which is a weird thought considering how far north we are19:22
*** dave-mccowan has quit IRC19:34
*** imacdonn has joined #openstack-keystone19:35
*** xek has quit IRC19:37
*** xek has joined #openstack-keystone19:37
lbragstadkmalloc jaosorior isn't this similar to what you were working on ?19:41
openstackLaunchpad bug 1457702 in python-keystoneclient "The default endpoint interface type for Keystone v3 should be 'public'" [Low,Won't fix]19:41
kmalloclbragstad: i think different that is ksc specific in how it processes the catalog19:43
jaosoriorlbragstad: what?19:43
*** jmlowe has joined #openstack-keystone19:44
hrybackikmalloc: for caching of fernet tokens -- do we need to have both `memcache_servers` /and/ `backend_argument` set to point at the memcache server?19:44
kmalloclbragstad: jaosorior is/was working on issues with hard-coded ednpoints19:44
jaosoriorlbragstad: it is kinda similar19:44
kmallochrybacki: no. it's two ways to configure the same thing19:44
jaosoriorI was planning to change the internal one I set to public19:44
jaosoriorbut hadn't had time.19:44
kmallocmemcache_servers takes priority19:44
kmallochrybacki: backend_argument is... flawed in many ways.19:45
hrybackikmalloc: hmm. let me verify something weird on my end before continuing19:45
kmallocbackend_argument is the "new way", but until recently didn't work at all19:46
kmallocespecially with the memcache backend.19:46
hrybackikmalloc: so in queens I'm seeing that backend_command is taking priority (backend = dogpile.cache.memcached)19:48
hrybackiI setup two instances of memcached running on 11211 and 11212 respectively to verify19:48
kmallocmemcached_servers implies dogpile.cache.memcache iirc.19:49
kmalloclet me check, I was almost certain we made memcache_servers take priority19:49
kmalloci might be wrong.19:50
hrybackikmalloc: can you point me at the section of code you look at?19:50
kmallocthis is from memory, when i wrote the code for keystone and then ported to oslo.cache :P19:51
kmalloclooking at the code now19:51
kmalloc looks like memcache_servers is the fallback19:52
kmallocand backend_argument is the primary.19:52
hrybackiI see. head spinny19:53
hrybackithanks kmalloc !19:53
*** aojea has joined #openstack-keystone19:59
kmallochrybacki: having issues with cache?20:00
hrybackikmalloc: finally drafting that caching doc we spoke about months ago20:02
hrybackiI shot you an email with the draft notes if you have a second (it's short)20:02
kmallocwill check20:05
kmalloc@rh email or @gmail?20:05
hrybackikmalloc: RH -- I can forward it to your personal though20:06
kmallocno worries20:06
kmallocjust making sure i'm looking at the right place20:06
hrybackiack, in general should I send things to one over the other?20:06
* kmalloc should go take some more cold meds first... it'll make groking caching documentation easier20:06
hrybackidon't cache a cold kmalloc20:07
*** jmlowe has quit IRC20:09
*** aojea has quit IRC20:12
lbragstadvishakha do you know where the patch was that fixed this for keystone?
openstackLaunchpad bug 1615076 in python-keystoneclient "Keystone server does not define "enabled" attribute for Region but mentions in v3" [Undecided,Fix released] - Assigned to Vishakha Agarwal (vishakha.agarwal)20:12
lbragstadfyi - here is a relatively trivial review that fixes a bug
*** aojea has joined #openstack-keystone20:20
*** aojea has quit IRC20:20
*** aojea has joined #openstack-keystone20:20
*** jmlowe has joined #openstack-keystone20:28
hrybackikmalloc: did anything major change in (token) caching between Newton and Queens that you recall?20:30
lbragstadI thought I remember the cache_on_issue functionality landing back then sometime20:31
kmallocyeah that sounds right20:33
kmallocotherwise i think it's no major changes20:33
kmalloccache_on_issue was a nice improvement20:33
*** whoami-rajat has quit IRC20:35
hrybackithanks kmalloc lbragstad -- cache_on_issue bumps performance a hair I would assume?20:38
lbragstadyeah - it just pre-caches tokens when they are created20:38
hrybackibless whoever made an intuitive name for that20:38
lbragstadsince the most common use case for them is people use them in other services immediately20:38
* hrybacki nods20:38
lbragstadare you still working on federation in tripleo?20:40
hrybackiby proxy of the team yeah. Pushing many things forward atm :)20:44
*** jmlowe has quit IRC20:50
ayounglbragstad, kmalloc so, this is what knikolla and I were going for with the per API RBAC
ayoungistio does it now, and it seems no one has complained about a cambrian explosion of URLs21:12
kmalloclbragstad: so.. with we have a test somewhere that verifies a standard user cannot get another user?21:13
ayoungwe could do something comparable with the app creds once cmurphy gets the feature in21:13
kmalloclbragstad: i'm inclined to add a similar test just in the common check *here* so it's clearly tested in the same place.21:13
kmalloclbragstad: otherwise that patch looks good.21:14
kmalloclbragstad: pinging you before scoring.21:14
lbragstadit's later in the chain21:14
kmalloclbragstad: then +2 on that one21:14
lbragstadall of those patches follow a basic pattern21:14
lbragstadkmalloc check for testing holes though, let me know if you find any missing negative tests21:15
kmallocright. for now this is looking good21:15
lbragstadmost of those patches implement system reader -> system member -> system admin -> domain functionality -> project functionality21:15
lbragstadwhich is why the tests initially start out as very system-specific21:16
kmallocayoung: my brain can only context switch so far.21:16
kmallocayoung: refresh me on the per-api RBAC thing21:16
kmallocayoung: rememebr i had 34 days to expunge all state knowledge of in-flight things in keystone from active memory21:17
ayoungkmalloc, impolied roles.  admin implies memeber implies GET /v3/users21:17
kmallockeep going.21:17
ayoungeforce RBAC in keystonemiddleware?21:18
ayoungsplit the role check from the scope check21:18
ayoungistio is doing just that, but for Kubernetes21:18
kmallocisn't this what we discussed with the app-cred restrictions?21:18
kmallocwith capabilities?21:18
ayoungthat is where they ended up landing21:18
kmallocyeah, i'm still not opposed to it... except the following concerns:21:19
ayoungjuyst showing that the mechanism is adopted out there21:19
kmalloc1) Nova doesn't communicate URLs to mioddleware.21:19
ayoungand it is fast becoming the norm21:19
kmallocso it's caveat emptor for the end user configuring21:20
kmallocaka, we can't validate if it would actually work... so just need to document it21:20
ayoungkmalloc, I'll let you convince yourself that is a non issue21:20
kmallocno need, just need clear documentation. it's long been my only requirement on that front21:20
ayoungon a conf call, and someone just unmuted and was playing rock21:21
ayounganyway, there are two competing products at Red Hat that are doing just this, but for the app layer21:21
kmalloc2) we have had the request for a way to limit the urls administratively -- so allowable templates21:21
ayoungthe JBoss one is 3scale, the K8S one is Istio21:21
kmallocbut that's been it.21:21
*** erus has quit IRC21:21
kmallociirc i've been a supporter of the limit-on-url mechanism from the original discussion21:22
ayoungThe thing that they don't realize is that they need to have the scope check, too21:22
kmallocscope check is something we do well.21:22
ayoungunless the apps are written with the scope in the URL...which leads me to wonder if that should be the norm21:22
*** erus has joined #openstack-keystone21:22
ayoungdefine "we"21:22
ayoungkeystoine, yes, openstack, not so much21:23
ayoungNova has it now.21:23 um21:23
kmallocthe core services do ok with it21:23
kmallocnova, neutron, glance, cinder.21:23
ayoungglance didnot last I looked.21:23
ayoungactually, none of them do21:24
ayoungthat is 96869621:24
kmallocok, there are two bits21:24
kmalloca fundamental scope check, and ignore RBAC policy is bad (we are fixing that)21:24
ayoungwell...nova does either or21:24
ayoungglance does nothing21:24
ayoungcinder... let me see...21:24
kmallocglance does check ownership21:24
kmallocadmin-ness not scoped is not an absence of a scope check21:25
kmallocwe are actively fixing that.21:25
kmallocsystem-scope and default roles.21:25
ayoungah, cinder is generating policy now.  excellent21:25
kmallocwe do a good job at scope checking in most services. we do not do a good job of limiting admin-access-bleed-through21:25
kmallocand that is 96869621:26
kmallocanyway. URL-matched restrictions is fine to add.21:26
ayoungoooh, and cinder checks is_admuin_propejct!21:26
ayoung  glance still broken21:27
ayoungno scope check21:27
lbragstaddon't url restrictions require us to map all service APIs to roles?21:27
kmallocglance is going to be more work for system scope.21:27
kmalloclbragstad: no21:27
ayounglbragstad, yeah, but there were catch alls21:27
kmallocwe can support templates if it helps (based on the conversations)21:28
ayoungthe default was to say admim implies anything not otherwise specified21:28
kmallocayoung: if you're conflating admin-project-scope-check with a pure scope check we're talking across each other21:28
kmallocand it doesn't seem to be in anyway relevant to the URL-based restriction code.21:29
kmallocand as i said, we are actively working on the admin-bleed-through issue(s)21:29
ayoungkmalloc, actually, I was not, just that the world seems to finally catch up with the need to fix policy.  I give lbragstad props for that21:29
kmallocglance is going to be one of the hardest to fix.21:29
ayoungand you...21:29
ayoungyeah, glance needs help. Last  I checked there was like 1 person actively working on it21:29
ayoungmaybe 3, but not much more21:30
kmalloci do apologize for a bit of the coarseness on irc today. on massive doses of cold meds :(21:30
lbragstadadding routes to roles in keystone is going to add some more complexity imo21:30
kmalloclbragstad: well, we're adding it at the app-cred layer afaik for now.21:30
lbragstadi think we should still push services to consume scope properly21:30
kmallocwe can extend to the roles once we have a mechanism to support it at an opt-in point21:30
lbragstadand remove hardcoded admin checks21:30
ayounglbragstad, absolutely21:30
kmallocthese are totally something to parallel21:31
kmallocapp-creds basically lead the features of basic roles (in my view) for long term enhancements21:31
lbragstadmy fear is that we will build a short circuit that doesn't require services to fix things "now"21:31
lbragstadand by "now" I just mean incrementally move in the same direction as a group of services21:31
kmallocsince it's always pure-opt in for adding a functionality to an app-cred...and app creds are immutable21:32
kmallocso no "opting in an active app-cred"21:32
lbragstadi could see the short-circuit getting used in some deployments and not in others, which might be super confusing for operators21:32
kmallocwe can also decide if an app-cred feature is worth pushing down to base roles.21:32
lbragstad(we also don't really know how to short circuit in middleware without scope information from the service)_21:33
kmallocso, i think the workflow is: 1) keep pushing on scope checks21:34
kmallocand enhance app-creds to be what we discussed.21:34
kmallocdecide if we want to expand features and checks work once services are doing things more correctly.21:35
*** erus has quit IRC21:41
*** erus has joined #openstack-keystone21:43
kmalloclbragstad: changes for some testing needed for the system_scope support, namely .cleanup_instance is needed21:45
kmalloccan be a followup, and will upgrade to +2 as needed21:45
kmallocthe domain one looks like it's a related failure to the change (in tempest) will need to be looked into.21:45
lbragstadyeah.. tempest is using domain admin == system admin :(21:46
lbragstadthrough a configuration option that defaults to true21:46
lbragstadso it assumes anyone with a domain-scoped token with 'admin' can do anything in the deployment21:46
lbragstadso that's where it's breaking21:46
lbragstadi have a patch for it21:47
lbragstadif i use depends on from the keystone patches, they pass21:47
*** erus has quit IRC21:49
*** erus has joined #openstack-keystone21:54
ayoungknikolla, OK, I am back to having nodes enabled22:00
ayoungfor exmaple:  87ff7a16-95f2-4349-af0d-30fc0a45fa43 | lapras.awx.fsi-moc   | ACTIVE | awx-private-net_network=, | rhel-guest-image-7.5-1a | m1.medium22:01
ayoungcan't ping, can't ssh22:01
ayoungtraceroute ends with22:01
ayoung14 (  3026.273 ms !H  3024.599 ms !H  3026.138 ms !H22:01
*** aojea has quit IRC22:03
ayoung$ openstack security group show awx-rdu-all-open -f json | fpaste22:03
ayoungUploading (2.0KiB)...22:03
ayounggah: sescaped json22:04
*** rcernin has joined #openstack-keystone22:13
*** rcernin has quit IRC22:15
*** rcernin has joined #openstack-keystone22:15
*** aojea has joined #openstack-keystone22:20
clarkbayoung: your icmp rule is a group rule so only works within the group22:23
clarkbthe tcp rule should allow for ssh from external though22:24
*** aojea has quit IRC22:24
*** aojea has joined #openstack-keystone22:25
clarkbthough that is ipv4 only, the ipv6 rules are group only, so ifyou are trying to hit it via ipv6 it would be sad too (doubt it based on the IPs above though)22:25
*** itlinux has quit IRC22:27
lbragstadalright - i just went through and updated milestones for old bug reports dating back to rocky22:27
lbragstadif anyone sees anything that is Fix Released/Fix Committed and milestone isn't set, just let me know22:28
*** aojea has quit IRC22:29
*** jmlowe has joined #openstack-keystone22:30
lbragstadlooking at the summaries in launchpad22:31
lbragstadthere was 70 bugs fixed in pike, 38 in queens, 60 in rocky, and we've fixed 36 so far in stein22:32
*** eandersson has quit IRC22:45
*** xek has quit IRC22:46
*** eandersson has joined #openstack-keystone22:46
*** tkajinam has joined #openstack-keystone23:09
*** bnemec has quit IRC23:37
*** erus1 has joined #openstack-keystone23:39
*** ianw is now known as ianw_pto23:42
*** mchlumsky has quit IRC23:46
*** imacdonn has quit IRC23:55
*** imacdonn has joined #openstack-keystone23:57

Generated by 2.15.3 by Marius Gedminas - find it at!