Tuesday, 2018-06-19

*** lifeless_ has joined #openstack-keystone00:05
*** lifeless has quit IRC00:05
*** Dinesh_Bhor has joined #openstack-keystone00:11
*** felipemonteiro has quit IRC00:17
kmalloclbragstad: ugh, we have a linger hard-coded "admin_rquireD" check in keystone00:51
*** felipemonteiro has joined #openstack-keystone00:54
*** zxy has joined #openstack-keystone00:59
*** yikun has joined #openstack-keystone01:06
*** masber has joined #openstack-keystone01:27
*** gyee has quit IRC01:33
openstackgerritMerged openstack/keystone master: Unified limit update APIs Refactor  https://review.openstack.org/55955201:34
openstackgerritMerged openstack/keystone master: Imported Translations from Zanata  https://review.openstack.org/57589601:34
wxylbragstad: kmalloc: We had the Dragon Boat Festival in China yesterday. I'll complete the follow-up patch about PK today.01:37
*** lifeless_ has quit IRC01:43
*** lifeless has joined #openstack-keystone01:45
openstackgerritwangxiyuan proposed openstack/keystone master: Api-ref: Refresh the Update APIs for limits  https://review.openstack.org/56974101:52
adriantkmalloc: I tried once to read through the @protected code to figure out why something was doing a thing. It was terrifying.01:52
kmallocwxy: sounds good, hope the festival was fun.01:57
kmallocadriant: yeah, I'm re-writing the whole thing. It's terrible what we have now.01:57
*** felipemonteiro has quit IRC02:17
*** mvenesio has joined #openstack-keystone02:21
*** liuzz has joined #openstack-keystone02:27
*** felipemonteiro has joined #openstack-keystone02:36
*** sapd_ has joined #openstack-keystone02:48
*** sapd has quit IRC02:48
*** blake has joined #openstack-keystone02:57
*** david-lyle has joined #openstack-keystone03:00
*** boris_42_ has quit IRC03:04
*** david-lyle has quit IRC03:09
*** mvenesio has quit IRC03:11
*** itlinux has joined #openstack-keystone03:15
*** felipemonteiro has quit IRC03:23
*** felipemonteiro has joined #openstack-keystone03:24
*** felipemonteiro has quit IRC03:36
*** ykarel|pto has joined #openstack-keystone04:01
*** blake has quit IRC04:03
*** blake has joined #openstack-keystone04:04
*** germs has quit IRC04:07
*** itlinux has quit IRC04:09
*** ykarel|pto is now known as ykarel04:26
*** jaosorior has joined #openstack-keystone04:32
*** pcichy_ has joined #openstack-keystone04:55
*** pcichy has quit IRC04:56
*** pcichy_ is now known as pcichy04:56
*** AlexeyAbashkin has joined #openstack-keystone05:01
*** markvoelker has quit IRC05:01
*** sonuk_ has joined #openstack-keystone05:05
*** sonuk has quit IRC05:05
*** links has joined #openstack-keystone05:33
*** AlexeyAbashkin has quit IRC05:34
*** AlexeyAbashkin has joined #openstack-keystone05:39
*** quiquell|off is now known as quiquell|rover05:44
*** pcichy has quit IRC05:49
*** josecastroleon has joined #openstack-keystone05:49
*** lifeless has quit IRC05:53
*** lifeless has joined #openstack-keystone05:53
*** blake has quit IRC06:02
*** AlexeyAbashkin has quit IRC06:07
*** gongysh has joined #openstack-keystone06:17
*** fiddletwix has quit IRC06:28
*** fiddletwix has joined #openstack-keystone06:28
*** ykarel_ has joined #openstack-keystone06:33
*** ykarel has quit IRC06:36
*** ykarel_ is now known as ykarel06:36
*** ispp has joined #openstack-keystone06:39
*** martinus__ has joined #openstack-keystone06:40
*** felipemonteiro has joined #openstack-keystone06:40
*** zxy has quit IRC06:49
*** felipemonteiro has quit IRC06:51
*** markvoelker has joined #openstack-keystone07:01
*** tesseract has joined #openstack-keystone07:02
*** rcernin has quit IRC07:05
*** amoralej|off is now known as amoralej07:06
*** jmlowe has quit IRC07:07
*** namnh has joined #openstack-keystone07:28
*** blake has joined #openstack-keystone07:28
*** tosky has joined #openstack-keystone07:34
*** markvoelker has quit IRC07:36
andreykurilinhi folks! Can anyone help me to figure out why it happens at stable/pike? http://logs.openstack.org/82/575682/2/check/neutron-rally-neutron/64d4cab/logs/devstacklog.txt.gz#_2018-06-18_14_49_45_03907:49
*** pcaruana has joined #openstack-keystone07:50
openstackgerritwangxiyuan proposed openstack/keystone master: Add auto increase primary key for unified limit  https://review.openstack.org/57602508:01
wxyandreykurilin: http://logs.openstack.org/82/575682/2/check/neutron-rally-neutron/64d4cab/logs/screen-keystone.txt.gz08:03
wxyandreykurilin: the oslo.config's version is less than 5.2.008:04
andreykurilinwxy: oh...do not know how I missed it08:04
andreykurilinthanks08:04
wxyandreykurilin: :)08:04
*** blake_ has joined #openstack-keystone08:05
*** blake has quit IRC08:08
*** blake_ has quit IRC08:09
*** nicolasbock has joined #openstack-keystone08:17
*** nicolasbock has quit IRC08:24
*** AlexeyAbashkin has joined #openstack-keystone08:25
*** lifeless has quit IRC08:28
*** markvoelker has joined #openstack-keystone08:33
*** jaosorior has quit IRC08:49
openstackgerritVishakha Agarwal proposed openstack/keystone master: Added check to avoid keyerror "user['name']"  https://review.openstack.org/57643308:53
*** markvoelker has quit IRC09:06
*** jaosorior has joined #openstack-keystone09:15
fricklerandreykurilin: I talked about that with slaweq yesterday. from my analysis the main issue is that rally_openstack requires dependencies that are too new for stable/pike09:17
fricklerandreykurilin: you also have a bug in your devstack plugin that hides the output from pip install here: http://logs.openstack.org/82/575682/2/check/neutron-rally-neutron/64d4cab/logs/devstacklog.txt.gz#_2018-06-18_14_49_13_40209:18
*** liuzz has quit IRC09:34
*** sonuk has joined #openstack-keystone09:36
*** sonuk_ has quit IRC09:39
*** Dinesh_Bhor has quit IRC09:49
*** gongysh has quit IRC09:55
*** markvoelker has joined #openstack-keystone10:03
*** namnh has quit IRC10:14
*** sonuk_ has joined #openstack-keystone10:20
*** sonuk has quit IRC10:23
*** annp has quit IRC10:36
*** markvoelker has quit IRC10:37
*** ykarel has quit IRC10:56
*** ykarel has joined #openstack-keystone10:56
*** slaweq has joined #openstack-keystone10:59
slaweqhi10:59
slaweqcan someone from keystone team take a look at errors which we have in neutron-rally-job for stable/pike and stable/ocata branches: http://logs.openstack.org/51/576451/1/check/neutron-rally-neutron/e320b1f/logs/screen-keystone.txt.gz11:00
slaweqmaybe You will quickly know from where this oslo.config>=5.2.0 comes from and how to fix it11:00
*** links has quit IRC11:06
*** lifeless has joined #openstack-keystone11:19
openstackgerritwangxiyuan proposed openstack/keystone master: [WIP]Add auto increase primary key for unified limit  https://review.openstack.org/57602511:20
*** links has joined #openstack-keystone11:22
*** lifeless has quit IRC11:28
*** lifeless has joined #openstack-keystone11:28
*** markvoelker has joined #openstack-keystone11:34
*** links has quit IRC11:39
*** quiquell|rover is now known as quique|rover|lch11:39
*** pcichy has joined #openstack-keystone11:48
*** edmondsw has joined #openstack-keystone11:50
*** amoralej is now known as amoralej|lunch11:57
*** markvoelker has quit IRC12:01
*** markvoelker has joined #openstack-keystone12:02
fricklerslaweq: as I said to andreykurilin earlier, rally installs newer packages that collide with stable/pike. they also hide the output of their pip install due to a bug here http://logs.openstack.org/51/576451/1/check/neutron-rally-neutron/e320b1f/logs/devstacklog.txt.gz#_2018-06-19_10_40_24_14312:04
fricklerthey execute "sudo pip install rally_openstack>=1.1.0" which results in the output landing in a file called "=1.1.0" instead of applying the version cap12:05
andreykurilinfrickler: yup. you are right. I'm working on this. soon, there will be a fix12:05
slaweqfrickler: yes, but I did small patch https://review.openstack.org/#/c/576451/ so rally should be installed in 0.10.1 version which has good requirements IMO12:05
slaweqand issue is still the same12:05
*** quique|rover|lch is now known as quiquell|rover12:05
andreykurilinhm..12:05
*** links has joined #openstack-keystone12:05
andreykurilinslaweq: that patch doesn't work12:07
andreykurilinhttp://logs.openstack.org/51/576451/1/check/neutron-rally-neutron/e320b1f/logs/devstacklog.txt.gz#_2018-06-19_10_40_24_14312:07
andreykurilinit is still master branch12:07
slaweqandreykurilin: ahh, so it installs rally from pip instead of git repo12:08
andreykurilinslaweq: not exactly12:08
andreykurilinslaweq: neutron job triggers opentack/rally devstack plugin and it's installs a package via pip. It was introduced in master branch and was not released yet. installing rally != master should work, but it did not work in your patch. do not know why12:09
fricklerslaweq: the branch tag on the enable_plugin has no effect if the repo already exists via required-projects12:10
slaweqfrickler: so should I remove rally from required-projects in job definition?12:10
*** raildo has joined #openstack-keystone12:10
*** sonuk_ has quit IRC12:12
openstackgerritwangxiyuan proposed openstack/keystone master: Add auto increase primary key for unified limit  https://review.openstack.org/57602512:12
fricklerslaweq: no, because devstack is not allowed to clone anything in that gate setup. I think you'd instead need to add the target branch here http://git.openstack.org/cgit/openstack/neutron/tree/.zuul.yaml?h=stable/pike#n7912:12
andreykurilinslaweq: https://docs.openstack.org/infra/zuul/user/config.html?highlight=secret#attr-job.required-projects.override-checkout12:12
andreykurilinreplace `- openstack/rally` by `- {"name": "openstack/rally", "override-checkout": "0.12.1"}`12:13
slaweqthx andreykurilin and frickler I will try12:13
slaweqbut should it be 0.12.1?12:13
slaweqisn't it too new?12:13
andreykurilinslaweq: no, it is not too new:)12:15
slaweqandreykurilin: are You sure? in 0.12.1 I see oslo.config>=5.1.0: https://github.com/openstack/rally/blob/0.12.1/requirements.txt#L1212:16
andreykurilinslaweq: I suppose devstack will install versions from u-c and since there is no `pip install` calls, nothing will be updated12:17
slaweqandreykurilin: ok, I will try 0.12.1 then12:18
slaweqandreykurilin: frickler thx a lot for help with this issue12:20
andreykurilinslaweq: sorry for introducing this issue. I did not expect that it can have such side-effect12:21
slaweqandreykurilin: no problem, it's normal :)12:21
slaweqbut I think that other projects also might be affected, it's not only related to neutron12:22
*** jmlowe has joined #openstack-keystone12:23
*** d0ugal has quit IRC12:24
*** jmlowe has quit IRC12:27
*** jistr is now known as jistr|mtg12:28
openstackgerritwangxiyuan proposed openstack/keystone master: Add policy for limit model protection  https://review.openstack.org/56271412:28
openstackgerritwangxiyuan proposed openstack/keystone master: Implement enforcement model logic in Manager  https://review.openstack.org/56271512:28
openstackgerritwangxiyuan proposed openstack/keystone master: Expose endpoint to return enforcement model  https://review.openstack.org/56271612:28
openstackgerritwangxiyuan proposed openstack/keystone master: Strict two level hierarchical limit  https://review.openstack.org/55769612:28
*** jmlowe has joined #openstack-keystone12:33
knikollao/12:41
*** d0ugal has joined #openstack-keystone12:44
*** amoralej|lunch is now known as amoralej12:52
*** mvenesio has joined #openstack-keystone12:52
*** s10 has joined #openstack-keystone13:01
*** devx is now known as DevX13:16
*** links has quit IRC13:22
ildikovknikolla: hi13:27
ildikovknikolla: I asked about the OPNFV Edge Cloud meeting and this week it's in an alternate slot which is 0300 UTC on Thursday13:28
ildikovknikolla: it seems highly inconvenient for both of us, so I requested a slot for the meeting next week, which I believe is 1300 UTC on Wednesday13:29
ildikovknikolla: would that work for you next week?13:29
knikollaildikov: hi13:32
knikollayeah, 1300 UTC next week works for me13:32
ildikovknikolla: great, I will confirm it and will send you a calendar invite13:32
ildikovknikolla: the Edge Computing Group call for this week s in ~25 minutes, I will give a highlight on what we discussed yesterday and will see if anyone has bandwidth to join13:33
ildikovknikolla: the call details are here if you would be interested to dial in: https://wiki.openstack.org/wiki/Edge_Computing_Group#Meetings13:33
ildikovknikolla: not a must, I don't want to fully fill up your calendar :)13:34
ildikovknikolla: we're usually taking notes on IRC too: #edge-computing-group13:34
knikollaildikov: thanks! but I can't make it today as I have to wrap up some things before an internal meeting.13:36
knikollai'll be sure to check the notes13:36
ildikovno worries13:36
ildikovwill ping you once I got a hold on a few people to work on this :)13:37
*** jistr|mtg is now known as jistr13:42
*** blake has joined #openstack-keystone13:59
*** spilla has joined #openstack-keystone14:03
*** wxy| has joined #openstack-keystone14:06
*** jeremyfreudberg has joined #openstack-keystone14:06
*** germs has joined #openstack-keystone14:08
*** germs has quit IRC14:08
*** germs has joined #openstack-keystone14:08
*** spilla has quit IRC14:08
*** spilla has joined #openstack-keystone14:08
jeremyfreudbergkmalloc / lbragstad: http://paste.openstack.org/raw/723829/ i just tried this on a fresh devstack, simple example for duplicated roles through trust14:10
lbragstadjeremyfreudberg: oh - cool14:12
lbragstadjeremyfreudberg: i'll see if i can recreate here in a minute14:12
*** germs has quit IRC14:15
*** felipemonteiro_ has joined #openstack-keystone14:15
*** felipemonteiro__ has joined #openstack-keystone14:17
*** felipemonteiro_ has quit IRC14:21
*** germs has joined #openstack-keystone14:22
*** mvk has quit IRC14:24
*** germs has quit IRC14:27
*** germs has joined #openstack-keystone14:31
*** germs has quit IRC14:31
*** germs has joined #openstack-keystone14:31
*** quiquell|rover is now known as quiquell|off14:35
kmallocjeremyfreudberg: nice.14:35
*** paul122345 has joined #openstack-keystone14:40
*** felipemonteiro__ has quit IRC14:43
*** felipemonteiro has joined #openstack-keystone14:49
*** biggles has joined #openstack-keystone14:52
*** efried has joined #openstack-keystone14:53
*** slaweq has quit IRC14:53
*** biggles has quit IRC14:53
efriedGood ugt morning folks.  Is now a good time to propose a ksa release to include https://review.openstack.org/#/c/574784/ ?14:55
*** nicolasbock has joined #openstack-keystone14:56
lbragstadefried: yeah - i think we can get around to one today or tomorrow, i'd like to get https://review.openstack.org/#/c/575685/15:01
openstackgerritJeremy Freudberg proposed openstack/keystone master: De-duplicate role list during trust creation  https://review.openstack.org/57654815:01
efriedlbragstad: Okay, sure.  Would you like it to be 3.8.1 or 3.9.0?15:01
efriedlbragstad: Hm, that patch won't show up in the release.  But meh, in the tag; and I'm in no special hurry.15:02
lbragstad3.9.0? since it's add a new kwarg?15:03
efriedwfm15:03
efriedI'll keep an eye on that other patch and roll one for the release once I see it merge.15:03
lbragstadefried: thanks15:03
lbragstadjeremyfreudberg: i just recreated that bug15:03
*** paul122345 has quit IRC15:04
lbragstadjeremyfreudberg: but i saw ['reader', 'member', 'reader'] in the list of roles returned, is that consistent with what you saw?15:04
jeremyfreudberglbragstad: yes15:05
lbragstadjeremyfreudberg: it looks like we can probably add a test to keystone/tests/unit/test_v3_auth.py15:11
lbragstadthere is a test class in there for trust specific behavior that'd be a good home for something like this i think15:11
ayoungkmalloc, lbragstad I want to add a "Token" service to the service catalog15:11
kmallocayoung: and what is this "Token" service?15:11
ayoungif you go to the auth_url, you get an unscoped token with a service catalog with only the token service in it15:12
ayoungand that is what you use to convert an unscoped token to a scoped one15:12
lbragstadjeremyfreudberg: https://git.openstack.org/cgit/openstack/keystone/tree/keystone/tests/unit/test_v3_auth.py#n376715:12
ayoungit lets us break the auth-url off the implicit knowledge that this is the keystone server15:12
kmallocdo we even need that? i mean.. can we just offer an endpoint that is the catalog without a token?15:12
ayoungkmalloc, same thing15:13
kmallocexcept you needed auth-url to get the unscoped token15:13
lbragstadjamielennox: was working on that15:13
ayoungif we do the catalog without the token, we need to give people a place to get a token15:13
lbragstadhe wanted to add a catalog with just the identity endpoint for unscoped tokens15:13
ayoungright, all you need is auth_url but Horizon uses that to do other keystoney things15:13
kmallocwhich we can encode (don't call it "token", call it "auth" or identity-auth)15:13
ayoungI want to be able to add additional auth_urls per IdP15:13
kmallocthis comes back to my desire to split auth and catalog from the /v3 suburl15:14
kmallocfor much the same reason. it could be promoted to it's own endpoint15:14
ayoungthe assumption is that you go to the Auth_url with the federated identity mechanism.  Token would only accept unscoped and issue scoped tokens15:14
ayoungright15:14
kmallocanyway, i prefer a "you can get a catalog without a token" and "here is where you auth"15:14
kmallocvs "i have an unscoped token with just a token-endpoint"15:14
ayoungkmalloc, works for me15:14
kmallocbut tl;dr ++15:15
kmallocand i'll bikeshed on name later ;)15:15
ayoung"here is where you auth" gives back a small catalog which only points to the catalog service and the token service15:15
kmalloc(paint it chartreuse_15:15
ayoungI like Chartreuse.  ++2A15:16
ayoungthing is, token service is read only15:16
ayoungI want to split the read parts of keystone from the write portions15:16
ayoungcatalog endpoint is readonly too15:16
kmallocyou mean it is POST/GET only15:17
kmallocfor tokens15:17
ayoungright15:17
kmallocnot "CRUD FOR OTHER THINGS"15:17
ayoungfrom a DB perspective, it is readonly.15:17
lbragstadyou can revoke tokens15:17
kmalloc^15:17
kmallocwell...15:17
ayounglbragstad, kill that, too15:17
kmallocno you can't15:17
kmallocrevoke api never was properly exposed15:17
ayoungkmalloc, can be done via a different service15:17
kmallocdelete sortof worked.15:17
*** r-daneel has joined #openstack-keystone15:18
kmallocanyway, uh, this is a major reason i wanted /auth vs /v3/auth15:18
kmallocso we could do things like this15:18
ayoung++15:18
kmallocbecause we can't break /v3/auth15:18
lbragstadPOST /v3/auth/tokens {X-Auth-Token: $GOOD_TOKEN} {X-Subject-Token: $BAD_TOKEN} writes a revocation event15:18
kmallocbut we can change how auth works under /auth to support such things15:18
lbragstads/POST/DELETE/15:19
kmalloclbragstad: ah15:19
kmallocayoung: for your work on @protected you just split out some of the check_protection wrapper bits, right?15:27
kmallocayoung: i am going to re-write the enforcer from the ground up in flask, but i wanted to make sure i knew what you were doing.15:28
*** jeremyfreudberg has quit IRC15:28
*** pcaruana has quit IRC15:29
kmalloclbragstad: just talked with dhellman and turns out any place we do msg = _(<message>) then pass that to logger and then raise an exception is wrong15:30
kmalloclbragstad: we should be duplicating the string and pass the untranslated bit to logger15:30
kmallocand the translated version to the exception.15:30
lbragstadah15:30
kmallocwe have a LOT of cleanup on that front to do15:30
lbragstadyeah...15:30
lbragstadwe should open a bug15:31
lbragstadthat would be good lhf15:31
kmallocit's mostly busy work, i'll open a bug and target R-3/RC15:31
*** felipemonteiro has quit IRC15:31
*** felipemonteiro has joined #openstack-keystone15:32
kmalloclbragstad: https://bugs.launchpad.net/keystone/+bug/177767115:33
openstackLaunchpad bug 1777671 in OpenStack Identity (keystone) "Incorrect use of translation _()" [Medium,Triaged]15:33
*** wxy|_ has joined #openstack-keystone15:38
*** wxy| has quit IRC15:39
openstackgerritwangxiyuan proposed openstack/keystone master: [WIP]Add auto increase primary key for unified limit  https://review.openstack.org/57602515:41
*** dklyle has joined #openstack-keystone15:41
*** ispp has quit IRC15:41
*** fiddletwix has quit IRC15:41
*** felipemonteiro has quit IRC15:42
*** ykarel is now known as ykarel|away15:43
*** felipemonteiro has joined #openstack-keystone15:44
*** dklyle has quit IRC15:46
*** felipemonteiro has quit IRC15:51
*** felipemonteiro has joined #openstack-keystone15:51
*** belmoreira has quit IRC15:53
*** ykarel|away has quit IRC15:54
*** asteroide has joined #openstack-keystone15:59
*** asteroide has quit IRC16:00
lbragstadkmalloc: keystone meeting?16:02
*** dklyle has joined #openstack-keystone16:11
*** knikolla has quit IRC16:19
*** felipemonteiro has quit IRC16:26
*** felipemonteiro has joined #openstack-keystone16:31
gagehugolbragstad I can take a swing at the documentation for case-insensitivity/what keystone expects16:38
lbragstadok16:38
*** blake has quit IRC16:43
*** blake has joined #openstack-keystone16:43
*** gyee has joined #openstack-keystone16:46
*** blake has quit IRC16:48
*** s10 has quit IRC16:49
*** AlexeyAbashkin has quit IRC16:52
*** blake has joined #openstack-keystone16:54
*** knikolla has joined #openstack-keystone16:55
*** blake has quit IRC16:55
*** blake has joined #openstack-keystone16:55
*** felipemonteiro has quit IRC16:56
*** blake has quit IRC17:00
hrybackihave to run to a lunch appt -- bbiab!17:00
* cmurphy afk17:00
lbragstad#startmeeting keystone-office-hours17:01
openstackMeeting started Tue Jun 19 17:01:03 2018 UTC and is due to finish in 60 minutes.  The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot.17:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.17:01
openstackThe meeting name has been set to 'keystone_office_hours'17:01
*** homeski has quit IRC17:01
kmalloclookit i spy a gyee17:01
*** homeski has joined #openstack-keystone17:01
lbragstadstepping away to grab lunch real quick and i'll back back to talk about migration options17:02
gyeekmalloc, IRC is my source of enlightenment :-)17:03
wxy|_I'll review the IRC log tomorrow. Then the patch will be ready tomorrow I think.17:03
*** wxy|_ has quit IRC17:04
*** pcaruana has joined #openstack-keystone17:05
*** blake has joined #openstack-keystone17:06
*** felipemonteiro has joined #openstack-keystone17:07
*** tesseract has quit IRC17:08
*** felipemonteiro has quit IRC17:08
*** lifeless_ has joined #openstack-keystone17:12
*** lifeless has quit IRC17:12
ildikovknikolla: cmurphy: there's a thread on the Edge ML on Keystone architectures which touches on federation17:20
ildikovknikolla: cmurphy: is either of you subscribed to that ML and could chime in?17:21
ildikovlbragstad: kmalloc: the question applies to you too in case you're interested ^^17:21
* cmurphy resubscribes17:30
lbragstadthanks wxy17:34
*** r-daneel has quit IRC17:37
*** jeremyfreudberg has joined #openstack-keystone17:47
*** jeremyfreudberg has quit IRC17:48
*** boris_42_ has joined #openstack-keystone17:52
*** amoralej is now known as amoralej|off17:53
*** dklyle has quit IRC18:10
lbragstadhttp://lists.openstack.org/pipermail/openstack-dev/2018-June/131630.html18:12
hrybackilbragstad++18:15
*** dklyle has joined #openstack-keystone18:32
*** jeremyfreudberg has joined #openstack-keystone18:51
openstackgerritJeremy Freudberg proposed openstack/keystone master: Expose duplicate role names bug in trusts  https://review.openstack.org/57661018:54
openstackgerritJeremy Freudberg proposed openstack/keystone master: Fix duplicate role names in trusts bug  https://review.openstack.org/57661118:54
jeremyfreudberg^ lbragstad, kmalloc: there it is18:55
*** r-daneel has joined #openstack-keystone19:00
lbragstadjeremyfreudberg: awesome19:03
lbragstadkmalloc: lemme know when you free to continue to migration stuff19:03
lbragstad(caught up in TC scrollback)19:04
*** r-daneel_ has joined #openstack-keystone19:05
*** r-daneel has quit IRC19:05
*** r-daneel_ is now known as r-daneel19:05
*** r-daneel_ has joined #openstack-keystone19:13
*** r-daneel has quit IRC19:14
*** r-daneel_ is now known as r-daneel19:14
*** felipemonteiro has joined #openstack-keystone19:14
*** AlexeyAbashkin has joined #openstack-keystone19:26
*** AlexeyAbashkin has quit IRC19:37
kmalloclbragstad: i am19:38
kmalloclbragstad: landlords just left (and I have 2 more errands to run today but they can wait)19:38
kmallocneed to get the AC from the storage unit and a 2nd one for my office.19:38
openstackgerritMerged openstack/keystoneauth master: Add minimum version for requirements  https://review.openstack.org/57568519:39
kmalloclbragstad: we need to fix @wip to be smarter19:39
kmalloclbragstad: i want it to handle explicit "what kind of error ot expect"19:39
kmallocnot just "an error"19:39
kmallocjeremyfreudberg: OH it's trust with implied roles that get expanded19:40
kmallocjeremyfreudberg: thats... wow.19:40
kmallocjeremyfreudberg: nice catch19:40
kmalloclbragstad: because i dislike that @wip doesn't lock us into a specific error. what if we change something and the error case changes... thats bad =/19:41
*** blake has quit IRC19:41
*** blake has joined #openstack-keystone19:42
jeremyfreudbergkmalloc: happy to help. since my newest patch prevents keystone from shooting itself in the foot, do you think we even need https://review.openstack.org/#/c/576548 (which covers up the foot shooting)19:43
kmallocnah, we don't need it. it isn't a bad case to use suspenders and a belt though :P19:43
*** blake has quit IRC19:47
*** aojea has joined #openstack-keystone19:48
lbragstadjeremyfreudberg: was that the cause of your issue in sahara?19:53
jeremyfreudberglbragstad: yes, this bug was the only cause (the case sensitivity stuff did NOT affect us)19:53
lbragstadso sahara was affected by duplicate role names in the token response?19:54
lbragstadstrange19:54
openstackgerritBen Nemec proposed openstack/oslo.policy master: Avoid redundant policy syntax checks  https://review.openstack.org/51142619:57
jeremyfreudberglbragstad: yes, because the token which had duplicate role names was used to create a second trust (in heat)19:57
lbragstadoh - so sahara was pulling that list, which included duplicates, and then put them in a request to build a new trust, which broken19:58
lbragstadbroke819:58
lbragstad/me sigh19:58
lbragstadbroke*19:58
jeremyfreudbergcorrect19:58
*** dklyle has quit IRC20:00
efriedlbragstad, kmalloc, mordred: https://review.openstack.org/576634 Release keystoneauth 3.9.020:03
* efried hopes that was done correctly, been a while20:03
kmallocefried: looks good to me20:04
lbragstadjeremyfreudberg: i assume you already have a patch that corrects that behavior in sahara?20:04
jeremyfreudberglbragstad: no, because it's all tied up in how sahara and heat interact20:05
jeremyfreudbergsahara passes a token to heat which just so happens to have this problem, and heat tries to create a trust with it20:05
openstackgerritMorgan Fainberg proposed openstack/keystone master: Add Flask-RESTful as a requirement  https://review.openstack.org/57441420:07
openstackgerritMorgan Fainberg proposed openstack/keystone master: Implement scaffolding for Flask-RESTful use  https://review.openstack.org/57441520:08
openstackgerritMorgan Fainberg proposed openstack/keystone master: Keystone adheres to public_endpoint opt only  https://review.openstack.org/57450220:08
openstackgerritMorgan Fainberg proposed openstack/keystone master: Convert json_home and version discovery to Flask  https://review.openstack.org/57473620:08
toskythe tl;dr version is that it's usually not sahara's fault, and that if sahara with the integration for all components works, then OpenStack works20:09
* tosky runs :)20:09
kmalloclbragstad: ^ did I get the minimum bit added ok on 574414?20:11
lbragstadthis is the start of a bug fix if anyone is interested in reviewing - https://review.openstack.org/#/c/562714/20:11
lbragstadkmalloc: looks ok to me20:11
*** felipemonteiro has quit IRC20:17
kmalloclbragstad: migrations?20:17
lbragstadsure20:17
lbragstadso - would we be able to take the traditional path for the migration?20:19
lbragstadcreate new table with correct schema, migrate data, drop old table?20:19
kmallocprobably20:20
kmallocit is likely to be a headache, because you have to do a lot of heavy lifting to determine which registered limit to FK to20:20
kmallocbecause name is non-unique20:21
kmallocso its, (name, service[, region]) for each migration to figure out what to FK to20:21
*** felipemonteiro has joined #openstack-keystone20:22
lbragstadname as in service name?20:22
lbragstadno, you mean resource_name20:23
kmallocresource_name20:23
kmallocyeah20:23
openstackgerritMorgan Fainberg proposed openstack/keystone master: Add support for before and after request functions  https://review.openstack.org/57663720:23
*** felipemonteiro has quit IRC20:26
lbragstadhmm20:27
*** jeremyfreudberg has left #openstack-keystone20:27
*** jeremyfreudberg has quit IRC20:27
hrybackilbragstad: question regarding DocumentedRuleDefaults20:29
*** pcaruana has quit IRC20:29
hrybackiso, what was the intended purpose of the description field?20:29
hrybackiI ask because Barbican uses the same rule for multiple parts of the API with similar but not the same purpose20:29
*** spilla has quit IRC20:30
lbragstadit's to describe the API in human terms as opposed to just being POST /v3/users20:30
lbragstadyou could say description='Create a new user'20:30
hrybackiright. I'm wondering if moving the description /into/ the operations dict might be useful20:30
*** felipemonteiro has joined #openstack-keystone20:30
hrybackiexample: https://docs.openstack.org/barbican/latest/api/reference/secret_metadata.html#put-v1-secrets-uuid-metadata-key and https://docs.openstack.org/barbican/latest/api/reference/secret_metadata.html#put-v1-secrets-uuid-metadata20:31
hrybackiutilize the same rule20:31
hrybackibut server different functions. We could 1) abstract the description to something that fits both 2) cover one but not the other 3) change DocumentedRuleDefault to handle description at the operation level20:32
hrybackiI'm guessing they aren't the only ones using the same rule for similar but different apis20:33
*** mvenesio has quit IRC20:34
*** mvenesio_ has joined #openstack-keystone20:34
lbragstadi'm not quite understanding i don't think20:34
lbragstadis the problem that they use the same rule to protect the policy?20:35
*** felipemonteiro has quit IRC20:36
hrybackilbragstad: lemme submit a PS and show you20:38
hrybackigimmie 5, wrapping up this one controller20:38
hrybackiokay lbragstad: so for their secretmeta controller they have two endpoints: /v1/secrets/{secret-id}/metadata and /v1/secrets/{secret-id}/metadata/{key-id}20:45
hrybackihttps://review.openstack.org/#/c/575218/8/barbican/common/policies/secretmeta.py20:45
hrybackifour rules, with 6 actions20:45
*** felipemonteiro has joined #openstack-keystone20:45
hrybackisecret_meta:put and secret_meta:get rules are used in both endpoints20:45
hrybackisimilar actions but not the same. However the way that they have laid out their poicy makes a single description field hard to use imo20:46
lbragstadoh20:46
hrybackiunless I'm being too verbose20:46
hrybackiyeah. and if they did it, you know others did too20:46
hrybackiI'm taking notes on re-occuring patterns20:46
*** spilla has joined #openstack-keystone20:46
lbragstadso - in that case, is there anything stopping barbican from creating a new policy for other of those endpoints?20:47
hrybackilbragstad: not that I am aware of20:47
lbragstadyou can keep the policy value/check_str the same20:47
hrybackiother than they don't want to make any 'policy changes' in Rocky20:47
*** edmondsw has quit IRC20:47
lbragstadbreaking them apart would let you be more descriptive about each policy since it's making each more specific20:48
hrybackiright. I'll make a note of that20:48
hrybackilbragstad: I've been using https://wiki.openstack.org/wiki/Barbican/Policy to help me track this20:49
hrybackiand expanding it quite a bit20:49
hrybackido we have anything akin to this in keystone?20:49
lbragstadthen you'd just change the barbican code to enforce one of the endpoint on the new policy20:49
* hrybacki nods20:49
lbragstadwe haven't used a wiki in ages20:49
hrybackiack20:49
lbragstadif an operator overrides the default20:49
lbragstadit can be aliased20:49
lbragstadhttps://docs.openstack.org/oslo.policy/queens/reference/api/oslo_policy.policy.html#oslo_policy.policy.DeprecatedRule20:50
lbragstadI wrote up an example in there - but you might not need something that involved20:50
hrybackioh that's nice lbragstad20:51
hrybackithey do have a decprecated api20:51
lbragstadi wrote an example of breaking a policy into more granular policies https://docs.openstack.org/oslo.policy/queens/reference/api/oslo_policy.policy.html#oslo_policy.policy.DocumentedRuleDefault20:51
lbragstadits the example right above ^20:51
* hrybacki nods20:51
openstackgerritMorgan Fainberg proposed openstack/keystone master: Implement base for new RBAC Enforcer  https://review.openstack.org/57663920:52
openstackgerritGage Hugo proposed openstack/keystone master: WIP Add docs for case-insensitivity in keystone  https://review.openstack.org/57664020:53
lbragstadkmalloc: pulling down the client patches quick for limits and checking some existing constraints20:56
*** martinus__ has quit IRC20:57
kmallocokie20:58
kmalloclbragstad: ... i hate our enforcement model20:58
kmallocit's such a PITA to work with20:58
kmalloci think i almost have it working without decorators.20:58
lbragstad?20:58
lbragstadoh... @controller.protected?20:59
kmallocyeah20:59
kmallochttps://review.openstack.org/#/c/576639/20:59
kmallocsee20:59
kmallocit's starting to come together in a much cleaner way20:59
kmallocall methods on flask dispatched requests will have an "after request" check to ensure enforcement was called21:00
kmallocand in the method it is the responsibility of the contributor to do "<new_enforcer>.enforce_call(<args>)21:01
kmallocwhich will do a lot of the magic @protected does now.21:01
kmallocbut eliminates all the stupid callback bits.21:01
*** felipemonteiro has quit IRC21:02
lbragstadhttp://paste.openstack.org/raw/723860/21:04
lbragstadso - for the migration21:04
lbragstad1.) create a new table called limits21:04
kmallocok21:04
lbragstadthat has the correct schema, FK relationship21:05
*** r-daneel has quit IRC21:06
kmallocyes21:06
kmallocand the new model object that is smarter/loads the right things.21:06
*** r-daneel has joined #openstack-keystone21:06
lbragstad2.) for every limit, check the limit.resource_name, limit.service_id, limit.region_id (if applicable)21:06
lbragstaduse those values to query the registered limit table21:07
lbragstadfinding the id of the registered limit those values apply to21:07
lbragstadand update the limit.registered_limit_id to the registered limit id of that query21:07
kmallocyes.21:08
lbragstadhmmm21:09
lbragstadcould this be done without another table?21:09
lbragstadwhat if step 1 is just adding a new column for limit.registered_limit_id21:09
lbragstad?21:09
kmallocwell, we are already doing a pivot for the PK change21:11
*** raildo has quit IRC21:11
lbragstadis that being done on the limit table though?21:11
kmallocyah21:11
kmallocon both21:11
lbragstador the registered limit table?21:11
lbragstadoh21:11
lbragstadhttps://etherpad.openstack.org/p/keystone-unified-limit-migration-notepad21:12
kmallocoh holy crap we're adding more triggers21:12
* kmalloc is going to cry21:12
* lbragstad pumps brakes21:12
kmallocwe need to stop adding triggers in mysql migrations21:12
kmallocthey are terrifying and horrible21:12
kmallochttps://review.openstack.org/#/c/576025/5/keystone/common/sql/expand_repo/versions/046_expand_update_pk_for_unified_limit.py21:13
lbragstadhold up, if we're able to isolate the migrations a bit, and do one of the in place, we might not have to deal with the triggers bit (since we're not creating a new table)21:13
kmallocand ultimately just cause tears21:13
kmallocthe PK change is going to need to be new tables21:14
kmalloci pretty much want to -2 that for the trigger reason only21:14
kmalloctriggers should never have been used21:14
lbragstadok - let's try and write down the changes we need21:14
lbragstadhttps://etherpad.openstack.org/p/keystone-unified-limit-migration-notepad21:14
kmallocthey are a terrible choice and are very very hard to debug.21:14
lbragstadand then see if we can break up the migrations to be a bit easier21:14
knikollalbragstad, kmalloc: i'm undecided on a review as to what score to give and what is considerent nitpicking or not. would be helpful to get your opinion21:18
lbragstadknikolla: what's up?21:18
knikollahttps://review.openstack.org/#/c/538371/6/keystone/tests/unit/limit/test_backends.py21:18
*** mvenesio_ has quit IRC21:18
knikollathe test correctly tests deletion of the limits, but it doesn't test that it didn't delete limits of other projects21:19
knikollathe WHERE part of the sql query.21:19
lbragstadoh - that seems like it would be a sane test case21:21
lbragstadif not in that patch, at least in a follow on21:22
knikollalbragstad: alright. didn't want to look like I was nitpicking.21:24
cmurphyi think it's worthwhile to consider the author - wxy is always responsive to feedback and isn't going to be discouraged by constructive criticism21:25
lbragstad++21:25
kmallocnit picking: This is a bad way to do X, please reimplement stylistically different -- or "your verbiage is not accurate".21:25
kmalloc"This needs an expanded test case like FOO" - not nit picking21:26
kmallocand very reasonable21:26
kmallocesp. for someone like wxy who gets it's not personal and is super responsive21:26
lbragstadknikolla: are you asking because of the ml culture change thread?21:26
knikollalbragstad: yes, in part.21:27
kmallocadvocating for further testing... is not nitpicking. you could easily also say "this can be a followup" and/or take on that followup yourself21:29
knikolla++21:30
knikollajust felt I should get some feedback first before I give feedback.21:30
knikollathanks!21:31
kmallocknikolla: also if you see my style of review where i say [NIT] <item>21:32
kmallocit helps clarify what you think is a trivial thing21:33
kmallocoften i might have a review with 10+ nits but still +221:33
kmallocall nits can be fixed down the line.21:33
knikollakmalloc: agree. most of my undecisiviness was in the score between +1/+221:33
kmallocoften +1 "everything looks good, but i have a serious question that needs answering, aka "what happens when X""21:34
kmallocimo21:34
kmalloc(from a core)21:34
kmalloci avoid no-score because it seems like those get lost21:34
kmallocso -1 = hey, I have a concern this might not do waht you want... or there is just no testing21:34
lbragstadi've been trying to do more no scores that i have in the past, but yeah... they are harder to see21:35
lbragstadunless i'm parsing emails from gerrit21:35
knikolla++21:35
kmalloc+1 = looks good, but expand [in the review, not even a patchset] on what i need answered21:35
*** nicolasbock has quit IRC21:35
kmalloc+2 = anything nits only or close enough that it wont cause issues21:35
knikollathat helps21:35
kmalloc-2 = "not just no, awwww heck no, but here is why we can't do it"21:35
kmalloci used to no-score anything that i just had questions about, vs +1,21:36
*** mchlumsky has quit IRC21:39
*** aojea has quit IRC21:41
*** blake has joined #openstack-keystone21:42
*** tellesnobrega has quit IRC21:42
*** tellesnobrega has joined #openstack-keystone21:43
lbragstadkmalloc: rewrote the proposed migration without triggers21:51
kmalloclooking21:52
kmallocyep.21:52
kmallocthat would be my preference.21:52
kmallocbut that sums up the alternative21:52
lbragstadso - that is going to require the database model to be cluttered for a release21:53
kmallocyes, it does.21:53
lbragstaddoes it also mean we have to run keystone at a specific version for a certain amount of time before moving to the next release?21:53
lbragstadnova had an issue with that with one of their migrations in the past i think, but there wasn't much they could do about it21:54
kmallocno.21:54
kmallocif you run migrate_data in Rocky, which we use to setup the FK, in stein we just stop referencing the old column.21:55
kmallocand we can drop it from the model and have a contract migration21:55
kmallocthat drops the actual column21:55
lbragstadwhat if i'm on Queens21:55
lbragstadi upgrade to rocky21:55
lbragstadand the immediately upgrade to Stein21:55
lbragstad(ff upgrades)21:55
kmallocdo we support upgrading to rocky and then to stien without running data_migrate?21:55
kmallocbecause it seems like i keep getting a different answer on this21:56
lbragstadno - i think running data migrate is required21:56
kmallocthen data migrate moves the data (fk) and stien doesn't care about old code.21:56
kmallocand old columns21:56
lbragstadwhen the nodes are down21:56
kmallocstien contract removes that old column21:57
lbragstadyep21:57
* lbragstad ventures into edge case land21:57
lbragstadlet's say i do all this without downtime21:57
kmallocwe are going to have to do a nullable allowed for the extra columns21:57
kmallocin rocky.21:57
kmallocbecause stien may or may not know about the columns21:57
lbragstadand a limit gets updated/created in the rocky migration21:57
lbragstadso the limit.registered_limit_id of that entry isn't populated21:58
kmallocright so21:58
lbragstadand then i upgrade to stein21:58
kmallocin rocky, registered_limit_id is nullable21:58
kmallocin rocky service_id, resource_name, region_id are all nullable21:58
kmalloc(code enforcement on if they are populated)21:58
kmallocin stien we drop service_id, resource_name, region_id, and make registered_limit_id not nullable21:59
kmallocstein*21:59
*** lifeless has joined #openstack-keystone21:59
*** lifeless_ has quit IRC21:59
lbragstadwhat happens if a limit sneaks in after the migration in rocky has been run to populate limit.registered_limit_ids?22:00
lbragstadon an older node?22:00
lbragstadlike a Queens node accepts the POST /v3/limits requests22:00
kmallocisn't migrate done when all nodes are rocky?22:01
lbragstadnope22:01
kmalloc*facepalm*22:01
lbragstadhttps://docs.openstack.org/keystone/latest/admin/identity-upgrading.html22:01
kmallocwe should fix that22:01
kmallocthat is bad.22:01
lbragstadhow do we fix that though?22:01
lbragstadyou need a place to transition things?22:01
kmallocmigrate happens once all nodes are on new code22:01
kmallocdata-migration22:01
kmallocexpand, upgrade code, migrate data, contract22:01
kmalloccode can run in n-1 scenario22:01
kmallocfor the schema22:02
lbragstadbut can you run new code if the data hasn't been migrated yet?22:02
kmallocapp code is smart22:02
lbragstadmmm i know where this is going22:02
lbragstadyou're going to make new code check all the places22:02
kmalloclbragstad: https://github.com/openstack/keystone/blob/03a616d1bf5715ac74756f2cb3aec1f09352de81/keystone/identity/backends/sql_model.py#L104-L11222:02
kmallocas an example22:03
kmallocpassword_hash is the new column22:03
kmallocpassword is the old22:03
kmalloccheck for the new, if not there, do the thing with the old22:03
kmallocwrites write to both locations [in password case, because of security, we had a config option for that]22:03
kmallocthis is not security related, so you can just blindly write to both locations with the relevant data22:04
kmallocmigrate is run when the code is 100% on <new>22:04
lbragstadyeah22:05
*** jrist has quit IRC22:05
lbragstadanother vesrion of that would be to have the new code check the old location22:05
lbragstadand make things consistent22:05
kmallocexcept you still need to make sure in Stein all data is in the new place22:06
lbragstadwhich would allow you to have old and new nodes running during migrateion22:06
kmallocin rocky, code does look in both places22:06
lbragstadyep22:06
kmallocbut migrate still has to occur.22:06
lbragstadmmmm22:07
lbragstaddamn22:07
lbragstadnevermind22:07
lbragstadfi - data problems are hard22:07
*** r-daneel has quit IRC22:07
kmallocyep22:08
openstackgerritMerged openstack/keystone master: Add policy for limit model protection  https://review.openstack.org/56271422:10
*** rcernin has joined #openstack-keystone22:12
*** r-daneel has joined #openstack-keystone22:16
lbragstadthat's going to be a total pain22:18
lbragstadwe're going to either need to change our upgrade process to make operators get all nodes on the new release before running migrate... or we break our rules and perform the migration in contract22:19
*** blake has quit IRC22:19
*** blake has joined #openstack-keystone22:20
kmalloci've done that a couple times22:23
kmallocbecause it was the only way to avoid breaking rules22:23
kmallocit sucked22:23
lbragstadif i think about it, it would easier to do that then change our upgrade process which likely breaks upgrade automation for operators :(22:23
*** blake has quit IRC22:24
kmallocwe should still change our upgrade process22:24
kmalloceven if it isn't this cycle22:25
kmalloctelegraph "we're changing to X"22:25
openstackgerritMorgan Fainberg proposed openstack/keystone master: Migrate all password hashes to the new location if needed  https://review.openstack.org/57666022:29
kmalloclbragstad: ^ should be an easy review22:29
lbragstadack22:29
kmalloclbragstad: that does the lifting of moving password to password_hash if password_hash is none [cleanup]22:30
kmallocand then in S, we can contract and drop the password.password column22:30
*** mvenesio has joined #openstack-keystone22:30
kmalloc:)22:30
kmalloclocal test succeeded, we'll see what zuul says22:30
*** felipemonteiro has joined #openstack-keystone22:42
*** dklyle has joined #openstack-keystone22:43
*** spilla has quit IRC22:43
*** david-lyle has joined #openstack-keystone22:45
*** blake has joined #openstack-keystone22:52
*** david-lyle has quit IRC23:04
lbragstadattempted to summarize the migrations https://etherpad.openstack.org/p/keystone-unified-limit-migration-notepad23:05
*** felipemonteiro has quit IRC23:12
*** felipemonteiro has joined #openstack-keystone23:13
*** jmlowe_ has joined #openstack-keystone23:14
*** jmlowe has quit IRC23:15
*** boris_42_ has quit IRC23:21
*** r-daneel has quit IRC23:37
*** lifeless_ has joined #openstack-keystone23:41
*** lifeless has quit IRC23:42
*** tosky has quit IRC23:45
*** r-daneel has joined #openstack-keystone23:50
*** r-daneel has quit IRC23:51

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!