Tuesday, 2017-03-28

eandersson_Having some weird behavior with keystoneauth1 and nova/compute. It's stuck at POST: https://keystone:35357/v3/auth/tokens for about 30 seconds00:11
eandersson_only when creating the os image (downloading & creating)00:11
*** thorst has joined #openstack-keystone00:12
*** thorst has quit IRC00:13
dstanekeandersson_: do you see any keystone logging?00:19
*** oomichi has quit IRC00:19
eandersson_nothing out of the ordinary - I don't have debug logs enabled unfortunately00:19
eandersson_I don't think this is a keystone issue though00:20
eandersson_requests are all successful00:20
eandersson_I added some logging here (before/after) the self.session.request()00:21
eandersson_https://github.com/openstack/keystoneauth/blob/stable/mitaka/keystoneauth1/session.py#L50300:21
dstanekeandersson_: how do you know that it's that post that is slow?00:21
eandersson_http://paste.openstack.org/show/aKgvCZKBbhsdLv5oIMSc/00:21
eandersson_That is the result00:21
eandersson_No other keystone calls are slow, and this happens while the image is downloaded from Glance, so I rather believe that it's some sort of threading issue.00:23
*** oomichi has joined #openstack-keystone00:23
dstanekeandersson_: it could be.. does the client use thread? i haven't looked into it that deeply00:25
eandersson_Yea - it's a threaded call00:25
eandersson_It's trying to create ports while creating/downloading the image.00:25
dstanekeandersson_: you could look at the apache logs for keystone to see if that POST succeeds before the client logs it00:25
eandersson_Good idea00:26
*** lucasxu has joined #openstack-keystone00:26
openstackgerritRichard Avelar proposed openstack/keystone master: Validate rolling upgrade is run in order  https://review.openstack.org/43744100:32
*** adrian_otto has quit IRC00:33
dstanekok, back to messing with the tests again00:34
*** catintheroof has joined #openstack-keystone00:35
*** thorst has joined #openstack-keystone00:43
*** nkinder has joined #openstack-keystone00:45
*** gyee has quit IRC00:47
eandersson_dstanek: looks like keystone answers as expected, but for some reason keystoneauth/requests keeps blocking for 30s00:52
eandersson_and when neutron finally can make it's call it's hitting the timeout that is currently set to 30s00:52
*** tovin07 has joined #openstack-keystone00:52
*** jamielennox is now known as jamielennox|away00:54
eandersson_It's such an odd issue... and for some reason the puppet module sets the requests timeout to 30s for neutron (but now where else?!)00:54
eandersson_https://github.com/openstack/puppet-nova/blob/master/manifests/network/neutron.pp#L2100:54
dstanekeandersson_: are you able to come up with a minimal command line example that i could debug?00:56
eandersson_Unfortunately not00:57
dstanekit is only through puppet?00:58
eandersson_Nah - so this is happening when I am creating a VM with an image that is large (e.g. Windows image).00:58
dstanekpuppet and i are no longer on speaking terms00:58
eandersson_hah00:58
*** jamielennox|away is now known as jamielennox00:59
dstanekeandersson_: i'll play around in a little bit and see it i can replicate00:59
eandersson_Thanks - sorry know it's an odd one00:59
eandersson_Probably not related to Keystone, or at the very least it seems more likely to be a threading issue.01:00
dstanekeandersson_: yeah, but it's got me interested :-)01:00
eandersson_https://github.com/openstack/nova/blob/stable/mitaka/nova/network/neutronv2/api.py#L25601:01
openstackgerritAnthony Washington proposed openstack/keystone master: Move user policies to DocumentedRuleDefault  https://review.openstack.org/44924001:01
eandersson_That is the api call that tries to make the two calls I posted earlier.01:02
*** thorst has quit IRC01:02
openstackgerritAnthony Washington proposed openstack/keystone master: Move identity provider to DocumentedRuleDefault  https://review.openstack.org/44927501:02
eandersson_Which originates from this call https://github.com/openstack/nova/blob/stable/mitaka/nova/compute/manager.py#L154501:03
*** namnh has joined #openstack-keystone01:03
*** liujiong has joined #openstack-keystone01:08
openstackgerritAnthony Washington proposed openstack/keystone master: Move endpoint group to DocumentedRuleDefault  https://review.openstack.org/44927301:09
*** thorst has joined #openstack-keystone01:11
*** thorst has quit IRC01:11
openstackgerritAnthony Washington proposed openstack/keystone master: Move implied role policies to DocumentedRuleDefault  https://review.openstack.org/44924601:12
*** markvoelker has quit IRC01:13
*** markvoelker has joined #openstack-keystone01:14
openstackgerritAnthony Washington proposed openstack/keystone master: Move auth to DocumentedRuleDefault  https://review.openstack.org/44933601:14
openstackgerritAnthony Washington proposed openstack/keystone master: Move policy association to DocumentedRuleDefault  https://review.openstack.org/44934401:19
*** knangia has quit IRC01:21
openstackgerritAnthony Washington proposed openstack/keystone master: Move group policies to DocumentedRuleDefault  https://review.openstack.org/44923701:23
openstackgerritAnthony Washington proposed openstack/keystone master: Move region policies to DocumentedRuleDefault  https://review.openstack.org/44921301:24
openstackgerritAnthony Washington proposed openstack/keystone master: Move trust to DocumentedRuleDefault  https://review.openstack.org/44927801:24
*** timburke has quit IRC01:25
*** AndyWojo has quit IRC01:25
*** hugokuo has quit IRC01:25
openstackgerritAnthony Washington proposed openstack/keystone master: Move role policies to DocumentedRuleDefault  https://review.openstack.org/44925101:25
openstackgerritAnthony Washington proposed openstack/keystone master: Move token revocation to DocumentedRuleDefault  https://review.openstack.org/44925501:27
*** AndyWojo has joined #openstack-keystone01:27
*** timburke has joined #openstack-keystone01:28
*** knangia has joined #openstack-keystone01:28
openstackgerritAnthony Washington proposed openstack/keystone master: Move credential policies to DocumentedRuleDefault  https://review.openstack.org/44923301:28
openstackgerritAnthony Washington proposed openstack/keystone master: Move ec2 credential policies to DocumentedRuleDefault  https://review.openstack.org/44923501:28
*** hugokuo has joined #openstack-keystone01:28
*** tovin07 has quit IRC01:28
openstackgerritAnthony Washington proposed openstack/keystone master: Move mapping to DocumentedRuleDefault  https://review.openstack.org/44934101:29
*** tovin07 has joined #openstack-keystone01:29
openstackgerritAnthony Washington proposed openstack/keystone master: Move protocol to DocumentedRuleDefault  https://review.openstack.org/44934501:30
openstackgerritAnthony Washington proposed openstack/keystone master: Move service provider to DocumentedRuleDefault  https://review.openstack.org/44934701:32
openstackgerritAnthony Washington proposed openstack/keystone master: Move consumer to DocumentedRuleDefault  https://review.openstack.org/44926901:32
openstackgerritAnthony Washington proposed openstack/keystone master: Move project endpoint to DocumentedRuleDefault  https://review.openstack.org/44927601:33
openstackgerritAnthony Washington proposed openstack/keystone master: Move role assignment to DocumentedRuleDefault  https://review.openstack.org/44925301:33
openstackgerritAnthony Washington proposed openstack/keystone master: Move grant policies to DocumentedRuleDefault  https://review.openstack.org/44924401:33
openstackgerritAnthony Washington proposed openstack/keystone master: Move service policies to DocumentedRuleDefault  https://review.openstack.org/44921401:33
openstackgerritAnthony Washington proposed openstack/keystone master: Move domain policies to DocumentedRuleDefault  https://review.openstack.org/44923401:34
openstackgerritAnthony Washington proposed openstack/keystone master: Move project policies to DocumentedRuleDefault  https://review.openstack.org/44923801:34
openstackgerritAnthony Washington proposed openstack/keystone master: Move endpoint policies to DocumentedRuleDefault  https://review.openstack.org/44921201:35
*** catintheroof has quit IRC01:41
*** markvoelker has quit IRC01:42
*** markvoelker has joined #openstack-keystone01:44
*** zhurong has joined #openstack-keystone01:53
*** nkinder has quit IRC01:53
*** agrebennikov has joined #openstack-keystone01:56
*** nicolasbock has quit IRC02:00
openstackgerritzhichao zhu proposed openstack/keystone master: Fix some reST field lists in docstrings  https://review.openstack.org/44989202:03
*** agrebennikov has quit IRC02:06
*** agrebennikov has joined #openstack-keystone02:07
*** thorst has joined #openstack-keystone02:12
*** Shunli has joined #openstack-keystone02:14
*** thorst has quit IRC02:17
*** ravelar has quit IRC02:18
*** oomichi has quit IRC02:28
*** oomichi has joined #openstack-keystone02:32
*** agrebennikov has quit IRC02:36
*** oomichi has quit IRC02:39
*** oomichi has joined #openstack-keystone02:42
*** markvoelker has quit IRC02:54
*** markvoelker has joined #openstack-keystone02:54
openstackgerritzhichao zhu proposed openstack/keystone master: Fix some reST field lists in docstrings  https://review.openstack.org/44989203:05
*** thorst has joined #openstack-keystone03:13
*** aojea has joined #openstack-keystone03:13
*** aojea has quit IRC03:18
*** rderose has quit IRC03:27
*** thorst has quit IRC03:33
*** lxnch has joined #openstack-keystone03:33
*** links has joined #openstack-keystone03:36
*** lxnch has quit IRC03:39
*** lxnch has joined #openstack-keystone03:39
*** prashkre has joined #openstack-keystone03:42
*** tovin07 has quit IRC03:53
*** zhurong has quit IRC03:57
*** jamielennox is now known as jamielennox|away04:09
*** prashkre has quit IRC04:19
*** prashkre has joined #openstack-keystone04:19
*** markvoelker has quit IRC04:21
*** markvoelker has joined #openstack-keystone04:23
*** prashkre has quit IRC04:29
*** thorst has joined #openstack-keystone04:30
*** dikonoor has joined #openstack-keystone04:30
*** jamielennox|away is now known as jamielennox04:30
*** thorst has quit IRC04:34
*** MarkMielke has quit IRC04:35
*** lucasxu has quit IRC04:35
*** nkinder has joined #openstack-keystone04:35
*** lucasxu has joined #openstack-keystone04:41
*** dikonoor has quit IRC04:42
*** dikonoor has joined #openstack-keystone04:43
*** lucasxu has quit IRC04:51
*** zhurong has joined #openstack-keystone04:56
*** dikonoor has quit IRC04:57
*** markvoelker has quit IRC05:04
*** dikonoor has joined #openstack-keystone05:08
*** lucasxu has joined #openstack-keystone05:12
*** lamt has joined #openstack-keystone05:19
*** aojea has joined #openstack-keystone05:26
*** aojea has quit IRC05:27
*** aojea has joined #openstack-keystone05:28
*** aojea has quit IRC05:32
*** dikonoo has joined #openstack-keystone05:36
*** dikonoor has quit IRC05:37
*** rcernin has joined #openstack-keystone05:38
*** bjornar_ has joined #openstack-keystone05:39
*** lucasxu has quit IRC05:42
*** richm has quit IRC05:43
*** aojea has joined #openstack-keystone05:52
*** bjornar_ has quit IRC05:53
*** lamt has quit IRC06:01
*** aojea has quit IRC06:03
*** lamt has joined #openstack-keystone06:05
*** prashkre has joined #openstack-keystone06:07
*** kfox1111 has quit IRC06:12
*** lamt has quit IRC06:19
*** thorst has joined #openstack-keystone06:31
*** thorst has quit IRC06:35
openstackgerritwangxiyuan proposed openstack/python-keystoneclient master: Init include_catalog for V3 client  https://review.openstack.org/44843706:36
*** jamielennox is now known as jamielennox|away06:36
*** lamt has joined #openstack-keystone06:42
*** lamt has quit IRC06:46
*** oomichi has quit IRC06:48
*** Aqsa has joined #openstack-keystone06:48
*** lamt has joined #openstack-keystone06:51
*** oomichi has joined #openstack-keystone06:52
*** tovin07 has joined #openstack-keystone06:52
*** mtreinish has quit IRC06:57
*** mtreinish has joined #openstack-keystone06:58
*** bjornar_ has joined #openstack-keystone07:05
*** lamt has quit IRC07:10
*** pcaruana has joined #openstack-keystone07:13
*** aojea has joined #openstack-keystone07:17
*** tesseract has joined #openstack-keystone07:18
*** jamielennox|away is now known as jamielennox07:21
*** dikonoo has quit IRC07:21
*** jaosorior has joined #openstack-keystone07:29
*** rcernin has quit IRC07:29
*** rcernin has joined #openstack-keystone07:47
*** prashkre has quit IRC07:48
*** aojea_ has joined #openstack-keystone07:55
*** aojea has quit IRC07:58
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-keystone08:00
*** dikonoo has joined #openstack-keystone08:03
*** rcernin has quit IRC08:06
*** rcernin has joined #openstack-keystone08:07
*** rcernin has quit IRC08:10
*** rcernin has joined #openstack-keystone08:10
*** Shunli has quit IRC08:17
*** mvk has quit IRC08:17
*** prashkre has joined #openstack-keystone08:19
*** dgonzalez has quit IRC08:24
*** dgonzalez has joined #openstack-keystone08:30
*** thorst has joined #openstack-keystone08:33
*** yuvalb has quit IRC08:36
*** fmarco76 has joined #openstack-keystone08:43
*** fmarco76 has quit IRC08:43
*** rcernin has quit IRC08:46
*** rcernin has joined #openstack-keystone08:46
*** mvk has joined #openstack-keystone08:47
*** rcernin_ has joined #openstack-keystone08:48
*** thorst has quit IRC08:52
*** tovin07 has quit IRC08:55
*** Aurelgad1o is now known as Aurelgadjo08:58
*** Aqsa has quit IRC09:11
*** knangia has quit IRC09:21
*** thorst has joined #openstack-keystone09:47
*** Aqsa has joined #openstack-keystone09:54
*** nicolasbock has joined #openstack-keystone10:11
*** namnh has quit IRC10:14
*** richm has joined #openstack-keystone10:14
*** mvk has quit IRC10:16
*** liujiong has quit IRC10:19
*** thorst has quit IRC10:19
*** zhurong has quit IRC10:20
*** zhurong has joined #openstack-keystone10:28
*** mvk has joined #openstack-keystone10:31
*** dikonoo has quit IRC10:49
*** raildo has joined #openstack-keystone10:59
*** oomichi has quit IRC11:09
*** oomichi has joined #openstack-keystone11:13
*** dikonoo has joined #openstack-keystone11:17
*** thorst has joined #openstack-keystone11:20
*** dikonoo has quit IRC11:24
*** zhurong has quit IRC11:25
*** thorst has quit IRC11:25
*** ma9_ has joined #openstack-keystone11:27
*** ma9_ has quit IRC11:29
*** jdennis1 has quit IRC11:31
*** jdennis has joined #openstack-keystone11:31
*** ma9_ has joined #openstack-keystone11:32
*** openstackgerrit has quit IRC11:33
*** ma9_ has quit IRC11:37
*** erlon has joined #openstack-keystone11:38
*** ma9_ has joined #openstack-keystone11:41
*** thorst has joined #openstack-keystone11:45
*** prashkre has quit IRC11:48
*** agrebennikov has joined #openstack-keystone11:57
*** ma9_ has quit IRC12:15
*** chlong has joined #openstack-keystone12:17
*** dave-mccowan has joined #openstack-keystone12:25
*** pnavarro has joined #openstack-keystone12:28
*** dikonoo has joined #openstack-keystone12:32
*** ma9_ has joined #openstack-keystone12:41
*** edmondsw has joined #openstack-keystone12:45
*** spilla has joined #openstack-keystone12:48
dstanekehlo keystone12:49
samueldmqdstanek: o/12:58
*** openstackgerrit has joined #openstack-keystone13:04
openstackgerritRodrigo Duarte proposed openstack/keystone master: Drop federated_user table foreign keys  https://review.openstack.org/44550513:04
*** lamt has joined #openstack-keystone13:14
lbragstado/13:18
rodrigodslbragstad, saw we have a huge stack of policy in code reviews13:19
rodrigodsi didn't review any of them yet, is there an order or something?13:19
*** links has quit IRC13:22
lbragstadrodrigods nope, they are all dependent on the final policy-in-code patch13:26
lbragstadbut once that merges, then they can all be rebased onto master13:26
lbragstadand reviewed in parallel13:26
rodrigodslbragstad, cool13:26
*** agrebennikov has quit IRC13:31
*** knangia has joined #openstack-keystone13:46
*** ma9_ has quit IRC13:46
*** markvoelker has joined #openstack-keystone13:49
openstackgerritAnthony Washington proposed openstack/keystone master: Move service provider to DocumentedRuleDefault  https://review.openstack.org/44934713:49
openstackgerritAnthony Washington proposed openstack/keystone master: Move group policies to DocumentedRuleDefault  https://review.openstack.org/44923713:52
*** lamt has quit IRC13:52
*** ma9_ has joined #openstack-keystone13:53
openstackgerritAnthony Washington proposed openstack/keystone master: Move user policies to DocumentedRuleDefault  https://review.openstack.org/44924013:54
*** agrebennikov has joined #openstack-keystone14:00
*** jlopezgu_ has joined #openstack-keystone14:01
*** edmondsw has quit IRC14:02
*** ma9_ has quit IRC14:03
*** lucasxu has joined #openstack-keystone14:04
*** edmondsw has joined #openstack-keystone14:05
*** ma9_ has joined #openstack-keystone14:10
*** jaosorior has quit IRC14:12
*** markvoelker has quit IRC14:17
openstackgerritMerged openstack/keystone master: Add a note to db_sync configuration section  https://review.openstack.org/44974414:29
*** sjain has joined #openstack-keystone14:32
openstackgerritLance Bragstad proposed openstack/keystone master: Remove policy file from source and refactor tests  https://review.openstack.org/44967514:37
*** edmondsw has quit IRC14:38
*** agrebennikov has quit IRC14:40
*** edmondsw has joined #openstack-keystone14:41
*** ravelar has joined #openstack-keystone14:41
*** rderose has joined #openstack-keystone14:45
openstackgerritMerged openstack/keystone master: Move release note from /keystone/releasenotes to /releasenotes  https://review.openstack.org/44979814:46
*** lamt has joined #openstack-keystone14:47
openstackgerritRichard Avelar proposed openstack/keystone master: Validate rolling upgrade is run in order  https://review.openstack.org/43744114:51
*** agrebennikov has joined #openstack-keystone14:53
openstackgerritRichard Avelar proposed openstack/keystone master: Validate rolling upgrade is run in order  https://review.openstack.org/43744114:56
knikollao/14:58
*** ma9_1 has joined #openstack-keystone15:02
*** ma9_ has quit IRC15:04
lbragstado/15:05
lbragstadravelar a published an edit on https://review.openstack.org/#/c/449675/15:05
lbragstadravelar which is now dependent on https://review.openstack.org/#/c/450818/15:05
ravelarlbragstad sweet15:06
ravelarsounds good :)15:06
ravelarthanks15:06
*** adrian_otto has joined #openstack-keystone15:06
lbragstadravelar i tested it locally with a fresh devstack and all the identity tempest tests passed15:06
lbragstadso - i'm hoping that's a good sign15:07
lbragstadravelar http://logs.openstack.org/75/449675/4/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/d78fd47/logs/devstacklog.txt.gz#_2017-03-27_18_29_25_334 was the thing causing you're patch to fail before15:07
ravelarlbragstad nice!15:14
*** lamt has quit IRC15:17
*** lamt has joined #openstack-keystone15:20
openstackgerritRichard Avelar proposed openstack/keystone master: Remove policy file from source and refactor tests  https://review.openstack.org/44967515:22
lbragstadravelar heads up - i left some comments on patch set 5 ^15:29
ravelarlbragstad ahh, see it, will do!15:29
*** rcernin has quit IRC15:31
*** rcernin_ has quit IRC15:31
*** sjain has quit IRC15:33
*** dikonoo has quit IRC15:41
openstackgerritAnthony Washington proposed openstack/keystone master: Move domain config to DocumentedRuleDefault  https://review.openstack.org/44933715:41
*** lamt has quit IRC15:41
openstackgerritDolph Mathews proposed openstack/keystone master: Doc db_sync --expand incurring downtime in upgrades to Newton  https://review.openstack.org/45086315:44
openstackgerritAnthony Washington proposed openstack/keystone master: Move trust to DocumentedRuleDefault  https://review.openstack.org/44927815:49
openstackgerritAnthony Washington proposed openstack/keystone master: Move trust to DocumentedRuleDefault  https://review.openstack.org/44927815:50
*** lamt has joined #openstack-keystone15:56
openstackgerritRichard Avelar proposed openstack/keystone master: Remove policy file from source and refactor tests  https://review.openstack.org/44967515:59
*** prashkre has joined #openstack-keystone15:59
*** aojea_ has quit IRC16:00
*** Aqsa has quit IRC16:00
prashkrelbragstad: Hi Lance. need a workflow on this https://review.openstack.org/#/c/450027/ to back port the fix to stable/ocata. could you please help me on this?16:03
*** erlon has quit IRC16:05
openstackgerritAnthony Washington proposed openstack/keystone master: Move policy policies to DocumentedRuleDefault  https://review.openstack.org/44924816:18
*** ma9_1 has quit IRC16:18
openstackgerritAnthony Washington proposed openstack/keystone master: Move access token to DocumentedRuleDefault  https://review.openstack.org/44926516:18
openstackgerritAnthony Washington proposed openstack/keystone master: Move revoke events to DocumentedRuleDefault  https://review.openstack.org/44934616:18
openstackgerritAnthony Washington proposed openstack/keystone master: Move implied role policies to DocumentedRuleDefault  https://review.openstack.org/44924616:18
openstackgerritAnthony Washington proposed openstack/keystone master: Move policy association to DocumentedRuleDefault  https://review.openstack.org/44934416:18
openstackgerritAnthony Washington proposed openstack/keystone master: Move endpoint group to DocumentedRuleDefault  https://review.openstack.org/44927316:19
openstackgerritAnthony Washington proposed openstack/keystone master: Move auth to DocumentedRuleDefault  https://review.openstack.org/44933616:19
openstackgerritAnthony Washington proposed openstack/keystone master: Move ec2 credential policies to DocumentedRuleDefault  https://review.openstack.org/44923516:19
lbragstadprashkre yeah - i can review16:19
openstackgerritAnthony Washington proposed openstack/keystone master: Move region policies to DocumentedRuleDefault  https://review.openstack.org/44921316:19
openstackgerritAnthony Washington proposed openstack/keystone master: Move credential policies to DocumentedRuleDefault  https://review.openstack.org/44923316:19
openstackgerritAnthony Washington proposed openstack/keystone master: Move mapping to DocumentedRuleDefault  https://review.openstack.org/44934116:19
openstackgerritAnthony Washington proposed openstack/keystone master: Move protocol to DocumentedRuleDefault  https://review.openstack.org/44934516:19
openstackgerritAnthony Washington proposed openstack/keystone master: Move token revocation to DocumentedRuleDefault  https://review.openstack.org/44925516:19
openstackgerritAnthony Washington proposed openstack/keystone master: Move endpoint policies to DocumentedRuleDefault  https://review.openstack.org/44921216:19
openstackgerritAnthony Washington proposed openstack/keystone master: Move role assignment to DocumentedRuleDefault  https://review.openstack.org/44925316:19
openstackgerritAnthony Washington proposed openstack/keystone master: Move role policies to DocumentedRuleDefault  https://review.openstack.org/44925116:20
openstackgerritAnthony Washington proposed openstack/keystone master: Move project policies to DocumentedRuleDefault  https://review.openstack.org/44923816:20
openstackgerritAnthony Washington proposed openstack/keystone master: Move service policies to DocumentedRuleDefault  https://review.openstack.org/44921416:20
openstackgerritAnthony Washington proposed openstack/keystone master: Move domain policies to DocumentedRuleDefault  https://review.openstack.org/44923416:20
openstackgerritAnthony Washington proposed openstack/keystone master: Move consumer to DocumentedRuleDefault  https://review.openstack.org/44926916:20
openstackgerritAnthony Washington proposed openstack/keystone master: Move grant policies to DocumentedRuleDefault  https://review.openstack.org/44924416:20
openstackgerritAnthony Washington proposed openstack/keystone master: Move project endpoint to DocumentedRuleDefault  https://review.openstack.org/44927616:20
openstackgerritAnthony Washington proposed openstack/keystone master: Move identity provider to DocumentedRuleDefault  https://review.openstack.org/44927516:20
openstackgerritAnthony Washington proposed openstack/keystone master: Move service provider to DocumentedRuleDefault  https://review.openstack.org/44934716:20
openstackgerritAnthony Washington proposed openstack/keystone master: Move group policies to DocumentedRuleDefault  https://review.openstack.org/44923716:20
openstackgerritAnthony Washington proposed openstack/keystone master: Move domain config to DocumentedRuleDefault  https://review.openstack.org/44933716:20
dstanekantwash: i thought you said those per safe to review :-)16:21
prashkrelbragstad: Thank you :)16:21
dolphmi was just wondering the same16:21
dstaneks/per/were/16:21
antwashdstanek: haha, just rebasing on top of master :)16:21
lbragstadantwash so those are all based individually on master?16:21
antwashlbragstad: yeah master is "9034755743875de39363dbdcd35477e82e37ea1b"16:22
dstanekantwash: unless there is a conflict don't worry about that16:22
dstanekit's just noise16:22
dstanekyou're harshing my buzz16:22
*** pcaruana has quit IRC16:26
lbragstaddolphm in case you're interested in doing a stable review - https://review.openstack.org/#/c/450027/16:29
*** lucasxu has quit IRC16:31
*** chlong has quit IRC16:34
antwashdstanek : about this bug, what should we mark it as https://bugs.launchpad.net/keystone/+bug/167467616:38
openstackLaunchpad bug 1674676 in OpenStack Identity (keystone) "The URL listed against the details of identity resources returns 404 Not Found error" [Medium,Confirmed] - Assigned to Anthony Washington (anthony-washington)16:38
*** lucasxu has joined #openstack-keystone16:40
knikollajust to make sure before i answer an email, keystone does not support rate limiting the api right?16:44
*** bjornar_ has quit IRC16:46
dstanekantwash: closed it16:48
*** chlong has joined #openstack-keystone16:49
dstanekantwash: i was expecting the person in here asking about it to close it :-( i thought it would be a good new comer exer cise16:49
antwashdstanek: haha, yeah it would be -- well at least bugs list is getting shorted :)16:50
antwashshorter lol16:50
dstanekantwash: my goal is for it to be in tehe 70s by the end of the week16:51
*** lamt has quit IRC16:52
knikollaantwash: RIP openstack CI with all those patches to check, haha.16:52
antwashknikolla: haha, I was just thinking the same thing!16:53
dstanekknikolla: keystone doesn't do any kind of rate limiting16:57
*** lamt has joined #openstack-keystone16:58
knikolladstanek: thanks, thought so. codesearch came up empty.16:59
openstackgerritKristi Nikolla proposed openstack/keystone master: Differentiate between dpkg and rpm for libssl-dev  https://review.openstack.org/45089117:06
*** lucasxu has quit IRC17:09
*** mvk has quit IRC17:13
*** MasterOfBugs has joined #openstack-keystone17:14
*** pramodrj07 has joined #openstack-keystone17:14
*** lucasxu has joined #openstack-keystone17:15
*** lucasxu has quit IRC17:15
*** lucasxu has joined #openstack-keystone17:16
*** lucasxu has quit IRC17:18
*** lucasxu has joined #openstack-keystone17:19
*** lucasxu has quit IRC17:20
*** lamt has quit IRC17:27
*** prashkre has quit IRC17:32
*** toddnni has quit IRC17:35
*** adrian_otto has quit IRC17:37
*** adrian_otto has joined #openstack-keystone17:40
lbragstadjust a friendly reminder that the weekly keystone meeting will be starting in #openstack-meeting in 20 minutes17:42
lbragstadfyi we're going to do a roll call for the next couple weeks so that we can get an accurate attendee list17:42
lbragstadstarting today17:42
*** toddnni has joined #openstack-keystone17:44
*** aojea has joined #openstack-keystone17:45
notmorganlbragstad: i might miss the role call(s), but i usually at least lurk most of the meetings17:47
notmorganlbragstad: fyi17:47
lbragstadnotmorgan ack - i'll include you in roll call today17:47
lbragstadnotmorgan thanks for the heads up17:48
*** bjornar_ has joined #openstack-keystone17:48
*** adrian_otto has quit IRC17:50
*** mvk has joined #openstack-keystone17:52
lbragstadantwash quick note on the commit message that can probably be applied to the rest of the patches - https://review.openstack.org/#/c/449237/5//COMMIT_MSG17:54
lbragstadantwash i didn't want to go through and apply the same comment to all your patches17:55
*** henrynash has joined #openstack-keystone18:02
*** adrian_otto has joined #openstack-keystone18:06
*** tesseract has quit IRC18:06
*** henrynash has quit IRC18:07
*** henrynash has joined #openstack-keystone18:08
*** henrynash has quit IRC18:11
*** Aqsa has joined #openstack-keystone18:11
*** henrynash has joined #openstack-keystone18:12
*** prashkre has joined #openstack-keystone18:18
*** lamt has joined #openstack-keystone18:20
*** adrian_otto has quit IRC18:21
*** gyee has joined #openstack-keystone18:26
*** aojea has quit IRC18:29
*** aojea has joined #openstack-keystone18:29
*** markvoelker has joined #openstack-keystone18:32
*** aojea has quit IRC18:34
*** eandersson_ is now known as eandersson18:35
openstackgerritMohammed Naser proposed openstack/keystonemiddleware master: Add python-memcached to requirements  https://review.openstack.org/28531518:43
*** lamt has quit IRC18:49
*** lamt has joined #openstack-keystone18:55
openstackgerritRichard Avelar proposed openstack/keystone master: Validate rolling upgrade is run in order  https://review.openstack.org/43744118:56
*** markvoelker has quit IRC18:57
lbragstaddstanek breton knikolla rderose if we patch trusts to honor the ephemeral group memberships of a token, is there any additional clean up required by the operators?19:02
dstaneklbragstad: yes, i think they'd still need to tell use when that user changes19:02
knikollathe trust cleanup19:03
knikollawhen group changes19:03
dstanekno matter what that has to happen19:04
*** henrynash has quit IRC19:04
dstaneklbragstad: making that change means that people can use the trusts without manually assigning to groups19:04
*** ravelar has quit IRC19:04
knikollai feel making groups persistent is a better option. as that can be more easily cleaned up via automation. it's hard to keep track of what created which trust and why.19:05
knikollaops can just compare groups in idp and keystone and update.19:06
dstanekknikolla: fair point19:07
lbragstadwhat if trusts validated the groups in the token at usage time?19:08
dstaneklbragstad: i think that's actually what the problem is. it does19:09
dstanekbut federated users are never in the groups19:09
bretonlbragstad: no. But my operators were ok with manual clean-up if group membership is persistent19:10
lbragstaddstanek what if we made it so trusts pulled the ephemeral groups instead of looking for persistent ones?19:11
dstaneklbragstad: that what i'm advocating for. just don't check group membership for federated trusts19:12
lbragstadah19:12
*** chris_hultin|AWA is now known as chris_hultin19:12
*** Aqsa has quit IRC19:14
openstackgerritAnthony Washington proposed openstack/keystone master: Remove unnecessary processing when deleting grant.  https://review.openstack.org/45093819:21
*** henrynash has joined #openstack-keystone19:26
openstackgerritAnthony Washington proposed openstack/keystone master: Move endpoint policies to DocumentedRuleDefault  https://review.openstack.org/44921219:29
*** henrynash has quit IRC19:31
*** aojea has joined #openstack-keystone19:32
knikollalbragstad: are we still doing keystone office hours fridays?19:32
lbragstadknikolla yessir19:32
*** lamt has quit IRC19:32
knikollalbragstad: sweet!19:33
*** ravelar has joined #openstack-keystone19:35
*** lamt has joined #openstack-keystone19:39
*** chris_hultin is now known as chris_hultin|AWA19:42
*** adrian_otto has joined #openstack-keystone20:08
*** chris_hultin|AWA is now known as chris_hultin20:14
*** Darren has joined #openstack-keystone20:20
openstackgerritRichard Avelar proposed openstack/keystone master: Validate rolling upgrade is run in order  https://review.openstack.org/43744120:40
notmorgandstanek, lbragstad answered your comments on the password hashing CR20:40
lbragstadnotmorgan thanks20:40
notmorganit still needs a rebase and a little work. but take a look at answers and reply/comment if needed before i do that work20:41
*** chris_hultin is now known as chris_hultin|AWA20:44
*** Darren has quit IRC20:50
lbragstadthe TC group is asking for keystone feedback on https://review.openstack.org/#/c/44703120:54
dstaneknotmorgan: sure20:56
*** markvoelker has joined #openstack-keystone20:59
*** markvoelker has quit IRC21:04
*** henrynash has joined #openstack-keystone21:05
*** henrynash has quit IRC21:06
*** aojea has quit IRC21:07
dstaneklbragstad: that's neato21:07
lbragstaddstanek the resolution?21:08
dstaneklbragstad: yes21:09
lbragstaddstanek you're good with it?21:11
*** spilla has quit IRC21:12
dstaneklbragstad: yes, i like the delegation call out21:13
lbragstadyeah21:14
lbragstadit's detailed, but i wonder how much detail there is suppose to be (going back to sdague's comment)21:15
*** ravelar1 has joined #openstack-keystone21:15
lbragstadif it's a mission statement, I'm not sure how much "How" needs to be included, but it would be useful21:15
*** ravelar has quit IRC21:16
dstanek'how' meaning deletegation?21:16
openstackgerritDavid Stanek proposed openstack/keystone master: Remove orphaned AuthTestMixin from test_v3  https://review.openstack.org/45098421:18
openstackgerritDavid Stanek proposed openstack/keystone master: Remove decorator for asserting validation errors  https://review.openstack.org/45098521:18
openstackgerritDavid Stanek proposed openstack/keystone master: Remove orphaned _create_context test helper  https://review.openstack.org/45098621:18
openstackgerritDavid Stanek proposed openstack/keystone master: Remove conflict guards in load_fixtures  https://review.openstack.org/45098721:18
openstackgerritDavid Stanek proposed openstack/keystone master: Consolidate and cleanup test_backend_ldap setup  https://review.openstack.org/45098821:18
openstackgerritDavid Stanek proposed openstack/keystone master: Reduce fixutre setup in test_backend_ldap  https://review.openstack.org/45098921:18
openstackgerritDavid Stanek proposed openstack/keystone master: Remove create_container_group from tests  https://review.openstack.org/45099021:18
openstackgerritDavid Stanek proposed openstack/keystone master: Removed domain conflict guard in load_fixtures  https://review.openstack.org/45099121:18
openstackgerritDavid Stanek proposed openstack/keystone master: Replace wip with skip  https://review.openstack.org/45099221:18
dstaneklbragstad: i couldn't push all of them, but that's good for starters21:19
rodrigods^ wow21:20
dstanekrodrigods: i had a productive Sunday21:20
rodrigodsdstanek, that you can call productive, for sure21:21
lbragstaddstanek yeah21:24
lbragstaddstanek nice patches!21:24
dstanekhopefully i'll have some time this weekend to work on it a bit more21:26
lbragstaddstanek well - i appreciate you working on it during the weekend21:28
*** aojea has joined #openstack-keystone21:29
openstackgerritOpenStack Proposal Bot proposed openstack/keystone master: Updated from global requirements  https://review.openstack.org/45101421:29
openstackgerritOpenStack Proposal Bot proposed openstack/keystonemiddleware master: Updated from global requirements  https://review.openstack.org/43931821:29
*** prashkre has quit IRC21:30
lbragstadantwash around?21:34
antwashlbragstad: yeah I am, what's up?21:34
dstaneklbragstad: deleting code is like stress relief21:34
lbragstadantwash did you happen to see my comment here? https://review.openstack.org/#/c/449237/5//COMMIT_MSG21:35
lbragstaddstanek yeah it is21:35
* dstanek needs to get ready for soccer practice21:35
lbragstaddstanek i specifically get a weird amount of satisfaction out of deleting if statements21:35
lbragstadantwash i wanted to follow up directly on that comment instead of going through and leaving it on every proposed patch :)21:36
antwashlbragstad: I just read it now, so I need to modify the commit message for all the patches?21:36
lbragstadantwash if you have to spin new versions of the patches anyway I would21:37
lbragstadonly because it makes the commit message more descriptive of what the change is actually doing21:37
lbragstadantwash does the comment make sense?21:37
*** edmondsw has quit IRC21:42
*** edmondsw has joined #openstack-keystone21:42
*** raildo has quit IRC21:43
antwashlbragstad: yeah, but I wasn't planning on pushing up any new versions except for the user and trust one honestly21:44
*** edmondsw has quit IRC21:47
*** henrynash has joined #openstack-keystone21:48
*** aojea has quit IRC21:49
*** aojea has joined #openstack-keystone21:49
*** thorst has quit IRC21:51
*** adrian_otto has quit IRC21:53
*** aojea has quit IRC21:54
*** markvoelker has joined #openstack-keystone21:59
*** mnaser has joined #openstack-keystone22:03
mnaseris there a way to configure a default base set of roles for all users?22:03
mnaseraka heat_stack_owner for everyone, as an example22:03
*** bjornar_ has quit IRC22:03
*** markvoelker has quit IRC22:06
lbragstadmnaser sounds like you want a default role for a user22:07
mnaserlbragstad isnt the default _member_ in keystone now?22:07
mnaserthen of course projects want to have their own roles to allow usage of specific services :<22:07
lbragstadmnaser yeah - that's hard to manage without having control to lay down specific policy files for a service22:08
mnaserlbragstad so i'm back to either play with policy files (rather not, because i fear that i'll miss a change with an upgrade)22:09
mnaseror i guess just add all these roles to users22:09
lbragstadmnaser we're currently working on some policy specs to do two things22:09
lbragstad1.) migrate all default policies into code22:09
lbragstad(and register them like we do configuration)22:10
lbragstad2.) document policies better22:10
lbragstadboth are targeted to pike22:10
mnaseri love the effort on 1.  nova policy file is empty now, love that :>22:10
lbragstadmnaser ++ we're doing the same thing22:10
lbragstadmnaser so maintenance is hopefully a little bit better22:10
mnaseryeah it just becomes a dropin22:11
lbragstadhttps://github.com/openstack/keystone/blob/master/etc/policy.json22:11
mnaserbut that doesnt solve the issue for all the other projects22:11
mnaseroh sweet22:11
lbragstadour current policy file is empty - soon to be removed22:11
lbragstadmnaser right - that's another issue we still have to step up and address22:11
lbragstadmnaser which has to deal with policy administration22:11
mnaseryeah i remember reading policies in keystone thing a while back22:12
*** edmondsw has joined #openstack-keystone22:12
lbragstadmnaser do you remember which thing that was? dynamic policy?22:12
mnaseryes22:13
lbragstadmnaser if openstack provided a better, more granular, set of roles by default, would that help you?22:13
mnaserlbragstad 100%.  as an operator, as we add new services, we constantly have to figure out what new roles it wants (or choose to modify policy files, which we dont want)22:14
lbragstadmnaser or are you looking for true policy administration via an API?22:14
mnaserso we gotta retroactively go back to all tenants and add that role22:14
mnaser#1, policy admin via API would be more of a complication imho, dropping in a file via config mgmt is a lot easier than dealing with an api in provisioning/etc22:15
mnaserex: standarized "user" role across all projects, rather than (so far in my experience): swift wanting a role, heat wanting a role, barbican wanting a role22:15
lbragstadsure22:15
lbragstadmnaser our first couple steps towards doing that were 1.) move policy into code 2.) document policy so it can be generated on the fly and take into account overrides22:16
*** edmondsw has quit IRC22:16
lbragstadbut we've been entertaining the idea of doing something along the lines of https://review.openstack.org/#/c/428454/22:16
lbragstadmnaser see lines 40 - 63 - https://review.openstack.org/#/c/428454/6/specs/keystone/ongoing/richer-policy-by-default.rst22:17
* mnaser reads22:17
lbragstadgranted - this is something we're trying to do in lock-step with nova, who has a similar spec proposed https://review.openstack.org/#/c/427872/22:18
mnaserlbragstad i like this a lot22:19
mnaserthat's very useful and gives a good set of starting points for new deployments22:19
lbragstadmnaser the overall idea would be that all projects (wanting to adhere to basic RBAC) would have to define those roles and map them to the operations for that project22:19
lbragstadso - out of the box, you'd have a little more flexibility as an operator to work with with respect to roles22:20
lbragstadi.e. being able to grant someone a role to be a project administrator and not global admin22:21
mnaserlbragstad id be on board with this.  hell even better if those basic roles get installed in the keystone bootstrap process22:21
lbragstadmnaser ++22:21
lbragstadmnaser i'm not totally sure this is going to be something we will be able to commit to for pike22:21
mnaserunderstandable22:21
lbragstadmnaser but there are plenty of interested parties willing to share the work22:21
mnaserits a big cross project cooperation for it to be useful22:21
lbragstadexactly22:22
lbragstadits a problem that spans across OpenStack, and those kinds of problems certainly don't get fixed over night :)22:22
mnaseran interesting issue is22:22
mnaserhow these things would get handled in upgrades22:22
lbragstadmnaser what specifically during the upgrade?22:23
mnaseri.e. just upgraded my keystone, everything is running smooth, upgrade nova, my new roles didnt get created22:23
lbragstadmnaser  ah22:23
mnasernova maps to role names that are defined in that spec22:23
mnasereverything stops (unless we start talking about "deprecated" role names, which can be very confusing for someone who says "i wanna just use my own system")22:23
mnasermaybe a keystone_authtoken option *shrug*22:24
lbragstadso during an upgrade if nova adds a new role - then something would have to happen during the upgrade process to create that role in keystone22:24
lbragstadbut from a definition standpoint, it would be handled in the code22:25
mnaseryeah but then you got the cases where $user has been running cloud since liberty with policy.json in place22:25
lbragstadmeaning you don't have to manually add newly defined operations/roles in the policy files anymore, you should only have to worry about the things you're overriding22:25
mnaserand the modifications you make in the code are being "overwritten"22:25
lbragstadmnaser we have some bits for that too22:26
lbragstadmnaser part of the tooling for moving policy into code allows you to pass oslo.policy a policy file and it spits out all the policies you can remove because they are the default22:26
lbragstadso it's a way to prune the policy file to only the things you care about, which are the operations you want to override22:27
mnaseryeah but i think you're going to have to make a lot of noises and warnings22:27
mnaseri think most deployers arent as involved with these big changes22:27
mnaser(i really like to think most people spend as much time as i do reading upgrade notes)22:27
mnaserbut i think that's something that people skip a lot heh22:27
lbragstadmnaser i'd believe it22:27
lbragstadmnaser we plan on honoring whatever is defined in the policy file though, so the upgrade would be backwards compatible22:28
lbragstad(not sure if i'm understanding that particular concern or not)22:28
mnaserlbragstad thinking about it you're right22:29
mnaserif they don't remove policy.json, the old policies stay and everything is ok22:29
lbragstadright22:29
mnaserif they dont have one, keystone (should) usually be upgraded before all services so as part of the upgrade it can add those roles22:29
lbragstadoperators can migrate away from the older policies to something more granular if/when they want to22:29
mnaseryeah this doesnt change any existing functionality fair enough22:29
lbragstadbut it allows you the ability to slim down your policy file to only what you need in your config management system22:30
lbragstadand then when you go to upgrade, you don't have to manually munge policy files together based on the overrides you care about and new operations that were added to the project22:31
mnaseryeah that really made things so nice for nova22:31
lbragstadi bet - i'm anxious for keystone to get to that point22:31
mnaserwe literally had to write an ansible json_file resource to make sure that we tweaked the right files instead of just uploading them22:31
lbragstadmnaser so it sounds like you need a way for projects to have specific roles incorporated by default22:46
*** lamt has quit IRC22:46
lbragstadmnaser but would also benefit from having standardization of some kind22:47
mnaseryes exactly22:48
*** david-lyle has quit IRC22:51
*** thorst has joined #openstack-keystone22:51
lbragstadmnaser awesome - we'd appreciate any of that feedback on the specs if you're willing to give it22:51
mnaserlemme put my name in there and as movement happens ill try to give comments/thoughts22:52
lbragstadmnaser that'd be fantastic22:52
mnaseradded on both, thanks lbragstad :)22:52
lbragstadmnaser thanks for sharing your opinions!22:53
mnaserlbragstad no problem, feel free to highlight me anytime for operator feedback or anything :>22:53
lbragstadmnaser if you have any other suggestions regarding the standardization or management of policy in particular, i'm all ears22:53
lbragstadmnaser will do (you might regret that statement!)22:53
mnaserlbragstad something i brought up a while ago was being able to shadow a user as an admin (but this is on a super unrelated note)22:53
mnaseras an operator it would be great if i can impersonate another user for a period of time22:54
lbragstadmnaser for a support role capacity?22:54
mnaserlbragstad yeah22:54
lbragstadmnaser to see things like a user sees them?22:54
mnaserwe could integrate with something like that to give temp. access to support staff for things rather than giving them full admin access22:55
lbragstadmnaser have you looked into trusts?22:55
*** thorst has quit IRC22:55
mnaserlbragstad but trusts seem like they are more longer lived22:57
mnaserand they require setting up revokes (or maybe im wrong and you can put a expiry time on them)22:58
mnaserand im not sure how easy it is to use openstack CLIs with trusts22:58
lbragstadmnaser trusts support an expiration22:58
lbragstadso you can create one for 10 minutes22:59
*** lamt has joined #openstack-keystone23:00
lbragstadmnaser it looks like osc supports trusts https://github.com/openstack/python-openstackclient/blob/1b3f953715ec592ee366b717c9eb6ab5c504acf9/doc/source/command-objects/trust.rst23:00
mnaserokay i guess i have somme more learning to do then :)23:01
*** markvoelker has joined #openstack-keystone23:02
lbragstadmnaser http://cdn.pasteraw.com/m1wnyry7k4syuydisnxlwtcif8xdmvo23:05
mnaserlbragstad ill have to try that out and see if we can somehow integrate with horizon23:05
lbragstadmnaser yeah - curious to get your feedback because I've heard the request before23:06
lbragstadspecifically for support use cases23:06
*** markvoelker has quit IRC23:06
*** thorst has joined #openstack-keystone23:08
mnaserlbragstad it'll be a fun horizon exercise23:09
*** adrian_otto has joined #openstack-keystone23:15
*** lamt has quit IRC23:17
*** ravelar1 has quit IRC23:20
*** thorst has quit IRC23:25
*** thorst has joined #openstack-keystone23:25
*** thorst has quit IRC23:29
*** dave-mccowan has quit IRC23:31
dstaneklbragstad: that's what i get for having a bunch of uncommited crap in my working directory. i have to eyeball pep8 because it always fails and then something slides through23:33
*** catintheroof has joined #openstack-keystone23:37
*** chrome0 has quit IRC23:40
*** gyee has quit IRC23:43
*** chrome0 has joined #openstack-keystone23:43
*** erlon has joined #openstack-keystone23:43
*** thorst has joined #openstack-keystone23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!