Monday, 2016-02-29

*** sdake has joined #openstack-keystone00:00
*** fpatwa_ has joined #openstack-keystone00:06
*** mylu has joined #openstack-keystone00:07
*** csoukup has quit IRC00:07
*** fpatwa_ has quit IRC00:10
*** wmolicki has joined #openstack-keystone00:17
*** roxanagh_ has quit IRC00:17
*** roxanaghe has joined #openstack-keystone00:18
*** jasonsb has joined #openstack-keystone00:20
*** jasonsb has quit IRC00:25
*** wmolicki has quit IRC00:28
*** wmolicki has joined #openstack-keystone00:28
*** roxanaghe has quit IRC00:32
*** roxanaghe has joined #openstack-keystone00:34
*** roxanaghe has quit IRC00:35
*** markvoelker has joined #openstack-keystone00:43
*** wmolicki has quit IRC00:44
*** sdake has quit IRC00:45
*** mylu has quit IRC00:46
*** markvoelker has quit IRC00:47
*** mylu has joined #openstack-keystone00:50
*** sdake has joined #openstack-keystone00:50
*** sdake has quit IRC01:06
*** EinstCrazy has joined #openstack-keystone01:06
*** mylu has quit IRC01:07
*** sdake has joined #openstack-keystone01:08
*** fpatwa_ has joined #openstack-keystone01:08
*** mylu has joined #openstack-keystone01:09
*** bjornar has quit IRC01:14
*** sdake has quit IRC01:43
*** vilobhmm11 has joined #openstack-keystone01:43
*** mylu has quit IRC01:44
*** markvoelker has joined #openstack-keystone01:44
*** mylu has joined #openstack-keystone01:46
*** markvoelker has quit IRC01:48
*** vilobhmm11 has quit IRC02:00
*** mylu has quit IRC02:18
*** mylu has joined #openstack-keystone02:20
*** mylu has quit IRC02:28
*** mylu has joined #openstack-keystone02:29
*** aginwala has joined #openstack-keystone02:29
*** sdake has joined #openstack-keystone02:31
*** mylu has quit IRC02:35
*** fpatwa_ has quit IRC02:36
*** mylu has joined #openstack-keystone02:37
*** spandhe has quit IRC02:37
*** mylu has quit IRC02:40
*** markvoelker has joined #openstack-keystone02:44
openstackgerritayoung proposed openstack/keystone: Remove unneeded revocation events
*** mdavidson has quit IRC02:46
*** mylu has joined #openstack-keystone02:46
*** lennyb__ has quit IRC02:46
*** richm has quit IRC02:47
*** lennyb__ has joined #openstack-keystone02:47
*** kfox1111 has quit IRC02:49
*** markvoelker has quit IRC02:49
*** richm has joined #openstack-keystone02:49
*** kfox1111 has joined #openstack-keystone02:49
*** mdavidson has joined #openstack-keystone02:50
*** boris-42 has joined #openstack-keystone02:50
*** roxanaghe has joined #openstack-keystone02:52
*** vilobhmm11 has joined #openstack-keystone02:58
*** spandhe has joined #openstack-keystone03:02
*** mylu has quit IRC03:07
*** mylu has joined #openstack-keystone03:07
*** roxanaghe has quit IRC03:14
*** sdake has quit IRC03:19
*** aginwala has quit IRC03:20
*** jasonsb has joined #openstack-keystone03:23
*** mylu has quit IRC03:23
*** aginwala has joined #openstack-keystone03:23
*** jasonsb has quit IRC03:28
*** roxanaghe has joined #openstack-keystone03:39
*** roxanaghe has quit IRC03:42
*** roxanaghe has joined #openstack-keystone03:42
*** dave-mccowan has quit IRC03:42
*** fangxu has joined #openstack-keystone03:43
*** Nirupama has joined #openstack-keystone03:43
*** links has joined #openstack-keystone03:44
*** sdake has joined #openstack-keystone03:47
*** aginwala has quit IRC03:51
*** topol_ has joined #openstack-keystone03:53
*** stevemar has joined #openstack-keystone03:53
*** Ephur has quit IRC03:59
*** links has quit IRC04:01
*** ChanServ sets mode: +o stevemar04:06
*** spandhe has quit IRC04:09
openstackgerritLance Bragstad proposed openstack/keystone: Return 404 instead of 401 for tokens w/o roles
*** david-lyle has joined #openstack-keystone04:13
*** links has joined #openstack-keystone04:18
*** david-lyle has quit IRC04:36
openstackgerritguang-yee proposed openstack/keystone: Return 404 instead of 401 for tokens w/o roles
*** markvoelker has joined #openstack-keystone04:45
*** markvoelker has quit IRC04:50
*** david-lyle has joined #openstack-keystone04:50
*** david-lyle has quit IRC04:56
*** spandhe has joined #openstack-keystone05:00
*** spandhe has quit IRC05:01
*** mylu has joined #openstack-keystone05:07
*** aginwala has joined #openstack-keystone05:38
*** fpatwa_ has joined #openstack-keystone05:44
*** roxanaghe has quit IRC05:44
*** roxanaghe has joined #openstack-keystone05:45
*** mylu has quit IRC05:53
*** fawadkhaliq has joined #openstack-keystone05:54
*** fpatwa_ has quit IRC06:09
*** aginwala has quit IRC06:19
*** fangxu has quit IRC06:19
*** jasonsb has joined #openstack-keystone06:27
*** fangxu has joined #openstack-keystone06:27
*** jasonsb has quit IRC06:32
*** richm has quit IRC06:36
*** aginwala has joined #openstack-keystone06:39
*** sdake has quit IRC06:43
*** rcernin has joined #openstack-keystone06:45
*** ianw has quit IRC06:45
*** markvoelker has joined #openstack-keystone06:46
*** rcernin has quit IRC06:49
*** ianw has joined #openstack-keystone06:50
*** markvoelker has quit IRC06:50
*** sdake has joined #openstack-keystone06:59
*** jaosorior has joined #openstack-keystone07:01
*** henrynash has joined #openstack-keystone07:13
*** ChanServ sets mode: +v henrynash07:13
*** fawadk has joined #openstack-keystone07:15
*** fawadkhaliq has quit IRC07:15
*** fawadkhaliq has joined #openstack-keystone07:15
*** roxanaghe has quit IRC07:17
*** sdake has quit IRC07:17
openstackgerritlei zhang proposed openstack/keystone: Make service type unique
*** fawadk has quit IRC07:20
openstackgerritPandiyan proposed openstack/keystone: Add driver details in architecture doc
*** rcernin has joined #openstack-keystone07:24
*** belmoreira has joined #openstack-keystone07:25
*** spandhe has joined #openstack-keystone07:32
*** pcaruana has joined #openstack-keystone07:43
*** fawadkhaliq has quit IRC07:43
*** gus has quit IRC07:50
*** gus has joined #openstack-keystone07:52
*** rk4n has joined #openstack-keystone08:02
*** spandhe has quit IRC08:05
*** rk4n has quit IRC08:05
*** fpatwa_ has joined #openstack-keystone08:09
*** fpatwa_ has quit IRC08:14
*** jed56 has joined #openstack-keystone08:20
*** fawadkhaliq has joined #openstack-keystone08:27
*** jasonsb has joined #openstack-keystone08:29
*** rcernin has quit IRC08:29
*** fangxu has quit IRC08:32
*** jasonsb has quit IRC08:33
*** rcernin has joined #openstack-keystone08:37
*** fawadkhaliq has quit IRC08:38
*** pnavarro has joined #openstack-keystone08:39
*** d0ugal_ has quit IRC08:46
*** d0ugal has joined #openstack-keystone08:46
*** markvoelker has joined #openstack-keystone08:47
*** markvoelker has quit IRC08:51
openstackgerritSergey Nikitin proposed openstack/keystone: Added .idea to the .gitignore
*** jistr has joined #openstack-keystone09:20
*** rk4n has joined #openstack-keystone09:21
*** huats is now known as Guest3387509:22
samueldmqmorning keystoners09:23
samueldmqbreton: o/09:25
*** vilobhmm11 has quit IRC09:31
henrynashsamueldmq: hi09:31
samueldmqhenrynash: hi09:33
henrynashsamuedlmq: on the update cascade….is someone putting up another patch (with a more complete solution as described in the docstring), or are we going with this one?09:34
samueldmqhenrynash: I'm working on another patch09:35
henrynashsamueldmq: ok, will review as soon as it is up09:35
samueldmqhenrynash: perfect09:35
samueldmqhenrynash: after that, I will dive into reseller's one09:35
henrynashsamuledmq: are you happy with teh projects acting as a domain patch? If so, could you at least +1 it to show you agree (or -1 if you don’t!!!)09:36
henrynashsamuedlmq: ah, ok09:36
openstackgerritSamuel de Medeiros Queiroz proposed openstack/keystone: API support for project cascade update
samueldmqhenrynash: just updated the docstring ... ^ will take a quick look in reseller before I leave for breakfast09:38
henrynashsamueldmq: thx09:39
*** e0ne has joined #openstack-keystone09:43
*** aginwala has quit IRC09:43
*** sdake has joined #openstack-keystone09:51
*** boris-42 has quit IRC09:54
*** e0ne has quit IRC09:55
*** henrynash has quit IRC10:03
*** sdake has quit IRC10:06
*** pnavarro has quit IRC10:09
*** fpatwa_ has joined #openstack-keystone10:10
*** EinstCrazy has quit IRC10:14
*** fpatwa_ has quit IRC10:15
*** daemontool_ has quit IRC10:23
*** daemontool has joined #openstack-keystone10:29
*** jasonsb has joined #openstack-keystone10:30
*** jasonsb has quit IRC10:36
*** daemontool_ has joined #openstack-keystone10:39
*** daemontool has quit IRC10:41
*** daemontool_ is now known as daemontool10:42
*** sdake has joined #openstack-keystone10:43
*** markvoelker has joined #openstack-keystone10:47
*** markvoelker has quit IRC10:52
*** sdake has quit IRC10:53
*** daemontool__ has joined #openstack-keystone10:56
*** flaper87 has quit IRC10:56
*** flaper87 has joined #openstack-keystone10:56
*** daemontool has quit IRC10:57
*** rk4n_ has joined #openstack-keystone11:01
*** rk4n has quit IRC11:04
*** richm has joined #openstack-keystone11:04
*** sdake has joined #openstack-keystone11:06
*** xek has joined #openstack-keystone11:12
*** pnavarro has joined #openstack-keystone11:12
*** sdake has quit IRC11:16
*** tellesnobrega is now known as tellesnobrega_af11:19
*** daemontool_ has joined #openstack-keystone11:23
*** daemontool__ has quit IRC11:27
zigonotmorgan: Hi, could you please remove intersphinx from the positional package?11:27
zigonotmorgan: Also, the image with references to external resources in the README.rst.11:28
zigoI had to patch all of these out in Debian...11:28
*** rk4n_ has quit IRC11:28
*** dims has joined #openstack-keystone11:29
*** dims has quit IRC11:32
*** dims has joined #openstack-keystone11:32
*** rk4n has joined #openstack-keystone11:33
openstackgerritSean Dague proposed openstack/keystone: Make keystone tests work on leap years
*** tellesnobrega_af is now known as tellesnobrega11:41
*** henrynash has joined #openstack-keystone11:47
*** ChanServ sets mode: +v henrynash11:47
*** raildo-afk is now known as raildo11:49
samueldmqhenrynash: hi11:55
*** dave-mccowan has joined #openstack-keystone11:55
henrynashsamueldmq: hi11:55
samueldmqhenrynash: where does the project_id in the url goes inside context?11:55
samueldmqhenrynash: the project passed as the body in an update goes into input_attr right?11:56
henrynashsamueldmq: i’ll have to remind myself!11:56
henrynashhold on11:56
samueldmqhenrynash: ok11:57
henrynashsamueldmq: so if a project_id is passed as a param, the entity is read from teh driver and place in ‘target’11:57
samueldmqhenrynash: as a param == in the url?11:58
*** htruta` is now known as htruta11:58
henrynashsamueldmq: oh, sorry….I thought you were talking abour aclling the policy engine11:58
samueldmqhenrynash: yes I am :)11:58
samueldmqhenrynash: let me submit another patchset, and we can discuss ther11:59
henrynashsamuedlmq: ok!11:59
*** ChanServ sets mode: +v topol_11:59
*** topol_ is now known as topol11:59
henrynashsamueldmq: as an aside, I’m still struggling a little with what the right thing is to do with that patch11:59
samueldmqhenrynash: I feel the same11:59
samueldmqhenrynash: and this implementation (as proposed now) seems a bit risky to merge this late?12:00
henrynashsamuedlmq: although at least won’t affect anything else (e.g. it’s localized)12:00
samueldmqhenrynash: yes, but wewould need, for example, test all token workflows involved12:01
samueldmqhenrynash: etc tec, it needs lots of testing12:01
henrynashsamueldmq: agrred…and federation12:01
samueldmqhenrynash: yes, let's rethink about it12:02
samueldmqhenrynash: if we have a different entry in the policy ...12:02
samueldmqhenrynash: and next cycle we decide to go with this check (as proposed now)12:02
samueldmqhenrynash: what would be the impact ?12:02
samueldmqhenrynash: need to deprecate things ? (since a policy entry would disappear?)12:02
henrynashsamueldmq: yes, I think we would had a config switch to use it or not (once we had a correct solution), and teh depreacte12:03
henrynash..and then deprecate12:03
samueldmqhenrynash: either we do this, or wait for a rc (or probably N) to get it in12:04
henrynashsamuedlmq: this feels like we need to do this in confunction with unified delgation….since it exposes the differences between regualr tokens, trusts and federation12:04
samueldmqhenrynash: yes, the fact they're different right now makes it harder12:05
henrynash(and inherited assiggnmnets)12:05
samueldmqhenrynash: for now I was considering skip trusts, as they don't support inherited role assignments12:05
samueldmqhenrynash: and the way it's proposed, it would be designed for working with inherited assignmetns12:05
henrynashsamuedlmq: …although would you explicitely stop trusts working? I guess you’d have to12:06
samueldmqhenrynash: as per my last version o the docstring12:06
-openstackstatus- NOTICE: Infra currently has a long backlog. Please be patient and where possible avoid rechecks while it catches up.12:06
samueldmqhenrynash: well, trust scoped tokens would only work (in the current proposal) if the policy was like:12:06
samueldmq"update_project": ""12:06
samueldmqor didn't check on target: like "update_rpoject": "role:admin"12:07
henrynashsamueldmq: is the user_id of a trust token the trustor or trustee (of neither)?12:08
henrynash(or neither)12:08
samueldmqhenrynash: I don't know, would need to check12:08
samueldmqhenrynash: also I am not 100% familiar with all tokens workflows12:08
henrynashsamuedlmq: if it is trustor or trustee…would the current code just kick in and try and run the current algorithm...12:09
samueldmqhenrynash: which increase prob of something going wrong12:09
henrynashsamueldmq: yep12:09
samueldmqhenrynash: yes, the current algorithm, in the worst case, tries to use the original token for all projects12:10
samueldmqhenrynash: so I think we're agreeing to postpone this a little bit12:10
samueldmqhenrynash: as it needs broader discussions12:10
henrynashsamuedlmq: if we do anything now, the only (easily depreacted) solution would be a separate policy endpoint….otherwise it’s going to be too confusing to migrate12:11
henrynashsamueldmq: what doees the current master code actually do?12:11
henrynashsamueldmq: i.e. if what will Mitaka code do if we don’t change anything12:11
samueldmqhenrynash: what do you mean by master?12:11
samueldmqhenrynash: the proposed code ?12:12
samueldmqhenrynash: so if we don't change anything, mitaka code doesnt expose the API for ?cascade12:12
henrynashsamueldmq: ah, ok12:12
henrynashsamueldmq: so it’s safe, at least12:12
samueldmqhenrynash: and honestly, otherwise someone really *needs* this feature now12:13
samueldmqhenrynash: I vote for postponing and having broader discussions at the summit12:13
henrynashsamueldmq: I tend to agree, given our uncertainty over teh correct policy decision, I support postponement12:14
samueldmqhenrynash: ++12:14
samueldmqstevemar: cc ^12:14
samueldmqstevemar: given that neither henrynash and myself are not 100% confident we are going towards the right direction in the current proposal for ?cascade operations12:15
samueldmqstevemar: we propose to postpone this feature; we believe this needs further discussion and may be influenced (positively) by unified delegation12:15
samueldmqhenrynash: rodrigods also had an interesting comment on patchset 25
henrynashsamueldmq: agreed12:17
*** EinstCrazy has joined #openstack-keystone12:17
samueldmqhenrynash: perfect, I am going to focus on reseller review now12:17
samueldmqhenrynash: which is much more important to get now imo12:17
henrynashsamuedlmq: ok, thanks12:18
rodrigodshenrynash, samueldmq, ++ to postpone and... regarding hierarchical quotas, i think this is how they are doing right now12:18
rodrigods"branch rights"12:18
*** daemontool_ has quit IRC12:18
samueldmqrodrigods: nice12:19
henrynashsamueldmq: one of the reasons I want to try and get it in is that we know many projects are designing  their nested quota systems….and having projects as a domain as part of their models is important12:19
samueldmqhenrynash: an immediate question that comes to my mind on that patch (12:19
samueldmqProjects acting as domains)12:19
*** clenimar has joined #openstack-keystone12:19
samueldmqhenrynash: do we have *all* domain calls at manager level equally tested with domain projects ?12:19
henrynashsamuedlmq: it’s a good question….I think most, but I’ll review that. thx12:20
*** daemontool has joined #openstack-keystone12:20
samueldmqhenrynash: nice, I will look as well12:22
henrynashsmauedlmq: looks well tested  - see test_v3_resource12:26
samueldmqhenrynash: looking12:27
henrynashsamueldmq: we mereged the tests in a previous patch12:27
samueldmqhenrynash: also I have something to mention ... something related (functional tests) that you may support for keystone as well12:27
samueldmqhenrynash: data driven functional tests (HTTP level)12:28
samueldmqhenrynash: see
henrynashsamueldmq: yes….had been thikning about that12:28
*** gordc has joined #openstack-keystone12:32
*** jasonsb has joined #openstack-keystone12:32
*** daemontool has quit IRC12:32
*** jaosorior has quit IRC12:33
*** jaosorior has joined #openstack-keystone12:34
*** daemontool has joined #openstack-keystone12:34
*** jasonsb has quit IRC12:37
openstackgerritDave Chen proposed openstack/keystone: Fix the migration issue for the user doesn't have a password
*** rk4n_ has joined #openstack-keystone12:39
*** rk4n has quit IRC12:42
*** Guest33875 has quit IRC12:45
*** nisha has joined #openstack-keystone12:45
*** huats_ has joined #openstack-keystone12:47
*** markvoelker has joined #openstack-keystone12:48
openstackgerritJacek Tomasiak proposed openstack/keystoneauth: Fix typos and improve formatting in migrating.rst
*** markvoelker has quit IRC12:53
*** pcaruana has quit IRC12:53
*** josecastroleon has joined #openstack-keystone12:54
*** daemontool has quit IRC12:55
*** daemontool has joined #openstack-keystone12:55
*** daemontool has quit IRC12:56
*** nisha_ has joined #openstack-keystone12:57
*** daemontool has joined #openstack-keystone12:59
*** nisha has quit IRC13:01
openstackgerritDave Chen proposed openstack/keystone: Fix the migration issue for the user doesn't have a password
*** nisha__ has joined #openstack-keystone13:03
*** Oku_OS has joined #openstack-keystone13:05
*** fpatwa_ has joined #openstack-keystone13:06
*** nisha_ has quit IRC13:06
*** pcaruana has joined #openstack-keystone13:08
raildoayoung: I loved the  "ValueError: day is out of range for month" error :P13:09
*** huats_ has quit IRC13:10
ayoungraildo, nothing is going to pass today.  I also wonder about all the systems out there running OpenStack.  It could be really bad13:10
*** fpatwa_ has quit IRC13:11
*** huats_ has joined #openstack-keystone13:11
*** huats_ has quit IRC13:11
*** huats_ has joined #openstack-keystone13:11
raildoayoung: yes, it's awkward, what if we send a patch changing the date for a valid year?13:12
*** nisha_ has joined #openstack-keystone13:12
samueldmqraildo: ayoung: sdague has a patch earlier today to fix our tests13:13
*** nisha__ has quit IRC13:16
*** links has quit IRC13:18
raildosamueldmq: awesome, thanks!13:18
samueldmqraildo: np13:20
*** nisha has joined #openstack-keystone13:20
samueldmqhenrynash: we have people using their own drivers in keystone right ?13:20
samueldmqhenrynash: I mean, do we really want to support legacy drivers for everything ? that's a lot of effort13:20
henrynashsamueldmq: well, that’s aways the theory!13:20
henrynashsamueldmq: and racksapce certainly used to13:21
samueldmqhenrynash: probably people have token drivers ? but I doubt for other backends like resource, etc13:21
henrynashsamueldmq: we’ve made that commitment, and, at least for now, I guess we have to conitnue (and yes, is a PITA)13:21
samueldmqhenrynash: we should probably consider asking people about that, and probably remove legacy support where we don't need to13:22
samueldmq(good for everyone at the end)13:22
samueldmqhenrynash: but yes, for now it's the commitment, can't change without asking13:22
*** nisha__ has joined #openstack-keystone13:22
henrynashsamueldmq: I guess one problem is that we might not know during cycle Y that someone had just built a driver based on cycle X…and then we break them13:23
*** nisha_ has quit IRC13:23
samueldmqhenrynash: could we ask -operators ? and see who effectively uses this13:24
samueldmqhenrynash: do other projects support htis too ?13:24
henrynashsamueldmq: so cinder and neutron would be interesting to ask, since they must do a LOT of this13:25
*** edmondsw has joined #openstack-keystone13:25
samueldmqhenrynash: ++13:25
samueldmqhenrynash: about reseller ..13:25
samueldmqhenrynash: did we support is_domain projs in L ?13:25
samueldmqhenrynash: nevermind, I am always confused with the purpose of our legacy driver13:26
henrynashsamueldmq: it was a defined attribute, but reserved for future use13:26
*** nisha has quit IRC13:26
samueldmqI got it13:26
samueldmqhenrynash: L1372
patchbotsamueldmq: patch 231289 - keystone - Projects acting as domains13:28
samueldmqhenrynash: the old driver didn't expect any project to appear inthe domain table, why do we have to query on both tables (domain and projs) ?13:28
*** nisha__ has quit IRC13:29
*** huats_ has quit IRC13:30
henrynashsamuedlmq: because we are responsind to a list_projects call from the manager…which DOES expect to (in the case of no filter) to see both, so we have to extract the match from both tables13:30
henrynashsamueldmq: the manager list_domains, calls list_projects_acting_as_domains, so that one is fine13:31
*** huats_ has joined #openstack-keystone13:31
samueldmqhenrynash: manager expects is_domain projects to be returned (and those are still in the domain table)13:34
samueldmqhenrynash: right?13:34
henrynashsamueldmq: so for list_projects, it expects both, no?13:34
henrynashsamueldmq: (which we don’t allow via the API today, but at the manager level we do)13:35
samueldmqhenrynash: okay, don't we need to do he same for list_projects_from_ids ?13:35
samueldmqhenrynash: since manager may be querying an is_domain project that is still at the legacy domain table ?13:35
henrynashsamuelmdq: we do…which is why we call get_project for each ID and it will get it from the right table13:36
samueldmqhenrynash: agreed13:36
samueldmqhenrynash: in the new manager, can I call list_projects_in_subtree(None) ?13:42
samueldmqhenrynash: in the old I could too, I guess13:42
*** Guest45139 is now known as amakarov13:43
*** fawadkhaliq has joined #openstack-keystone13:43
samueldmqhenrynash: left a couple of comments/questions13:46
henrynashsamueldmq: project_id==None is caught by _assert_valid_project_id13:46
henrynashsamueldmq: ok, will look , thx13:46
*** markvoelker has joined #openstack-keystone13:47
*** ninag has joined #openstack-keystone13:47
*** daemontool has quit IRC13:48
*** henrynash has quit IRC13:51
*** daemontool has joined #openstack-keystone13:54
*** daemontool_ has joined #openstack-keystone13:55
*** links has joined #openstack-keystone13:58
*** daemontool has quit IRC13:59
*** mhickey has joined #openstack-keystone14:06
edmondswhave y'all seen UTs failing today because of the leap year?14:09
edmondswthings like datetime.datetime.utcnow().replace(year=2030) fail with "ValueError: day is out of range for month" because there is no Feb 29 in 203014:11
edmondswworking on a patch if nobody else already is14:11
patchbotsamueldmq: patch 285987 - keystone - Make keystone tests work on leap years14:11
edmondswtx samueldmq14:12
*** Nirupama has quit IRC14:12
samueldmqedmondsw: np14:13
edmondswsamueldmq hmm... looks like he only caught some of the places14:16
edmondsw2030 isn't a leap year either14:16
edmondswand that's used in at least 3 places I saw14:16
*** links has quit IRC14:16
*** boris-42 has joined #openstack-keystone14:16
*** diazjf has joined #openstack-keystone14:17
*** bknudson has left #openstack-keystone14:19
*** bknudson has joined #openstack-keystone14:19
*** ChanServ sets mode: +v bknudson14:19
*** pauloewerton has joined #openstack-keystone14:20
*** andrewbogott has quit IRC14:21
*** andrewbogott has joined #openstack-keystone14:21
edmondswhenrynash samueldmq that patch is already +workflow... but I don't think it can merge until the 2030 cases14:23
edmondswshould I throw up another patch there, or do we need to do something like remove the +workflow first?14:23
bknudsonedmondsw: if you post a new revision it will remove it from the merge queue14:24
bknudsonalso, if the unit tests aren't going to pass it's not going to merge14:25
lupineseems like today is a very bad day for me to try to get the tests passing locally :D14:26
marekddolphm: for the patch it's not only the tests that miss14:28
marekdbut some logic as well14:28
marekddolphm: i am working on this14:28
marekddolphm: we will need to actually fetch role assignments for user and mix them with roles for groups14:28
marekdotherwise we will not be backwards compatible.14:29
dolphmmarekd: ++ that's exactly what i was hoping to get out of the test14:29
*** jsavak has joined #openstack-keystone14:30
dolphmmarekd: thanks for your help!14:30
bknudsonunit tests are still failing even with
patchbotbknudson: patch 285987 - keystone - Make keystone tests work on leap years14:32
*** mylu has joined #openstack-keystone14:34
*** sdake has joined #openstack-keystone14:35
bknudsonedmondsw: you working on a fix for the unit tests?14:35
edmondswyeah, tox recreate is just taking a while14:35
bknudsonfailing tests are keystone.tests.unit.test_v3_auth.TestTrustRedelegation.test_redelegate_with_role_by_name keystone.tests.unit.test_v3_auth.TestTrustRedelegation.test_roles_subset keystone.tests.unit.test_v3_auth.TestTrustRedelegation.test_redelegation_terminator14:36
*** mylu has quit IRC14:38
*** pcaruana has quit IRC14:38
openstackgerritMatthew Edmonds proposed openstack/keystone: Make keystone tests work on leap years
marekddolphm: no problem.14:40
edmondswsamueldmq henrynash bknudson ^ that should fix the tests14:41
*** sigmavirus24_awa is now known as sigmavirus2414:42
*** mylu has joined #openstack-keystone14:42
*** knikolla has joined #openstack-keystone14:42
*** woodster_ has joined #openstack-keystone14:49
*** pcaruana has joined #openstack-keystone14:52
*** rk4n_ has quit IRC14:55
openstackgerritRaildo Mascena proposed openstack/keystone: Constraint to prevent duplicate endpoints
*** rk4n has joined #openstack-keystone14:56
*** doug-fish has joined #openstack-keystone14:56
*** sdake has quit IRC14:58
*** EinstCra_ has joined #openstack-keystone15:01
*** EinstCra_ has quit IRC15:02
*** EinstCrazy has quit IRC15:02
*** lennyb__ is now known as lennyb15:03
*** EinstCrazy has joined #openstack-keystone15:03
*** nisha has joined #openstack-keystone15:03
*** henrynash has joined #openstack-keystone15:06
*** ChanServ sets mode: +v henrynash15:06
*** fpatwa_ has joined #openstack-keystone15:07
openstackgerritMarek Denis proposed openstack/keystone: Shadow users - Shadow federated users
*** fpatwa_ has quit IRC15:12
openstackgerritMarek Denis proposed openstack/keystone: Shadow users - Concrete role assignments for federated users
*** permalac has joined #openstack-keystone15:19
*** permalac has quit IRC15:19
*** permalac has joined #openstack-keystone15:20
openstackgerritRon De Rose proposed openstack/keystone: Shadow users - Shadow federated users
*** phalmos has joined #openstack-keystone15:24
openstackgerritRon De Rose proposed openstack/keystone: Shadow users - Concrete role assignments for federated users
ayoungedmondsw, I Wonder why the tests with 2030 don;'t fail either.15:27
ayoungedmondsw, because I just saw them failing when I rebased my most recent patch on them15:28
*** jorge_munoz has joined #openstack-keystone15:28
*** mylu has quit IRC15:28
edmondswayoung they do fail... or rather, did until I fixed them15:28
ayoungedmondsw, ah, maybe I rebased on the earlier patch15:29
henrynashdstanek: ping15:29
ayoungedmondsw, cool I'll grab yours. THanks15:29
edmondsw2031 references were earlier in each method, so those were hit first and it didn't get to 2030. But when you fix the 2031 references and rerun the UTs, the 2030 references fail15:29
*** timcline has quit IRC15:29
edmondswayoung ^15:30
ayoungedmondsw, we should be using real Calendar tools.15:30
edmondswayoung, absolutely this isn't the best way to fix it... just the quickest15:31
edmondswto get things unblocked15:31
ayoungedmondsw, well I wish I had known the sdagues patch was untested.  Easy enough to test.  I assume you ran the whole battery?15:32
notmorganreally leap years fail?15:32
edmondswayoung, I just reran the failing tests, but yeah, I ran each of those15:32
notmorganis this a python screwup or a keystone screwup?15:32
ayoungnotmorgan, a little of both15:32
ayoungnotmorgan, we are using datetime, and changing the year15:33
ayoung this fix is still wonky, but if it gets through the tests, we'll do a better one15:33
notmorganso strictly us15:33
edmondswnotmorgan yes15:33
notmorganyou coule we could also do datetime(day=20, year=2032)15:33
notmorgani would recommend that rather than being "smart"15:34
notmorganand trying to cover edge cases - use a known day15:34
ayoungnotmorgan, year == utcnow.year + 2015:34
ayoungkeep rolling it forward15:34
notmorganright until you overrun unixtime and hit another edge case.15:34
stevemarkeystone doesn't do leap days, that's just awesome15:35
notmorganlets not be "clever" here. lets use a fixed date, if we have to fix the tests in 2030, we did something very right15:35
*** mylu has joined #openstack-keystone15:36
bknudsonjust try to be retired by 203015:36
notmorgani -115:36
notmorgan'd the review with the comment15:36
notmorganfeel free to still +A it15:36
notmorganjust a dissenting opinion15:36
dstanekhenrynash: pong15:37
samueldmqstevemar: I had read something like: "make keystone works in leap years"; and thought wow hehe15:37
*** jaosorior is now known as jaosorior_away15:37
henrynashdstanek: hi - wanted to check in on the projects acting as a domain….are you in agreement that we are at least provding the right level of backward compatibility (albiet you might prefer that we push the logic into teh driver layer)15:38
samueldmqnotmorgan: you mean a completly fixed date vs current day/month in a different year?15:38
notmorgansamueldmq: yep15:38
ayoungnotmorgan, let's force it past unixtime at a minimum....15:39
samueldmqnotmorgan: yeah, makes sense to me too15:39
notmorganayoung: works for me, just a dissenting opinion on the fix to not be "today in 2032" or whaever, it isn't a good test, if we want to check every date, we write a test to check every date :P15:40
notmorganayoung: like i said, feel free to +A the change to unblock things as well, I wouldn't hold it up.15:40
ayoungnotmorgan, no, you are right on, but if we can't handle Unix time yet, we need to address that first, and then make expiry something unsurpring in our tests second15:40
ayoungnotmorgan, I need your help on something15:40
notmorganayoung: i shall try15:40
notmorganunrelated, i get my 802.11s network today! :)15:41
ayoungnotmorgan, I was working on reducing the revocation events, and instead re-validating the auth data on token validation15:41
ayoungin doing so, I had to remove the MEMOIZE calls in the token provider15:41
ayoungis this a real problem, or will we be OK due to caching in the other drivers?15:41
notmorganayoung: in the token provider or in the token persistence?15:41
ayoungnotmorgan, I'll link:15:42
dstanekhenrynash: i think so, but i haven't had time to go in depth. if others think it's fine i wouldn't hold that up15:42
notmorganit's going to hurt because that is a lot of offloaded logic to memoization (depending on what was removed)15:42
henrynashdstanek: ok, just wanted to check, thx15:42
notmorganbut it can probably be re-spun or shuffled around a little.15:42
*** timcline has joined #openstack-keystone15:42
patchbotayoung: patch 285134 - keystone - Remove unneeded revocation events15:43
ayounglook at line 304 in the new15:43
notmorganayoung: that is going to make repeate token validation way way way more expensive :(15:43
notmorganwhat was the issue you ran into w/ removing the events?15:43
notmorganalso invalidate_individual_token_cache is only relevant in revoke_by_audit_id/ID15:44
ayoungnotmorgan, basically that if you memoize, you don't see changes like project inactivation or role removal15:44
ayoungnotmorgan, so, the checks really should be done against the independant backends, not against the snapshot of the token15:44
*** henrynash has quit IRC15:45
ayoungwe could do it as a cache that gets invalidated when the changes come in.15:45
*** pushkaru has joined #openstack-keystone15:45
notmorgannah. that wont be super useful.15:45
ayoungBut I suspect that there will be so many changes the cache will just be always invalidating, and that might hurt more15:45
notmorganas long as we have caching on the other subsystems it'll be not as bad as it could be15:46
*** nisha_ has joined #openstack-keystone15:47
ayoungnotmorgan, right, and if there are multiple threads, is the MEMOIZE going to actually do anything?15:47
ayoungor was that done assuming a greenlet impl15:47
notmorganyes, because @memoize must go to a shared backend15:47
notmorganwe do not support per-thread caching15:47
*** nisha_ has quit IRC15:48
notmorganso you must have a shared memcache for all keystones if caching is enabled, otherwise invalidates of the cache aren't shared15:48
*** nisha_ has joined #openstack-keystone15:48
*** nisha_ has quit IRC15:48
notmorganall keystones and all threads/processes.15:48
*** nisha_ has joined #openstack-keystone15:48
*** nisha_ has quit IRC15:48
ayoungnotmorgan, so even with caching, a token validation is not going to hit memcache several times for each validation15:49
notmorganayoung: will offload some of the added pain here15:49
patchbotnotmorgan: patch 272007 - keystone - Use requst local in-process cache per request15:49
notmorganayoung: correct.15:49
notmorganayoung: with your change that is.15:49
notmorganayoung: previously a single token could be cached therefore subsequent validates will be a single hit. the request local cache helps limit the number of trips in a single request to memcache/sql as well.15:50
openstackgerritRon De Rose proposed openstack/keystone: Shadow users - Concrete role assignments for federated users
notmorgani'm ok with this, but you'll need to add a releasenote15:50
*** nisha has quit IRC15:50
notmorganspecifically about this15:50
*** nisha has joined #openstack-keystone15:51
ayoungSo the revocation events approach optimizes based on the assumption that the token is likely to be valid.  The validate-whole-token approach optimized based on the assumption that something is going to change, or that multiple validations of the token are rare15:53
*** wolsen has joined #openstack-keystone15:53
*** rderose has joined #openstack-keystone15:53
ayoungnotmorgan, for my needs, I don't actually give Rodents Hindquarters about revocations15:54
ayoungit is actually to solve a different problem, which is to drop the validation based on the cached token data for UUID tokens15:54
ayoungbut jorge was seeing the increase size and performance issues with revocation events checking.  So maybe there is no good solution here.15:55
*** EinstCrazy has quit IRC15:55
notmorganrequest local cache will help some, and clearly calling out that tokens validates themselves are not cached.15:55
notmorgannot that they were cached post rev event processing anyway15:56
notmorganthat caching is needed on the other subsystems to ensure smooth operation vs just token now (a fair recommendation)15:56
*** slberger has joined #openstack-keystone15:58
*** roxanaghe has joined #openstack-keystone15:58
ayoungtox -e py27  -- keystone.tests.unit.test_v2.V2TestCase.test_remove_role_revokes_token  passes16:02
ayoungtox -e py27  -- keystone.tests.unit.test_v2.V2TestCase passes16:02
ayoungtox -e py27  -- keystone.tests.unit.test_v2 fails on the above test16:02
ayoungdouble you tee eff16:02
*** jsavak has quit IRC16:03
*** jsavak has joined #openstack-keystone16:04
ayoungGAH HEISENBUG16:05
notmorganayoung: hehe16:05
ayoungnotmorgan, actually, might be the same test is run in multiple suites16:06
notmorganyeah that happens16:06
notmorganless heisenbug more "different setup = different results"16:07
ayoungwell, change it for one, it fails in the other.  But I thought I saw a com,plete tox -epy27 run last night16:07
*** henrynash has joined #openstack-keystone16:10
*** ChanServ sets mode: +v henrynash16:10
*** browne has joined #openstack-keystone16:11
ayoungand this leads to even more code efficiency!  ooh, I like this16:15
*** rcernin has quit IRC16:16
*** fawadkhaliq has quit IRC16:17
*** pcaruana has quit IRC16:17
ayoungnotmorgan,  what is the right way to do  expires = datetime.something.('2032, 2, 29')16:23
notmorganayoung: explain?16:23
notmorgannot sure what you're asking16:23
ayoungnotmorgan, addressing your comment16:23
samueldmqayoung: notmorgan: I am doing it with just putting the string directly '2031-02-18T18:10:00Z'16:24
ayoungI'm going to make our test date be leap years day in 203216:24
notmorgandatetime.datetime(day=<day>, month=<month>, year=<year>)16:24
openstackgerritJacek Tomasiak proposed openstack/keystoneauth: Fix typos and improve formatting in migrating.rst
samueldmqtaht can be passed to new_trust_ref16:24
notmorganwould be the best choice.16:24
*** belmoreira has quit IRC16:24
notmorganbut you can also strftime it or is it strptime16:24
ayoungsamueldmq, that works.16:24
notmorganwhatever there is a way to STR->DATETIMe easier to use the string16:24
*** sigmavirus24 is now known as sigmavirus24_awa16:25
* samueldmq is running the tests16:25
ayoungsamueldmq, set it once in the core test file and use16:25
*** sigmavirus24_awa is now known as sigmavirus2416:25
notmorganayoung: next monday I am swinging through Boston btw16:25
ayoungnotmorgan, excellent!16:25
*** mylu has quit IRC16:25
ayoungnotmorgan, breakfast, lunch, coffee, or dinner?16:26
notmorganayoung: dunno if i'm going to NH that day or just bumming around until i catch a flight home.16:26
notmorganone of those for sure!16:26
notmorganif i'm headed to NH, i'll also be back through on wed16:26
ayoungWhere in NH?16:26
notmorganunfortunately, my friends are in the bahamas atm, so hard to coordinate16:26
notmorganmight be going snowboarding for a day on tuesday16:27
notmorganfigured i was on the east coast anyway, so what's an extra day? ;)16:27
ayoungcool.  Its been warm.  Snowboarding might not be worth it.16:28
*** mylu has joined #openstack-keystone16:28
notmorganright now it looks like i wont make it to NH, if i can't get ahold of my friends, just gonna book my flight home from BOS16:28
ayoungit was 50 degress when I last checked down here...same yesterday16:28
notmorganon monday16:28
*** rodrigods has quit IRC16:29
* notmorgan taps foot and waits for fedex >.<16:29
*** rodrigods has joined #openstack-keystone16:29
notmorgantopol: you've been awfully quiet.16:29
*** timcline has quit IRC16:30
ayoungsamueldmq, what call are you making to do the string parsing?16:31
ayoungOh, we can just do16:31
notmorganayoung: internally it does strptime16:32
ayoung ref['expires_at'] =  '2031-02-18T18:10:00Z'16:32
notmorganif it's a known string, so yeah16:32
samueldmqayoung: yes16:32
ayoungnotmorgan, we have16:32
ayoungref['expires_at'] = datetime.datetime.utcnow().replace(16:32
ayoung                year=2032).strftime(unit.TIME_FORMAT)16:32
samueldmqayoung: that's what I am doing16:32
ayoungso yeah, IO think that is s astring16:32
* samueldmq is running tests16:32
notmorganit can be a followup patch btw samueldmq16:32
notmorganif you want to just +A the current fix.16:33
notmorganso things can gate16:33
ayoungnotmorgan, will do16:33
ayoungit is holding me up16:33
openstackgerritSteve Martinelli proposed openstack/keystone: add hints to list_services for templated backend
openstackgerritSteve Martinelli proposed openstack/keystone: add hints to list_services for templated backend
samueldmqnotmorgan: done16:34
samueldmqayoung: oops16:34
openstackgerritayoung proposed openstack/keystone: Remove unneeded revocation events
*** nkinder has joined #openstack-keystone16:38
openstackgerritayoung proposed openstack/keystone: Remove unneeded revocation events
ayoungooh that is nice!16:39
ayoungrebase in the webui16:39
stevemarjeez, our tests for the templated backend are broken as hell16:39
*** mylu has quit IRC16:42
stevemarbknudson: thanks for releasing a new ksc16:42
openstackgerritSamuel de Medeiros Queiroz proposed openstack/keystone: Use a fixed expiry date in keystone tests
notmorganstevemar: the whole concept of the templated backend is broken16:43
notmorganstevemar: maybe i'll write the yaml based one today and we can just deprecate the current one?16:43
samueldmqayoung: notmorgan ^ still need an update; can't use something fixed as expires_at is part of the unique constraint16:43
bknudsonstevemar: y, I don't know what we do to get meta info in the list results16:43
samueldmqso tests are giving duplicated erros hehe16:43
bknudsonstevemar: since whatever we do seems to break the world16:43
ayoungwe don't make it easy16:43
samueldmqayoung: :-)16:44
*** mhickey has quit IRC16:45
stevemarbknudson: don't rage quit on me now16:45
*** amakarov has quit IRC16:46
*** pkarikh has quit IRC16:46
* notmorgan rage quits16:46
notmorganoh wait16:46
ayoung2032 will get the tests to pass.  But..are we dealing with unix time here?   Will adding 3 decades break things16:47
*** pkarikh has joined #openstack-keystone16:47
notmorganthe datetime object wont16:48
notmorganbut... fernet might?16:48
notmorgan>>> datetime.datetime(month=2, day=20, year=3000)16:49
notmorgandatetime.datetime(3000, 2, 20, 0, 0)16:49
notmorgansamueldmq: ok, so use a fixed day/month/year and let the time be "flexible"16:51
notmorgansamueldmq: that is closer to a real production use-case anyway16:52
*** amakarov has joined #openstack-keystone16:52
marekddolphm: erm....16:53
marekddolphm: i'd rather want you take a close look at some implementation details of def shadow users because i think this means touching lots of sensitive logic.16:54
marekddolphm: we will need to relax some constraints when it comes to popularing roles assigned to users and their groups16:54
samueldmqnotmorgan: ayoung: def get_future_expiry_str(hour, minute, second) in unit core ?16:54
dolphmmarekd: rderose: how so?16:54
samueldmqwith fixed day/month:year16:54
notmorgansamueldmq: *shrug*16:55
marekddolphm: and since it means relaxing constraints it means throughtful thinking about potential breaking security stuff.16:55
ayoungsamueldmq, nah...I'm giving up on this16:55
dolphmmarekd: (what constraints?)16:55
*** petertr7 is now known as petertr7_away16:55
*** daemontool_ has quit IRC16:55
marekddolphm: we need to support role assignments from a) role assignment between fed user and a project/domain but also role assignments between groups and project/domains16:55
dolphmmarekd: gotcha, ++16:56
marekddolphm: and roles must be additive, otherwise we will badly break whole federation assumption we ve been talking about past 2+ years16:56
dolphmmarekd: how would they be non-additive?16:56
marekddolphm: either 'classic role assignments' or 'groups only'16:57
marekddolphm: look, what i need to do now is: populate roles for user_id where user is a shadow user and later populate roles for groups assigned as mapping effect16:57
dolphmmarekd: i don't know if it's possible for mitaka, but i'd like federated group memberships to become 'classic group memberships' (so the output of mapping is "here's a user ID, it's already been added to 3 groups in SQL"16:58
marekdnow, guess what happens if the user doesn't have role assignment for a project. it raises Unauthorized16:58
marekdbecause this is how we treat normal users16:58
marekdno role assignemnt -> 40116:58
marekdhm i can catch exception16:58
marekdbut it's smelly :(16:58
rderosedolphm: reading...16:59
*** petertr7_away is now known as petertr717:01
marekddolphm: i doubt it is possible, because my understandind we still need to preserve old behaviour17:03
notmorganstevemar: something like that as the templated catalog input?17:06
*** sdake has joined #openstack-keystone17:06
notmorganstevemar: rather than "source a blob of json and string replace and hope it works"17:06
dolphmmarekd: the old behavior of ephemeral group memberships?17:06
dolphmmarekd: or?17:06
marekddolphm: old behavior17:06
dolphmmarekd: old behavior of what?17:06
notmorganstevemar: /me was going to try and be clever and do node references as well.. but ugh17:06
marekddolphm: dynamic assignment of groups and resolving roles assignment based on group membership.17:07
notmorganstevemar: or so we really jus tnot care about the templated catalog.17:08
notmorgancause it was/is dropped from devstack17:08
stevemarnotmorgan: i don't like user defined IDs :|17:08
dolphmmarekd: we can now have concrete group memberships - we don't need to maintain the old ephemeral model17:08
*** jaosorior_away is now known as jaosorior17:08
*** fpatwa_ has joined #openstack-keystone17:08
dolphmmarekd: have the opportunity* to use17:08
*** sheel has joined #openstack-keystone17:08
stevemarnotmorgan: it was dropped from devstack cause it was broken17:08
notmorganstevemar: and should never be re-added17:08
stevemarit was broken cause we don't care about it17:08
notmorganstevemar: so, if we care about having a CMS managed catalog17:09
notmorganit should be something like ^17:09
notmorganas waht the CMS manages17:09
marekddolphm: so we can just remove mechanism without any notification?17:09
notmorganif we don't we should just deprecate the catalog17:09
openstackgerrithenry-nash proposed openstack/keystone: Projects acting as domains
marekddolphm: tomorrow dynamic assignments are not working any more, please update your shadow users accordingly?17:09
notmorgan"you're on your own using this, we wont remove it but..... good luck:17:09
dolphmmarekd: ? i'm lost on what feature we're potentially losing17:09
henrynashsamuedlmq: have updated:
*** dan_nguyen has joined #openstack-keystone17:10
henrynashayoung: would be good to get a thumbs up (or not) from you on that too:
dolphmmarekd: after a mapping is applied to a saml doc, you get a user identity and a list of groups, right?17:10
marekddolphm: in the mapping you specify list of groups the autenticated member will become a member17:10
dolphmmarekd: in liberty, for example17:10
ayounghenrynash, happy to look.17:10
marekddolphm: yep17:10
sheelstevemar: hi there17:11
marekdand based on the group are fetching roles17:11
marekdand match with projects etc17:11
stevemarsheel: hey17:11
dolphmmarekd: so, the next step is to actually call for group_id in mapped_group_memberships: assignments_api.add_user_to_group(shadowed_user_id, group_id)17:11
dolphmmarekd: and actually assign that user ID into all those groups17:11
sheelstevemar: its regarding one approval required17:11
*** henrynash has quit IRC17:11
dolphmmarekd: no more ephemeral group / role management17:11
sheelstevemar: i tries to find you on openstack client17:12
marekddolphm: is that already implemented anywhere?17:12
sheelstevemar: so is it ok to raise request here for
sheelstevemar: could you please have a look and approve it?17:12
stevemarsheel: there is no openstackclient channel, dtroyer and others are in #openstack-sdks for all things openstackclient17:12
dolphmmarekd: no, i'm saying that's the missing piece of that shadow user chain, and it would be required to pass the test that is outlined in
patchbotdolphm: patch 284943 - keystone - Shadow users - Concrete role assignments for feder...17:12
sheelstevemar: ohk... i just searched on ops17:13
dolphmmarekd: we have the infrastructure to support concrete role assignments at that point, but no concrete role assignments for shadow users17:13
*** fpatwa_ has quit IRC17:13
stevemarsheel: let's go to #openstack-sdks and talk there17:13
marekddolphm: what if admin removes group from the mapping? today they would expect user user no longer to be assiged group17:13
sheelstevemar: yeah sure17:13
*** boris-42 has quit IRC17:14
dolphmmarekd: good question ; did the SAML doc contain an expiration on the attribute that ultimately resulted in the group membership?17:14
ayoungdolphm, so, I think I want to back off pushing for Fernet to be the default in Mitaka. Target  that at Newton,  and instead file a slew of bugs for the things that keep Fernet from being the default, and try to knock out as many of those as possible before the M release.17:14
dolphmmarekd: if so, the correct solution would be to result in a group membership that will expire at that point, but can be refreshed with future federated auth flows17:15
ayoungThe effort to make it the default has been a stellar bug  reveal.17:15
lbragstadayoung FYI -
patchbotlbragstad: patch 278693 - keystone - Make fernet support trust auth against v2.017:15
dolphmayoung: ++17:15
marekddolphm: you are asking for attributes expiration?17:15
ayounglbragstad, Cool.  I have been deep in the token/trust code this weekend.17:16
lbragstadayoung - those failures give me the willies17:16
dolphmmarekd: yes, saml has a validUntil property, if i remember correctly?17:16
ayounglbragstad,  see
patchbotayoung: patch 285134 - keystone - Remove unneeded revocation events17:16
lbragstadayoung everything passes locally - but it's tempest stuff... so something tells me it's timing related :(17:16
ayounglbragstad, it might be, but there are other things I have found....l;ets see17:16
marekddolphm: something like that.17:17
ayoungah,  34 vs 27...17:17
marekdbut it looks like slightly less trivial than 3 lines of Python and I think we have deadline today?17:17
*** openstackgerrit has quit IRC17:18
marekddolphm: i am not sure what's your personal goal for Mitaka?17:18
*** openstackgerrit has joined #openstack-keystone17:18
marekdhave a working shadow users?17:18
dolphmmarekd: yes17:18
dolphmmarekd: local, unified authorization management for federated users17:18
marekdso we are missing only code manages group membership from mapping engine and ideally handles group membership expiration :-)17:19
lbragstadayoung have we been tagging all the bugs we come up with as 'fernet'?17:19
dolphm"federated users should be able to consume local role assignments just like locally-managed users can "
lbragstadayoung if we're not going to be able to land fernet as the default this release then I want to prep it so that we can land it as soon as Newton opens for dev17:20
ayounglbragstad, ++17:20
*** jasonsb has joined #openstack-keystone17:20
dolphmmarekd: rderose: i wasn't sure how much code that last step was going to take; if it's a non-trivial effort from where the current patchset stands, we should make the call to punt the spec to newton now17:20
dolphmwe're on step 3 from the original work items:
dolphmstevemar: ^ punt shadow users to newton?17:21
marekddolphm: i will see if i can propose something, and it will be up to you to decide17:21
*** sdake has quit IRC17:22
marekdbut dont expect fireworks17:22
dolphmwork items 4 and 5 can likely wait (subsequent notifications impact is just an anticipated bug-fix, and item 5 is correcting tech debt created by the spec)17:22
stevemardolphm: i was actually going to approve it today17:22
dolphmmarekd: lol17:22
dolphmstevemar: approve what?17:23
*** bjornar has joined #openstack-keystone17:23
*** sdake has joined #openstack-keystone17:23
ayoungOK...singing out for a while...back later on tonight17:23
*** ayoung has quit IRC17:23
stevemardolphm: the latest patch rderose had to store authed users to a backend17:23
*** sigmavirus24 is now known as sigmavirus24_awa17:26
*** jasonsb has quit IRC17:26
*** daemontool_ has joined #openstack-keystone17:26
*** jistr has quit IRC17:26
dolphmstevemar: correct, but can those users consume local role assignments? do we still have a need for federated fernet tokens, ephemeral group memberships, etc?17:27
* lbragstad hopes we can kill federated fernet tokens17:27
stevemarlbragstad: you and me both!17:27
openstackgerritMarek Denis proposed openstack/keystone: Role assignment resolution for shadow users.
marekddolphm: it passes test_v3_federation tests17:28
dolphmmarekd: is this my fireworks?17:28
stevemardolphm: so those are all fine to answer in newton, administratively i'd like to leave that spec in mitaka and propose follow up blueprints or bugs for newton17:28
marekddolphm: no17:29
marekdthis should make you pass tests from
patchbotmarekd: patch 284943 - keystone - Shadow users - Concrete role assignments for feder...17:29
marekdand old tests from test_v3_federation.17:29
* dolphm is looking at the new test17:30
*** david-lyle has joined #openstack-keystone17:30
dolphmmarekd: cool! you could make more assertions about the token that's returned (like it specifically contains the role you just assigned)17:32
dolphmmarekd: i'd also like to see a group membership added in the same test (one that is not provided via mapping)17:32
*** jsavak has quit IRC17:33
marekdhave you seen ?17:34
patchbotmarekd: patch 286169 - keystone - Role assignment resolution for shadow users.17:34
*** jsavak has joined #openstack-keystone17:34
marekdi assume tests failing due to Feb 29 are okay17:35
*** csoukup has joined #openstack-keystone17:35
marekddolphm: otherwise this should pass our auth tests.17:35
dstanekrderose: pep8 doesn't check the commit messages17:35
marekddolphm: i need to leave for a while17:36
dolphmmarekd: thank you sir!17:36
dolphmmarekd: going to be back on "today"?17:36
dolphmmarekd: i'm going to be AFK next two days17:36
*** fawadkhaliq has joined #openstack-keystone17:36
marekdyes, in a 2hrs17:36
dolphmmarekd: cool, ping me17:36
marekdneed to do sth before they close the office17:36
dolphm(or Ron)17:36
dolphmrderose: *(17:36
openstackgerritRon De Rose proposed openstack/keystone: Shadow users - Shadow federated users
rderosemarekd Added your test case back:17:39
patchbotrderose: patch 279162 - keystone - Shadow users - Shadow federated users17:39
rderosemarekd Took it out because the new FederatedUserTests class was a work-in-progress and wasn't sure I wanted to keep it.  But see your point and as I said, have added it back.  Thanks for your help.17:39
dolphmrderose: what do you mean by "marekd's test case?"17:40
rderose"Shadow federated users" is ready for review. I believe I have addressed all feedback this can be release independent of "concrete role assignments"17:40
rderosemarekd push a new patch with an additional unit test17:41
*** pece has joined #openstack-keystone17:41
*** nisha_ has joined #openstack-keystone17:41
* dolphm is stepping away for food real quick17:42
bjornarIs any effort beeing done to make database calls more efficient in keystone?17:42
notmorganbjornar: it's a very slow process because we have a lot of "can't break" previous behaviors17:43
openstackgerritSamuel de Medeiros Queiroz proposed openstack/keystone: Use a fixed expiry date in keystone tests
*** nisha has quit IRC17:43
notmorganbjornar: the shortest fix is to ensure a given entity is only queried from the backend once per request chain. I have a patch to handle that. the next is really about fixing how we query the backends, but remember each subsystem is independant, you can't join resource to identity for example, as the backend owning the data is potentially different17:44
samueldmqstevemar: replied your question in patch 28614817:44
patchbotsamueldmq: - keystone - Use a fixed expiry date in keystone tests17:44
notmorganbjornar: so there will always be *some* extra chattyness because it's not all run by a single driver.17:45
bjornarnotmorgan, looking at token issue now, a total of 86 queries (including rollback/commit/select 1)17:45
bjornar23 "real" queries17:45
notmorganignore the rollback/select/1 ones17:45
samueldmqstevemar: but it is still giving an error (I copy-psted it there) which made me curious17:45
bjornarnotmorgan, its still packets back and forth, but ok17:45
notmorganright. and we might be able to eliminate ~10 of those?17:45
notmorganhere let me link you my patch and you can look at how it lightens the load17:46
notmorganbjornar: and enable caching (you don't need a shared backend for this)17:46
patchbotnotmorgan: patch 272007 - keystone - Use requst local in-process cache per request17:46
notmorganbjornar: but that should ensure we query the backends exactly 1 time for a given entitiy. it is a dodge/not a final fix17:46
bjornarBecause looking at the queries it looks like it could be more or less one query with proper joins :/17:46
bjornarreally ugly17:47
notmorganbjornar: again, if you are crossing subsystem boundries you can't join17:47
notmorganbjornar: if you see a call for user, then role, then project17:47
notmorganthose have to be assumed to not be in the same backend17:47
notmorganby architecture17:47
bjornarnotmorgan, but keystone should know17:47
notmorganbjornar: we *do* know, but the issue is if it isn't in the same backend we have massively different logic17:48
*** spandhe has joined #openstack-keystone17:48
notmorganor we're hooking in to deep-dark-voodoo of SQLA to make it talk to different backends in wierd ways17:48
notmorganlike ORM -> LDAP17:48
notmorganthat is scary stuff17:48
bjornarBut I mean.. some >X % is using sql17:48
notmorganwe opted for "use the same code path when getting entities from different backends"17:48
notmorganso we can support driver splits without needing an insane test matrix and a lot of edge cases17:49
notmorganbjornar: it isn't really feasible to use joins if everything is SQL but not use joins if things are in LDAP17:50
notmorganor in something else17:50
notmorganwe have to treat any system with it's own driver config as separate.17:50
bjornarnotmorgan, But it is even repeating the same queries17:51
notmorganbjornar: so first step there is look at the patch i linked you17:51
notmorganbjornar: that one minimizes duplicate queries as long as caching is turned on.17:52
bjornarBut dont you guys take this seriously, sorry, but this is crappy17:52
*** jsavak has quit IRC17:53
notmorganbjornar: look, i'm trying. seriously. the fix i proposed works in any/all memoized values even if you don't configure memcache (make the dogpile backend null). it is specifically designed to avoid queryin the backend multiple times as the first step17:54
*** sdake has quit IRC17:54
bjornarnotmorgan, I mean, its even reapeating the exact same query two times in a row!17:55
notmorganbjornar: the hard part is, i am only one person. I can only do so much. so if you want to dump on us for "not taking it seriously", feel free to. but it isn't the case. We also have other architectural fixes that are aimed to move us in the right direction17:55
*** david-lyle has quit IRC17:55
bjornarSELECT AS role_id, AS role_name, role.domain_id AS role_domain_id, role.extra AS role_extra FROM role WHERE = '629f506286cd4c3191c291f8ac2f9a25'17:55
*** david-lyle_ has joined #openstack-keystone17:55
bjornarnotmorgan, Sorry, I dont mean to be rude. But someone should be blamed for writing this code in the first place17:56
notmorganbjornar: it wasn't "just written" like that17:56
bjornarI mean.. this is not a "pet-project" anymore, and keystone is some huge percentage if the total "delay"17:56
notmorganbjornar:  it evolved17:56
notmorganand WE have been working to clean this up17:56
bjornarnotmorgan, but if devs dont log queries..17:56
*** sdake has joined #openstack-keystone17:57
bjornarnotmorgan, I remember I sent you a big query trace like 2 years ago... if anything, its worse now!17:57
notmorganbjornar: what is the "huge" delay?17:57
notmorganbjornar: define a "huge" delay for me17:57
bjornarhuge is anything above 10ms17:57
*** jsavak has joined #openstack-keystone17:57
notmorganbecause 50ms is so bad when it takes 30seconds for something else to process17:58
notmorganor even 100ms17:58
notmorgananyway, seriously, i am trying to address this in the shortest way possible17:58
notmorganhence my proposed patch17:58
bjornarI will look at it.17:59
notmorganuntil we can unwind some other things like make it so token validation doesn't have 5 or 6 different paths it could go down17:59
notmorganif we have 1 or 2 paths we can start ensuring that we're not making silly calls for no reason.17:59
dstanekrderose: did a first pass17:59
bjornarnotmorgan, Where can I find the document that describes what queries are done for what api calls and statistics about what api calls take the most hits?17:59
notmorganbut if we have 5 or 6 and 3 or 4 of those are called sometimes but no others, it's problematic18:00
notmorganbjornar: osprofiler is a WIP and likely will be able to be used to do that in Newton18:00
rderosedstanek: cool, thx18:00
dstanekdid keystone really break the gate for a week?18:00
notmorganbjornar: we have grown to a point where we need it, but its tooling that is being developed18:00
notmorgandstanek: yes, for kolla, the admin_auth_token thing18:01
notmorgandstanek: iirc18:01
bjornarnotmorgan, because that is where to start I think. Document the performance impact18:01
*** Guest67703 is now known as me_18:01
*** agireud has quit IRC18:01
*** me_ is now known as med_18:01
bjornarnotmorgan, I can perhaps do some initial work there18:01
*** med_ has quit IRC18:01
*** med_ has joined #openstack-keystone18:01
notmorganbjornar: so there is osprofiler, and it will do exactly what you want - it isn't something we can "document" by hand, we need tooling. like i said, that is a WIP18:01
notmorganbjornar: and please do help! :)18:01
notmorganbjornar: i expect osprofiler to be landed in Newton fwiw.18:02
notmorganbjornar: support for it that is.18:02
lbragstadbjornar what did you use to gather your metrics?18:02
notmorganand it can outline all the queries and everything done, including time spend in the cache18:02
bjornarlbragstad, query logs18:02
*** henrynash has joined #openstack-keystone18:02
*** ChanServ sets mode: +v henrynash18:02
*** agireud has joined #openstack-keystone18:03
bjornarnotmorgan, one good start would be to "tag" queries18:03
notmorganbjornar: right now we also have trace level logging in keystone that will tell you how long a manager (identity_api, assigment_api, etc) will take if enabled18:03
dstaneknotmorgan: bummer..i didn't know that18:03
notmorganbjornar: which might also help.18:03
notmorgandstanek: they fixed it on their end before we fixed it on ours18:03
notmorgandstanek: and they also did approve of our choice, just hit in an unfortunate way18:04
dstaneknotmorgan: what took so long? just nobody to work on it?18:04
bjornarnotmorgan, so a: SELECT /* trace details */ foo...18:04
*** aginwala has joined #openstack-keystone18:04
notmorgandstanek: it was more of a "hey if the gate is broke don't land things" message18:04
notmorgandstanek: we (keystone) didn't know. and they went about fixing it.18:04
notmorganbjornar: if you want to work on adding that into the queires with SQLAlchemy, please do, i am not sure how to do that off the top of my head18:05
dstaneknotmorgan: ah, ok. the recent email made it seem like we knew and didn't do anything about it18:05
notmorganbjornar: the ORM somewhat makes this a bit harder.18:05
notmorgandstanek: nope, we didsomething about it. this was critizing "push code through w/o things working"18:05
openstackgerritTin Lam proposed openstack/keystoneauth: Properly set ClientException message
notmorgandstanek: vs "fix gate then push things through". since they can't test always in the gerrit CI they have external/nonvote/etc things to deal with18:06
notmorgandstanek: technically people can push through with a fail vote as i understand it18:06
notmorganbjornar: also, just make sure if you're proposing adding the trace details to the queries, that is is only enabled in "debug" mode.18:06
bjornarnotmorgan, sure!18:07
notmorganbjornar: for obvious reasons (or with a flag), this may also be something that needs to go in oslo.db18:07
notmorganbjornar: so it might be easier there. take a look in both places (our core sql code is in keystone.common.sql18:07
bjornarnotmorgan, actually it depends how "small" one makes the trace18:07
bjornarbecause if its just a "few" bytes, it might make sense to leave it on unless some flag.. I dont know18:08
notmorganbjornar: i'll tell you that as long as the trace is opt-in, i'm not opposed18:08
notmorganbut if it's always on, i'm going to need more justification / proof that is is low impact when passed through the ORM18:08
notmorganand that it doesn't do wonky things.18:08
dstanekrderose: let me know if you have questions18:08
*** aginwala has quit IRC18:09
notmorganbjornar: also here is os-profiler: which will summarize a lot of this stuff when it lands18:09
patchbotnotmorgan: patch 103368 - keystone - Integrate OSprofiler in Keystone18:09
bjornarnotmorgan, Thanks, I'll check it out.18:09
notmorganwe now have the tech to set options in the libraries from inside keystone, so that will be a newton thing i'm sure.18:10
notmorgani should poke DinaBelova about that patch see if we can getthe bits done quickly18:10
bjornarabout the templated "bug", 1550742 .. has it been this way since early 2015?18:10
notmorganbug 155074218:10
openstackbug 1550742 in OpenStack Identity (keystone) "bug listing services when using templated backend (v3)" [Medium,In progress] - Assigned to Steve Martinelli (stevemar)18:11
notmorganbjornar: yeah the templated catalog is broken and has been for a long time18:11
notmorganbjornar: we're probably better off just deprecating it tbh.18:11
bjornarnotmorgan, so noone is using it? Why?18:11
bjornarIs peoples catalogs changing on a daily basis?18:11
notmorganbjornar: people use it, but it is very badly designed. and should be redesigned if we want to keep it18:12
notmorganbjornar: basically right now it's a json blob on disk with substitutions in code.18:12
notmorganbjornar: which means typos/strings being slightly off, etc on the CMS side can break everything18:12
bjornaryeah.. but thats fine :)18:12
notmorganbecause it becomes invalid/partially valid/a totally different in-memory structure due to json load18:12
notmorganthat is terribad18:13
notmorganif we want to support that, we should have structured yaml or even json that is just the elements18:13
notmorganlet keystone format the catalog18:13
notmorgansorry, wasn't clear18:13
*** nisha_ has quit IRC18:13
notmorganit is the complete catalog as though it was rendered by the API18:13
*** petertr7 is now known as petertr7_away18:13
notmorganand we just substitute some values in18:13
*** nisha has joined #openstack-keystone18:13
notmorganso it's like saying get me the catalog, sorting that result to disk via your CMS18:13
notmorganthat is not correct :P18:14
notmorganloading some values from json/yaml and formatting it sanely would be correct18:14
bjornarnotmorgan, should not be too hard18:14
notmorganbut for the most part no one uses it, since updates to your catalog shouldn't require keystone restarts, and the DB is shared across servers18:14
notmorganso you don't need to coordinate CMS runs/restarts in the hopes that you don't get inconsistent catalogs18:15
notmorganbjornar: all solvable/workable things, but with limited use it has a low priority18:15
*** bjornar has quit IRC18:15
stevemarnotmorgan: bjornar__ actually i think that just recently broke when dstanek removed the kvs backend18:17
notmorganstevemar: likely18:17
dstanekstevemar: broke the templated catalog?18:17
*** bjornar has joined #openstack-keystone18:18
dstanekit should work fine with the exception that we not don't allow write perations18:18
*** lhcheng has joined #openstack-keystone18:24
*** ChanServ sets mode: +v lhcheng18:24
*** Guest63721 is now known as mfisch18:24
*** mfisch is now known as Guest200418:25
*** petertr7_away is now known as petertr718:26
*** raildo is now known as raildo-afk18:27
stevemardstanek: refer to bug 155074218:27
openstackbug 1550742 in OpenStack Identity (keystone) "bug listing services when using templated backend (v3)" [Medium,In progress] - Assigned to Steve Martinelli (stevemar)18:27
*** Guest2004 is now known as mfisch18:28
*** mfisch has quit IRC18:28
*** mfisch has joined #openstack-keystone18:28
*** raildo-afk is now known as raildo18:29
marekddolphm: thanks for the review.18:30
*** boris-42 has joined #openstack-keystone18:30
marekdI did swallow the exceptions because it was the quickies yet kind of robust thing to do but I don't actually think this should be merged in a way it's implemented. On the other hand I am not sure we can just remove raising Unauthorized from methods like populate_roles()18:31
*** doug-fish has quit IRC18:32
*** huats_ has quit IRC18:33
*** huats_ has joined #openstack-keystone18:33
*** huats_ has quit IRC18:33
*** huats_ has joined #openstack-keystone18:33
openstackgerritMerged openstack/python-keystoneclient: Change tests to pass session to Client
openstackgerritMerged openstack/python-keystoneclient: Link to AccessInfoV3 returned from get_raw_token_from_identity_service
*** browne has quit IRC18:33
openstackgerritMerged openstack/python-keystoneclient: Tests stop using deprecated HTTPClient.get()
*** pece has quit IRC18:34
sbezverkHello, any idea when gets merged??18:37
patchbotsbezverk: patch 285987 - keystone - Make keystone tests work on leap years18:37
stevemarsbezverk: it's still going through the check queue:
sbezverk@stevemar thank you18:39
dstanekstevemar: wow, how it that not caught in the tests?18:41
stevemardstanek: have you seen the tests?!18:41
openstackgerritRon De Rose proposed openstack/keystone: Shadow users - Shadow federated users
openstackgerritMichael Krotscheck proposed openstack/keystone: Moved CORS middleware configuration into oslo-config-generator
krotscheckbknudson: Update for ya ^^. Most notable change is that the defaults are now also set during app initialization.18:43
*** sdake has quit IRC18:43
dstanekstevemar: yep, unfortunately :-P18:43
krotscheckDoh, forgot the subdomain18:44
dstanekstevemar: actually it looks like i don't use the list_services or _list_services ever. i thought it would have been used to get the catalog18:44
dstanekstevemar: should be a super easy fix18:44
openstackgerritMichael Krotscheck proposed openstack/keystone: Moved CORS middleware configuration into oslo-config-generator
*** clenimar has quit IRC18:44
*** nisha has quit IRC18:45
dstanekstevemar: use remove 'hints' from _list_services :-)18:45
*** rk4n has quit IRC18:45
*** doug-fish has joined #openstack-keystone18:46
*** doug-fish has quit IRC18:46
*** doug-fish has joined #openstack-keystone18:47
stevemardstanek: true18:47
stevemardstanek: did you want to re-propose?18:47
dstanekstevemar: it is likely only there because i copied 'list_services' and renamed it to '_list_services' so that i could use it twice. probably not, i think the get catalog may got more confusing, but feel free to try it out and see how it looks18:48
*** vilobhmm11 has joined #openstack-keystone18:49
* dolphm is back18:49
*** pnavarro has quit IRC18:50
*** e0ne has joined #openstack-keystone18:51
*** sigmavirus24_awa is now known as sigmavirus2418:53
*** spzala has joined #openstack-keystone18:53
stevemar*welcomes dolphm back*18:54
*** jaosorior has quit IRC18:54
openstackgerritAndreas Jaeger proposed openstack/pycadf: Remove unused pngmath Sphinx extension
*** jaosorior has joined #openstack-keystone18:58
openstackgerritSteve Martinelli proposed openstack/keystone: Shadow users - Shadow federated users
dolphmrderose: how goes it?19:03
*** doug-fis_ has joined #openstack-keystone19:03
stevemarrderose: did we ditch the 'update federated user' method?19:04
henrynashayoung: (let me know if you have questions on, trying to get a thumbs up from all the cores who are listed as authors!)19:04
patchbothenrynash: patch 231289 - keystone - Projects acting as domains19:04
rderoseI hope not :)  Did I overwrite it?  Looking now...19:04
*** doug-fi__ has joined #openstack-keystone19:06
*** doug-fi__ has quit IRC19:06
*** doug-fi__ has joined #openstack-keystone19:07
rderosestevemar it's there19:07
rderosedolphm was just responding to dstanek's comments19:07
*** jaosorior has quit IRC19:07
rderosedolphm making little progress on "concrete role assignment"19:07
*** doug-fish has quit IRC19:07
*** clenimar has joined #openstack-keystone19:08
rderosedolphm I know you are hesitant about merging “shadow federated users” without having “concrete role assignments”, but I’m concerned that I won’t have time to get this in by the end of the week.  Just have tons of meetings this week.19:08
*** doug-fis_ has quit IRC19:08
rderosedolphm I do feel strongly that “shadow federated users” can be merged without "concrete role assignment"19:08
dolphmrderose: i primarily wanted it to be proposed and to see what type of work it would take to make the tests pass there; i think it has achieved that19:09
*** fpatwa_ has joined #openstack-keystone19:09
dolphmrderose: yes, the patches can be merged independently, but a revised / smaller spec needs to be proposed to newton now is all19:09
dolphmto go along with the narrower scope of work19:09
dolphm(i'll work on that right now)19:09
rderosedolphm I see, cool19:10
marekddolphm: so we are abandoning ?19:10
patchbotmarekd: patch 286169 - keystone - Role assignment resolution for shadow users.19:10
dolphmmarekd: no!19:10
stevemarrderose: i was looking at the wrong #19:11
*** dims_ has joined #openstack-keystone19:11
openstackgerritMarek Denis proposed openstack/keystone: Role assignment resolution for shadow users.
*** dims has quit IRC19:11
henrynashbknudson: I think I covered off all the points you had on (other than refactoring the migration select, which I may come back in and do)….are you Ok with this now?19:12
patchbothenrynash: patch 231289 - keystone - Projects acting as domains19:12
bknudsonhenrynash: I'll add it to my list of things to look at19:13
henrynashbknudson: thx19:13
*** fpatwa_ has quit IRC19:13
*** henrynash has quit IRC19:14
stevemarbknudson: it's on my list too :O19:16
*** doug-fi__ is now known as doug-fish19:16
*** jsavak has quit IRC19:16
openstackgerritJacek Tomasiak proposed openstack/python-keystoneclient: Fix invalidation of tokens stored by DefaultCLI plugin
*** jsavak has joined #openstack-keystone19:17
rderosemarekd: appreciate the help, but I have a patched started for this already:19:17
patchbotrderose: patch 286169 - keystone - Role assignment resolution for shadow users.19:17
patchbotrderose: patch 284943 - keystone - Shadow users - Concrete role assignments for feder...19:17
rderosemarekd what's up, are we duplicating work?19:18
*** e0ne has quit IRC19:20
dstanekrderose: i don't remember don't remember the postgres auto increment behavior, but i thought it was like oracle and used independent sequence objects19:21
rderosedstanek I see...  Wouldn't tests fail if this were the case though?19:22
dstanekrderose: are you testing on postgres?19:22
dstanekrderose: i haven't used postgres for several years so i don't quite remember what the issue was and google isn't helping too much19:23
rderosedstanek just thought that the functional tests would run the test against all support databases19:24
dstanekrderose: not that i know of19:24
rderosedstanek I'll google this some more as well19:24
rderosedstanek okay19:24
dstaneki can fire up pg in a bit too for some manual testing19:24
dstanekstevemar: do you know the answer to that?19:24
rderosedstanek that would be great19:25
stevemardstanek: whats the concern with pg?19:26
stevemarrderose: dstanek we do not test at all with postgres atm19:26
dstanekstevemar: does sqlalchemy do auto increments in pg the same as it does for mysql19:26
marekdrderose: dolph suggest could merge both patches into one, so unless you are going to do it now I will submit new patch.19:27
stevemardstanek: excellent question!19:27
stevemardstanek: i think zzzeek may know the answer to that question :)19:27
rderosemarekd okay, go for it.  in the future, it would be nice to get a heads up19:28
marekdrderose: what heads up?19:28
dstanekrderose: why are we splitting the shadow users driver from the users backends?19:28
marekddolphm: you called "catch Exception, e" as old style exception catching, where the problem was ",e" part, right?19:30
marekdinstead of "catch Exception as e"19:30
rderosedstanek because we would need to support the new methods in the ldap driver as well and that didn't make sense19:32
rderosedstanek for ldap driver, I'd have to create stub methods or something19:33
rderosedstanek so in the end it just seemed better to create a new, separate driver to support all of the shadow user functionality19:33
*** knikolla has quit IRC19:34
dstanekrderose: feels weird to me that we have two different sets of backends that access the same tables. now you can misconfigure it and break it on accident.19:34
dstanekmarekd: can the mapped auth plugin be used with the ldap identity backend?19:36
dstanekstevemar: ^19:36
marekddstanek: today mapped plugin doesnt really need any info about users from any backend19:37
rderosedstanek 2 backends, but all of the ORM mapping is still being done in backends/sql.py19:37
marekddstanek: well, it can be used as a mean of authentication and later data is read from whatever backend keystone has configured19:37
rderosedstanek we just have a separate driver for the shadow user functionality is all19:38
dstanekmarekd: ok so can a keystone instance that is setup to do ldap identity be used in a federation?19:38
dstanekrderose: what if they are not using the sql identity backend?19:39
dstanekrderose: so i guess my concern is that if it is possible for keystone to have federated authentication using the ldap backend then what is the upgrade path for those deployments? if it's even possible19:41
marekddstanek: that's what we have at cern19:41
marekdour users are stored in ldap19:41
marekdand we also provide federated access19:42
rderosedstanek if they are not using sql identity backend, shadow users driver is, and will still shadow federated users19:42
openstackgerritAlexander Makarov proposed openstack/keystone: WIP/DNM Closure table for HMT
dstanekrderose: so, will all of the tables be correctly created so that cern will still work after the upgrade?19:43
marekddstanek: i'd assume they'd need to be in sql backend19:43
marekdas ldap is ro ?19:43
marekdwhat we have is: users/grops in ldap, rest in dedicated sql (projects, domains, roles, role assignments)19:44
rderosedstanek federated auth hasn't been changed, the only difference is were shadowing federated users19:44
rderosedstanek hmm...19:45
*** fangxu has joined #openstack-keystone19:45
marekdnotmorgan: LDAP access is/will be RO ?19:45
marekdnotmorgan: i think you were driving that change.19:46
notmorganmarekd: nope wasn't me driving it. Henry and a few others were. But that is the long term plan19:46
rderosedstanek what do you mean by all of the tables?19:47
marekdnotmorgan: but it's safe to assume that no new features should rely on RW LDAP?19:47
notmorganThat would be a safe assumption19:47
marekdnotmorgan: thanks.19:47
rderosedstanek there is only the federated_user table in this patch and it does get created with the migration19:47
dstanekrderose: does an ldap deployment have all of the tables and they just are not used?19:50
rderosedstanek yes, I believe so19:51
*** knikolla has joined #openstack-keystone19:51
rderosedstanek stevemar I missed the comment about versioning the driver:
patchbotrderose: patch 279162 - keystone - Shadow users - Shadow federated users19:51
rderosedstanek stevemar: how important is this?19:52
dstanekrderose: we need to version our drivers so that we can properly change interfaces in the future19:52
*** pnavarro has joined #openstack-keystone19:54
rderosedstanek so is it just a matter of adding V9 at the end of the class name?19:54
dstanekrderose: i don't know enough about the read-only ldap identity backend to say it there'd be any weird issues.19:54
rderosedstanek I don't think there would be.  those tables should still exist in sql and will be populated for federated users.19:56
rderosedstanek and eventually we will shadow ldap users as well19:56
dstanekrderose: and it's OK if identity doesn't use those tables?19:56
*** dims_ has quit IRC19:58
*** spzala has quit IRC19:59
rderosedstanek for this patch, yes it wouldn't matter.  because the only thing we are doing is creating a local identity for federated users19:59
dstanekrderose: i think dolphm would be much better at identifying corner cases in this particular area that me19:59
rderosedstanek you raise some interesting points that we will need to consider and test20:00
*** Guest71383 is now known as redrobot20:01
dstanekrderose: yes, to you question about just adding V920:01
*** sdake has joined #openstack-keystone20:04
*** shaleh has joined #openstack-keystone20:04
*** dims has joined #openstack-keystone20:12
*** david-lyle_ is now known as david-lyle20:12
dolphmmarekd: yes, "catch" statements should use "as" instead of commas, because in py3 that's just interpreted as a tuple of exception types, where "e" is not an object that has been imported20:13
*** hockeynut_afk is now known as hockeynut20:13
*** pushkaru has quit IRC20:13
*** pushkaru has joined #openstack-keystone20:13
openstackgerritRon De Rose proposed openstack/keystone: Shadow users - Shadow federated users
*** mhickey has joined #openstack-keystone20:16
rderosestevemar dstanek added "V9" to the shadow users driver20:17
*** nkinder has quit IRC20:19
*** sdake has quit IRC20:22
marekddolphm: yeah20:22
*** fawadkhaliq has quit IRC20:22
marekdthat's what I've been asking about20:22
openstackgerritSteve Martinelli proposed openstack/keystone: V2 operations create default domain on demand
dolphmstevemar: did we stop creating the default domain as part of db_sync ?20:23
dolphmthere's no bug report or anything there20:23
*** doug-fish has quit IRC20:24
dstanekdolphm: i remember seeing somewhere that we did20:24
dolphmdstanek: new version
patchbotdolphm: patch 279162 - keystone - Shadow users - Shadow federated users20:27
notmorgandolphm: we just recently did20:28
dolphmnotmorgan: why?20:28
notmorgandolphm: bootstrap should be used instead (at least that was the reasoning) rather than baking it into a db_sync. but if we rely on it being there, it is actually better to create it on demand if it doesn't exist20:29
notmorganif you get down to it20:29
*** dims has quit IRC20:29
notmorganhaving custom/non-recreatable things baked into our migration scripts probably wasn't the best idea20:29
notmorganthen again, removing it from the scripts also may have been bad. *shrug*20:30
*** doug-fish has joined #openstack-keystone20:30
* notmorgan is only recounting history here not advocating one way or another.20:30
*** doug-fis_ has joined #openstack-keystone20:32
openstackgerritRon De Rose proposed openstack/keystone: Shadow users - Shadow federated users
*** doug-fish has quit IRC20:34
*** yarkot_ has joined #openstack-keystone20:36
dolphmnotmorgan: ah, cool20:36
*** mylu has joined #openstack-keystone20:37
*** doug-fis_ has quit IRC20:37
dolphmnotmorgan: existing liberty deployments shouldn't run bootstrap, right?20:37
notmorgandolphm: correct20:37
*** yarkot_ has quit IRC20:38
notmorgandolphm: and if existing de[ployments riun bootstrap the worst it can do is create data in the DB.20:44
notmorgandolphm: if they specify already existing roles/users/etc it does nothing20:44
notmorganexcept warn that it didn't need to create things20:44
*** knikolla has quit IRC20:45
stevemardolphm: bknudson tossed up a patch to create one if one is not created (but the user does a user/project list operation)20:46
bknudsonseems like we should be able to deprecate default_domain_id20:48
stevemarbknudson: true20:48
bknudsonif v2 creation methods are deprecated20:48
*** jsavak has quit IRC20:48
stevemarbknudson: they already are20:51
*** jsavak has joined #openstack-keystone20:54
openstackgerritMarek Denis proposed openstack/keystone: Shadow users - Concrete role assignments for federated users
stevemarmarekd: rderose going to rebase ^ on the latest... come on rebase button! don't fail me now20:59
stevemarwomp womp20:59
marekdstevemar: should be easy21:00
*** chlong_ has joined #openstack-keystone21:01
stevemarmarekd: thank you21:01
marekdstevemar: no problemo.21:03
*** rderose has quit IRC21:06
openstackgerritMerged openstack/python-keystoneclient: Correct test running instructions
openstackgerritMerged openstack/python-keystoneclient: Fix reference to ClientException
*** knikolla has joined #openstack-keystone21:07
*** pushkaru has quit IRC21:09
*** fpatwa_ has joined #openstack-keystone21:10
*** raildo is now known as raildo-afk21:12
*** fpatwa_ has quit IRC21:14
*** henrynash has joined #openstack-keystone21:16
*** ChanServ sets mode: +v henrynash21:16
*** ayoung has joined #openstack-keystone21:17
*** ChanServ sets mode: +v ayoung21:17
*** sheel has quit IRC21:17
ayounghtruta, I'm looking at  and am pertty close to approving.  One question:  how immutable it a domain?  Why do we even support update Project and or Domain?  It should be just the description field, right?21:18
patchbotayoung: patch 231289 - keystone - Projects acting as domains21:18
ayoungpatchbot ++21:18
*** jsavak has quit IRC21:19
*** jsavak has joined #openstack-keystone21:19
stevemarayoung: i believe name and description are mutable21:20
stevemarayoung: id/domain_id should not be21:20
stevemarparent_id should be mutable..21:20
ayoungserverascode, its the parent and domain stuff I want to be sure of21:20
ayoungwhy should parent be mutable?21:21
ayoungstevemar, that sounds wrong21:21
*** dims has joined #openstack-keystone21:23
henrynashayoung: you can’t move a project within a hierarchy21:24
ayounghenrynash, so what is mutable?21:24
ayounghenrynash, I would be happier if we just made projects immutable and left it there21:24
ayoungwhat would that do, besides make it harder to add a description after the fact?21:25
henrynashayoung: name, description…..and under very strict cirumstances you can change domain_id (for backward compatibiity reasons)21:25
ayoungah, name we want to be abkle to change for the cleanup21:25
ayoungwhy domain_id?  And where is the logic that protects that?21:25
henrynashayoung: you used to be able to “move” a project betwen domains by changeing the domain_id….this functionality is deprecated, but not removed21:26
ayounghenrynash, hmmm.  Don't recall that21:26
krotscheckIrony, thy name is bknudson ;)21:26
krotscheckbknudson: I'm guessing you script-patched everything in oslo because of feature freeze?21:26
krotscheckSorry- script -2'd.21:26
henrynashayoung: so (until we can formally remove that…in N I think), we let a project who’s immediate parent is a domain, but has not children have their domain_id changed21:27
bknudsonkrotscheck: yes, oslo will not have any new releases except bug fixes in M21:28
ayounghenrynash, now, there is no way a project can become a domain by accident, right?21:28
krotscheckbknudson: Serious question- I'd like to make sure set_defaults is called during api initialization, where's the right place to put that?21:28
henrynashayoung: is_domain is definitely immutable21:30
bknudsonkrotscheck: -- in keystone.server.common.configure()21:30
krotscheckbknudson: Thank you, much obliged.21:31
bknudsonkrotscheck: this change is llke config.set_default_for_default_log_levels()21:31
ayounghenrynash, not sure what it would break if a project accidentally became a domain anyway21:31
bknudsonkrotscheck: so maybe create a config.set_defaults() and change server.common.configure to call that21:31
henrynashayoung: with only top level domains, probably nothing21:31
ayounghenrynash, I'm a +2A that one.21:31
ayoungits got enough +1s and it has been around 62 revisions.  I really should take my name off as co-author21:32
ayoungmy changes are gone21:32
henrynashayoung: ok, I think many of the changes from the string of authors are gone!21:33
openstackgerritMichael Krotscheck proposed openstack/keystone: Moved CORS middleware configuration into oslo-config-generator
*** browne has joined #openstack-keystone21:33
samueldmqhenrynash: patch 231289 is on the gate21:37
patchbotsamueldmq: - keystone - Projects acting as domains21:37
samueldmqhenrynash: congrats, nicely done21:37
henrynashsamueldmq: yep, cool :-)21:37
henrynashsamueldmq: thx for all your help on it21:37
openstackgerritMichael Krotscheck proposed openstack/keystone: Consolidate configuration default overrides
krotscheckbknudson: And there's the followup that collects all of the method invocations into one ^^21:38
bknudsonkrotscheck: how do I test out the cors change? look for headers when I do a request?21:38
krotscheckbknudson: The tests should fail....21:39
krotscheckbknudson: But yes- switch out allowed_origin with a uri (, then do a curl OPTIONS request with the Origin: header.21:39
*** dims has quit IRC21:40
krotscheckWaaait a sec. Where did my tests go.21:41
krotscheckOh, discussion told me not to add tests for middleware.21:42
*** dims has joined #openstack-keystone21:44
samueldmqhenrynash: np, my pleasure :)21:46
samueldmqhenrynash: glad to see it moving21:46
henrynashsamueldmq: been a long time coming....21:46
*** browne has quit IRC21:47
samueldmqhenrynash: yep :)21:49
*** fawadkhaliq has joined #openstack-keystone21:50
*** hambuergaer has joined #openstack-keystone21:53
*** edmondsw has quit IRC21:53
stevemarayoung: ++ on approving that patch, i was just going to do the same21:53
stevemarayoung: samueldmq henrynash depending on what gets merged first (reseller or shadow users) we will need to rename 093 to 09421:53
*** pnavarro has quit IRC21:54
henrynashstevemar: understand21:54
*** hambuergaer has quit IRC21:55
stevemarsamueldmq: henrynash last patch needed for mitaka-3, cascade, whats going on there?21:56
samueldmqstevemar: so, I and henrynash agree that it needs further discussion21:57
samueldmqstevemar: because we are not 100% sure it's the right way to do it, as proposed21:58
stevemarsamueldmq: henrynash from an API perspective, i think it's fine and working... just the policy may need tweaking21:58
samueldmqstevemar: and it also may benefit from unified delegation21:58
henrynashstevemar: samueldmq and I are for postponing this one…there is still much uncertaintiy over exactly what policy rules we shoudl follow…especially when you try and think how it should work for normal tokens, trusts and federation21:58
*** vilobhmm11 has quit IRC21:58
stevemarhenrynash: samueldmq anyway we can make it strict right now? and loosen the policy if we think it is necessary ?21:59
*** vilobhmm11 has joined #openstack-keystone21:59
SamYaplehello my keystone people! question about bootstrap. without admin_token i can create the user/role/project, basically enough to get me a token. I can then use that token to create service/endpoint and then auth like normal21:59
samueldmqstevemar: is it better to make strict and relax or the opposite ?21:59
henrynashstevemar: if we want something now, I’d suggest the separate polciy endpoint option, then in teh future we can deprecate that in place of an agreed hierarchical approach22:00
*** vilobhmm11 has quit IRC22:00
SamYaplemy question is, is there a non-deprecated way to do that via python? says non-session auth is deprecated22:00
stevemarsamueldmq: one is easier to reverse than the other :)22:00
samueldmqstevemar: henrynash also the implementation would require a lot of tests for different types of tokens22:00
*** vilobhmm11 has joined #openstack-keystone22:00
samueldmqstevemar: yes, but in this case (a new policy entry) is a more relaxed approach; but if we need to change that in the future, we would need to deprecate the policy entry22:01
samueldmqas henrynash said22:01
stevemarhenrynash: i like that suggestion, separate policy for that API call... but we've never deprecated a policy entry22:01
rodrigods^ i like this one22:01
samueldmqstevemar: and based on the fact of thinking on a possible deprecation of something that is still being proposed ... it's not good22:01
rodrigodsbut not the idea of proposing to deprecate later22:02
stevemarhenrynash: samueldmq if you guys are okay with bumping it to N, we can do that22:02
rodrigodsi like it overall, because the concept of "branch" operation22:02
henrynashstevemar: true, I think once we had a hierarchial approach agreed, we would first create a config option that woudl determine which policy entry we use, then depracte the separate entry pint22:02
*** pauloewerton has quit IRC22:02
stevemarhenrynash: if we're creating all these plug points just to squeeze this in for M, then I'm OK with bumping to N22:03
samueldmqstevemar: henrynash: my heart says that, if we're going to do it, let's do it right from the start22:03
stevemarsamueldmq: henrynash hehe, all that code for handling cascade and we fall just short of the finish line :)22:04
samueldmqstevemar: yes, imo this and all the hmt things need broader discussions22:05
*** sdake has joined #openstack-keystone22:05
henrynashstevemar: I think there is a lesson here….we often tend to consider policy enforcment of API changes as an implemenation detail, whereas we probably should have a section explicitely in the specs on it22:06
*** doug-fish has joined #openstack-keystone22:06
stevemarhenrynash: definitely22:06
samueldmqstevemar: henrynash: it's basically what's happening now, like glance has support for hierarchical proejcts, and others don't, sometimes they don't even agree in the approach22:06
stevemaresepecially when it's non-obvious, like this one22:06
samueldmqhenrynash: stevemar: ++22:06
samueldmqand specifically on this one, we even asked the api-wg for advices :)22:07
* samueldmq things every project should have a member in the api-wg22:07
henrynashstevemar: separate question….was does SKIPPED mean under tests for patch in teh gate? (which is what projects as a domain now says)…22:08
*** knikolla has quit IRC22:09
SamYapleayoung: you around?22:09
ayoungSamYaple, I'm around22:09
*** dims has quit IRC22:09
SamYaplehey. posted a question above I cant find a good answer too.22:10
SamYapleany non-deprecated way to bootstrap endpoint and service without a cli tool?22:10
ayoung"hello my keystone people! question about bootstrap. without admin_token i can create the user/role/project, basically enough to get me a token. I can then use that token to create service/endpoint and then auth like normal"22:10
ayoungSamYaple, Um. Why22:10
SamYaplefollowed by "my question is, is there a non-deprecated way to do that via python? says non-session auth is deprecated"22:10
ayoungwhy is CLI a problem?22:11
SamYaplefor kolla with ansible I need to know if something has changed22:11
SamYaplealso so i dont recreate teh same service and endpoints over and over22:11
*** browne has joined #openstack-keystone22:11
SamYapleopenstack service create is not idempotent22:11
*** browne has quit IRC22:12
samueldmqhenrynash: looking22:12
samueldmqhenrynash: zuul?22:12
henrynashsamueldmq: yes….in teh gate queue22:12
*** browne has joined #openstack-keystone22:12
SamYapleayoung: basically im trying to avoid doing this non-sense
patchbotSamYaple: patch 285625 - kolla - Remove keystone admin token22:13
*** mylu_ has joined #openstack-keystone22:13
henrynashsamueldmq: seaerch for 23128922:13
*** vilobhmm11 has left #openstack-keystone22:13
*** mylu has quit IRC22:13
ayoungSamYaple, bug notmorgan about that.  Idempotentcy and not destroying a working install should be a baseline22:14
*** sdake has quit IRC22:14
*** petertr7 is now known as petertr7_away22:15
*** jsavak has quit IRC22:15
SamYaplenotmorgan: bug22:15
samueldmqhenrynash: I'm looking for answer on -infra22:15
*** jsavak has joined #openstack-keystone22:15
henrynashsamueldmq: thx22:16
*** browne has quit IRC22:17
*** sdake has joined #openstack-keystone22:18
samueldmqhenrynash: there is a black point on the side of the gate22:19
samueldmqhenrynash: hover over it22:19
samueldmqhenrynash: merge conflict! thanks to anteaya for clarifying22:19
henrynashsamueldmq: ah! got it!22:19
henrynashsamuedlmq: damn!22:20
anteayahover over the black dot22:20
openstackgerritMerged openstack/keystone: Make keystone tests work on leap years
samueldmqanteaya: thanks, appreciate your help22:21
*** browne has joined #openstack-keystone22:27
stevemarbknudson: poke about
patchbotstevemar: patch 285152 - keystone - Fix the migration issue for the user doesn't have ...22:27
bknudsonstevemar: what about it?22:27
SamYaplesee if it were me i would have just said everyone gets a day off every 4 years...
stevemarbknudson: i think davechen is looking for a bit of guidance on that patch22:28
bknudsonstevemar: I'll try it out and see if it works.22:30
samueldmqhenrynash: it needs  rebase22:32
openstackgerritBrant Knudson proposed openstack/keystone: Update developer docs for ubuntu 15.10
*** shaleh has quit IRC22:35
*** doug-fis_ has joined #openstack-keystone22:35
*** doug-fish has quit IRC22:37
henrynashsamueldmq: tried that, seems no chanegs required…puzzling22:38
openstackgerrithenry-nash proposed openstack/keystone: Projects acting as domains
*** ChanServ sets mode: +v samueldmq22:42
htrutahenrynash, samueldmq: wow. Look who's merging22:44
henrynashstevemar, ayoung, samueldmq: just had to rebase, feel free to add in some +2s so it doesn’t look like I;m forcing this in on my own :-)22:45
htrutawe should've put "Co-authored by: keystone team"22:45
*** fpatwa_ has joined #openstack-keystone22:45
*** sdake has quit IRC22:46
*** ninag has quit IRC22:47
*** doug-fis_ has quit IRC22:48
openstackgerritMerged openstack/python-keystoneclient: Document session as an argument to v3.Client
*** doug-fish has joined #openstack-keystone22:48
*** fpatwa_ has quit IRC22:50
*** jsavak has quit IRC22:50
*** diazjf has quit IRC22:52
*** doug-fish has quit IRC22:53
*** doug-fish has joined #openstack-keystone22:53
stevemarhtruta: hehe22:53
stevemarhenrynash: i'll rebase it on shadow users, since they are both using 093... and then punt it through22:55
stevemarthen baby sit the gate :)22:55
henrynashstevmar: shadow users is failing in teh check queue…so it won’t make itthrough23:01
henrynashstevemar: without changes, not sure what the problem is23:02
henrynashstevemar: shadow users is failing p34 and legacy drivers23:03
*** sdake has joined #openstack-keystone23:03
*** browne has quit IRC23:09
*** phalmos has quit IRC23:10
bretonmitaka-3 feature freeze on FEB 2923:12
bretonare we freezed?23:12
samueldmqbreton: almost :) just waiting on last changes to merge23:12
*** akscram has quit IRC23:12
samueldmqbreton: as far as I know, stevemar's right person to confirm :)23:13
*** akscram has joined #openstack-keystone23:13
*** browne has joined #openstack-keystone23:14
*** browne has quit IRC23:14
*** dims has joined #openstack-keystone23:14
*** ninag has joined #openstack-keystone23:18
*** ninag has quit IRC23:22
stevemarbreton: just about frozen. i'll be pushing shadow users and projects-as-domains tonight23:25
stevemarand tagging the master branch23:25
stevemarhenrynash: what the heckaroony, why is it failing py34 and legacy tests23:27
*** spandhe has quit IRC23:28
*** mhickey has quit IRC23:29
bknudson which is just about to merge conflicts with Projects acting as domains23:31
patchbotbknudson: patch 272007 - keystone - Use requst local in-process cache per request23:31
henrynashstevemar: py27 failed too…looks like coding error in for from_dict() at line 3023:34
patchbothenrynash: patch 279162 - keystone - Shadow users - Shadow federated users23:34
*** EinstCrazy has joined #openstack-keystone23:34
*** EinstCrazy has quit IRC23:35
*** slberger has left #openstack-keystone23:37
*** spandhe has joined #openstack-keystone23:39
*** browne has joined #openstack-keystone23:43
henrynashbknudson, stevemar: I’ll fix the merge issue with  and projects as a domain….but looks like shadow users actually has a coding error23:43
patchbothenrynash: patch 272007 - keystone - Use requst local in-process cache per request23:43
*** gordc has quit IRC23:50
*** mylu_ has quit IRC23:52
*** csoukup has quit IRC23:55
*** mylu has joined #openstack-keystone23:55

Generated by 2.14.0 by Marius Gedminas - find it at!