18:07:05 #startmeeting keystone 18:07:06 Meeting started Tue Aug 27 18:07:05 2013 UTC and is due to finish in 60 minutes. The chair is henrynash. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:07:07 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:07:09 The meeting name has been set to 'keystone' 18:07:16 I'm glad we're not on the marketing committee 18:07:20 #topic Feature Freeze 18:07:31 indeed 18:07:47 Aug 28 18:07:47 who has changes that they are trying to get in 18:07:57 (says me who has 2 or maybe 3) 18:08:01 o/ 18:08:01 pretty much everybody :) 18:08:06 Not me... 18:08:18 I learned my lesson the hard way 18:08:20 do we -2 anything that looks like a new feature? 18:08:31 bknudson, yep 18:08:50 bknudson: if it's not in the bp that was agreed, then yes 18:08:53 it will be allowed to merge as soon as the Havana RC branch is created. Master becomes Icehouse 18:09:13 #action dolphm publish when we can start Icehouse development 18:09:30 ayoung: is meetbot not listening to you? 18:09:35 milestone 3 cute is sept 4, so I assume it will happen around then 18:09:38 can somebody review fabio's endpoint filtering extension? 18:09:42 that one's real close 18:09:45 morganfainberg, doesn't matter, it will show up in the log 18:09:47 should be able to start icehouse development now, just don't expect it to be merged. 18:09:51 gyee, url? 18:10:04 #link https://review.openstack.org/#/c/33118/31 18:10:14 gyee: it's on my todo today, it is def. close. 18:10:29 thanks guys 18:10:59 so I have two definites that need to go in: 18:11:05 31? Bah. Wake me once he breaks 50. That when things get interesting 18:11:07 #link https://review.openstack.org/#/c/38308/ 18:11:09 right stevemar ? 18:11:10 henrynash, I am reviewing them 18:11:28 #link https://review.openstack.org/#/c/43257/ 18:12:09 and I need to check with dolphm on #link https://review.openstack.org/#/c/43581/ 18:12:14 I think I can Approve endpoitn filtering, 18:12:28 ayoung, thank you sir 18:12:54 I also have a few that need some hard looks. 18:12:55 ..which is ready, but want to get his view on whether we go ahead with the originally contemplated page sematics, or where we leave that to IceHouse as it was somewhat contentious 18:12:58 ayoung: thanks 18:13:06 target entities was real close, too. henrynash is jenkins failure spurious? 18:13:19 ayoung: yes, think so 18:13:34 fabiogia, understand, I say that because it is an extension, and should be disabled by default. Low risk. 18:13:50 ayoung: gyee suggested I add a configuration.rst section on policy rules, which I think is a good idea 18:13:50 henrynash's is higher risk, but I've looked at it a few times and know the scope 18:14:05 ayoung: I'll do that after this meeting 18:14:17 filter support is a little riskier 18:14:45 henrynash +1 on adding a section on policy rules. Defintely would like a rosetta stone for that 18:15:11 ++ 18:15:13 I would be motivated to postpone filters to icehouse. That would make jaypipes sad, but I'm used to doing that to him 18:15:16 ayoung: I think we have to put filtering in - I'll commit to ramping on the testing ahead of RC 18:15:18 ayoung, I like the admin domain concept, its a much cleaner deployment pattern 18:15:35 gyee, me too 18:15:41 gyee +1 18:15:43 ayoung: lol :) that's fine with me. better to be done right. 18:16:08 ayoung: I'd really like filters in there….there doesn't seem a debate about it…just the testing that we need to do 18:16:09 what is the admin domain concept? other than what is described by the name 18:16:28 ayoung: if we are pushing filtering to Icehouse, i'll want to propose a couple more patchsets for caching around the list methods. 18:16:28 just an expected deployment pattern 18:16:29 henrynash, it isn't a question of whether the approach is right...it is the risk 18:16:31 ayoung, for filters, passing query params to the drivers is a good start 18:17:03 we probably will need to make some changes along the way, but at least give us a starting point 18:17:16 ayoung: agreed. and we have an extra week of testing since we are freezing early 18:17:23 gyee, ... I think we are SQL stupid in OpenStack in general and Keystone specifically. I suspect that we do way too mcuh sql generation,. and I don't want to open Keystone up for an injection attack 18:17:42 I'd be happier if we forced everything to use sored procedures 18:17:51 heh 18:17:53 stored 18:17:53 ayoung, we are doing param binding 18:18:04 so sql injection is unlikely 18:18:36 "unlikely" 18:18:42 yeah, lets punt 18:18:48 jaypipes: (fyi, I agree with deferring pagination to IceHouse to make we sure we get the api right, even though that is ready to go) 18:19:01 ayoung: let's punt, what? 18:19:08 pagination to IceHouse, fine with me 18:19:10 punt on what? 18:19:12 henrynash, lets punt filtering to Icehouse. 18:19:15 but can we at least get filters in? 18:19:20 pagination to icehouse sounds good. 18:19:28 ayoung: not sure I'm ready to agree 18:19:28 gyee, I won't block it, but I'm not going to approve it. 18:19:30 agree 18:19:42 ayoungL pagination to IceHouse, yep 18:19:57 ayoung: that's fair 18:20:05 henrynash, filtering is a prereq for pagination, right? 18:20:14 ayoung: yes 18:20:21 why dont we have enough runway that you feel comfortable with filtering? 18:20:33 now, you make the argument that filtering is not an API change, just an implementation of what is in the spec 18:20:40 ayoung: they are separate patches 18:20:41 I think that breaks the H2 API freeze rule 18:20:58 ayoung: how come? 18:21:08 all I care is get the query params into the drivers so I can do my thing 18:21:26 can we split out the part that gets the query params into the drivers? 18:21:31 gyee, you are not going to get them for LDAP anyway. The best we are going to get is SQL this go round 18:21:58 I'll let y'all make the choice, but I think it is a mistake, and suggest we let filtering land in Icehouse 1 18:22:28 bknduson: well we already do filtering, we just do it at a high-level (controller)…. all this does is let the drives implement that same semantics 18:22:29 what filtering do we allow? 18:22:32 what's in the spec? 18:22:33 bknudson, that sound like a good idea 18:22:34 i like bknudson's plan. get the query params down to the driver, but maybe not implment the filtering 18:22:49 at the driver. if someone wants to add that in on their own, they at least have it. 18:23:09 fyi, I have some custom (internal) drivers 18:23:24 henrynash: gyee: it sounds we've already got what gyee wants. 18:23:25 morganfainberg, yes 18:23:31 s/implement/do anything with it yet/ 18:23:34 bknudson: yes 18:23:48 so we will break filtering for LDAP then, since the LDAP drivers don't implement it and we are going to remove it from the controller? 18:23:59 it's not removed from the controller 18:24:08 ayoung: that isn't the way it works 18:24:08 the controller handles what the backend doesn't 18:24:09 So we are going to execute the filter twice? 18:24:15 bknudson: exactly 18:24:36 ayoung: no, the driver removes the filter from the query_dict if it has satisfied it 18:24:41 the controller modifies the query string to remove those that it handles. 18:24:53 exactly 18:24:55 bknudson the driver 18:25:08 sorry, should have said driver and not controller there. 18:25:20 I'll step out of the argument: I've said my piece 18:25:40 henrynash: drivers don't do any filtering at this point? 18:25:59 bkndson: the sql one for assignment and identity does 18:26:30 that's the part that gyee and ayoung are concerned about 18:27:03 bknduson: that was the whole point of this bp, that we started at the last summit, we desoped to pull out just filtering.... 18:27:56 so without the filtering option arent we seeing big problems in production environments with performance? 18:28:09 topol, yeah. 18:28:10 topol, I 18:28:12 topol, yeah 18:28:14 topol: yep. 18:28:20 ve heard that list users is a big one from horizon 18:28:31 we dont' want LDAP to take hours and return you a few thousand entries :) 18:28:31 I am a fan of filtering, just not a 1/2 solution at the last minute 18:28:48 ayoung: why is it 1/2 a solution 18:28:51 gyee, we don't have a filtering patch for LDAP ready to go, and I think that one is the real problem, 18:28:51 1/2 solution > no solution 18:28:58 henrynash, LDAP 18:29:00 do we support partial matches? 18:29:15 bknudson: i think that code is there, but not enabled. 18:29:20 I didn't look a the filtering spec so don't know what was expected. 18:29:21 bknduson: I agreed that should go to ice house, there is a separate review for that 18:29:42 Im confused. is it viewed as a half solution or just not enough time to test? 18:30:09 topol, both 18:30:39 I like simo's argument on KDS better, code ain't going to mature to sitting there :) 18:30:39 topol: so the framework is the full solution, then any given driver can whose to satisfy filters to not 18:30:49 s/to/by/ 18:31:13 gyee, are you saying lets get it out there so we can kick the tires on it? 18:31:21 topol: today we have the sql assignment and identity drivers that do this, the LDAP one does not (which is OK for compatibility, but you'll get no performance increase for LDAp either) 18:31:21 I don't think we want different backends handling partial match queries differently. 18:31:39 OK...we should move on 18:31:49 bknduson: there is no partial backend support yet enabled 18:31:54 Quotas is on the BP list, and I think it has not had any activitry in a while 18:31:57 ayoung: agreed 18:32:03 https://review.openstack.org/#/c/40568/ 18:32:27 oh, wait, 26 Aug... 18:32:37 also, he's been working on the API spec. 18:32:49 anyone know Dmitri's IRC handle? 18:32:50 which looks close. 18:32:58 looks like no love from Jenkins either 18:33:09 -1s all around 18:33:18 ayoung: nope 18:33:24 it is work in progress 18:34:03 so it would seem to be on track to be deferred 18:34:04 , no? 18:34:28 topol, unless he can get it done by code freeze 18:34:35 topol: it is looking like early Icehouse to me. but he might get it done. 18:34:45 yeah...extension, and would be good to have in, but no other core project is going to be consuming it in Havana 18:34:57 retarget? 18:35:15 based on the comments inline i'd suggest push to I 18:35:27 jamielennox +1 18:35:28 ayoung: +1, aim for early icehouse so core projects can work to consume it 18:35:41 henrynash, since meetbot doesn't listen to me, can you try #action ing that? 18:35:56 ayoung: +1, re-target 18:36:06 #action retarget quotas to Icehouse 1 18:36:06 #action defer https://review.openstack.org/#/c/40568/ to I 18:36:15 meh 18:36:24 it might be getting it... 18:36:25 * topol meetbot needs cpr 18:36:31 doesn't listen to me either 18:36:38 topol, maybe, or it might be doing it silently 18:36:46 we'll find out when we read the minutes 18:37:12 #action defer pagination to I 18:37:15 somebody doing screen capture right? 18:37:19 just in case 18:37:32 henrynash, the only high bugs on there have your name on them, 18:37:33 theres always the meeting log right? 18:37:45 ah, wait 18:37:51 bknudson, you have an AD related one 18:37:53 ayoung: so I'll be on them next 18:38:03 ayoung: https://review.openstack.org/#/c/41515/ 18:38:04 #link https://launchpad.net/keystone/+milestone/havana-3 18:38:20 #link https://review.openstack.org/#/c/41208/ 18:38:32 bknudson, looking now 18:38:35 if people are willing to put eyes on caching, i'd appreciate it. 18:38:55 morganfainberg, you got it 18:39:05 morganfainberg, me 2 18:39:08 and the dependent changesets. 18:39:22 morganfainberg, will look today 18:39:34 morganfainberg: agreed 18:39:36 i expect to have 1 or 2 more small changesets in the next day to add to the chain, but i need to sync w/ henrynash about filtering 18:40:01 filtering, you thinking caching filtering? 18:40:32 gyee: talked with Dolphm, the get_* methods will be cached and aggressively invalidated as needed 18:40:53 gyee: the list_* methods will be cached, but may return slightly stale data (low cache_times, configurable) 18:40:54 please don't cache filtering 18:41:11 gyee: what's the deal on endpoint filtering? https://review.openstack.org/#/c/33118/ 18:41:13 you are going to blow up the cache in the hurry 18:41:26 gyee: is it going in? 18:41:32 stevemar, yeah. 18:41:35 stevemar, yes 18:41:48 gyee: i can avoid caching filters specifically. i'll talk w/ you after meeting so i'm on the same page. 18:41:54 stevemar, I'm giving it one last look after the meeting and then I'll approve, assuming all is well 18:42:17 ayoung: cool, it needs some +2s if we want it in 18:42:39 gyee: it's why i didn't implment for list methods yet. 18:42:57 morganfainberg, yeah, list_* will have heavy impact on the cache 18:43:40 if you are going to do it, make it optional and configurable 18:44:04 henrynash, I have someone that work on trying to replicate https://bugs.launchpad.net/keystone/+bug/1211445 if you have been unable to 18:44:06 Launchpad bug 1211445 in keystone "deleting an unassigned role causes 500" [High,Confirmed] 18:44:58 ayoung: (hey, hello meetbot) that would be good..both Dolphm and I failed to reproduce 18:45:45 henrynash, OK. I'll reassign to him. 18:46:15 ayoung: although maybe that one ISN"t relate (as suggested) to the other…(which is the one we could not reproduce: https://bugs.launchpad.net/keystone/+bug/1210590 18:46:17 Launchpad bug 1210590 in keystone "Split backend crashes with AttributeError" [High,Invalid] 18:47:09 henrynash, I'll have him focus on just the "listing projects..." bug 18:47:16 anyting else high priority? 18:48:07 ayoung, give me 5 min at the end for client stuff 18:48:09 gyee, filter endpoint based on scope https://blueprints.launchpad.net/keystone/+spec/endpoint-filtering is covered by fabiogia 's patch right? 18:48:27 (oops!) getting a new keystone contributor up to speed and just realized what time it was... hope ya'll are being productive ;) 18:48:31 showing as slow progress, but it should be about to get in 18:48:34 dolphm_, very 18:48:48 we were lost without our fearless leader 18:48:50 dolphm_, we voted to postpone quotas to I1. 18:48:57 there are 34 high priority bugs, or which about half have fixes: https://bugs.launchpad.net/keystone/+bugs?search=Search&field.importance=High&field.status=New&field.status=Incomplete&field.status=Confirmed&field.status=Triaged&field.status=In+Progress&field.status=Fix+Committed 18:49:22 bugs can be worked on between FF and RC? 18:49:26 ayoung, yes 18:49:43 dolphm_, since quotas is still taggedas WIP, doubt it will be finished by tomorrow 18:49:53 morganfainberg, yes 18:49:54 morganfainberg: i would hope so 18:49:58 ayoung: interesting 18:50:09 morganfainberg, they become blockers 18:50:20 ayoung: nod. 18:50:41 i expect to have soem time to start working on bugs here once we're past the mad rush to the FF 18:50:46 dolphm_, yeah. Seems like quotas is close, but I don't think the other projects will be able to consume it, so it makes sense to defer 18:51:17 dolphm_, unless he was all ready to go and just working out minor kinks...do you know Dmitri's IRC handle? we can ask him 18:52:52 I think we are thorugh the high-priority items for H3? 18:54:50 dolphm_, we spend a good amount of time discussing filters. My vote is to postpone them, as we will only have SQL and not LDAP, and because it is, as I see it, high risk. I won't -2 it, but recommend we postpone. THere are many voices that disagree 18:55:10 dolphm: we agreed that pagination should be deferred, but there wa disagreement on filtering 18:55:47 alright, so i realize that this will be a bit of a quiet period for client because everyone has a lot of reviews anyway but i want to poke people again to look at client as well (no feature freeze) 18:55:47 morganfainberg, if we cache at the driver level and filter at the controller level, we'll have a workable solution, no? 18:56:01 ayoung: i think so. 18:56:09 ayoung: it makes the caching easier too. 18:56:14 particularly https://blueprints.launchpad.net/python-keystoneclient/+spec/auth-plugins which is the base stuff for aababilov's APIClient work 18:56:38 jamielennox, +1 18:56:44 ayoung: but it doesn't solve the long load really hard hit back end (beyond some basic level) for the list_* methoids, like list_users 18:56:55 it'd be good if people could at least have a skim through, the first two base patches are up and i think it's a good direction - it will make us much easier to consume 18:56:56 jamielennox, that's the same impl as Nova's right? 18:57:07 gyee, that's the intention 18:57:20 morganfainberg, it will be expensive the first time it is hit, and then cached, and then expensive again when the cache is invalidated, right? 18:57:33 i want it developed in keystone first but and then we'll try to make it nova compatible - if we have to tweak 1 or 2 things in nova it should be ok 18:57:33 morganfainberg: yeah, we can't have a caching solution the does not allow filtering at the driver level (whether are do that in H or I) 18:57:40 stevemar, I am guessing oath will make use of it? 18:58:03 ayoung: yes. 18:58:37 gyee: make use of caching? 18:58:46 2 mins to go.... 18:58:50 stevemar, no, make use of pluggable auth in the client side 18:59:08 the one jamielennox is working on 18:59:14 gyee, stevemar i haven't looked at that 18:59:16 gyee: oh i'm a bit behind, sounds like it would be a great fit 18:59:20 morganfainberg, I suspect also that the caching will not be used by Apache. 18:59:28 it sounds like a good fit though 18:59:52 ayoung: we can discuss that after the meeting, but not sure on that front. 19:00:05 so this is a a Puppet review, but topol and other HTTPD fans should look at it https://review.openstack.org/#/c/29059/ 19:00:06 jamielennox, i'm knee deep in keystoneclient right now, learning a lot, hopefully i can help you out a bit, soon 19:00:15 ok timesup 19:00:18 stevemar, sounds good 19:00:28 ayoung, I will look 19:00:31 ayoung: thanks! sounds rational.. I'm only on my phone, I'll probably have questions when I get back to my desk 19:00:35 #endmeeting