17:59:16 #startmeeting keystone 17:59:17 Meeting started Tue Jun 18 17:59:16 2013 UTC. The chair is ayoung. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:59:18 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:59:20 The meeting name has been set to 'keystone' 17:59:30 KEYSTONE! 17:59:33 hey 17:59:34 o/ 17:59:35 Hi! 17:59:37 hi 17:59:39 \o 17:59:45 ayoung: lol i'm around today, btw 17:59:48 dolphm_, thought you were not going to be here this week 17:59:48 o/ 17:59:52 hey 17:59:54 ayoung: next two weeks 18:00:03 ayoung: don't let me stop you :) 18:00:15 ayoung: in fact, want to run next week's meeting as well? henrynash can do july 2 18:00:20 I'll keep the home fires burning 18:00:23 Hello 18:00:30 We'll make it work. Henry and I can work together 18:00:34 you'll also have to attend the release status meeting 3 hours after this 18:00:48 ayoung: indeed 18:00:50 dolph gets a vacation???? I dont remember that in the brochure... 18:00:58 :( 18:01:08 not a vacation? 18:01:30 So looking at the agenda 18:01:32 mini vacation i suppose 18:01:33 Reminder: Havana milestone 2 cut & API-level feature freeze July 16th 18:01:52 dolphm_, care to spell out exactly what we will not allow after the 16th? 18:02:13 changes to the identity-api or anything that doc describes need to wait until icehouse after the 16th 18:02:29 dolphm_, how about configuration file changes? 18:02:45 changes can still land in identity-api during milestone 3, but they should be marked as "New in version 3.2" (to go with icehouse, rather than havana) 18:02:56 ayoung: config is fine 18:03:03 ayoung: just a light feature freeze 18:03:28 OK, so just changes that would force other services or the CLI to modify how they talk to Keystone, but changes that would affect installers etc are Ok. 18:03:31 non-api impacting features are still fair game in milestone 3 18:03:50 ayoung: as long as they're backwards compatible changes that would affect installers :) 18:04:20 how about API changes that are backward compatible? :) 18:04:22 blueprints are targeted accordingly -- there's no blueprints targeted at m3 that affect api 18:04:23 dolphm_, split identity is going to have some impact there. We can discuss if it looks like it is going to slip past H2, but right now it is looking likely to get in 18:04:28 gyee: no 18:04:35 figures 18:04:45 gyee, those can be 3.2, just not 3.1 18:04:55 gotcha 18:04:57 I think that makes it easier on everyone 18:04:58 gyee: catalog-option is milestone 2 or icehouse, for example 18:05:01 optional* 18:05:02 Cool. Next item 18:05:12 High priority bugs or immediate issues? 18:05:14 ayoung: use #topic 18:05:38 #topic High priority bugs or immediate issues 18:05:45 i'm not aware of anything new 18:05:53 All critical bugs have fix committed 18:06:02 27 Bugs marked "High" 18:06:29 5 with fix committed 18:06:47 6 with "In Progress" 18:06:58 The rest Triaged or confirmed 18:07:08 keystone-manage db_sync fails updating from migrate_version 5 is incomplete 18:07:41 I've done a lot of keystone-manage db_sync lately and haven't had problems 18:08:04 bknudson1, ISAM MySql? 18:08:10 ayoung: i think that's another innodb vs myisam failure 18:08:17 well, MyISAM is not working 18:08:45 migration 26 fails because it tries to drop FKs that aren't there. 18:08:59 Right. I have a fix for that, but need to clean it up to pass code review. 18:09:01 because MyISAM doesn't suport FKs? 18:09:11 Need to straighten out my Postgres setup to retest 18:09:23 bknudson1: ayoung: who wrote the fix to explicitly set innodb on all migrations? 18:09:25 ayoung: I put a similar change into my fix for switching all tables to InnoDB 18:09:26 did that merge? 18:09:34 https://review.openstack.org/#/c/32510/ 18:09:42 dolphm_, no, abandoned, I need to fix 18:09:59 restored and rebased 18:10:00 dolphm_: it's not done yet... I'm working on changing it so that the change is only in a new migration 18:10:14 looks like just a Pep 8 fix... 18:10:33 dolphm_: and then I ran into some weird problem where migration 7 downgrade failed. 18:10:56 bknudson1: do we not need both parts? fix unspecific migrations and migrate broken schemas properly in 23? 18:11:25 dolphm_: but I figured that out and now it's a matter of creating the FKs that should have been there in the new migration 27 18:11:37 bknudson1: ah 18:12:12 dolphm_: I should have this ready by end of day. 18:12:53 dolphm_, the appraoch we should go with for, say, DB2 support is that we are willing to make changes to support it, but they should be changes that run for all (almost all) RDBMSs 18:13:02 so that the DB2 code doesn't bitrot. 18:13:14 ayoung: I made that update to the DB2 migrations 18:13:33 bknudson1, thanks. 18:13:59 ayoung: made extensive use of the constraints helper, so that's been handy 18:14:22 Good 18:14:45 Yeah, in general we should be moving duplicated code in the migrations into helpers 18:14:57 #topic Unified Client authentication 18:15:12 And yes, that is also How do we encourage the other CLI clients to use our v3 auth client class 18:15:26 i guessed as much 18:15:27 i am working on a code update for broken credential schema in sqlite which is no-op in 23 18:15:29 ayoung: make keystoneclient good and contribute back to other clients :) 18:15:48 So, jamielennox has been battling the Kerberos and X509 type auth, and what has become obvious is that each of the clients reimplement it 18:15:56 this is code duplication, and we should fix 18:16:08 one solution is to have the other clients consume the keystone client for auth 18:16:20 dolphm_, yes, we will do Keystoneclient first 18:16:26 our users around here are also interested in having all the clis be able to do v3 auth 18:16:30 ayoung: so i though nova and glane use our client class? 18:16:43 henrynash, not in the CLI 18:16:45 (i.e., use domains) 18:16:52 henrynash, you are thinkg middleware, and yes they do 18:17:02 auth_token middleware is in the client library. 18:17:25 ayoung: I'd swear we had a discussion on this on the mailing list and it was implied the cli's do as well 18:17:35 keystoneclient or openstackclient? 18:17:42 nova , glance, etc. 18:17:45 gyee: novaclient 18:18:00 henrynash, I had two different engineers look at it. Let me see if I can get one of them here 18:18:11 I thought keystoneclient is on its way to retirement no? 18:18:17 in favor of openstackclient? 18:18:21 the keystone command line utility is 18:18:29 keystoneclient python lib will remain 18:18:39 gyee: just the CLI 18:18:40 openstack client is just the CLI 18:19:03 i c 18:19:15 rcrit, you looked at the clients. None of the other clients are useing keystoneclient that you saw, right? 18:19:41 I didn't look at all of them, just nova and glance 18:20:30 I believe cinder users keystoneclient 18:20:35 rcrit, and they do their own auth, right? 18:20:35 uses* 18:20:48 martitia_, thanks, good to know there is a precedent 18:21:00 so chmouel and joeH seems to think they use the keystone auth class 18:21:24 IIRC they do a lot of their own username/password handling, for example 18:21:37 I was looking into how to add another auth protocol and it would have required updating each client separately 18:21:39 how do you pass --user-domain-id, --project-domain-id to the other clis? 18:22:16 bknudson1, do they even support those fields? 18:22:29 ayoung: not yet, but they'll have to to do v3 auth, right? 18:22:42 We need to bring this up at the Overall meeting later on tonight 18:22:49 dolphm_, is that OK? 18:23:00 bknudson1, young: so I assume what has to happen is that they DO need change the command lines to get the new bits of auth info 18:23:19 ayoung: bring up what, exactly? 18:23:28 or one passes the parser to keystone, it adds the available auth options, and returns it 18:23:34 …but that they should be using the v2/client or access objects from keystone client….and they need to upgrade to using v3 18:23:35 henrynash: they need to change, but should they be re-architected so that they get the options from keystoneclient 18:23:53 dolphm_, in order to do Kerberos or X509 client auth as part of, say, nova, we need to modify their CLIs. They should be consuming Keystone client to do that. 18:24:04 It will take a cross-project effort 18:24:27 bknudson1: oh, you want them to reuse keystone client to get the cli parameters as well (not just pass them to a keystone auth class)? 18:24:35 ayoung: until keystoneclient has a way to *allow* other clients to consume options and stuff from us, there's nothing to bring up 18:24:42 You can always do an explicit keystone token-get and pass that to the other CLIs. 18:24:56 henrynash: yeah, that's been on the community wishlist for a long time 18:25:18 ayoung: good suggestion 18:25:22 Is there a blueprint? 18:25:38 ayoung: I think Jamie Lennox recently started a bp 18:26:01 bknudson1, heh, I meant for the common command line stuff 18:26:02 https://blueprints.launchpad.net/python-keystoneclient/+spec/consolidate-cli-auth 18:26:22 openstackclient might have something documented 18:26:44 dolphm_: ayoung: yes, that one. 18:26:58 I'm concerned if we need to wait for parameter re-use before we get Grizzly features into the other cli clients 18:27:07 dolphm_, so, if we do this, it is not going to be released on the Havana schedule anyway. Is there any specific time gate to hit, or is it just "done when it is done"? 18:27:30 ayoung: clients are not held to the 6 month schedule 18:27:48 dolphm_, right, and they don't have their own schedule, either, right? 18:27:56 ayoung: we've done at least two releases since grizzly shipped, for example, and i'd like to do another in the next week or two 18:28:00 ayoung: no 18:28:31 OK, so once we get something reasonable into Keystone, we can start working with one project at a time to get them up to speed on it 18:28:37 ayoung: ++ 18:28:52 dolphm_, ayoung: you mean, once we have v3 auth in keystone client….? 18:29:03 henrynash, that, too 18:29:11 some of this could be done in parallel. 18:29:17 henrynash: yeah, i'd like to release keystoneclient 0.3.0 when that happens 18:29:27 dolphm_: ++ 18:29:30 keystoneclient could provide the cli options and we add v3 when ready 18:29:31 #link https://review.openstack.org/#/c/21942/ 18:29:37 Is that sufficient? 18:30:15 ayoung: i haven't done a full review, but yes.. when that merges and no one screams, i'll tag v0.3.0 18:30:16 dolphm_,a young: then I would suggest novaclient and glanceclient in that order 18:30:26 I'm thinking we can merge 21942... I haven't looked at it since my last comment. 18:30:46 So if we make this work for Keystone client, we probably should do the work for Glance and Nova in parallel, to make sure that the code is organized to support them for Kerberso, etc. 18:31:11 Ready to move on, then? 18:31:32 bknudson1: to review that change, i'm just going to write something like sample_data.sh in python based on the v3 api 18:31:43 dolphm_, +1 18:32:17 yay v3 samples 18:32:26 #topic Using CADF for notification framework 18:32:30 ayoung: i'd pick a single client to pick on integrating with first, and then once that merges, you can point all the other client contributors back to it and say "we want to do this to your client next" 18:32:32 dolphm_: I'll take a quick look at it again, too. 18:32:39 dolphm_, sounds good 18:32:55 +1 on CADF 18:33:00 #link https://wiki.openstack.org/wiki/Ceilometer/blueprints/support-standard-audit-formats#Provide_support_for_auditing_events_in_standardized_formats 18:33:14 Hi, Matt Rutkowski here as bp submitter if there are any questions 18:33:35 mrutkows, what will the impact be on Keystone? 18:33:48 so since we're looking at implementing notifications, seemed useful to have a standard format for the messages (e.g., CADF) 18:34:05 The wonderful thing about standards... 18:34:16 in the ref. blueprint, our Havana goal was to establish a notification path thru Ceilometer that would allow us to audit any openstack component's APIs, starting with Nova, but after connecting with Henry, Brant and Lance we see that our work could be used beyond just the Nova component... 18:34:50 whcih would mean more oslo-incubator work 18:35:00 We understand that our filter would have to work with keystone as it has other things that may need to be logged 18:35:23 mrutkows, so the primary thing I can think of that should be common is audit logging the policy layer 18:35:31 I assume that would be done in common 18:35:59 ayoung, yes our goal would be to take the audit filter to common 18:36:03 for notifications like we are talking about in Keystone, where we want to tell other services that a project has been deactivated, does it apply? 18:36:09 as soon as we veryf it works with keystone APIs 18:36:47 mrutkows, so, what would have to change in Keystone? 18:36:49 ayoung, yes, in fact it was our hope to log/audit keystone / security events 18:37:18 ayoung, mrutkows: think you are talking at cross purposes on "common", I think young meant you modify the openstack/common/policy engine to log those events 18:37:41 mrutkows, CADF just provides a common fomat. What does the actual notifying? ceilometer? 18:37:52 topol: +1 18:37:55 ayoung, I need to work with Brant and Lance, but as long as we can remain a common middleware filter/notifier, hopefully nothing will need to change 18:38:13 topol, I would think it would be "put a message in this format onto a specific queue" from Keystone's perspectiv 18:38:21 wouldn't the notifier have to be implemented in each project from oslo-incubator? 18:38:35 the CADF format is coded under Ceilometer and the audit middleware filter uses it and has an established "audit" message type 18:38:36 i'm confused on if this is a solution to supersede bp notifications or not 18:38:48 mrutkows, please come up with a non-eventlet based approach. 18:38:51 https://blueprints.launchpad.net/keystone/+spec/notifications 18:39:02 dolphm_: I don't think it's superceding, it's picking the format for the notifications 18:39:04 or if this is just a desired format for notifications 18:39:25 dolph, it is a normative standard format 18:39:38 because the blueprint doesn't specify a format 18:39:46 that other cloud providers or companies can also use 18:40:08 fair enough, but bp notifications currently blocked and won't land in havana, so what's the goal for discussion today? 18:40:13 I think what I am concerned about is populating the fields of the log message. If all of that can be deduced from a simple LOG.error, fine. Need to parse that spec to see if the places we need to log will need to provide additional info and where that info comes from 18:40:29 mrutkows, in your experience, how hard is it to come up with the data for a log point? 18:40:35 notification point 18:40:40 mrutkows, so what folks are trying to figure out is do they still have to use some queue capability and all you provide is a std format or do they leverare ceilometer to get the queue capability 18:40:56 ayoung, happy to have a side call to review what CADF has in it 18:41:10 it is extensible 18:41:14 comes down to what can celilomter provide us infrastrcuture wise 18:41:16 mrutkows, is is simple 18:41:23 topol, look at the link 18:41:47 What, when, Who, OnWhat, Where, FromWhere, ToWhere. 18:41:58 there 18:42:10 I'm less worried about extending it as making is simple to fill out that data 18:42:27 it was designed by security architects from many companies to be ISO/NIST audit compliant 18:42:36 mrutkows, so a simple "how to guide" for that will be helpful. 18:42:37 along with other audit frameworks 18:42:46 topol: ++ 18:42:53 ayoung, +100, need time... 18:42:54 ayoung, just did. It mentions leveraging CADF. +1 on that. I was curious if ceilometer plays a role here or not. looks like no 18:43:14 my daughter gets married this weekend so am out for a week or so 18:43:28 topol, more likely Keystone produce events, and ceilometer plays middleman. 18:43:34 Mazel Tov 18:43:36 mrutkows, congrats 18:43:55 ayoung, agree, the api path can be a good start 18:43:59 ayoung, yep, ceilometer will just listen for the event notifications similar to how it does with other projects. 18:44:01 ayoung, agreed! 18:44:03 and the notifier can be called apart from the filter 18:44:06 Ok...good stuff. Moving on 18:44:17 do we have a BP thats shows how everything fits together? 18:44:20 #topic Gyee's patch 18:44:28 yes, I am taking liberties, but we are running out of time 18:44:45 #link https://review.openstack.org/#/c/29021/ 18:45:06 ayoung, the pluggable token one? sure I can break it up if you guys can't review that much code 18:45:07 This is, I think, pretty important to get in, but I have concerns about its, let say "reviewability" 18:45:15 just have had the time the last few days 18:45:18 haven't 18:45:19 of course 18:45:48 whatever make you happy boss :) 18:46:07 gyee: I started looking at it but it's hard to get enough time to get through it all. 18:46:19 gyee, so aside from any bug fixes that slipped in there, like the JSON policy one that can be split out stand alone 18:46:28 bknudson1, I can break it up into chunks as ayoung suggested 18:46:40 I'd like to see the reordering that does no functionality change as a stand alone 18:46:57 Does't really matter the size so long as we can say "It should behave the same before as after" 18:47:17 ayoung, yeah, I will have to get the dependencies correct 18:47:30 like v2 changes depends on v3 changes, etc 18:48:23 gyee, sounds good. 18:48:39 ayoung, thanks for bring it up 18:48:48 gyee, I know jamielennox looked at it, in the context again of the Kerberos stuff, trying to make sure we only have to get it "right" at one point 18:49:03 #topic Get /catalog behaviour in opt-out of service catalog blueprint 18:49:19 #link https://blueprints.launchpad.net/keystone/+spec/catalog-optional 18:49:27 (i think i answered this one in the bp?) 18:49:27 is everybody OK with requiring token for GET /catalog? 18:49:35 sounds like an easy decision 18:49:44 assigned to guang yee, but I suspect you do not have bandwith for it 18:49:51 can someone else p[ick it up? topol? 18:50:06 <[1]fabio> I am already working on it 18:50:06 I know simo was having some concern last week about requiring token for GET /catalog 18:50:20 ayoung, what needs picked up? 18:50:27 topol, you too slow 18:50:31 ayoung, fabio is working on it 18:50:33 [1]fabio, status? 18:50:42 indeed 18:50:45 I think he should have a review ready this week 18:50:56 <[1]fabio> I have implemented the part that removes the catalog from the token request 18:51:13 <[1]fabio> and I asked clarifications for the get /catalog part 18:51:29 <[1]fabio> so now that we have consensus I will continue 18:51:35 what is your launchpad id [1]fabio ? 18:51:39 <[1]fabio> hopefully get something by next meeting 18:52:03 cool 18:52:22 I'll update the BP 18:52:33 #topic High priority code reviews 18:52:41 Role assignment API w/ inheritance 18:52:48 OK 18:52:51 #link https://review.openstack.org/#/c/29781/ 18:52:59 (i'd like to make this a permanent feature on the meeting agenda, btw) 18:53:20 +1 18:53:42 not a bad idea 18:53:53 <[1]fabio> ayoung: my full name in launch pad is Fabio Giannetti 18:53:55 is this just a list of reviews to look at? 18:53:56 OK, so this one is getting attention. Anything more need to be said? 18:54:00 so as everyone (should) know, this bp is the "stepping stone" one…assuming we can't get the whole "role assignment as a 1st class entity)_ in in time 18:54:00 henrynash, I am fine with implementing the proposed APIs as extension for now and revisit role-assignment in icehouse 18:54:20 gyee +1 18:54:43 <[1]fabio> gyee +1 18:54:49 however, while this one might be ok as an extension, the other one probably should be core 18:54:51 [1]fabio, you are officialy on the hook! 18:55:01 henrynash, the other one being... 18:55:13 <[1]fabio> ayoung: Ok, thanks :-) 18:55:14 yes, I think that one is aling with role-assignment 18:55:26 https://review.openstack.org/#/c/32394/ 18:55:36 #link https://review.openstack.org/#/c/32394/ 18:55:50 this is a replacement api for the two broken ones 18:56:10 …to find out what role assignments a user/project/domain has 18:56:25 henrynash, +1 on https://review.openstack.org/#/c/32394/ 18:56:35 OK, will look at both of them. 4 minutes remainint 18:56:36 This should be core, I think, (agreed the inheritance bit would only be active with the extesinsion) 18:56:48 that one is very useful 18:57:14 henrynash +1 18:57:18 #topic Open discussion 18:57:46 Does the keystone server advertise its version? 18:57:46 henrynash, role-assignment or role_assignment? 18:57:52 bknudson1, yes 18:57:55 dash or underscore 18:57:58 so that a user knows that /role-assignments is available? 18:57:59 you can get the versions from the / url 18:58:06 gyee: hmm, good point 18:58:12 well, the version of the APIs that it supports 18:58:15 so it'll say 3.1 in H? 18:58:17 so - weird 'bug' for you guys that I haven't really been able to sort out 18:58:35 bknudson1: yes, GET / 18:58:43 apparently, attempting to run keystone unittests via tox on a devstack node fails 18:58:43 gyeeL your view? 18:58:47 that makes NO SENSE of course 18:58:48 gyee: dashes 18:58:59 dolphm_, gyee: ok 18:59:01 but I figured I'd toss it you guys' way in case anyone gets bored 18:59:01 mordred, I'll look after the meeting 18:59:10 dolphm_, ok that's fine, just want to make sure 18:59:16 ayoung: I can come up with zero reasons why it should matter 18:59:33 mordred: what fails? 18:59:36 mordred, its code, not magic. Much as they may appear similar 18:59:38 mordred, which bug? 18:59:53 mordred: details? 18:59:56 gyee, dolphm_: ok, i;ll redraft both of those bps with the extension in mind for inheritance and the GEt role-assignments in core 19:00:05 fetching the old clients? that's a weird thing keystone does. 19:00:07 BTW, I need a re-review of the LDAP Shim b gone patch, as I had to reduce it a little in scope upon rebase 19:00:10 dolphm_, bknudson1, ayoung, gyee: anteaya ran in to it 19:00:14 henrynash, sounds good 19:00:18 how about /roleAssignment 19:00:20 and I dont think it fails anywhere obviously like fetchin gthings 19:00:21 henrynash, bknudson1 gyee dolphm_, can you take a look 19:00:28 no dash no underscore 19:00:28 https://bugs.launchpad.net/keystone/+bug/1191999 ? 19:00:29 o/ 19:00:30 Launchpad bug 1191999 in keystone "unittests should not require internet access" [Undecided,New] 19:00:33 anteaya: have you filed a keystone bug about the test failures? 19:00:39 ah, henrynash alread -1ed. Cool 19:00:41 OK, times up 19:00:44 atiwari: no 19:00:47 #endmeeting