18:01:19 #startmeeting keystone 18:01:20 Meeting started Tue May 14 18:01:19 2013 UTC. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:21 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:24 The meeting name has been set to 'keystone' 18:01:24 #link https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting#Agenda_for_next_meeting 18:01:34 the agenda is giant today 18:01:53 let's just talk about ldap. 18:01:57 and sort of a mess 18:01:59 dolphm, yeah, that is my doing. I girue I would make people aware of those isues, not expecting them to be resolved 18:02:09 no, we can't just talk about LDAP 18:02:17 sure we can ;) 18:02:18 but first 18:02:20 Can we get some feedback on https://review.openstack.org/#/c/25517 18:02:22 wow, dolphm, I forgot to add the pluggable token management to the agenda 18:02:26 there is a lot going on, and we need core to be engaged across the board 18:02:29 #topic Havana milestone 1 18:02:30 I posted the code yesterday 18:02:32 #link https://launchpad.net/keystone/+milestone/havana-1 18:02:38 #link https://wiki.openstack.org/wiki/Havana_Release_Schedule 18:03:12 just an announcement/reminder- havana-m1 is about two weeks out, if you don't think a bp will make it, or are working on a bp should be targeted to m1, let me know 18:03:15 dolphm, so the anony binding is the riks issue there, since it isn't started 18:03:44 and I think topolino is on vacation? 18:03:44 i believe the milestone will be forked on may 28th 18:03:56 bknudson: yep, he is 18:03:58 #topic High priority issues? 18:04:06 anything not on the agenda already? 18:04:08 bknudson, the actual impl is pretty small, can someone else from IBM pick it up? 18:04:15 ayoung: noted 18:04:33 topol is the one with the minions. cat's away kind of thing. 18:04:41 and no topol present 18:04:48 and sahdev is gone, too 18:04:49 i also remember it being a very simple bp 18:04:53 thereotically 18:04:59 it should be like 10 lines of code. 18:05:13 dolphm, for anonymous, yeah, the one thing I asked is that instead of deducing anonymous, he make it an explicit enumeration value...I can take that one 18:05:18 I hate to promise to do something and then find I'm too busy, but I think I could pick it up. 18:05:26 bknudson, do you have LDAP exp? 18:05:38 ayoung: I've got some LDAP exp. Worked on IBM's server for a few years. 18:05:43 if you take it, I can help you. 18:05:45 bknudson: ayoung: first one to code review wins 18:05:55 ok, I'll put it on my list. 18:05:59 which one? 18:06:03 I seem to be better about creating work than finishing it. 18:06:15 bp keystone-ldap-anonymous-binding 18:06:19 https://blueprints.launchpad.net/keystone/+spec/keystone-ldap-anonymous-binding 18:06:46 yeah, should be easy 18:07:08 token flush is under review. the one thing that might mitigate is if we get some agreement on the revoked token table 18:07:22 bknudson: it'd be tough to beat ayoung's stream of consciousness blueprinting ;) 18:07:31 #link https://review.openstack.org/#/c/28859/ 18:07:36 I tend to report bugs. 18:07:45 #topic High priority code reviews 18:07:59 bknudson: nothing wrong with that! 18:07:59 dolphm, pluggable token management 18:08:08 https://review.openstack.org/#/c/25517 18:08:14 #link https://review.openstack.org/#/c/29021/ 18:08:22 gyee: ^ 18:08:32 yeah, 1 KLOC worth of shit 18:08:59 rkanade, I'll look after the meeting, promise 18:09:11 #action ayoung to review https://review.openstack.org/#/c/29021/ right after meeting 18:09:13 thanks adam :) 18:09:15 gyee: excellent progress on this…I'll go through it in detail 18:09:17 there it is in the notes! 18:09:21 obviously code reviews for/blocking bp's have highest priority, so besides gyee's that also includes: 18:09:23 #link https://review.openstack.org/#/c/28133/ 18:09:28 we've got competing fixes for the atomic problem... jay pipes had one too, I think. 18:09:28 #link https://review.openstack.org/#/c/27881/ 18:09:42 bknudson: have links to those? 18:09:48 Jay pipes review for atomic issue doesnt work 18:10:01 rkanade: link to review? 18:10:20 https://review.openstack.org/#/c/27507/1 18:10:25 #link https://review.openstack.org/#/c/27507/ 18:10:28 mentioned the issue on that review 18:10:29 jay pipes is abandoned. 18:10:41 code review Tuesday, keep'em coming :) 18:10:44 rkanade, aside from "user create needs to be atomic" what bug is this really fixing? 18:10:55 bknudson: not intentionally though 18:11:19 don't have to answer here, but update the bug report with an actualy "this is causing problem blah..." 18:11:37 it is fixing other 4 apis too 1. update_user_project 2. delete_domain 3. delete_project 4. delete_user 18:12:08 what is update_user_project? 18:12:10 because, while I am sympathetic, LDAP is not going to play well. And I don't think it is smart to treat sql as a more equal backend 18:12:20 it is an api call, it needs to be atomic 18:12:23 ayoung: +1 18:12:34 all these are api calls which need to be atomic 18:12:49 rkanade, understand, I am propsing that we split identity into 3 parts 18:13:02 and so domains, users, and projects/roles might all be in 3 separate backends 18:13:16 so, kiss your integrity constriants goodbye 18:13:26 ayoung: +1 18:13:31 nice 18:13:43 3 sql backend instances ? 18:13:47 gyee, if you quote me, I want full attribution 18:13:51 rkanade, nope 18:14:05 rkanade, I would see domains in a flat file, idenitity in LDAP, and projects in SQL 18:14:15 being them most common impl down the road 18:14:23 So no sql backend for identity ? 18:14:58 why split up into multiple stores before the single store models work? 18:14:59 ayoung: that's essentially what the bp: https://blueprints.launchpad.net/keystone/+spec/auth-domain-architecture 18:15:02 rkanade, it will be there, but don't expect the calls to be atomic across domains, idenityt, and projects 18:15:18 ayoung: except that I don't necessarily agree that domains will be a flat file 18:15:39 ok, but that wont fix the atomicity issue right ? 18:15:41 henrynash, sure 18:16:09 rkanade, so long as it is atomic within a single backend, I am OK with it...but I roles assignments are going to be separate from users 18:16:18 update_user_project can't be atomic 18:16:26 i see 18:16:29 delete_user probabny not either 18:16:43 delete_domain will span 3 backends 18:16:45 maybe 18:16:45 asgreene: it's an issue of whether identity needs to be defined and tested…ie. where authentication and authorisation needs to take place….and it may well be in non-openstack systems 18:16:51 atomicity will still be problem, but the fix won't be "put it in a single db transaction" 18:17:04 ok, i can fix issue for single backend for now , is the session code in the review alright? 18:17:10 henrynash, to your point, domains will have multiple backends. I just like the flat file cuz I'm a control freak 18:17:25 and not a "flat" file, but kindof textured and bumpy and json like 18:17:26 bknudson: agreed…we need to make a s/w resilient and expectant to find integrity issues 18:17:31 @bknudson : can you provide some alternative ? 18:17:38 bknudson: +1 18:17:43 ayoung: :-) 18:18:22 ayoung: sounds like yaml 18:18:31 rkanade: I don't think there's an obvious fix for it. 18:18:35 bknudson: +1 18:18:36 dolphm, btw, I'll communicate with jamielennox tonight, and we should have an updated token flush for tomorrow 18:18:48 ayoung: awesome, thanks 18:18:52 ayoung: i'm looking forward to that 18:18:56 any specific reason why db transactions arent ok? 18:18:58 dolphm, maybe. Guess it is time to learn "yet another..." 18:19:01 yay, token flush! 18:19:10 rkanade, they are fine, just they can't span multiple backends 18:19:25 rkanade, lets talk after this, and I can help you scope in the patch, ok? 18:19:33 ok sure 18:19:55 dolphm, I'd like to target "split crednetials" for H1 18:20:09 #link https://review.openstack.org/#/c/28372/ 18:20:27 is there a bp? 18:20:28 speaking of credentials, when are we migrating ec2_credential table over to credential? 18:20:32 should be fairly close. 18:20:39 no reason to keep both 18:20:52 dolphm, it is linked to a larger BP, but not one that would be completed for H1, so nevermind 18:20:58 gyee: +1 18:21:09 gyee, +1 18:21:27 gyee, that might also help out with some of the issues we have with the ec2 middleware 18:21:36 ayoung: like what? 18:21:46 dolphm, ayoung, should I file a bp and get it done? 18:22:22 #link https://review.openstack.org/#/c/27563/ 18:22:30 dolphm: how about: https://blueprints.launchpad.net/keystone/+spec/inherited-domain-roles 18:22:30 gyee: that'd be great 18:22:38 dolphm: I could get that done for H1 18:22:47 dolphm, thanks 18:22:57 yeah waht are we doing about domains? 18:23:01 er, regions 18:23:02 dolphm: although we haven't actually approved it 18:23:07 henrynash: let me know when you have an implementation, and if it looks like we can merge in h1, we'll bump it up? 18:23:23 dolphm: ok 18:23:36 dolphm, what's the deal with this? https://review.openstack.org/#/c/27563/ 18:23:38 aside from termie are we all agreed that jaypipes bp is correct? 18:23:42 henrynash: direction is approved, just no spec or anything to approve the bp design 18:23:51 https://blueprints.launchpad.net/keystone/+spec/first-class-regions 18:23:54 #link https://blueprints.launchpad.net/keystone/+spec/first-class-regions 18:24:00 lol. 18:24:01 I hate to agree with termie, but I think he's onto something 18:24:03 henrynash: spec == code review* 18:24:04 :) 18:24:09 * jaypipes had pretty much just given up on it all. 18:24:12 he did write up a doc spec 18:24:15 dolphm: ;-) 18:24:21 I like a general approach for endpoint lookups 18:24:52 an endpoint is just a url and a bunch a attributes 18:24:52 is the "regions" abstraction general enough to cover all of the use cases we have? 18:25:06 namely: what endpoints to link to a project, and so on? 18:25:07 we should be able to construct any kind of filters 18:25:20 ayoung: it sounded like it was general enough for the discussions we had in the regions session 18:25:21 just like LDAP filters 18:25:50 jaypipes: since there's no impact on existing api's, i'm wondering if this should be an OS-REGION extension on top of v3 18:25:51 dolphm, yep, that was my take. 18:26:23 dolphm, the one issue that has come up (and I think it covers) is for things that are not necessarily keystone concepts, like cells. 18:26:26 jaypipes: and limit the implementation impact to it's own middleware 18:26:52 dolphm: I don't see why it would be an extension... the concept is already in the existing API -- it's just not a first-class citizen. 18:26:54 ayoung: i don't think it's intended to overlap with cells, afaik 18:27:13 ayoung: just everyone's various interpretations of "regions" and availability zones 18:27:19 dolphm, so, it was not intended to, but *could* it was the question 18:27:29 ayoung: cells are private to an implementation -- i.e. they are not discoverable,. 18:27:48 jaypipes: termie will certainly be much more agreeable if it's a discrete extension 18:27:51 ayoung: in other words, a user can't ask that a resource be spun up in a cell. 18:27:58 if we say that Regions is the abstraction for grouping resources, and ,if you ever need to expose something new, such as cells (in the future) I think there is a clear mapping 18:28:09 jaypipes, not today. But what about tomorrow? 18:28:11 dolphm: extension of the service catalog or extension of the API in general? 18:28:13 ayoung: tenants/projects are the abstraction for grouping resources 18:28:22 dolphm, but not by locale 18:28:25 jaypipes: of the api in general 18:28:31 ayoung: I'll pay you for that hamburger then. 18:28:35 dolphm, I hear you, this is nova stuff, and not our abstractions 18:28:39 jaypipes, you misunderstand 18:28:50 I am saying that "yes, this abstraction is powerful enough" 18:29:06 dolphm: I'm fine putting it in an extension if that's what folks want, sure. 18:29:08 so we *should* go with regions as designed. I don't think that was clear. 18:29:17 jaypipes, it can't be an extension 18:29:22 ayoung: ah, gotcha 18:29:24 it needs to be part of the service catalog 18:29:28 ayoung: ? 18:29:42 lol, when you guys figure that out, lemme know and I will code it up. ;) 18:29:44 dolphm, regions are going to be how we group things in the service catalog 18:30:06 jaypipes, this is when I go to bat for you and say "lets cook this" 18:30:07 ayoung: sure, but that doesn't mean it needs to be in the catalog controller/driver 18:30:31 dolphm, OK, I can see it as a separate back end...maybe, if I squint 18:30:51 ayoung: then it should be just as easy to see it as an extension with it's own backend (please continue squinting) 18:31:00 Maybe there should be a Group concept like the one for users 18:31:06 ayoung, with the pluggable token management backend, you can have you own catalog if you want 18:31:09 just saying 18:31:14 ayoung: there's a bunch of other reviews on the meeting agenda, any others your want to mention? 18:31:21 dolphm, ok, so a related issue is: splitting migration back ends 18:31:29 keeping it in the scoped o jaypipes work 18:31:39 say he needs a DB backend 18:31:56 we shouldn't be oputting that in with the rest of the common sql code 18:32:20 currently db_sync will stand up tables for any/all extensions, i don't see an immediate reason to change that, although it'd be ncie 18:32:21 so we need to make it the first backend that has its own revision repo... 18:32:23 it's not a blocker 18:32:43 dolphm, extensions should not be in core, otherwise, "extensions" means nothing 18:32:48 but we already have the mech 18:33:18 ayoung, I think "extensions" is more significant to the API then impl 18:33:22 dolphm, I think we need to be able to enumerate through the "active extensions" from the config file, and call db_syn on each one 18:33:33 gyee, I'll link, one sec 18:33:44 #link https://wiki.openstack.org/wiki/Keystone/sql-migrate-extensions 18:34:07 #link https://github.com/openstack/keystone/blob/master/keystone/cli.py#L51 18:34:22 so we call db_sync for each backend, even though they are all common code 18:34:36 but we have no way for doing it for an 3rd party extension 18:34:42 ayoung: so we'd need some way for extension to register db_sync. 18:34:52 bknudson, right...so how about 18:35:09 extensions = regions, oauth.... 18:35:20 and then we pull that out of the config file and enumerate? 18:35:54 extensions is a config option? it's not just they add middleware? 18:36:06 [pipeline:public_api] 18:36:17 ok 18:36:17 bknudson, it is done like that for now^^ 18:36:37 ayoung: i'm just saying that's massive scope creep for this, considering we're way past that 18:36:42 but not everything that is registered that way would end up getting a db_sync 18:36:45 ayoung: nice to have yes, but not a blocker for this 18:36:49 dolphm, no, this is fixing something we've messed up 18:36:57 dolphm, oauth is going to have the same thing 18:37:07 ayoung: right, but it's already broken 18:37:13 it also would have been cleaner for trusts...I'm truying to learn from mistakes of the past 18:37:19 so for all extensions, we need this 18:37:32 we are making the rule "do it in an extension first" 18:37:37 so this is out side of the bargain 18:37:42 out->our 18:37:59 dolphm, I can code it up...I'll take this, and jaypipes can consume it 18:38:09 it going to be messy, if I remove an extension from the pipeline, do I run db_sync to downgrade? 18:38:21 ayoung: sure, but it's not fair to block his implementation if he gets there first 18:38:22 gyee, you don't have to, but you can 18:38:27 gyee: +1 18:38:38 I think this is going to be extremely messy 18:38:44 db_sync ? 18:38:49 gyee: +1 18:38:59 gyee, there is actually already a mechanism in place for it, we just don't use it 18:39:06 see the BP 18:39:15 how the deployers need to have knowledge of the backend 18:39:16 just needs a unique repo patch 18:39:25 is the repository path in the extension? 18:39:39 gyee, no, they need to register it as an extension. we can have the rgions extension enabled by default if we decided 18:39:46 bknudson, sort of 18:39:50 bknudson, it is in the DB table 18:40:01 and the migrations themselves are then in the extension 18:40:07 we don't even support multiple sql backends, much less this 18:40:16 ayoung: yes, that's what I was asking. 18:40:28 dolphm, we do support this already, for the most part 18:40:36 we just have only one migrtation scheme, in common 18:40:39 but in the db: 18:40:53 select * from migrate_version; 18:40:58 keystone | /opt/stack/keystone/keystone/common/sql/migrate_repo | 22 18:41:07 ayoung: post a demo of two backends using separate sql connection strings, and an API call that consumes both 18:41:39 dolphm, this would be in a single backend. smaller use case 18:41:52 (since this was on the agenda) 18:41:52 #topic splitting migration back ends 18:42:02 thanks 18:42:33 dolphm, anyway, enough people have the context, I'll work with the people writing extensions to make it happen 18:42:51 ayoung: alright 18:42:53 #topic IPv6 and eventlet 18:42:55 bknudson: ? 18:43:01 It is very late here in India, please give your comments on the review itself https://review.openstack.org/#/c/25517 . Thanks :) , cya all . 18:43:06 rkanade: o/ 18:43:07 gnighht rkanade 18:43:10 dolphm: I added this just to mention the bug report... 18:43:11 gn 18:43:18 #link https://bugs.launchpad.net/keystone/+bug/1178732 18:43:19 Launchpad bug 1178732 in keystone "Eventlet dnspython monkey-patching problems" [Undecided,New] 18:43:29 so we are back on an even keel with IPv6 and eventlet for gating, right? 18:43:35 there's also: 18:43:37 #link https://bugs.launchpad.net/keystone/+bug/1176204 18:43:38 Launchpad bug 1176204 in keystone "keystone ipv6 tests fail" [Undecided,Fix committed] 18:43:48 I think what I'll go with is moving the wsgi.Server code to its own part 18:43:54 bknudson: ayoung: what's the current status of all this? 18:44:05 so jamielennox is working on something that might help, too 18:44:14 Where we are now is that the tox.ini sets the env var for eventlet 18:44:28 This is another change that we should be able to get rid of. 18:44:43 jamielennox had posted code for reveiw, but it was too huge... 18:44:54 it is now abandoned and he is reworking into smaller patches 18:45:07 one goal, the main one, is to remove eventlet from the server test code, and use webtest 18:45:14 let me find the link 18:45:37 #link https://review.openstack.org/#/c/28387/ 18:46:02 he has split the v2 and v3 tests out, but the main thing, the webtest one, is still being reworked 18:46:11 the major problem was the tests, so changing to webtest would make it easier 18:46:29 but I assume the keystone-all server still uses eventlet? 18:46:32 #link https://blueprints.launchpad.net/keystone/+spec/extract-eventlet 18:46:44 #link https://review.openstack.org/#/c/29049/ 18:46:53 interesting approach 18:47:23 do we use eventlet when run under apache? 18:47:25 bknudson: i would assume so based on this patch 18:47:29 bknudson: no 18:47:41 bknudson, as much as possible, no 18:47:55 most of the eventlet patching was moved into the startup code, and apache start up bypasses that 18:48:08 nginx as well 18:48:10 then I'm not sure what the blueprint is talking about with apache/nginx? 18:48:25 https://github.com/openstack/keystone/blob/master/bin/keystone-all#L109 18:48:41 bknudson, trying to make sure we are testing what would actually be run in them, for one thing 18:49:26 so is the blueprint about changing keystone-all to not use eventlet, too? 18:49:47 bknudson, no, the blueprint is to make the final changes to make the tests and client and middleware don't require it 18:49:56 targetted bp to m2 18:50:02 targeted* 18:50:09 dolphm, thanks 18:50:12 ok, well, maybe I'll get my change in before that. 18:50:25 I don't think it conflicts. 18:51:06 dolphm, so the test re-org is a *nice to have* not a need-to-have 18:51:42 but it comes from jamielennox trying to separate out the eventlet code in the testing, and getting frustrated with the organic nature of the code 18:51:57 I think the v3 one looks good, suspect that the v2 one is probably right, as well 18:52:34 anyway, instead of it coming by fiat, everyone should look and provide feed back, with an eye to keeping the tests as maintainable as possible 18:52:52 dolphm, can we talk Token table revocations list now? 18:53:16 ayoung: is that on the agenda somewhere? 18:53:25 dolphm, yep 18:53:39 dolphm, right under "Atomic API " 18:53:49 was hoping to get to notifications (but bknudson also disappeared) 18:53:57 diasppeared? 18:53:57 bknudson: er, maybe not 18:54:15 bknudson: maybe my client was acting weird 18:54:28 ayoung: i thought we talked about atomic stuff 18:54:41 ayoung: pm me 18:54:41 dolphm, so Token revocation is a real bug, and is stonewalled 18:54:42 #topic Notifications 18:54:44 is there a BP for revocations? 18:55:05 dwaite, see the agenda, links to the reviews, and bps off of them 18:55:29 we were wondering if you dolphm could use some help on the notifications, to speed it up 18:55:31 dolphm, just need people to keep that process moving. termie objects, but I think we can't meet his ideals on this one\ 18:55:40 I've got a guy on my team here ( lbragstad ) who could help out 18:55:41 bknudson: what kind of help? 18:55:51 dolphm: implement it. 18:56:03 i'd love to, but not sure if i can hit m1 18:56:26 bknudson: oh wait, i misread that 18:56:26 dolphm: I think it's scheduled for m3, with his help could probably move up to m2. 18:56:33 bknudson: you want him to implement it? 18:56:43 dolphm: yes, have him implement it. 18:56:52 bknudson: i don't have any issue with that 18:56:57 dolphm: great! 18:57:03 bknudson: i've only put a little thought into how i would implement, but haven't written any code 18:57:07 dolphm: awesome thanks! 18:57:34 that was it for notifications. 18:57:52 notifications are an argument to split project fro identity, too. 18:57:53 bknudson: why i had notifications targetted to m3- it's not api-impacting, so that's just the latest it could possibly merge; i'd like it to depend on oslo.notify, which is still in progress 18:58:15 lbragstad: get caught up on oslo.notify! 18:58:37 i haven't seen any blockers in those discussions for us 18:58:41 dolphm: yep, I was working with a core from oslo on that for a while yesterday 18:58:44 awesome 18:58:50 lbragstad: keep me posted 18:59:01 dolphm, last thing I'd like to hit is: gate support for MySQL and Postgres Unit tests 18:59:11 I guess the only question I had was if notify doesn't have a maintainer, does that effect us 18:59:11 #topic gate support for MySQL and Postgres Unit tests 18:59:17 err eff 18:59:19 1 minute 18:59:19 we can run the unit tests against the real DBS, we should be running that in the gate. Agreed? 18:59:41 #link https://review.openstack.org/#/c/27878/ 18:59:44 henrynash, gyee ? 18:59:50 "unit" tests, no lol 18:59:55 ayoung, on it 19:00:15 #endmeeting