18:01:11 #startmeeting keystone 18:01:12 Meeting started Tue Apr 1 18:01:11 2014 UTC and is due to finish in 60 minutes. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:13 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:15 The meeting name has been set to 'keystone' 18:01:17 #topic Reminder: don't be a fool 18:01:20 #topic Reminder: Design summit session proposals open until April 20th 18:01:36 o/ 18:01:52 we should delete v2 as april fool's joke 18:01:59 lol 18:02:02 Don't we have like 3000 sessions submitted for Keystone? 18:02:05 bknudson, heh 18:02:06 there was also an announcement on the -dev that cross-project sessions will have an earlier deadline than the rest of the summit proposals, so get those in super early 18:02:14 ayoung: i'm afraid to look 18:02:27 dolphm, there are a bunch 18:02:37 I have more to add 18:03:19 we're also one session slot short this year versus the last few years (8 vs 9) 18:03:42 dolphm, we'll end up making buckets and collecting up topics into them, then\ 18:03:50 ayoung: of course 18:04:04 anyway, the fun part: 18:04:06 #topic Announcement: icehouse-rc1 now available! 18:04:17 gyee, instead of adding, lets pick the set of buckets and figure out what goes where 18:04:22 * dolphm "Like during the Havana cycle, Keystone is again the first project to publish a release candidate in preparation for the Icehouse release! Congratulations to the Keystone development team for reaching that milestone first." -ttx 18:04:26 ayoung, absolutely 18:04:26 congrats 18:04:31 how do we get fixes into icehouse now? 18:04:40 nice 18:04:41 use the milestone-proposed branch? 18:04:43 bknudson: so now we have a milestone-proposed branch 18:04:51 We are at 22 submissions 18:04:55 fixes need to land in master first, and milestone-proposed is treated as a stable/ branch 18:05:00 dolphm, ayoung, so you guys going to come up with the bucket list? 18:05:06 and then well cut another rc? 18:05:06 bknudson,dolphm, do we have any outstanding rc2 potentials? 18:05:06 Heh 18:05:08 if needed? 18:05:10 currently we only have one patchset that i believe needs to get into milestone-proposed, and warrants an rc2 18:05:16 gyee, I can make a first pass, sure 18:05:29 #link https://review.openstack.org/#/c/84457/ 18:05:48 the +0, -28607 there is not an april fools joke :) 18:05:49 we couldn't even get a clean backport after a week 18:06:06 dolphm, sure. 18:06:10 bknudson: yeah, the automated job in master made it messy, but thankfully it's just a script to recreate the patch 18:06:21 is there another patch to add the xlations back? 18:06:33 bknudson: this only removes dupes 18:06:39 bknudson: so, that's not necessary 18:06:43 that patchset so feels like an april fools joke. really? really? 18:07:30 bknudson: rather, just read the script :P https://gist.github.com/dolph/9915293 18:07:52 i should add a comment there as to where it came from -- it was posted as a comment in another bug report 18:08:07 it would be nice if we just had the files we wanted... don't they generate the po files? just check those in to replace the existing 18:08:42 bknudson: yeah, i'm not sure what the better long term answer is, but there's clearly a problem we'll need to solve moving forward 18:08:59 dolphm: what bug was it posted in? 18:09:00 and i'm guessing it's *not* "run this script to fix things periodically" 18:09:04 there must be a set of "master" files somewhere 18:09:39 lbragstad: https://bugs.launchpad.net/ironic/+bug/1298645/comments/2 18:09:40 Launchpad bug 1298645 in ironic "translated .po files do not contain any translation text" [High,Fix committed] 18:10:13 do we want the milestone-proposed files to match the files currently in master ? 18:10:23 bknudson: master is already ahead 18:10:27 bknudson: so, definitely not 18:10:41 bknudson: i.e. new strings, etc, have landed in master since milestone-proposed was cut 18:10:42 they're already translating juno? 18:10:53 bknudson, no just new / changed string sets 18:11:10 bknudson, by default the translation is the same as the original string. 18:12:00 horizon seems to be the only project where the translation folks have put in a significant amount of effort 18:12:13 ok, I'll see if I can run the script and regenerate and compare with master. 18:12:14 the rest is mostly getting i18n infrastructure in place 18:13:10 i've been keeping an eye on launchpad, but if anyone is aware of any issues in milestone-proposed that may warrant an RC2 - please file them / bring them to my attention ASAP 18:13:21 Here is a proposal for Buckets: Backends, Hierarchy, HTTP, IdP, Performance, Tokens. Heh, that is only 6 18:14:02 ayoung, tokens, i think you need to mention tokens again, because tokens. 18:14:07 ayoung, ;) 18:14:14 ayoung, Client & Middleware? 18:14:20 dolphm, I'm not certain that translations belong anywhere but Horizon. There is a pretty strong argument that it is easier to Google for the issue if iti is in only one language than all 200 18:14:35 gyee, client and middleware would be a good additional bucket 18:14:35 ayoung: take it to the mailing list :) 18:14:45 dolphm, ++ 18:14:53 ayoung, i'd support that personally 18:14:58 ayoung: there's a couple existing threads on the topic 18:15:03 ayoung, though responses from the API might want to be translated 18:15:03 what translation, just read the fantastical code! 18:15:10 ayoung, e.g. messages (not errors) 18:15:23 if any. 18:15:37 *shrug*. i like the idea of not translating for ease of error handlings 18:15:42 erm, error searching 18:16:07 funny how translations could actually make a problem harder to identify instead of easier 18:16:45 our lack of logging makes it hard to figure out what a problem is. 18:17:04 bknudson, ++ 18:17:09 one of my plans is to try some things and see if I can figure out the problem from the logs 18:17:10 better logging would help. 18:17:12 bknudson: ++ 18:17:13 stevemar2, is http://summit.openstack.org/cfp/details/71 server or client side plugins? 18:17:53 bknudson, somethine worth chatting about at the summit (informal probably) 18:18:16 I'll just yell it out in the developer lounge. 18:18:24 bknudson, ++ :) 18:18:53 bknudson, that is unless you have something together before then. 18:19:39 #topic Icehouse release notes 18:19:56 so, as soon as we cut RC1 i ran off and got us started on some release notes... 18:20:01 #link https://wiki.openstack.org/wiki/ReleaseNotes/Icehouse#OpenStack_Identity_.28Keystone.29 18:20:36 please review, amend & edit as necessary if i missed anything worthy of advertising to deployers, etc 18:20:40 I dont think I have ever heard bknudson yell... 18:21:06 topol, maybe he yells all the time, just not around you. 18:21:11 i'm most concerned about changes in keystone.conf that i didn't capture as upgrade notes 18:21:37 and any tricky / heavy migrations we produced 18:22:53 dolphm, i think the only complex migration is the grant table one 18:23:08 morganfainberg: ooh, that's a good one to note 18:23:35 config file changes, i think mostly it was additions for new features. 18:24:15 or removal of explicity set options and defaulting to the config default...did that change any defaults? 18:24:17 any changes of default values, beyond the default token duration? 18:24:18 there might have been some deprecations for the move to oslo-incubator db 18:24:26 bknudson, ++ 18:24:35 bknudson, those would be good to capture 18:24:55 dolphm, oh the mutable domain_id option 18:25:01 dolphm, that is worth capturing. 18:25:23 morganfainberg: ooh, non-ideal defaults is a good theme 18:25:31 things we want to see changed in juno 18:25:43 API validation framework? 18:26:08 dolphm, mutable domain_id is false by default we opted (iirc) to say "this is security related" so if osmeone relies on that they need to re-enable it (please, let no one rely on that) 18:26:38 it is one of the few cases i think we ran into a "we're going to break this" decisions 18:27:08 dolphm, ++ on non-optimal defaults 18:27:51 added a couple TODO's to the release notes based on the above ^ 18:28:50 i'd love to get real Python 3 support completed in Juno 18:28:59 dstanek, ++ 18:29:07 dstanek: ++ 18:29:12 dstanek: apparently they're going to get it done a pycon 18:29:28 at pycon 18:29:28 bknudson: what project? 18:29:38 dstanek: there is more than just eventlet that is broken with py3 18:29:40 bknudson: get'r* done 18:29:46 jamielennox: lots :-( 18:29:52 i couldn't make the oslo-incubator db stuff work with py3 18:30:02 sqla-migrate is also broken 18:30:08 lots of olso won't run on 3 18:30:19 jamielennox: i think the next release of sqlalchemy-migrate addresses that 18:30:21 Ouch 18:30:28 i just fixed gettextutils this mornin 18:30:30 jamielennox: they held it back since we were shipping icehouse, IIRC 18:30:41 jamielennox, i'm looking at alembic again since i;m going to do the migration collapse here shortly 18:30:59 What was the comment about everyone shipping non-backwards cmpat releases of Python libraries? 18:31:09 morganfainberg: yea, i switched kite from sqla-m to alembic for py3 - didn't end up mattering because of the oslodb stuff 18:31:23 jamielennox, yeah. 18:31:32 morganfainberg: sort of an expected surprise, but https://bugs.launchpad.net/neutron/+bug/1288427 18:31:34 Launchpad bug 1288427 in neutron "Unsequenced alembic migration files block the gate." [Critical,Fix released] 18:31:34 there is no common oslo.db librray yet is there? 18:31:43 jamielennox: no 18:32:00 I think the oslo.db is next on the list for oslo library 18:32:07 dolphm, yep, was going to aim for 100% ordered, as much as they "want" to use unordered magic 18:32:19 ayoung: ? 18:32:33 dolphm, that bug specifically was a concern i wanted to avoid. 18:32:39 dolphm, looking for the origianal quote, but the punchline was "pycon" 18:33:37 "How I know it's the week before pycon: everyone is releasing non backwards compatible python library releases breaking OpenStack" Sean Dague 18:33:53 lol 18:33:55 lol 18:34:11 ayoung, OpenStack Gate: The Python Library Regression Test 18:34:26 we should freeze our deps 18:34:34 jamielennox: https://wiki.openstack.org/wiki/Python3 18:34:39 bknudson, you mean cap all of them? 18:34:52 i'm going to update that today so that it's current 18:34:57 morganfainberg: yes, to protect ourselves 18:35:02 bknudson, i think that poses a lot of operational headaches. 18:35:23 bknudson, at one point we capped a lot, but then projects got out of sync and we installed conflicting versions 18:35:36 bknudson, it's an issue with the way pip/pypi works (not unsolvable) 18:35:36 bknudson: it would cause havoc on the distros 18:35:52 just the week before pycon 18:35:53 bknudson, and what jamielennox said 18:35:56 LOL 18:37:12 dstanek: that page isn't as reassuring as i would have liked - i thought we were a bit closer than that 18:37:26 dstanek: i understand the servers but oslo-incubator should gate on py3 18:37:57 jamielennox: they likely need to gate before we can :( 18:38:11 if we're going to pull in changes from oslo-incubator to keystoneclient... 18:38:30 they'll need to work on py3 18:38:33 dolphm: i'm going to port the olso stuff we currently use to push them in the right direction 18:38:42 dstanek: awesome! 18:39:01 #topic Document V3 features (vs V2) 18:39:03 morganfainberg: o/ 18:39:23 bknudson: not just that - i understand how hard it is to convert existing projects, but i was having to start kite again in a new repo so wanted to have it gated on py3 from the start and it just can't 18:39:34 So I was chatting with the nova folks and the comment that there was no "list" of what V3 provides over V2 in keystone's api was brought uop 18:39:48 related: i should also throw out there that, if you missed it, we reverted the deprecation on v2 last week - it's "stable" in milestone-proposed, as is v3 18:39:53 short of the "go look at the identity-api repo" 18:39:59 I think the biggest thing projects are running into with v3 is the catalog format change in the token 18:40:22 they have their own ServiceCatalog class that doesn't support the v3 catalog 18:40:22 bknudson: which they shouldn't be parsing manually anyway 18:40:27 bknudson: auth_token and the client in general should protect everyone from that change :( 18:40:32 jamielennox: right 18:40:40 There should be a list of features / differences / reasons to move targeted at projects etc that want to move (this was a request from jogo, and i feel totally reasonable) 18:40:49 dolphm: though we have to pass X_SERVICE_CATALOG as a string - which does make them try it 18:40:50 dolphm, and if not, then they should use the python-keystoneclient catalog API 18:41:11 jamielennox: and a different string with v2 vs v3 -- we need to provide an option to give them the same string either way 18:41:39 dolphm: interesting - but i'm not sure if that's a benefit to just making them parse correctly 18:41:48 jamielennox, how about passing the ServiceCatalog object in the request environ 18:41:55 there is no great advantage of v3 catalog over v2 18:42:01 gyee, can you.. do that cleanly? 18:42:06 jamielennox: it's worse, IMO 18:42:14 morganfainberg, sure 18:42:22 gyee: aparently its not allowed, things coming out of middleware shouldn't be python specific 18:42:26 gyee, ok (I haven't looked at that extensively lately) 18:42:33 alternatively, we could provide both v2 and v3 catalog in the token response 18:42:36 however the context middleware and others do it i'm suer 18:42:39 jamielennox, says who? 18:42:46 bknudson: no, it's big enough 18:42:48 would need compression for that. 18:42:53 bknudson, that is bad, the token is massive as is. 18:42:54 that's how Swift passing their logger object 18:42:59 bknudson, even w/ compression i think it's ugly. 18:43:21 you can pass whatever you want through the wsgi environment, auth_token uses strings for historical / backwards compatibility 18:43:22 bknudson, though it might be the "real" solution 18:43:33 historical reasons* 18:43:41 I think there are already changes to get the other clients using keystoneclient ServiceCatalog... 18:43:45 there was someone here working on it. 18:43:45 dolphm, right, lets make history by passing service catalog object :) 18:43:51 bknudson: right i've had a go 18:43:52 bknudson, we got cinder there 18:43:54 bknudson, :) 18:44:02 bknudson: the main problem is volume_service_name in a few places 18:44:18 i've added service_name to keystoneclient (not sure if that's merged yet) 18:44:42 i'm not sure if i need to add some sort of selection to others 18:44:53 gyee: you could just pass a keystoneclient session and call it a day 18:45:03 dolphm, ++ 18:45:04 ++ 18:45:11 dolphm: ++ that's the hope 18:45:27 no-one should ever need to manage there own service catalog 18:45:36 jamielennox ++++++++ 18:45:41 other projects seem to be somewhat interested in domains. 18:45:45 not sure if that's a good thing 18:46:17 this is the volume_service_name issue: https://github.com/openstack/python-cinderclient/blob/master/cinderclient/service_catalog.py#L59-L71 18:46:45 i'm not sure yet if we can work around it in keystoneclient.service_catalog 18:46:51 bknudson: ++ 18:47:44 I don't know that we'll have much to say to other projects to use V3 over V2... 18:48:00 but I think our users will prefer V3 over V2. 18:48:00 jamielennox: that's a mess 18:48:04 bknudson, if we want to convince them to move to V3, we probably need to sell it. 18:48:08 dolphm: yep 18:48:23 bknudson, even if the selling is "new auth everyone wants in v3, use it" 18:48:23 morganfainberg, we'll need to put up the code for them 18:48:27 and v2 is frozen and will receive new features. 18:48:33 erm wont 18:48:33 bknudson: i don't think it's to the advantage of the projects, it's to the advantage of deployers 18:48:42 I think the best way to sell it is if they don't have to make any changes to use it. 18:48:58 bknudson, ++ but there are some cases where they will need some changes. 18:49:09 bknudson, i would hope auth_token would just "do it" for them 18:49:09 bknudson: or to use it, they delete a lot of code and replace it with a few lines using keystoneclient :) 18:49:16 ++ 18:49:23 dolphm, ++ 18:49:27 dolphm: yes, they should be happy to have to maintain less code. 18:49:27 morganfainberg: it's damn near ready... three outstanding issues i know of 18:49:34 dolphm, cool. 18:49:50 yeah, keystoneclient is a hell lot easier to use now 18:49:57 I need to circle back around on the compressed tokens change 18:49:58 thanks to jamielennox 18:50:10 in either case if there is a change that is needed (even if we contribute the code) we need to still say "hey guys, merge this" 18:50:29 morganfainberg: incompatible service catalog when switching to v3, we still require v2 to authenticate auth_token itself, and i don't think we're using v3/OS-SIMPLECERT/ atm (?) -- then it's just a matter of reversing the default from v2 to v3 18:50:30 morganfainberg: to make other projects use client you mean? 18:50:34 jamielennox, yeah 18:50:44 ayoung: target juno-m1 for that 18:50:48 morganfainberg: yea - i plan to go and do a lot of that 18:50:49 ayoung: if not before the summit 18:50:59 dolphm, yeah, for the server. But needs to be in client before that 18:51:17 dolphm: the v2 call in auth_token will be fixed by the plugins from CONF stuff 18:51:19 jamielennox: ping me throughout that process as well 18:51:34 jamielennox, as long as we have a clear reason why they should support V3, that is an easier sell 18:51:52 jamielennox, as long as we have the clear message, i think we're on the right track. 18:52:05 ayoung: i hopefully only have a few more days of icehouse stuff to worry about, and i can help you get that landed on both sides 18:52:08 jamielennox, you can keep me in the loop as well, I am doing the same thing 18:52:10 and that is what is being looked for. why should they move the api even if we supply the code 18:52:15 cool 18:52:53 morganfainberg: if you can get a doc started, i'd be happy to contribute to it as i think of things 18:53:03 dolphm, sounds good. I'll get a wiki page up 18:53:41 #action morganfainberg to setup wiki for reasons/features of V3 keystone api 18:53:52 dolphm, 7 mins 18:53:55 morganfainberg: i was also wondering why it needed to be discrete from the how-we-get-to-v3 bp? 18:54:02 morganfainberg: it seems like this would just be the itnro 18:54:27 Why v3 & How v3 18:54:33 dolphm, sure, but if we're looking for a concise list of differences, it makes sense to have it separate 18:54:42 The biggest "why" is domains 18:54:54 e.g. "why this, and here is what it provides" 18:55:00 ayoung, I can help write that part if you want 18:55:01 vs the "you should do this because users like it" 18:55:01 as far as how...that is the script stuff that we've been adding to the recent reviews 18:55:09 if we have hierarchical projects in v3 then that will affect other projects 18:55:23 dolphm, but yes, it's all part of the same documentation 18:55:36 https://review.openstack.org/#/c/82687/ and so on 18:55:55 dolphm, though the v3 specific changes might also be useful for deployers in absence of the "how to move to v3 as a openstack project" 18:56:07 bknudson, it will be a good discussion in the upcoming summit 18:56:17 we need to get a handle on the hierarchical projects...the IDs should not be what determines the hierarchy... 18:56:53 ayoung: regions have the same problem 18:57:01 I can't keep up with the discussion or the sample code flying around. 18:57:09 (and IMO more quickly relevant) 18:57:18 jamielennox, yeah, but regions are already implmented in a somewhat sane way, no? 18:57:37 ayoung: from client side - how am i supposed to know what region belongs to another? 18:57:42 regions also don't have an immediately profound effect on every other project 18:58:21 * dolphm 2 min 18:58:26 you mean we need an API to select subregions? 18:58:31 ayoung: specifically from a service catalog if you ask for a region and a sub-region of that is present how am i supposed to know that that one is usable? 18:58:39 ayoung: that's basically GET /v3/regions 18:58:53 dolphm, agreed, but a good impl that we can say "see this is how hierarchy should be done" would help the project discussion 18:58:55 jamielennox: /v3/regions reflects the hierarchy 18:59:05 dolphm: i can't go querying /v3/regions every time i want to query a catalog 18:59:09 so a recommendation, it would be great to focus on one stakeholder project to try out v3 and deal with the kinks, bugs, etc and then use that as a reference success story for the other projects 18:59:20 which means Nova 18:59:26 jamielennox: there's just no correlation between /regions and regions in the service catalog yet ;) 18:59:28 topol, ++ 18:59:38 topol: ++ 18:59:40 time! 18:59:41 dolphm: take US > US-EAST, if US-EAST is present in a service catalog and someone asks for something in the US region - how do it know? 19:00:03 times up 19:00:08 back to #openstack-keystone :) 19:00:08 so for that one project we go all hands on deck and hold their hands to make is successful 19:00:10 #endmeeting