18:00:34 #startmeeting keystone 18:00:35 Meeting started Tue Nov 17 18:00:34 2015 UTC and is due to finish in 60 minutes. The chair is stevemar_. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:36 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:00:39 The meeting name has been set to 'keystone' 18:00:44 #link https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting 18:00:58 hi 18:01:07 * shaleh waves 18:01:08 henrynash: you had to put the biggest item as #1 18:01:17 moi? 18:01:17 #topic HMT Phase 1 & Phase 2 18:01:23 henrynash: go! 18:01:37 ok, so this is a follow on from the discussion at the summit 18:01:43 we agreed Phase 1 18:01:56 (store domains as projects, bt only top level domains) 18:01:58 hi 18:02:04 o/ 18:02:05 Phase 2 is the actual resller functionality 18:02:17 So long as domain names are globally unique, it does not mattewhere they are in the tree 18:02:21 request from Morgan was….go see if we can make federation cover this use case 18:02:57 and as per my mail to the list (http://lists.openstack.org/pipermail/openstack-dev/2015-October/078063.html) 18:03:02 …this doesn’t really cut it 18:03:18 so proposal is (as ayoung just stated) 18:03:25 hmm, morgan seems to be afk 18:03:46 I still think the uniqueness thing is going to be an issue 18:03:51 henrynash: how long will Phase 1 take? 18:04:03 marekd: what do you mean by long? 18:04:09 so we ahd all the code for that ready for Liberty 18:04:11 htruta: days, months 18:04:23 ayoung: domains unique globally? 18:04:25 marekd, 21 story points 18:04:29 hey all! 18:04:31 marekd: it is already implemented. how long it takes depend on reviews 18:04:31 and is ready for review now….it;s in pretty good shape haveing been beaten about 18:04:32 marekd, yes 18:04:51 ayoung: on uniquness 18:04:54 marekd, if I want to "squat" on the domain name "CERN" can I cretea domain under my project with that name? 18:05:02 ayoung: so yes...there will be a problem :-) 18:05:09 1) Yes, all domains names stay unique 18:05:11 unless we use some existing systems like DNS :P 18:05:54 http://tomatobubble.com/economist_magazine_cover.html 18:06:13 ignore ^ 18:06:21 lol 18:06:21 2) I also believe we can avoid the origional issue of there being two child projects of teh same name (with one being a reguakr project, one being a domain)…we can jsut ensure that is not allowed…and you can’t get into that issue via migration 18:06:45 SO 18:06:50 proposal to team is 18:07:11 Use domains for HMT…but with the restrictions of: 18:07:18 1) all domain names unique 18:07:43 2) NO inheritance across domains…i.e. we are just using domain parent as an indiactor of owner 18:08:37 This is as the oriinal spec was written and agreed for Liberty 18:09:15 questions: ? 18:09:28 what's the domain parent comes from if no inheritance is allowed? 18:09:30 is this the spec? http://specs.openstack.org/openstack/keystone-specs/specs/backlog/reseller.html 18:09:31 inheritance of what? 18:09:32 and this is already implemented 18:09:56 stevemar_: yes 18:10:06 role assignment inheritance? 18:10:09 domain is anywhere in the tree? 18:10:14 davechen: parent is just ownership, where you inherite assignments down the tree is a different matter 18:10:20 lots of things changed since the spec... maybe we need to update it 18:10:32 bknduson_: only at top….parent of a domin must be another domai 18:10:42 stevemar_: but if it helps, henrynash has already updated the api 18:10:56 ok, domain parent can't be a project 18:11:14 https://review.openstack.org/#/c/200624/ 18:11:26 bknduson_: true 18:11:42 henrynash, I don't like it 18:11:48 ayoung: :-) 18:11:50 it violates the scope fo project assignments 18:11:55 and we're still doing a domain is a special type of project? 18:11:57 the global naming thing really complicates issues 18:12:15 bknudson_: yes, that’s wjhat phase 1 is 18:12:16 think from an RBAC perspective 18:12:34 what would you need to have as a role in order to create a domain under a project? 18:13:09 we don't support creating a domain under a project 18:13:14 bkundson_: (so strictly the parent of a project acting as a domin must be another poject acting as a domain) 18:13:14 ayoung: hu? we don’t allow that 18:13:56 henrynash, so domains not under a project? Nested domains still problematic. 18:14:06 gyee: what's about he domain is actually a project, project inheritence is allowed, right? 18:14:25 ayoung: projects acting as domains only under other projects that act as domains 18:14:38 davechen, we support D -> D -> D but not D -> P -> D 18:14:39 henrynash, so, this is why I said at one point "put domains only yunder other domains, and then give an approach for nesting the names" 18:15:03 henrynash, let me write it up in email and I think I can make it clearer 18:15:05 ayoung: we are putting domain under other domains… 18:15:25 gyee: when the reseller is done, that's still not possible? 18:15:37 Ok, so I already briefly talked with stevemar_ at the summit about my idea to introduce a unit test for checking for incompatible sql db changes 18:15:38 henrynash, domains under domains will only work with a DNS style naming 18:15:45 ^ ignore that 18:16:06 ayoung: what’s DNS got to do with it? 18:16:14 henrynash, unique naming 18:16:21 ayoung: that might be our next steps... to remove the uniquness of domain name across the cloud 18:16:21 "coke" vs "pepsi" 18:16:32 pepsi should not be able to create a domain named "coke" 18:16:39 davechen, no, we can't have D -> P -> D or this is going to be a bloody mess 18:16:43 not that it hasn't already 18:16:51 only ["pepsi","coke"] etc 18:17:35 ayoung: we can always look to relax that rule later 18:17:44 i think the take away here is we need to better explain the problem and proposed solution 18:17:50 ayoung: let’s not fix everything at once! 18:18:05 henrynash, relax what rule? We can't allow nesting without breaking things at the policy level 18:18:06 stevemar_, ++, still a bunch of mysteries :) 18:18:08 theres a lot of moving parts here 18:18:16 yeah, i'm having trouble keeping track of it all 18:18:19 henrynash, let me write it up..too much to explain here 18:18:22 henrynash: so,you don't have a solution for domain uniqieness? 18:18:30 henrynash: i thought you had said you have 18:19:01 marekd: I didn;t try and solve that…since people (at the summit) demanded that domains sat unique! 18:19:16 henrynash: ok 18:19:23 stevemar_: ++ me too 18:19:26 I think updating the spec is a good idea 18:19:28 henrynash: no, wait...'sat unique' ? 18:19:33 henrynash, domains has to be unique, or we'll need to introduce Keystone V4 18:19:35 (stay unique) 18:19:36 henrynash: what's that mean? 18:19:48 sit and stay sitting 18:19:59 ok 18:20:04 Ok, so sounds like we need a new spec? 18:20:36 for resellet in general we should be moving in small steps; I think we should get reseller, but: i) domains only under domains and ii) it should be allowed to get project scoped tokens in is-domain projects, only is-domain-project scoped tokens 18:20:43 henrynash, this is a reseller issue. I think that requesting a new domain should be workflow, but not subject to HMT....its a superblock, not a mountpoint 18:21:17 henrynash: not necessarily a new spec 18:21:29 damn, i'll settle for a paste or an email 18:21:37 update the exisitng backlog spec 18:21:47 something that clears things up 18:21:51 so teh exiisting spec mixes pahse 1 and phase 2 18:22:03 henrynash: so we need to first fix that 18:22:31 i have no problem ripping the completed parts of phase1 from the backspec and marking them as completed 18:23:14 o/ 18:23:32 we gotta keep chugging along, i'll ping you after the meeting henrynash 18:23:35 Ok, so I’ll write up teh propased pahse 2 solution…maybe as a spec 18:23:47 henrynash: we can figure out the logistics together 18:23:47 ok, I yield :-) 18:23:51 :) 18:23:55 #topic New reno-based process for release notes 18:23:59 bknudson_: you're up! 18:24:11 so openstack has a new tool for doing release notes 18:24:12 reno 18:24:30 instead of doing release notes on the wiki at the end of the release we can propose release notes with the patch 18:24:56 and the proposal is that we use this new approach 18:25:23 bknudson_, link? 18:25:24 which means a change to how we do reviews -- essentially if release notes for the change are required then we need to make sure they're included 18:25:46 bknudson_: i think it's still author's responsibility 18:26:03 and reviewers' 18:26:03 what's kind of thing should include release notes in the patchset? 18:26:10 davechen: yes 18:26:14 -1 if you think it should be in release notes 18:26:19 http://docs.openstack.org/developer/reno/ 18:26:21 so I think this is great…and in fact I’ve already written my first one…but wasn’t sure on teh scope of each RN….is it just that one feature? you are implementing (see https://review.openstack.org/#/c/242853/14/releasenotes/notes/Assignment_V9_driver-c22be069f7baccb0.yaml) 18:26:35 For output, see http://docs.openstack.org/releasenotes/neutron/ (good initial content) or http://docs.openstack.org/releasenotes/keystone (empty) 18:26:41 there's one review in particular that says when to add stuff: https://review.openstack.org/#/c/246455/ 18:26:43 davechen, incompatible config changes is one example 18:27:14 stevemar_: i didnt't get it. 18:27:17 #link http://docs.openstack.org/developer/keystone/developing.html#release-notes 18:27:22 i've proposed our liberty release notes here: https://review.openstack.org/#/c/246145/ stable-cores please review :) 18:28:04 our developer guidelines have a section saying what the changes are that should have release notes 18:28:15 davechen: oops, it says so here https://review.openstack.org/#/c/246455/ 18:28:32 cool, thanks, check it out later. 18:28:43 Also, I started an etherpad with the changes that have already merged and should have release notes: https://etherpad.openstack.org/p/keystone-mitaka-release-notes 18:29:17 bknudson_: oh nice 18:29:30 bkundson_:, stevemar_: but it sill isn’t clear to me which section you fill out in the yaml doc for your partiuclar change…there’s one RN doc per change? 18:29:49 RN? 18:29:55 oh release note 18:29:58 or do you update a release RN 18:30:09 i.e. teh Mitaka RN? 18:30:13 we just want the release notes to look right by the time we're at the end of the release. 18:30:16 henrynash: one release note per bug/blueprint/backwards incompatible thing 18:30:26 I don't think it has to be one release note per patch 18:30:37 you could update an existing release note if you're changing something 18:30:45 not ALL bugs need release notes, just if they impact operators 18:30:46 Not all bugs deserve release notes 18:30:56 dolphm: thanks :) 18:31:04 orend-users, etc 18:31:06 sure, get that…. 18:31:07 or* 18:31:13 if you care about ALL bugs, refer to LP 18:31:22 hopefully the guidelines here make it clear that not all bugs require a patch : http://docs.openstack.org/developer/keystone/developing.html#release-notes 18:31:27 require a release note 18:31:43 henrynash: so yeah, just do: reno new , then fill in the section that matters 18:31:43 henrynash: if you have a series of changes, add the release note to the last one that "finishes" the implementation. 18:31:59 the only bugs the guidelines say need a release note are security bugs. 18:32:11 so, i may say it's a little subjective, but good to go. 18:32:11 the build process lumps the sections together, so all "features" are together and all "upgrades" are together 18:32:36 AJaegar: actually no, I build the release note incremental as my patches add changes 18:32:52 stevemar_: ahh, right THAT was my question…how doe sthat happen 18:32:59 henrynash: magic :) 18:33:13 specifically build-sphinx-magic 18:33:22 it's a branch off of sorcery 18:33:23 henrynash: that's another option of doing it ;) 18:33:34 but then you must wriet your text assuming they are distributed amongst other RNs 18:33:47 which i didn’t do 18:34:04 henrynash: right 18:34:14 henrynash: looking at your patch, i would remove the "Nones" 18:34:41 ok 18:35:00 thx 18:35:18 henrynash: maybe look at https://github.com/openstack/neutron/tree/stable/liberty/releasenotes/notes and compare to http://docs.openstack.org/releasenotes/neutron/liberty.html 18:35:50 perfect! 18:35:53 ^ should make it clear how the build will look like 18:35:58 \o/ 18:36:00 pretty 18:36:08 yep, absolutley answerits it. 18:36:16 alright 18:36:20 #topic Online Schema Migration 18:36:23 ok, that's me 18:36:28 (henry can’t type straight anyway) 18:36:31 hi xek! 18:36:38 the idea is to introduce a unit test for checking for incompatible sql db changes 18:36:56 after the unit test is merged, it's easy for reviewers to spot any incompatibilities 18:37:06 incompatible with what? 18:37:22 incopatible with previous versions 18:37:31 so an upgrade without downtime is possible 18:37:43 breton suggested a spec is needed, so I made one 18:37:55 we're never had this requirement before 18:37:55 I think it's a good idea to get more eyeballs on the issue 18:38:06 and I don't see how it could catch any incompatibility 18:38:24 because we could just change the way we interpret a field 18:38:27 well it just checks for drops and alters 18:38:34 xek: how we know it's incompatiblity with previous versions? 18:38:44 we can never drop a table or column? 18:38:50 the tables will just keep getting wider and wider 18:39:17 dropping a column or changing a name will obviously be incompatible with the previous sqlalchemy model 18:39:23 is there a specific issue that this is trying to prevent? 18:39:41 this limits what db changes one can make between releases 18:39:44 xek: but it's always what migration does. 18:39:51 so a schema change, which previously took one release, can take 3 18:39:55 we should be able to drop a column if we've stopped using it for a release 18:40:29 Is there a spec proposed? 18:40:38 as far as I understand, xek wants to be able to do "git checkout master; k-m db_sync;" and have no downtime 18:40:48 bknudson_, yes, https://review.openstack.org/#/c/245186/ 18:40:49 bknudson_: i think so, there is spec. 18:40:55 xek: so what’s the goal, code upgrade withut simultaneious schema change, or schema upgrade without simultaneious code change? 18:41:29 partial schema change before upgrading the code, then both versions - the old one, and the new one running simultaneously 18:42:00 so, the goal is to be able to update schema without having to restart database nodes, right? 18:42:03 the change looks like for rolling upgrade, and reduce the downtime? 18:42:10 xek: ok, right so now we getting to the issue..so you can roll your update out across your OS deploymet 18:42:26 if we're going to do this then we should have tests for it. 18:42:29 henrynash, yep 18:42:45 shouldn't this be a grenade requirement as oppose to service-specific? 18:42:49 bknudson_, probably grenade multi-node tests 18:42:50 as in, run tempest on stable keystone after migrating to master db 18:44:01 and maybe we could even have unit test for it somehow. 18:44:27 wouldn't you have to test that you can use a service, update the api node, then update the db 18:44:29 xek, have you tried it today to see how badly say kilo -> liberty went 18:44:52 I hope we don't have to support new code with old db, too. 18:44:58 shaleh, I checked that there were a couple of patches which dropped things 18:45:18 shaleh, that's why I want to start with Mitaka 18:45:23 bknudson_ wouldn't your api layer have to know how to deal with both versions of the schema before you can drop the old schema? 18:45:26 bknudon, I hope not :) 18:45:27 xek, but have you tried to perform this style of upgrade and documented the failures? 18:45:39 but we will have to support old code writing to db and new code using it. 18:45:52 xek: how do we add features is alter isn't allowed? 18:46:06 dstanek: alter column 18:46:12 dstanek, you can add things, and drop them the next release 18:46:32 xek but the api layer has logic that knows how to handle both cases of the schema 18:46:57 so the idea is to deprecte the feature/db model vefore droping table/column whatever? 18:47:02 don't we do this already ? 18:47:11 i think not 18:47:25 bknudson_: we have had columns altered in the last release (?) to support feature 18:47:26 s 18:47:32 I think we do, I can't think of a thing we remove without deprecation 18:48:44 if we already do it the unit test will just be an additional hint for the code reviewers 18:49:25 https://review.openstack.org/#/c/241603/ - link to the unit test 18:49:30 if this is what our users want then I think we should give it to them. 18:49:51 xek have you talked to odyssey4me about this? 18:49:53 a unit test is only the start of it 18:50:08 lbragstad, not yet 18:50:14 bknudson_, perhaps we do a dry run of this for mitaka and implement it fully in N? 18:50:18 xek he has a lot of the same ideas 18:50:58 lbragstad, I'll get in touch 18:50:58 shaleh bknudson_ I'd like to make some changes to the revocation_event table, we could try it there 18:51:09 xek you can find him in #openstack-ansible 18:51:37 I would like to see someone run the upgrade like xek suggests and document just how well (or not) it goes 18:51:38 lbragstad: dropping and altering columns? 18:51:38 xek he wanted to discuss this today in the meeting, but is based in the uk 18:51:49 to really support 0-downtime we have to deal with a multiple phase migration....i think i need to read the spec on this to understand it better 18:52:01 bknudson_ I want to remove the datetime format usage in sql 18:52:18 that's going to break everything 18:52:22 xek: we're gonna have to continue in -keystone, i want to give davechen a few minutes 18:52:30 you're going to need to do both for a release 18:52:35 ++ :) 18:52:37 stevemar_, ok, I welcome any reviews 18:52:43 both formats in separate columns 18:52:44 bknudson_ it will be a long running migration, yes 18:52:46 stevemar_: my topic is pretty light. 18:52:53 bknudson_ yeah, exactly 18:53:05 #topic Should return the endpoints from endpoint_group when using "endpoint_filter.sql" as catalog's backend driver? 18:53:11 okay. 18:53:11 davechen: what's up? 18:53:18 i filed a bug there. 18:53:39 i just found when using endpoint_filter.sql as the backend for catalog 18:53:43 #link https://bugs.launchpad.net/keystone/+bug/1516469 18:53:43 Launchpad bug 1516469 in OpenStack Identity (keystone) "endpoints not show correctly when using "endpoint_filter.sql" as catalog's backend driver" [Undecided,New] - Assigned to Dave Chen (wei-d-chen) 18:53:46 davechen, yes 18:53:53 gyee: yes? 18:53:57 yes for what? 18:54:08 i am not sure whtether it's a bug or not 18:54:30 since I am not sure about the fearure implemented long time ago 18:54:32 that is a bug 18:54:51 but seem not many people use it. 18:54:54 we should be filtering the endpoints when the endpoint_filter.sql backend is used 18:55:07 it's hanging around for so long time, no one found it? 18:55:14 both of those calls should be going through the same backend method so should get the same result :( 18:55:30 yes. but it's currently not. 18:55:39 I ran into some of this when working on Mercador. 18:55:43 there might be some code to be changed. 18:55:46 davechen: i doubt many folks are using endpoint filter 18:55:52 I think they are 18:55:58 but's it not hard to so this, or to fix this. 18:56:01 keeps catalog managable 18:56:15 stevemar_: but no report for it so far? 18:56:22 davechen: looks like you found it 18:56:43 okay, not that all of you experts agree it's a bug, i am going to fix it shortly. 18:56:48 now that 18:57:01 you mean bug or feature? 18:57:08 the bug 18:57:14 it's a bug 18:57:17 its a "special feature" 18:57:20 i think it's should be a bug instead of a feature. 18:57:22 :P 18:57:26 so yes, its a bug! 18:57:34 alright alright gyee 18:57:35 gyee: you are asking for spec for review :) 18:57:36 ? 18:57:45 davechen: he's failing at being funny :) 18:57:53 sorry 18:57:57 davechen: thanks for staying up! 18:58:01 i know it's late there 18:58:11 nope, it's good time for me, i am in US 18:58:24 davechen: :O 18:58:25 davechen, you in Santa Clara? 18:58:30 stevemar_: thanks for your kindness. 18:58:32 we should go grab coffee 18:58:34 SAT 18:58:44 yes, I am done 18:58:45 #topic REVIEW SPECS! 18:58:46 thanks guys 18:58:53 everyone review specs, go go go 18:59:00 * gyee runs 18:59:01 we need more eyes 18:59:07 eyes on all the specs! 18:59:17 looking at you bknudson_ 18:59:21 not reviewing enough 18:59:26 that is all 18:59:27 I don't care about specs 18:59:28 thank you everyone 18:59:32 hah 18:59:36 bknudson_: the specs care about you <3 18:59:44 #endmeeting