16:12:04 <vishy> #startmeeting Hierarchical Multitenancy
16:12:05 <openstack> Meeting started Fri Feb  7 16:12:04 2014 UTC and is due to finish in 60 minutes.  The chair is vishy. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:12:06 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:12:09 <openstack> The meeting name has been set to 'hierarchical_multitenancy'
16:12:22 <vishy> #topic Role Call
16:12:29 <vishy> who all is here?
16:12:42 <VINOD_> Vinod here
16:12:49 <schwicke> Ulrich here
16:13:03 <tiamar> Tiago here
16:13:04 <raildo> raildo here
16:13:07 <tellesnobrega> Telles here
16:14:04 <vishy> #topic POC Discussion
16:14:21 <vishy> has anyone had a chance to experiment with my POC?
16:14:34 <VINOD_> yes
16:14:42 <VINOD_> i have looked into it
16:15:05 <vishy> comments/feedvack/issues?
16:16:10 <VINOD_> I think the main issue is how nova will come to know the complete hierarchy
16:16:39 <vishy> VINOD_: why does it need to know the complete hierarchy?
16:17:31 <VINOD_> I guess it has to
16:18:06 <VINOD_> lets say i have authenticated by generating a token at level 2...now i want to get some information from level 4...Am i going to put all the ids/names upto level 4 in the URL..
16:18:09 <raildo> When you list the instances, for example, will be listed all the hierarchy, or just design it is authenticated?
16:18:25 <VINOD_> or if i am asking blindly given me something at level4
16:18:43 <VINOD_> my token is at level 2...then how nova knows that i am asking something at level 4
16:18:55 <VINOD_> and the token is at level 2 and so prefix will match
16:19:35 <vishy> if you want to limit your scope to level4 then you should reautenticate to level4
16:19:58 <vishy> although this could also be solved by allowing you to limit the scope of a token you have
16:20:22 <vishy> but in general reauthenticating seems like it might be easier
16:20:30 <VINOD_> Then if i want to manage all the children from level 2...i need to every time switch levels
16:20:48 <vishy> it depends on what you mean by manage
16:21:50 <VINOD_> I mean lets say i want to run a query at sub projects of level 3...then i have to switch the context for each sub-project at level 3..i cannot have the token at level 2 and do all the things without switching
16:21:57 <VINOD_> Then what about quotas
16:22:06 <schwicke> a token issued at level 2 will be valid for all operations on level two AND below, right ?
16:22:17 <tellesnobrega> VINOD_: ++
16:22:20 <VINOD_> But here it is not happening
16:22:45 <VINOD_> i have to switch from one child to another child of level two and limit my scope
16:22:47 <vishy> the point is that you can do everything for the projects below
16:23:05 <VINOD_> How..if nova does not know the hierarchy...
16:23:05 <vishy> the only command that works across multiple instances at once is list
16:23:15 <VINOD_> the same problem i am facing right now in setting the quots
16:23:26 <vishy> so quotas can use prefixes as well
16:23:36 <VINOD_> in nova, the quota of a project does not have information about to which domain it belongs
16:23:48 <vishy> it would in my model
16:23:55 <VINOD_> ok
16:24:06 <vishy> all of the things in the system would move from project_id -> hierarchical.project_id
16:24:18 <VINOD_> ok
16:24:22 <vishy> so you just walk back up the list achecking quotas
16:25:09 <vishy> VINOD_: to your first point, if we do have other actions on multiple instances then the rescoping starts to become painful
16:25:22 <vishy> but as it is today, I don't think it is too bad
16:26:05 <VINOD_> For example, i want to list all VMs created by sub-projects of a parent project....will it be possible
16:26:25 <VINOD_> if i authenticate and set my scope at parent project
16:26:31 <vishy> VINOD_: well it would be more of a client side filter in this case
16:26:52 <vishy> i think the id vs. name thing is a bit of a problem
16:27:10 <vishy> since if you can list across multiple projects you need to as a user see which instances belong to which project
16:27:19 <VINOD_> If we use name, the name can be repeated at different branches...which will create more problem
16:27:26 <vishy> and a long.string.of.ids
16:27:32 <vishy> is not readable
16:28:02 <vishy> i'm thinking that we may need to keep track of scope_name in addition to scope_id
16:28:09 <raildo> vishy: +1
16:28:13 <vishy> so that we can use the name for display for the user
16:28:40 <VINOD_> ok
16:28:56 <schwicke> +1
16:28:58 <vishy> then the client could limit the scope for a query
16:29:04 <tellesnobrega> vishy: can you give an example on how will this work?
16:29:07 <vishy> and in fact it could be added to the list command
16:29:17 <vishy> tellesnobrega: which specifically
16:29:36 <vishy> so the filter could be done server side for certain objects
16:29:50 <tellesnobrega> vishy: how the names would be shown to the user, keeping track of scoped_name and scoped_id
16:29:51 <vishy> right now we have nova list --all-tenants
16:29:55 <vishy> we could have
16:30:10 <vishy> nova list --scope=orga
16:30:13 <vishy> or something
16:30:40 <VINOD_> and will be there any config option to limit the maximum hierarchy level that one can go upto
16:31:07 <vishy> so you would see a field called something like "scope" in the instance list
16:31:21 <vishy> VINOD_: that seems interesting
16:31:26 <vishy> like max depth
16:31:38 <VINOD_> Yes..otherwise it can become messy
16:31:47 <VINOD_> with long list of subprojects ids..or names....
16:31:48 <vishy> VINOD_: although that isn't particularly easy to do in the db
16:31:59 <tiamar> +1
16:32:02 <vishy> so it might have to be a python filter
16:32:16 <tellesnobrega> vishy: +1
16:32:25 <vishy> some of them support regex so you could have a regex filter in addition to the prefix match
16:32:35 <vishy> to only allow a certain number of separators
16:32:41 <VINOD_> ok
16:32:56 <tiamar> the root could store the number of leaves or depth
16:33:24 <schwicke> the maximum depth should be configurable though rather than hard coded
16:33:40 <vishy> tellesnobrega: so I would think we would have to keep the name in the database as well in that case which is nasty
16:33:44 <schwicke> was discussing this recently with Vinod. I think it is a must
16:33:54 <vishy> tellesnobrega: for the name question. I really don't have a good answer for this
16:34:52 <vishy> schwicke: it could be kind of like ldap
16:35:08 <vishy> where you can use SCOPE_BASE, SCOPE_ONELEVEL, or SCOPE_SUBTREE
16:35:29 <vishy> or we could have it be an arbitrary depth number
16:36:06 <schwicke> vishy: actually, we were thinking of a configuration parameter in nova.conf which can only be changed by the cloud owner
16:36:16 <schwicke> to keep it simple
16:36:22 <vishy> schwicke: as in maximum number of depth allowed?
16:36:31 <vishy> for search or for project creation?
16:36:33 <schwicke> vishy: yes
16:36:43 <schwicke> for project creation
16:36:46 <tellesnobrega> vishy: schwicke, why is it so important to have a max depth? keep balance in the tree?
16:37:06 <vishy> schwicke: that makes sense
16:37:17 <schwicke> motivated by experience from other services with our users
16:37:33 <vishy> i ssupect it is useful at the very least because some projects may have limits about the size of the field used to store the ids
16:37:33 <schwicke> who are complex organizations by themselves
16:37:34 <tiamar> schiwicke : that seems like a quota for domain that was proposed a few wweks ago
16:38:06 <vishy> ok so I will have to think about the name thing a bit more
16:38:14 <VINOD_> Vishy: I think, we have two ways for nova to know the hierarchy. One is nova always asks keystone to known the hierarchy whenever is required or every service keep the hierarchy information with them..But both have some advantages and disadvantaes..
16:38:21 <schwicke> tiamar: except that in our case introducing domains is not enough
16:38:27 <vishy> one option is to just make the owner field based on names instead of ids as ayoung suggested in his email
16:39:14 <vishy> VINOD_: i have been trying to avoid keeping that information out of the services
16:39:38 <VINOD_> I think its not a question of ids or names...its all about deciding whether a query fired by a user with a scope can run that query or not...
16:40:07 <VINOD_> if you store it with ids or names, any way the service has to talk to keystone to see the query fired for some project comes under the scope or not
16:40:12 <vishy> the whole point of doing it as a hierarchy is you can make that decision without knowing the whole hierarchy
16:40:22 <vishy> via prefix matching
16:40:47 <vishy> if you get rid of that basic idea you might as well do the whole thing as a bunch of tags
16:40:54 <raildo> I believe it is a problem Nova to keep asking the  keystone due notifications that are not well implemented.
16:41:55 <VINOD_> Let me give an example,  project A -> project A1 -> project A2 -> project A3. Now if lets say a user has created a VM at project A3. The nova will have only information that a VM exists for project A3. Now i have authenticated with a scope at Project A1.
16:42:21 <vishy> the "owener " of the VM is A1.A2.A3
16:42:32 <VINOD_> If i want to list VMs comes under all levels of project A1...nova first has to talk to the keystone..to get which are all the project lies beneath project A1
16:42:33 <vishy> so what is the problem?
16:42:44 <vishy> no it doesn't
16:42:49 <vishy> it does a prefix match
16:42:57 <vishy> that is the whole idea of the POC
16:43:58 <vishy> ok we only have 15 minutes left
16:44:02 <vishy> so next steps
16:44:20 <tiamar> so the table for instances stores A1.A2.A3 ?
16:44:23 <vishy> #topic Next Steps
16:44:28 <VINOD_> That's exactly my point...
16:44:31 <vishy> tiamar: correct
16:44:38 <VINOD_> ok
16:44:51 <vishy> the poc does this today
16:45:02 <vishy> but it uses ids.stuck.together
16:45:21 <vishy> which means there is no good way to display to the user which project it is from
16:45:55 <vishy> there are other places where we need the project name in nova so perhaps the right way here is just to ask keystone for the name of the project when we need it and cache the value
16:46:16 <vishy> i'm a little bit happier about that than I am about asking for the whole hierarchy
16:46:29 <VINOD_> ok
16:46:46 <vishy> so the purpose of the poc was to create something so we could see what else is missing
16:46:56 <vishy> so we could add a few features / refine it
16:47:11 <vishy> for example, someone could attempt to add the project_name query and caching
16:47:27 <vishy> or someone could attempt to implement a single level of quotas
16:47:44 <vishy> someone could actually modify keystone to send scope in the token separately from project_id
16:47:58 <vishy> someone could try making keystone have nested projects and send a bigger scope
16:48:12 <vishy> these could all be useful things to take to the summit in Atlanta
16:48:24 <vishy> does anyone feel like tackling any/all of these things
16:48:40 <vishy> the alternative is we create a more detailed design explaining the solution
16:48:49 <vishy> and have a couple of discussions in atlanta
16:49:25 <VINOD_> We could do some thing on quota...
16:49:32 <VINOD_> Tiago: what do you say
16:49:47 <VINOD_> we have already done some implementation of quotas with domains...
16:49:54 <VINOD_> and projects and users
16:50:13 <schwicke> maybe we can extend that to the POC
16:50:39 <tiamar> VINOD_, schwicke : i think extending to the poc is a good idea
16:50:42 <VINOD_> Yes, definitely myself schwicke can do it
16:50:57 <VINOD_> i will try to do it
16:51:00 <VINOD_> any deadline?
16:51:04 <tiamar> I'm thinking on the keystone part
16:51:09 <schwicke> next meeting I guess :)
16:51:14 <vishy> no deadline
16:51:28 <tiamar> the new policy json already define some important roles for multitenancy
16:51:28 <tellesnobrega> im not sure yet, but maybe me and raildo can tackle one of the solutions
16:51:37 <tiamar> yeap
16:51:37 <raildo> +1
16:51:50 <VINOD_> So, i will check on quotas
16:51:50 <tiamar> tellesnobrega, raildo : on keystone, right? good.
16:51:58 <raildo> yes
16:52:00 <tellesnobrega> tiamar: yeap
16:52:15 <vishy> #action schwicke, VINOD_ to try to stick hierarchical quotas in the poc
16:52:31 <vishy> #action vishy to experiment with name caching
16:53:08 <vishy> ok next week we can take a look at anything that is produced
16:53:41 <VINOD_> ok
16:53:42 <tiamar> #action raildo tellesnobrega to work on nested projects
16:53:43 <tellesnobrega> ok
16:53:45 <VINOD_> thank you vishy
16:53:47 <vishy> putting more levels in keystone and actually passing the scope separately would be cool
16:53:50 <vishy> alright
16:54:00 <vishy> I will see you guys next week!
16:54:04 <tellesnobrega> see you
16:54:06 <schwicke> see you!
16:54:10 <VINOD_> see you....bye...
16:54:14 <raildo> bye
16:54:23 <VINOD_> Wishing all a nice weekend
16:54:34 <vishy> #endmeeting