13:00:04 #startmeeting nova api 13:00:05 Meeting started Wed Feb 8 13:00:04 2017 UTC and is due to finish in 60 minutes. The chair is alex_xu. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:00:06 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 13:00:09 The meeting name has been set to 'nova_api' 13:00:11 who is here today? 13:00:11 o/ 13:00:35 o/ 13:01:19 cdent: gmann let us wait johnthetubaguy and sdague if they are around 13:01:20 * johnthetubaguy waves 13:02:26 let us start the meeting 13:02:33 #topic Pike ideas 13:02:42 #link https://etherpad.openstack.org/p/nova-api-pike 13:03:08 ^ I created an etherpad to collect the idea for Pike 13:03:29 nice, thanks 13:03:48 did we want to discussion the policy ideas, I am curious what folks think 13:03:53 Matt helps to input some items 13:03:57 also big thank you to alex_xu for reviewing that 13:04:11 johnthetubaguy: np 13:04:26 sure, we can discuss the policy ideas 13:04:35 yea 13:04:53 so operator feedback first 13:04:54 most folks struggle editing policy right now 13:05:17 many of them are forced into re-writing all rules because the want more RBAC levels than we have in the default policy file 13:05:46 my thinking is we define a rough end goal for where we want to be, and get that reviewed by operators 13:05:59 then we go through each rule, one by one, and make it match the plan 13:06:11 ...so to the "plan" I have suggested 13:06:26 'reviewed by operators' means sending email to operator ML? 13:06:33 basically we add more roles, like observer roles, roles for neutron and cinder, operator rules, etc 13:06:45 alex_xu: hopefully means talking to them at an ops meetup, at a minimum 13:06:57 alex_xu: but thats part of it 13:07:01 i see but there might be lot of roles depends on each operator 13:07:06 I really want something we agree on first 13:07:07 or we add common one 13:07:15 so there are two things here 13:07:21 the name of the role, and the common patterns 13:07:35 if we can cover 80% of deployments with the default, thats great 13:07:55 folks will just need to say what the role name is in their keystone, and we take care of the rest, for the 80% 13:08:08 for the 20% we need to add doc strings on each of the rules, so they know what they control 13:08:18 o/ 13:08:38 basically, my idea is stop most folks having to change much of the policy file 13:08:53 whether the name of role need to get more widly agreement, so other project can use the same name 13:09:02 but also give folks who need to change everything (or audit it) a bit of a helping hand (i.e. no need to read code to work out what each rule means) 13:09:34 alex_xu: we actually we need it to be project specific, like nova_operator etc, but yes common patterns are good 13:09:46 so I am talking to keystone folks a bit about this, they have started a policy meeting 13:10:01 they have a proposal for some roles we could create 13:10:02 johnthetubaguy: can you be more specific on folks that are changing policy so much? 13:10:30 sdague: I think they want observer roles, and operator roles that are not admin everywhere 13:10:32 johnthetubaguy: yeh, I honestly thought we had 5 roles decided on back in tokyo 13:10:41 sdague: I am hoping we can get access to some specific ones 13:10:57 sdague: yeah, its basically those 13:11:22 ok, what stands between those hallway conversations and just doing it? 13:11:38 upgrade story, the spec I wrote out tried to cover that 13:12:12 I think I have the OSIC folks who are willing to do the work to implement it 13:12:32 johnthetubaguy: ok, so... every time I brought this up at ops meetups, people mostly didn't change much in policy 13:12:32 kinda a follow on from config work that was related 13:12:33 ++OSIC folks 13:12:53 sdague: I was told they tried, it failed, and so gave up 13:13:02 johnthetubaguy: I guess I need to understand the spec more 13:13:09 at the manchester meetup 13:13:38 maybe the issue is the following, today, we do policy as "policy_point: can be done by this complicated formula of things" 13:13:49 vs the way people would think about things: 13:13:58 giant list of policy points... 13:14:19 observer_can: p1, p2, p4, p8 13:15:04 thats a good point, this is still reversed in my spec 13:15:10 and... it almost seems like you could add role stanzas like that as an overlay to the existing content 13:15:12 p1: observer and admin can, etc 13:15:23 so that the current stuff is just the current stuff 13:15:31 sdague: yeah, it should be equivalent 13:15:32 and the new stuff is new stanzas 13:15:49 which... makes the upgrade a lot more viable for users, right? 13:16:28 maybe... need more visualization on it 13:17:00 johnthetubaguy: ok 13:17:04 I was also talking about renaming the rules (via deprecation aids) to better match the REST API rather than the code 13:17:16 yeh, which I'm all for 13:17:28 but if there is a different upgrade issue, we should probably address that 13:17:54 maybe the staging is (1) fix names and docs, (2) look at making what each role does more visible (3) think about adding extra roles 13:18:25 it was mostly the "we need to evolve like config" upgrade issue that I was referring too 13:18:58 johnthetubaguy: ok cool 13:19:17 sorry, rewrapping my head on the whole thing 13:19:36 sdague: totally, been going through that a bit myself 13:20:05 FWIW, we really have made a lot of progress at this point 13:20:24 no policy file needed by default, defaults in the code, reduced the number of rules by half 13:20:30 yeh 13:20:37 we are in a much better place to move forward now, which is kinda awesome 13:20:39 there is still a docs gap though, right? 13:20:54 pretty much the policy rules only make sense if you go and read the code 13:20:55 yeah, and a naming to avoid the docs having to say everything 13:20:58 yeah 13:21:06 well, the docs should be there anyway 13:21:13 names are never as obvious as you think :) 13:21:27 yeah, I tried it, its hard 13:21:54 doc for rules will be as description right ? so that it can be seen from sample file like config one 13:22:03 yeh 13:22:08 my current leaning is towards compute:servers:action:live-migration 13:22:20 gmann: turns out we have two doc strings in our sample file already 13:22:34 yea :) 13:22:40 johnthetubaguy: do we have anything published web side? 13:22:44 #link http://docs.openstack.org/developer/nova/sample_policy.html 13:23:08 um... I wouldn't call that docs :) 13:23:26 I mean two of them have strings 13:23:30 What I really expect is: 13:23:41 but its totally not docs 13:23:43 - os_compute_api:os-aggregates:set_metadata 13:24:18 this allows the roles in question to set metadata on aggregates. 13:24:20 how about embeding policy rule inside api-ref for corresponding APIs 13:24:34 sdague: I was meaning the rule near os_compute_api:os-attach-interfaces:create has added some docs already 13:24:36 default_permissions: admin role only 13:24:48 ok 13:24:53 gmann: thats mixing operator and API user though 13:25:04 it's not api-ref for sure 13:25:06 yeah, we need to render the default better, like the config stuff 13:25:09 ah yea 13:25:14 but it should end up with/near the config guide 13:25:30 but we all seem agreed with the same as config, you shouldn't need to read the code to understand it 13:25:35 yeh 13:25:55 yes 13:26:16 if we go and add docs, I am tempted to consider a rename, but maybe that is premature? 13:26:33 johnthetubaguy: honestly, we can document things first, that would be of immediate help to people 13:26:41 sdague: thats fair 13:26:52 I was wanting to avoid visiting all the rules twice 13:26:54 I'd rather get us in the habbit of writing down what is instead of assuming we can make a change that means we don't need docs 13:26:56 but we have less rules now 13:27:15 sdague: yeah, we totally need the docs, even with the most perfect of names 13:27:30 do we have other topics? I don't want to take over the whole meeting with this 13:27:49 yeah, I think we should review the other ideas 13:27:53 other items in https://etherpad.openstack.org/p/nova-api-pike 13:28:18 well, before we leave... 13:28:25 I can add a spec for adding in the docs 13:28:34 so folks stop needing to read the code 13:28:48 johnthetubaguy: that would be cool, I would be up for helping get that all sorted 13:28:57 sdague: cool, thanks 13:29:10 I think the next step might be making the rules mean something (actually adding targets) 13:29:32 like admin_or_owner is largely a no op today 13:29:40 because of the deafault rule target 13:29:48 we are safe due to the hardcoded DB checks 13:30:09 it feels like we should add in the targets, then consider removing the hard coded things once we are happy with our unit tests 13:30:15 but thats a separate discussion 13:30:19 johnthetubaguy: I thought the hardcoded db checks were mostly gone 13:30:37 sdague: they are gone for is_admin, they are everywhere for membership in the project 13:30:54 oh, you mean owner 13:31:06 sorry, yeah, owner 13:31:20 that is not checked in the policy later, even though the rule says its checking 13:31:28 right 13:31:39 we just check context.project_id == context.project_id right now, which is amuzing 13:32:04 heh 13:32:15 yeh, well the project scoping like that goes back years 13:32:18 although there is a separate discussion about that being a good idea, as it keeps policy from making the API interoperable 13:32:33 so we totally said we should move on lol, we should do that... 13:33:28 :) 13:33:38 turns out that was a conversation stopper 13:33:44 what are team members excited about for the next cycle? 13:33:50 what would they most like to solve 13:34:20 https://review.openstack.org/#/c/386555/ on capabilities would be awesome 13:34:47 yea that is nice to do 13:34:53 query param schema excites my less, but would be nice 13:35:16 do we feel like capabilities can come to a resolution on design for the PTG? 13:35:22 query param schema is just a refactor, so it can be background work for new people 13:35:27 so +1 for capabilities 13:35:41 and cleanup extension things as sdague started, make code more easy 13:36:13 sdague: so turns out we didn't have many people from keystone or API-wg in the summit session, if we can fix that, maybe? 13:36:24 I guess the concern I have about capabilities... it relies a lot on link following 13:36:34 do we know of any sdk tooling that's regularlly doing that? 13:37:04 there is a proposal to remove the link following, for the global scope and per type scope 13:37:10 that might be a good idea 13:37:16 spending some time poking on some other APIs recently, they tend not to expose anything on their API until the SDK also implements it 13:37:43 so I keep thinking about to horizon and enabling buttons in the GUI 13:37:54 johnthetubaguy: sure, but they aren't writing their own client 13:38:10 so... I'm fine exposing capabilities 13:38:11 like ironic and libvirt based servers, choose which turn on 13:38:18 I think it's hugely needed 13:38:31 sdague: they do, I was more thinking efficient calling patterns down that thought path 13:38:38 link follwing is expensive 13:38:58 I mostly get concerned that it is useful only if we have sdk drops with the functions 13:39:26 thats true, would this be adaptive help in the SDK? 13:39:51 I guess I am struggling on the consuming part, if I don't think about horizon 13:40:49 yeh, I don't know. 13:40:52 who were the original people or group that said "we need capabilities"? 13:41:04 cdent: definitely people making guis 13:41:30 ironic snapshots always fail, why not hide the button, etc 13:41:38 the other side was live-resize 13:41:52 we expect folks to turn that off, and want that to be discoverable when talking to multiple clouds 13:42:11 but thats a little bit of a vanity use case, in some ways 13:42:22 in the stuff I've read, that particular use case is not made as clear as it potentially could be. Instead we end up with discussions on how to make capabilities as (uh) capable as possible 13:42:50 I know that mordred ended up basically writing crazy autoconf like stuff where he tries a bunch of things in shade because there is no way to figure out that he can't do a thing in advance 13:42:58 and doesn't want to fail deep in complicated logic 13:43:13 this is mostly about OpenStack clouds could do A 13:43:33 but some clouds turn off B, C, D for performance, security, hypervisor support reasons 13:43:51 right, the how do I get an external IP on my server question 13:44:47 cdent: so perhaps the issue is framing the use case a bit better 13:44:52 maybe shade is the SDK use case 13:45:00 the way I imagined this was more along the fact that 13:45:33 as a user I'd prefer to ask a "capabilities service" for that information, not go grubbing around in all the projects at several urls. perhaps a service could register it's capabilities at startup or regularly (not deployment) 13:46:23 cdent: yeh 13:47:02 cdent: that works for top level constructs for sure. It doesn't solve the "this cloud has baremetal and libvirt computes" 13:47:11 and this compute you have can't be live migrated 13:47:47 I think anything that can be started as "this cloud..." ought to be able to be "registered" 13:48:02 cdent: sure 13:48:13 and I think anything that says "this " should be in the representation of that entity itself 13:48:28 cdent: yep, no disagreement there 13:48:53 would aggregated centrally work better, i.e. exposed in each service, but also aggregated (via checking everything in the service catalog, etc), or does that mean we do the work twice? 13:48:56 ok... so where does that leave us, because this seems circular 13:49:08 which probably means the thing being carved off is too big 13:49:17 and maybe it can be started as something smaller 13:50:02 maybe per endpoint global, and per endpoint type only? 13:50:15 leave the aggregation / central piece for later? 13:50:50 or, find a specific problem, like a chunk of code in shade or horizon we'd like to go away, and work backwards on solution to that 13:50:52 stop the per resource capabilitiy thing at first, just go for the shade and horizon use cases first 13:51:29 johnthetubaguy: yeh 13:51:38 so horizon just shows all buttons based on VM state, and shade just hard codes 13:51:51 so it would be improving both of those to some degree I guess 13:52:00 maybe just do the horizon one to start with? 13:52:14 ironic and VM mixed cloud 13:52:17 get the buttons right 13:52:26 * alex_xu reminds 8 mins left 13:52:34 (for user operations) 13:52:50 cdent: from an api-wg perspective, how would you like to / expect to constrain scope so we can get feedback and iterate on something real? 13:54:15 sdague: that's an excellent question. I don't really have a good answer. I agree with you that the current scope is too large, and have said as much on the ongoing review of the capabilities spec. In the context of the api-wg what matters is what a capability looks like and where it is retrieved from 13:54:52 * mordred waves 13:54:55 (and of course that whatever it is, everyone is doing the same thing) 13:55:08 cdent: OK with doing it per endpoint first, as s stepping stone towards centralised? 13:55:44 johnthetubaguy: I think that's probably the only way it is likely to happen, but I don't like it that much because it means it will never truly be centralised :) 13:56:00 sdague, cdent: so, for several of these things (although admittedly not enough of them) we have flags in os-client-config so that discovery code can be skipped for known clouds 13:56:29 * mordred reminds self to go make a "this cloud requires floating ips" flag 13:56:36 mordred: code reference? 13:56:41 one sec ... 13:56:56 I think that shade might be more concrete than horizon for these discovery routines 13:56:57 cdent: yeah, I am with you on that 13:57:32 sdague: assuming they are not harder questions to answer, +1 13:57:42 sdague: line 14 and line 16 are both concrete examples: http://git.openstack.org/cgit/openstack/os-client-config/tree/os_client_config/defaults.json#n14 13:57:44 cdent: I think that if we can convince keystone folks that more things should flow into service catalog, there is a centralization plan 13:57:53 sdague: yes 13:58:43 I think we could argue thats more useful than exposing policy, and so it might be what they are wanting to do 13:58:45 cdent: so, I think that's the centralizing plan 13:59:09 mordred: it makes me slightly sad that all the api versions are hardcoded in shade 13:59:14 ok, we need to close the meeting now 13:59:16 sdague: those are just defaults 13:59:17 sure 13:59:23 we should back to nova channel 13:59:30 sdague: they're overridable in config - and they also can be auto-discovered 13:59:31 mordred: ah, ok 13:59:32 sdague: but yes 13:59:42 and johnthetubaguy sdague there is patch looking for feedback https://review.openstack.org/430497, that is also mordred faced problem 13:59:52 sdague: like, if your config says "image api version 2" and the cloud only has 1, we'll use 1 but give you a warning 14:00:00 #endmeeting