12:00:09 #startmeeting nova api 12:00:09 Meeting started Tue Oct 20 12:00:09 2015 UTC and is due to finish in 60 minutes. The chair is alex_xu. Information about MeetBot at http://wiki.debian.org/MeetBot. 12:00:10 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 12:00:12 The meeting name has been set to 'nova_api' 12:00:18 who is here today? 12:00:24 o/ 12:00:25 hi 12:00:28 hi 12:00:42 hello everyone! 12:00:51 o/ 12:01:05 #topic actions from last meeting 12:01:22 Action: alex_xu file bug for wadl doc problem, and work with Kevin_Zheng on fix them 12:01:31 also gmann help on this 12:01:38 #link https://etherpad.openstack.org/p/nova-v2.1-api-doc 12:01:50 all the problem which I found, already fixed 12:01:57 also part of them already merged 12:02:12 Kevin_Zheng, gmann thanks for fixing 12:02:20 sorry for late 12:02:23 Your welcome 12:02:26 ah, great stuff 12:02:35 Action: alex_xu create bp and etherpad to track the concept doc work 12:02:35 you're 12:02:44 #link Action: alex_xu create bp and etherpad to track the concept doc work 12:02:49 oops 12:02:51 #link https://blueprints.launchpad.net/nova/+spec/complete-todo-in-api-concept-doc 12:03:03 ^ I just created the bp 12:03:29 please free feel to take any todo from api concept doc 12:03:43 just approved it 12:03:53 johnthetubaguy: thanks! 12:04:16 we probably we dicussion how to get more people join those works, we have a lot of doc work in M 12:04:23 yea 12:04:27 +1 its a good thing to talk about that the summit 12:04:37 Action: gmann_ will creae BP and etherpad to track the work merge api sample test and run all the extensions 12:04:42 we have a low hanging fruit doc, but this is not really that 12:04:48 did today 12:04:50 gmann_: how was that? 12:04:53 #link https://blueprints.launchpad.net/nova/+spec/api-sample-tests-with-all-extensions 12:04:55 johnthetubaguy: yea 12:05:05 gmann_: cool, thanks 12:05:23 I remember there already have a patch up 12:05:33 johnthetubaguy, please aprove that too :) 12:05:43 i also started base patch on that- https://review.openstack.org/#/c/237400/ 12:05:46 -1067 nice 12:06:00 gmann_: cool 12:06:11 did those removed tests just become redundant I guess? 12:06:52 oh, I see now, cools 12:06:58 johnthetubaguy, yea redundant tests written for separate extensions will be removed 12:07:09 approved that blueprint 12:07:25 and best will be we will get rid of all extension list flag setting for v2 12:07:32 I'm thinking how we test bdm or scheduler-hints thing 12:07:35 johnthetubaguy, Thanks 12:08:01 will all bdm and scheduler-hints showing one single servers api sample test? 12:08:21 alex_xu, may be , i need to check those 12:08:43 gmann_: yea, we only have one sample test, but we need show all of those parameters 12:08:59 yes 12:09:15 ok, cool, let's move on 12:09:17 for each resource single tests will all param in request response 12:09:17 Action: johnthetubaguy to sort out etherpads for API session, and send link to alex_xu 12:09:34 #link https://etherpad.openstack.org/p/mitaka-nova-api 12:09:44 johnthetubaguy: thanks for the etherpad 12:09:59 hey folks (sorry, was working on etherpads) :) 12:10:05 also thanks to sdague, he already list all the things in the etherpad 12:10:06 alex_xu, johnthetubaguy cool 12:10:21 sdague: yea, cool etherpad :) 12:10:22 Thanks guys, it will be great 12:10:42 I didn't found anymore I can add to etherpad 12:10:52 johnthetubaguy: you happy with that etherpad agenda? 12:11:06 #topic Summit API Session 12:11:13 I'll probably delete the little agenda bit at the bottom 12:11:13 * alex_xu jump the topic directly 12:11:26 sdague: looks close, I keep meaning to loop back around all of those etherpads to see where we are add 12:11:46 sdague: that was more a consitency thing, I think we just move pre-reading to the top, and the rest is your agenda 12:11:51 johnthetubaguy: ok, I think we're probably not going to get to the feature classification bits yet 12:11:55 yeh, that's fine 12:12:22 sdague: yeah, I am thinking about adding a testing session to replace the final unconference session 12:12:37 we need to push on testing, thinking about how to make it happen 12:12:52 I also think I found another person in our team that's interested in helping here, including on the docs side. I think trying to recruit more folks to help on docs this cycle will be an important part of this. 12:13:06 sdague: +1 12:13:12 sdague, yea 12:13:27 looks like we still didn't have microversion test on tempest 12:13:36 alex_xu: we should both talk to the OSIC people about making this a priority 12:13:40 yeah, right 12:13:44 alex_xu: that would be a good add, at least one of those 12:13:45 alex_xu, yea, we have 1 session for that 12:13:49 and we have qa-session for microversion test 12:13:51 johnthetubaguy: yea 12:14:04 gmann_: oomichi cool 12:14:25 I have WIP patch for microversion top bottom and changed testing 12:14:29 oomichi: ah, OK, can you report back to the nova meetup on the outcome of that maybe? 12:14:31 #link https://etherpad.openstack.org/p/mitaka-nova-api 12:14:37 oops, wrong 12:14:39 gmann: you mean in tempest or nova? 12:14:43 #link http://mitakadesignsummit.sched.org/event/10827a8275cb824c9e2a0fd2ee9c4ae1#.ViYwIm48qaw 12:14:51 in tempest 12:14:53 johnthetubaguy, on nova 12:15:04 gmann: do we have a blueprint for that? 12:15:24 as we discussed in Vancouver that nova functional test should tests top bottm an changed version 12:15:33 not yet, 12:15:38 gmann: agreed with the direction, just thinking about a specless blueprint to track it 12:15:49 sure will do tomorow 12:15:49 gmann: send me a link, and I can get that approved for you 12:15:57 johnthetubaguy, sure 12:16:00 :) 12:16:04 johnthetubaguy, Thanks 12:16:17 brb, baby waking up. 12:16:28 sdague, :) 12:16:32 sdague: cool :-) 12:16:36 #action gmann_ create bp for microversion top bottom and changed testing 12:16:48 alex_xu, Thansk. 12:16:53 sdague: heh, baby is urgent thing 12:16:58 gmann_: np 12:17:26 any more question on summit api session? 12:17:37 * edleafe doesn't miss having babies run my schedule 12:17:43 sounds like a good set of things, thanks for the prep on that 12:18:15 something like remove v3 cleanup looks like needn't list at there 12:18:28 yea, really thanks to sdague 12:18:46 so let's move on 12:18:50 #topic API Documentation 12:19:21 for generate swagger doc, current we have two poc 12:19:40 one is generated from nova code base, we already talk about that in last week 12:19:46 #link  https://review.openstack.org/233446 12:20:07 another one is generate from tempest log, which thanks to oomichi 12:20:12 #link  https://review.openstack.org/235092 12:20:22 oomichi: ^ do you want to talk about that? 12:20:38 alex_xu: not so many. 12:20:41 so my thoughts on this stuff are quite nova specific 12:20:51 look at these two, I like this approach 12:20:58 (1) generate docs from the code 12:21:09 * sdague is back 12:21:09 alex_xu: maybe it is nice to make the combination of both for covering all cases 12:21:13 (2) have tests that run to check the docs 12:21:32 (3) have a way to check tempest coverage vs the docs 12:21:38 or something like that 12:21:46 johnthetubaguy: nice idea 12:21:52 basically, +1 to what oomichi said about the combo 12:22:00 oomichi: yea 12:22:04 yea that will be cool thing 12:22:05 johnthetubaguy: sounds cool 12:22:12 johnthetubaguy: and it is nice to improve test coverage also for tempset by using both:-) 12:22:14 I like that you commit the code in nova, and we get some docs and testing automatically in that commit 12:22:17 i like 3 part more 12:22:25 alex_xu: instead of using a decorator, can we use a docstring? 12:22:32 sdague: +1 12:22:39 sdague: yea, we can 12:22:51 even if that docstring needs a marker in it like: ::api_doc:: 12:22:53 sdague: or your mean all the docs, like for input, output, error code? 12:23:04 and the chunk after it is what we consider for swagger 12:23:18 I feel like instead of @wsgi.doc 12:23:27 in looking at https://review.openstack.org/#/c/233446/6/nova/api/openstack/compute/keypairs.py,cm 12:23:42 sdague: I wonderd about removing the param, use first line as the short title, and the rest as the description 12:23:56 johnthetubaguy: yeh, maybe 12:23:56 sdague: yea, that's one also want to docstr to instead 12:24:15 johnthetubaguy: +1 12:24:16 sdague: decorators are easier to understand than some nested docstring. What is the concern with them? 12:24:22 or how about adding description in request schema 12:24:25 +1 for docstring 12:24:38 edleafe: its prose in a decorator, its super hard to read 12:24:39 on my PoC also, docstring is used 12:24:46 if we have schema for all APIs 12:24:53 for explaining overview of API 12:25:03 edleafe: yeh, being in a decorator means that it's going to push people towards brevity 12:25:19 johnthetubaguy: ah, thanks. Didn't know you were aiming for verbosity 12:25:31 being in a docstring is going to allow for people to expand and even add things like examples 12:25:46 we have have each param description in schema along with general one 12:25:59 actually we need doc for request schema, response schema, url params, query params, error status code, success code, currently implement looks mess, all of those thing spread in many place 12:26:12 yeah, I like the params being described in the schmea 12:26:32 ye, so we have type and their description all in one place 12:26:36 so I think it needs to be close to where things are modified 12:26:48 yes, it is nice to write the explanation of each param at close place 12:26:49 when you change the param, you change the schema, so that feels OK 12:27:17 johnthetubaguy: +1, nice to maintain the code 12:27:18 johnthetubaguy: yea, that is good point, also think about that in the beginning 12:27:22 but honestly, I think we just need to pick one and try it, and evolve it as we go 12:27:32 yea 12:27:46 cool 12:27:57 so, personally, I think we should focus on the docs close to the code. The schema should just be enforcement and not description. 12:27:59 just, yeah, my preference is close to where you normally make modifications, for the best hope of folks noticing the need for changing the doc 12:28:44 anyway - https://review.openstack.org/#/c/233446/6/nova/api/openstack/compute/keypairs.py,cm if it used docstrings instead of decorators would be super useful I think 12:28:55 sdague, but param type and thir checking on schema side than code 12:29:00 sdague: I am thinking the schema is close enough, as we probably change the schema if the docs need to change, but yeah, if it could get in the code, then maybe thats better 12:29:12 sdague: what "not description" mean? 12:29:50 johnthetubaguy: yeah, schema is nice place to write the description for each param 12:29:50 oomichi: describing each parameter inside the schema seems like the wrong places to do that 12:30:18 sdague: discussion point 12:30:19 because the schema is really the test 12:30:23 in some ways 12:30:31 you want to describe your intent elsewhere 12:30:37 and then the schema enforces it 12:30:55 well the schema drives the param checking code, the response schema is much less clear 12:31:00 if you only describe your intent in the enforcement layer, you tend to slip up and change enforcement an intent 12:31:17 hmm, true 12:31:18 sdague, schema has all their type range etc defined 12:31:30 sdague: uhm, difficult 12:31:32 gmann_: right, sure, and that's fine 12:31:49 anyways, I think this is one we need to see in code first 12:32:01 sdague: most info of each param is written in schema 12:32:05 gmann_: yeah 12:32:07 need it to look non-horrible, ideally 12:32:16 yeh, anyway, lets look at how it all comes together 12:32:20 so more poc 12:32:25 and see what each stage looks like 12:32:25 yeah 12:32:30 ok, cool 12:32:31 yea 12:32:37 nice step 12:32:48 lets layer on the param bits, see how it looks 12:32:49 because it's important that the generated docs feel coherent 12:32:56 sdague: +1 12:32:58 and it's important the improving the docs is straight forward 12:33:10 because if it's editing in a million places, no one will improve it 12:33:25 sdague: big +1 12:33:37 we want to grow a class of contributors that can improve our docs without understanding our code 12:33:48 nice api doc will make us avoid wrong API usages 12:33:53 sdague: +1 12:33:54 agreed, certainly for clarity, etc 12:33:57 sdague, yup 12:34:16 ok, cool, so let's move on? 12:34:19 and can implement tests with those doc without code reading which i always do :( 12:34:19 yep 12:34:28 gmann_: +1 12:34:35 heh 12:34:37 #topic Remove project_id from URL 12:34:47 this is another important work 12:34:53 sdague: do you want to talk about it? 12:35:18 I added this into the agenda, thinking of this is something should tracked by api team 12:35:41 alex_xu: sure 12:36:04 so in support of the service catalog tng work, I spent last week trying to see what's required to remove project_id from nova 12:36:09 it's not too bad 12:36:27 #link https://etherpad.openstack.org/p/removing-project-id-in-nova-urls 12:36:28 * johnthetubaguy wonders if we should rename to making project_id optional in the URL 12:36:32 that was my working document 12:36:46 johnthetubaguy: yeh, that's fair, it's making it optional 12:36:48 sdague: cool 12:36:49 sdague, and we need to support url with project_id too by ignoring them ? or m missing something 12:37:14 we aren't removing url with new microversion? 12:37:15 the only place we care about project_id, mostly, is during one enforcement check 12:37:55 alex_xu: so I think we need to backport this change to previous versions, but thats maybe a different discussion 12:38:03 https://github.com/openstack/nova/blob/81fbb34e2544b26ae72f7eee3841b4024d22896f/nova/api/openstack/wsgi.py#L813-L821 12:38:11 johnthetubaguy: ok 12:38:28 yeh, we have to figure out our version signaling on this 12:38:40 easy to implement it with ProjectMapper instead of PlainMapper 12:38:49 because response contain those in namespace or links etc 12:38:52 my feeling right now is that we should add a microversion to indicate that project_id is optional from here on out 12:39:11 sdague: +1 my gut says the same thing 12:39:22 that's one of the reasons the patch is Do Not Merge 12:39:25 sdague, +1 for version to publish 12:39:53 sdague: if specified microversion doesn't support it but client doesn't pass project-id, nova will return 400? 12:40:01 the other thing, is we probably don't want to remove this in a microversion, because of all the folks using the old service catalog entries, as folks transition 12:40:07 oomichi: no 12:40:20 oomichi: its just optional, I think 12:40:26 sdague: the microversion is just for publishing 12:40:29 oomichi: or at least that wasn't my intent 12:40:36 because this involves changing the mapper 12:40:45 so it's kind of hard to do it runtime 12:40:48 sdague: yeah, right 12:41:01 sdague: from implemenation side 12:41:06 yeh 12:41:29 just for publishing is good idea 12:41:36 it's a compromise, but unless we want to massively redo Routes, I don't think we can get perfect microversion behavior 12:41:50 so its something we though about doing in v3 right, but dropped it, making it optional/ignored seems like a better solution 12:41:51 so just using the microversion as the signaling element seemed fine 12:41:56 yeah 12:42:05 johnthetubaguy: yeh, it was not going to be there in v3 12:42:23 sdague: yeah, my small concern is that that doesn't seems perfect for microversion 12:42:25 so thinking about signalling, I guess the big signal is actually in the service catalog? 12:42:46 johnthetubaguy: yes 12:43:00 except, so many applications hard code past the service catalog 12:43:08 so we can't just depend on that 12:43:24 hummm 12:43:27 yeah, +1 12:43:42 many signals good 12:44:04 the first session of the design summit is a double session on the service catalog tng work 12:44:21 I'm mostly going to leave this patch in the DNM state until after summit 12:44:30 yeah, +1 12:44:30 and just make sure we have broad agreement on the big plan 12:44:44 agree 12:44:48 project_ids being optional is an early bit that projects will need to do 12:44:51 yeah, I'd like to hear all the concerns 12:45:11 so assuming no giant roadblocks, we should be able to clean this up and merge by M-1 12:45:35 https://etherpad.openstack.org/p/mitaka-service-catalog-session is the etherpad for service catalog session 12:46:07 cool 12:46:27 so for context, we use Repose in front of our API, I think there will need to be changes to drop the project_id in the URL 12:46:37 johnthetubaguy: right, probably 12:46:59 yea, in links it is added 12:47:09 sdague: do we have a nova-spec for that now? 12:47:18 but honestly, I consider that all a bit evil, but its a heads up 12:47:29 repose that is 12:47:30 oomichi: no nova-spec, we're going to use the openstack spec + nova blueprint 12:47:44 sdague: ah, I see. 12:47:55 sdague: nice combination 12:47:56 yeah, cross project consistency FTW 12:48:20 cool, if no more question, looks it's time to move on 12:48:26 sounds good 12:48:34 #topic API futures - specs 12:48:47 anyone people want to bring up? 12:49:11 did you all see the spec etherpad? 12:49:21 #link https://etherpad.openstack.org/p/mitaka-nova-spec-review-tracking 12:49:25 there is an API section in there 12:49:26 johnthetubaguy: yea 12:49:43 ok let me bring up one 12:49:43 feel free to use that in ways you all find useful 12:49:53 johnthetubaguy, cool 12:49:54 johnthetubaguy: ok 12:49:57 #link #link https://review.openstack.org/#/c/227274/1 12:50:04 This spec is going to correct a lot of status code. Like 200->204, 200->201 and 200->202 12:50:06 johnthetubaguy: oh, thanks 12:50:11 The question do we welcome this kind of change? I feel only 200->202 useful. 12:50:18 other fix is more about consistent 12:50:32 if we think the fix is ok, And do we think it is ok fix them all in one microversion? 12:50:40 alex_xu: that is based on api-wg guideline? 12:50:48 oomichi: yea 12:50:57 alex_xu: cool :) 12:51:14 alex_xu, if changing all then this seems huge change so increase in x? 12:51:20 for x.y 12:51:32 I feel like we should bring all the API wg changes in in one microversion 12:51:43 gmann_: yea, one microversion change a lot of apis 12:51:44 and it should include the new header as well 12:51:52 sdague, +1 that will be great 12:51:56 sdague: its tempting 12:52:19 so the thing that worries me, is making it harder for existing users to use new features 12:52:22 but, before we do that, we should make sure we've got reasonable way of showing documentation for microversions 12:52:26 johnthetubaguy, it avoids having lot of microversion for same kinda fix 12:52:31 any its very marginal benifit 12:52:39 johnthetubaguy: right, I don't think it's a huge rush 12:52:44 sdague: yeah, actually thats a very good way of look at it 12:52:52 lets make sure we have a docs before / after story that looks good first 12:53:06 so I think its something we should push until next cycle 12:53:18 johnthetubaguy: yeh, I'm fine with that 12:53:19 what do you all think? 12:53:31 or late this cycle if the doc display is all sorted 12:53:43 sounds ok for me 12:53:44 but doing it before then is going to mostly create confusion 12:53:57 sounds good after doc thing 12:54:29 sdague, johnthetubaguy anytime we gong to change x in microversion? 12:54:45 gmann_: I don't think so 12:55:03 * alex_xu mis-reading gmann_ works in previous 12:55:05 I never believed X was meant to be changed 12:55:14 because we have not shorted out the y arting point of any x 12:55:17 yea 12:55:19 yeah, I don't think we defined what that would mean 12:55:24 it will be 2.y forever 12:55:24 arting -> starting 12:55:32 sdague: +1 12:55:33 yea, my feeling too 12:55:38 sdague: yeah, that is 12:55:56 people kept confusing this for semver :) it's not, it's negotiation. 12:56:21 sdague: that is common misunderstand on current microversion anyway 12:56:22 another one 12:56:24 :) 12:56:30 right, its a different concept 12:56:31 #link https://review.openstack.org/#/c/209917/ 12:56:54 it's add project_id and user_id to server-group response 12:57:04 alex_xu, ahh 12:57:12 the only think I don't like is only show project_id and user_id when admin context is used 12:57:23 I think we should just show those two attrs in all the times 12:57:32 and everything is simple 12:57:38 +1 12:57:39 what would you think guys? 12:57:54 will it be more useful in create response / 12:57:55 it will make no harm I guess 12:58:02 do we show user_id for server, I can't remember now? 12:58:02 alex_xu: +1 12:58:25 in create we do not show 12:58:30 alex_xu: +1 12:58:37 johnthetubaguy, I remember yes, at least nothing just for admin user 12:58:37 alex_xu: nice to make consistent API and there is not demerit for doing that I guess 12:58:41 alex_xu: yeh, showing it all the time seems the right call 12:58:57 ok, cool, thanks guys for having agreement 12:59:09 so I have enough more today 12:59:14 cool 12:59:22 oops, I have nothing more today 12:59:26 :) 12:59:29 I had a question 12:59:29 alex_xu: there was some talk of getting more people involved in API doc work. I'd like to get involved. 12:59:29 cool, almost out of time 12:59:31 sorry I'm laste 12:59:32 late 12:59:38 annegentle: :) 12:59:39 yeah ccarmack! 12:59:42 #topic open 12:59:44 ccarmack: awesome 12:59:51 annegentle: what's up? 12:59:55 ccarmack: cool 13:00:08 annegentle, sdague: I could do PoC or spec implemenation 13:00:13 alex_xu: any way to use the verbage from the existing api doc? I put that in my review and hadn't gone back to see your response yet 13:00:19 sorry, annegentle ccarmack, it's time to close the meeting :( 13:00:34 can we discuss on -nova? 13:00:34 lets jump into #openstack-nova quickly 13:00:35 ok, let's go to #openstack-nova for follow ups 13:00:36 sure 13:00:37 yes 13:00:44 #endmeeting