16:00:40 #startmeeting api wg 16:00:41 Meeting started Thu Feb 11 16:00:40 2016 UTC and is due to finish in 60 minutes. The chair is elmiko. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:42 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:00:45 The meeting name has been set to 'api_wg' 16:00:49 #chair cdent etoews 16:00:53 Current chairs: cdent elmiko etoews 16:00:58 hi 16:00:59 hello o/ 16:01:03 hello 16:01:09 howdy 16:01:14 * elmiko hands mic to etoews 16:01:51 * etoews drops mic 16:01:59 * cdent was waiting for that 16:02:05 haha 16:02:15 #link https://wiki.openstack.org/wiki/Meetings/API-WG#Agenda 16:02:25 * etoews fumbles about for mic 16:02:29 #topic previous meeting action items 16:02:30 so: we had disco balls and glitter in the service catlog meeting, this meeting has a high bar to meet 16:02:34 ha 16:02:40 lol, nice 16:02:59 #link http://eavesdrop.openstack.org/meetings/api_wg/2016/api_wg.2016-01-28-16.00.html 16:03:30 well the summit submission is done so +1 16:03:31 so, i got all of mine. the server-side traceback guideline merged today 16:03:43 \o/ 16:03:49 huzzah! 16:03:59 motion 16:04:19 nice! 16:04:20 it actually got a really solid response since the rework, nicely done jaypipes 16:04:23 ah i haven't reached out directly to the CPLs about the errors guideline but have been working with the magnum team. 16:04:32 cool 16:04:38 more on that later (if we get to it) 16:04:45 k 16:04:55 let's dig in to the meat and potatoes 16:04:57 #topic service type vs. project name for use in headers 16:05:03 #link http://lists.openstack.org/pipermail/openstack-dev/2016-January/085145.html 16:05:12 #link http://eavesdrop.openstack.org/meetings/tc/2016/tc.2016-02-02-20.01.log.html#l-263 16:05:21 etoews 16:05:41 i managed to read through the email thread yesterday 16:05:47 \o/ 16:06:20 that's an accomplishment 16:06:21 the first three agenda items are all really a piece of the same thing 16:06:32 cdent: yea, pretty much 16:06:32 ya 16:06:51 i'm in agreement with cdent on that thread fwiw 16:06:56 So I would be curious to get etoews' reaction to the whole pile 16:07:05 :) 16:07:12 \o/ 16:07:14 heh there you have it 16:07:25 * etoews warms his hands on the fire in cdent's belly 16:07:32 so, basically, we should have a registry based on service types and that we should curate it? 16:07:36 lol 16:08:01 * cdent has been experiencing a bit of discomfort lately 16:08:05 hold on. let me check something. 16:08:42 or am i jumping ahead? 16:08:45 i was thinking primarily of this 16:08:57 "I think that's pretty weak sauce and we need to take both a more assertive and more aggressive stance with regard to achieving quality and consistency in the APIs[1]. Reaching consistency is the primary mission of the group but consistent crap is still crap and our API consumers deserve better." 16:09:23 ok, cool 16:09:37 and yea, i agree with cdent on this one too. i'm just timid about how we achieve it... 16:09:41 that statement puts us oh so mildly in conflict with sdague's belly's fire 16:10:11 What I keep sensing is "if we had a good API docs site we would have discoverability of conflicts" 16:10:13 If I understand him correctly he doesn't want us striving for purity for purity's sake at the cost of "breaking" existing things. 16:10:19 but can't boil the ocean of course 16:10:20 cdent: right 16:10:30 we did that once 16:10:35 it was called Nova v3.0 16:10:54 after two years we had to throw the whole thing out because no one was ever going to drop v2 if we did that 16:10:57 the real question then becomes, where to draw the line between purity and absurdity 16:11:12 yeah, I think we can find a middle ground 16:11:30 What's important to me is that we don't _always_ use precendent as truth 16:11:32 imo, we have been trying to do that by doing api evaluations before we create guidelines 16:11:34 there's a lot of stuff that is just wrong 16:11:40 elmiko: yes 16:11:41 agreed 16:12:07 right, but my particular point here was the 4 characters coming out of the header we being deleted for no real reason than purity 16:12:15 it's the standard deprecation dance. right it comes down to a question of how long anyone is willing to support previous versions. 16:12:35 etoews: it's not standard deprecation dance when it comes to APIs 16:13:00 in what regrad? 16:13:05 you can use the original ec2 api as it was in 2006 16:13:19 heh sdague well. 16:13:54 sdague: I'm not disputing the truth of that, but I guess why we've decided replicating that behavior is a goal? 16:13:55 right. aws has the willingness (and developer workforce) to more or less indefinitely support apis. 16:13:57 if we are looking for adoption on the openstack front, compatibility is key 16:14:13 i'm in full agreement. 16:14:34 totally agree, what would change that would be concerning sdague? 16:14:47 annegentle_: the mailing list post I had 16:14:51 a v3 is conceivable as long as we're willing to support v2 (ideally indefinitely) 16:15:07 about changing a header name for no good reason 16:15:11 sdague: ah ok 16:15:20 i'm a little stumped on the whole "remove API_" issue, it makes sense to me to remove it from an outside perspective, but i acknowledge sdague's point as well. it's not clear for me how to draw the line. 16:15:31 http://lists.openstack.org/pipermail/openstack-dev/2016-February/085670.html 16:15:41 we're in a different world though where contributors want to have experimental APIs. what can we do about that? 16:16:04 can we save experimental apis for a bit later? 16:16:12 etoews: sure 16:16:14 +1 16:16:23 yeh, lets sort this one on the microversion spec 16:17:00 sdague: so the other side that I see tho is "just cuz nova did it first we all have to agree?" 16:17:01 I feel like etoews has more to say on the general topic. Do you? 16:17:04 basically I didn't want gratuitious header change. I agree service type instead of code name is important, but dropping API- is gratuitious 16:17:09 sdague: I'm not arguing, I'm pointing out the other view 16:17:10 annegentle_: yea, that's kinda my question too 16:17:34 let me start by saying i'm willing to back off on having to remove API- 16:17:45 sdague: but I can't tell from reading if it's the four characters or the nova start point? 16:17:51 etoews: agreed, but i think it makes a good pinata for now 16:18:05 annegentle_: it's because we have multiple things out in the field that do it. 16:18:14 but it really point to what annegentle_ brought up above "just cuz nova did it first we all have to agree?" 16:18:23 telling released services you have to break your users needs a real benefit 16:18:35 not just cuz someone thinks it's nicer 16:18:59 i don't feel like we're advocating for a reverse in direction for those who implement it with the API- 16:19:08 we're trying to forge a new way forward 16:19:09 sdague: so, I'm working with SDK devs who are implementing microversions now 16:19:19 elmiko: so instead you'll have to keep a decoder ring of headers 16:19:39 sdague: it's going fine but we are having to explain in one:one conversations. This spec helps immensely of course. 16:19:48 sdague: i feel like we'll have to do that either way, imo it's part of the evolutionary process 16:19:55 because you can't OpenStack-%s-API-Minimum-Version % service 16:19:56 #idea: For future reference resist standardizing on header types with meaingful and changeable identifiers in the header left hand side. Bad for flexibility and bad HTTP. 16:20:37 elmiko: see, I feel like this was going to be OpenStack-%s-API-Minimum-Version % service in the code, which is fine. 16:20:43 interesting though cdent, how do we write guidelines then? 16:20:45 sdague: nope. step 1. add new header to nova. step 2. remove all documentation on old header. step 3. likely nothing (support both headers server side forever) 16:21:02 etoews: you *CAN'T* removal all documentat 16:21:20 there are clouds out there, publicly deployed, where this is the interface 16:21:21 elmiko: generic left hand side, multi-variant right hand side 16:21:27 and the other header isn't accepted 16:21:56 this is a key part of the bootstrapping 16:22:08 cdent: maybe, probably, i'm missing something but how would we avoid the API- no API- question? 16:22:27 (elmiko let's punt on that for a minute) 16:22:32 k 16:22:51 sdague: I think part of the thinking here is that the number of user and clouds that don't exist yet are much bigger than those that do 16:23:06 cdent: that's always the theory 16:23:14 hope, perhaps? 16:23:18 lol 16:23:26 definitely hope ;) 16:23:29 until you piss off all your current users and they go elsewhere and take your future users with them 16:23:29 (btw, I'm not really arguing against sdague, just trying to flesh out the concerns) 16:23:48 sdague: thing is, we don't have a docs site that can document microversions yet. 16:23:53 same, i'm not opposed to what sdague is talking about. but i want to better understand the problem space. 16:23:59 github keeps changing shit in their api and I haven't left yet? 16:24:02 sdague: so to me, and I sound like I'm single minded, the docs space is the problem space. 16:24:21 sdague: we can't support without docs 16:24:24 annegentle_: we also have a docs issue, and I'll agree with that 16:24:26 it's a significant aspect of it for sure 16:24:29 annegentle_: we're already supporting it 16:24:34 annegentle_: agreed to a large extent, having good, *fresh*, api docs help alleviate some of these issues. 16:24:35 people are already writing software using this 16:24:44 we're already using it between services today 16:24:44 sdague: yes, I talk to them a lot :) 16:24:52 sdague: about how hard it is to find out what to do :) 16:24:56 :) 16:25:02 annegentle_: sure 16:25:36 but a huge part of fixing all of that has been held up on getting this base spec sorted out, which is hung up on breaking users that have figured out how to use our stuff 16:26:36 anyway, the point is, in this case, this is a really really critical part of the bootstrapping process. There will be production clouds which don't work with the new header out there for the next 5+ years 16:26:41 I think we have general consensus that we can be flexible abot the -API thing, but the more general issue is still live. 16:26:54 cdent: +1 16:26:57 even though documentation hasn't fully caught up, when it does, it's important it works for all clouds 16:27:06 cdent: the more general issue being? 16:27:07 cdent: +1 16:27:08 i'm still struggling to figure out how we guide this process in future 16:27:33 are we allowed to advise changing the namespace for a header, etc... 16:27:53 do we need a header registry guideline of some sort? 16:28:16 elmiko: no, we need to not promulgate headers easily 16:28:23 or even a header name guideline, it just feels kind murky to me 16:28:24 there are much better and correct ways to do http 16:28:38 cdent: ok, interesting. i'd like to hear more about that 16:29:00 (also, meeting halfway point approaching) 16:29:21 is the general issue nova instead of compute? or header names changing generally? 16:29:25 i've been working with swift metadata lately and the headers are an dumping ground of inconsistency. 16:29:27 it's too late to go back in time, but the correct way to have done microversions could have been content-type parameters or a single headler incidcating microversion with the service type on the rhs 16:29:38 etoews: all bets are off with swift 16:29:43 we shouldn't even try there 16:29:43 true 16:29:45 :( 16:29:56 annegentle_: my issue was header names in general 16:30:08 also swift has over 70 headers and nova has 2 16:30:19 cdent: so, are you saying we should back off header advise in general, when possible? 16:30:31 for existing headers we should advise 16:30:46 ? 16:30:57 and when new ones are proposed we should consider ways to avoid the creation of more headers, or ways to consolidate multiple header propositions under one header 16:31:18 we do already advise against the X- naming 16:31:20 cdent: maybe a general header guideline to address some of these thoughts would be helpful? 16:31:26 so we are in there doing this as guideance 16:31:28 annegentle_: right, i was kinda thinking about that 16:31:37 heh I said swift which is "accio notmyname!" 16:31:46 heh, yeah 16:31:53 just checking IRC before I get on the bus :-) 16:31:54 cdent: I'm totally open if you'd like to propose content type negotiation as a follow on instead of changing to OpenStack-[Service-Type]-API-Min-Version 16:32:05 i like the idea of capturing the thoughts cdent is talking about, namely alternative to headers and why you would/should use them 16:32:08 but I don't think we should go from name -> service type -> something else 16:32:19 I can write up the resist-headers idea 16:32:23 we get kind of one correction, and carrying around the 2 things forever 16:32:39 correction, then another correction, and that's just churn 16:32:46 sdague: I don't think we should change the mechanics of microversions now, I'm just saying for other headers people might like to come up with there are other ways 16:32:53 cdent: ok 16:32:53 that all sounds reasonable to me. 16:32:58 sdague: agreed, and i'm happy to decide on one direction for the microvers. headers and not changing that (header naming-wise) 16:32:59 so just name -> service-type 16:33:04 sdague: it's only 2 not 72 16:33:20 and a pattern 16:33:21 we actually have a start on header guidelines here http://specs.openstack.org/openstack/api-wg/guidelines/headers.html 16:33:23 annegentle_: sure, but I'm not using 70 as my success bar :) 16:33:29 hell to the no :) 16:33:37 cdent: ok, I'm fine with that then 16:33:39 #action: cdent writes up his ideas on resist headers 16:33:50 etoews: yea, i was thinking about capturing the more indepth ideas that cdent is talking about 16:34:26 should we talk about the registry then? 16:34:44 or get more into naming? 16:34:59 one sec. 16:35:09 i feel like we are agreed about the service_type being the choice for naming 16:35:15 so do we comment on https://review.openstack.org/#/c/243429/4/guidelines/microversion_specification.rst and ask to put API- back in? 16:35:34 sadly, i feel like we have to for consistency sake 16:35:43 I think that's the compromise we reached 16:35:49 I ... Uh. 16:35:55 an API is a collection of operations? 16:35:57 i hate to make alex_xu_ go through that again... 16:36:02 or is an API a single interface to the service? 16:36:18 our precision with this term "API" is odd. 16:36:20 I think it will be fine, I can respin for him even 16:36:29 though, the experimental thing is still the problematic one 16:36:36 and honetly to me it's about nova/compute 16:36:41 honestly even 16:36:42 annegentle_: i think the feeling on the review was that adding API- to the header was kind of redundant 16:36:52 elmiko: ok wasn't just me then 16:37:07 sdague: yeah... that design space without breaking users is really where the conflict lies 16:37:15 because I *really* don't think we should be endorsing putting experimental things into the APIs, once things are out in the world, they are used 16:37:20 the main problem with that is that it thrashes the few microversion impls that are actually out there 16:37:51 sdague: i have an issue about the experimental stuff 16:37:51 sdague: Me. too. 16:37:58 can we nail down exactly what the action is on moving forward on https://review.openstack.org/#/c/243429/4/guidelines/microversion_specification.rst and who will take that action? 16:37:59 annegentle_: right, this is just a case where we did it already one way. And the only harm was 4 bytes. 16:38:23 four nova bytes or api- four bytes? 16:38:23 etoews: it sounds like sdague volunteered to adjust the spec by adding API- back in 16:38:31 I concur with elmiko 16:38:33 annegentle_: api- , as i understand 16:38:33 elmiko: yes +1 16:38:36 ok 16:38:54 #action sdage to update https://review.openstack.org/#/c/243429 by adding API- back in 16:39:00 #action sdague to 16:39:06 sorry 16:39:06 oops. 16:39:20 that Dage guy will be busy 16:39:23 no no. i meant to delete that and hit enter instead 16:39:24 do we need an undo on the last one? 16:39:34 thanks and let's carry on to a new topic 16:39:55 we seemed to be naturally veering into experimental territory or shall we go to registry? 16:39:55 ok, experimental apis isn't on the agenda, but i feel like we should discuss 16:40:01 yea.. 16:40:09 +1 16:40:11 #undo 16:40:12 #topic experimental api inclusion 16:40:12 Removing item from minutes: 16:40:23 ok, experimental stuffs 16:40:36 #link: https://review.openstack.org/#/c/273158/ 16:40:47 i like this, because i'm working a new api version for sahara and it helps. but i could live without it 16:41:08 i can also see how having pieces of experimental stuff in a fully production api might be discordant 16:41:08 It is absoluately critical to the health of openstack that there is a way to make experimental apis available to real users. The mechanism of how that is done is what's up for discussion, right? 16:41:26 cdent: i feel so, but i think sdague has other ideas 16:41:34 * cdent looks at sdague 16:41:40 * elmiko looks at sdague 16:41:43 ;) 16:41:48 i think one of the reasons we have suboptimal api designs is because openstack projects seemingly have to come to the table with a fully baked api. 16:42:07 etoews: yeah, my belly wrote about that too. It's a huge problem. 16:42:15 it feels to me that experimental API should hang off a different endpoint 16:42:20 and i agree with cdent's position 16:42:23 I'm okay with different endpoint 16:42:24 without getting an api infront of users in some beta form that's just not doable. 16:42:37 i'm fine with advising different endpoint as well 16:42:51 new endpoint ++ 16:42:52 cdent: yes, before we allow for bad API evolution with the guise of microversions.... Some projects would want APIs for an entire feature to be experimental until they get feedback and the feature stabilizes.. 16:42:55 like 'compute' is nova API. There could be a 'compute-experimental' that only advertizes experimental resources 16:42:56 but, i really like the header option too because it forces acknowledgement 16:43:21 agreed 16:43:24 elmiko: if people are using service catalog to find endpoints, isn't it the same thing? 16:43:31 and we want them to use service catalog... 16:43:43 it is a different amount of work 16:43:47 there's more acknowledgement from a new entry in the service catalog in my mind 16:43:48 cdent: would the experimental api have a separate entry in the catalog? 16:43:55 * cdent considers making endpoints have opaque urls :) 16:44:01 and different than bob in IT setting experimental true in their shade fork 16:44:32 i'm trying to think from a development standpoint here. 16:44:36 I also think that experimental in the main API is going to be just like javascript github projects 16:44:43 meh, why release, just pull from master 16:44:47 if i am working on a experimental feature, would i need to adjust the service catalog to acces it? 16:44:48 when we say "different endpoint" doesn't that imply a different entry in the catalog? 16:44:51 how about identifing the experimental nature in the service catalog? 16:44:53 cdent: yes 16:44:56 sdague: lol! 16:45:27 type: compute 16:45:27 beta: true 16:45:30 cdent: i guess it didn't for me 16:45:53 i was thinking different endpoint like using /v2/... instead of /v1.1/.... 16:46:05 etoews: that came up at summit and was considered...dangerous? 16:46:09 elmiko: no, this needs to be != existing service types 16:46:16 where "that" == "beta: true" 16:46:26 can you give the sahara example? 16:46:28 yeh, that means different things to different people 16:46:35 right, an example might be helpful 16:46:39 sdague: service catalog would be like, type="data-processing-experimental" ? 16:46:44 elmiko: yeh 16:46:52 interesting, and i like it 16:47:03 and, importantly, that endpoint *should not* include resources in the main API 16:47:09 suffix is better than a separate field in the structure? 16:47:10 i definitely agree that driving folks to the service catalog is better than gating through headers 16:47:24 etoews: yeah, because of the lookup process 16:47:32 etoews: it should not be discoverable as a data-processing endpoint 16:47:33 it is not 16:47:55 ok, this makes sense to me 16:48:00 "hey I want to do some data processing, whatcha got?" 16:48:06 this aligns with my growing sense of service catalog uber alles 16:48:07 realistically this should be about many many many signals that this bit isn't ready 16:48:12 ah. it's the namespace collision that's the concern. got it. 16:48:44 sdague: service catalog entry and header... ;) 16:49:03 elmiko: and it doesn't contain the main API resources 16:49:07 you have to unlock all the deadbolts 16:49:10 so you can't just point your app at the other endpoint 16:49:13 sdague: yup, agreed 16:49:19 you have to explicitly be calling both 16:49:20 ++ for data-processing-experimental 16:49:26 here's a real world example 16:49:27 gouthamr: thought? 16:49:30 * elmiko waves at SergeyLukjanov 16:49:33 thoughts? 16:49:42 we have old sahara at http://host:port/v1.1/... 16:49:52 we have new sahara at http://host:port/v2/... 16:50:00 etoews: I'm still racking my brains about how resource isolation can be achieved.. what if a feature has DB implications.. 16:50:10 service catalog shows the first for data-processing, the second for data-processing-experimental 16:50:13 does that sound right? 16:50:22 are we sorta letting teams be bad at API design a little longer? Or is it not really like that? 16:50:23 elmiko: you could do it that way 16:50:43 * annegentle_ recalls the termie talk in Vancouver was it? 16:50:51 but, honestly, I also think that major API revisions are things projects should just not do any more 16:51:05 sdague: why not? 16:51:14 we're 3 years into migrating from cinder v1 -> v2, glance v1 -> v2, keystone v2 -> v3 16:51:20 annegentle_: in some sense i guess. give them space/time to figure out better designs without having to worry about backwards compat. 16:51:20 sdague: yeah I feel like we don't have that luxury because our users lose 16:51:43 sdague: We can be 10 years in and have good user outcomes though. 16:51:46 sdague: fair point, but our api could really use some major overhauls... 16:51:56 sdague: it's not a time concern as it is a "does it work well" concern 16:52:04 elmiko: right, but if you can get to it in steps, that's what microversions let you do 16:52:08 and following semver, i can't see how to get use there from here 16:52:18 trust me, we went down this road in Nova 16:52:22 so... I think we are be far to accepting of what the past has shown and letting us drive decision too much. It should certainly inform, but should not control 16:52:22 spent 2 years on it 16:52:26 then lit it all on fire 16:52:37 hmm 16:52:39 Nova is a nightmare of horrible process that should not control the rest of the projects 16:52:50 * elmiko thinks "is sdague predicting my next 2 years" 16:53:00 cdent: sure, many things aren't great 16:53:07 Desperately unhealthy is nova. Other projects may be better. 16:53:08 however, glance v2 16:53:18 yeah, all these "old" projects 16:53:21 cinder v2 16:53:25 keystone v3 16:53:36 i feel like we can get away with this in sahara though 16:53:43 elmiko: sure, maybe 16:53:54 sometimes evolution needs to be revolutionary. 16:53:58 show us the way elmiko :) 16:54:00 no project every has though in openstack 16:54:02 Different contexts have different concerns 16:54:04 i totally understand how migrating nova or keystone might be really tough 16:54:18 etoews: i'm trying... ;) 16:54:25 ok, 5 min left 16:54:28 i think magnum is looking at a v2 too 16:54:31 and we are way off in left field 16:54:42 do we want to talk registry quickly? 16:54:44 so yeah: we should talk naming registry stuff a little bit 16:54:49 jinx buy me a cooke 16:54:51 i think it was a necessary detour. 16:54:54 #topic service type name registry 16:54:54 ya 16:54:57 etoews: agreed 16:55:17 ok, so yes, registry, woo, party-time, excellent! 16:55:21 seque here is that an experimental api should never show up in the service registry... 16:55:33 haha! 16:55:37 sigh... 16:55:59 * elmiko labels cdent "habitual line-stepper" 16:56:06 I still want to ask, if we had our docs ducks in a row, would we need this separate registry? 16:56:17 sorry, I like continuity 16:56:22 annegentle_: yes 16:56:24 i think yes, if only to help new projects 16:56:27 or could the docs site discover / preach truth 16:56:30 no, yes entirely 16:56:35 heh 16:56:43 so the service registry is a pre-doc doc 16:56:48 lookup list 16:56:53 congress grabbed 'policy' as service type 16:57:00 which is just crazy 16:57:20 we do need to mitigate "good names" here 16:57:33 otherwise we preclude future services in openstack 16:57:40 a registry here is important 16:57:40 now, if a team grabs a name and takes on a mission, what if they don't deliver on it? 16:57:43 i can't remember how the email chain ended, sdague were you planning on generating a spec for the registry or is this something we could work on through the api-wg? 16:58:01 annegentle_: i liked the idea about checking the registry at regular intervals to drop kruft 16:58:11 elmiko: I ended with I would create this repo - https://review.openstack.org/#/c/278612/ 16:58:22 also, cdent's idea about projects only grabbing the name when they acutally *need* it 16:58:33 sdague: oh, awesome! 16:58:45 in the service catalog tng group we're sketching basic format 16:58:48 and thank you =) 16:58:54 ok, yeah, the timing is good too. Only take a name when you really have a service running 16:58:55 and I think we'll just review bits in 16:59:05 yeah that was the happy dance party 16:59:06 annegentle_: right, no squatting 16:59:38 1min 16:59:52 http://lists.openstack.org/pipermail/openstack-dev/2016-February/086269.html 17:00:00 #link https://review.openstack.org/#/c/278612/ 17:00:02 no squatting, no cookie licking 17:00:11 definitely no cookie licking lol 17:00:18 ok, thanks everybody! 17:00:20 we're still kind of dancing around the problem of an API needing to be something close to done before being able to be "real" (where real means persisting in the registry and the service catalog) 17:00:35 * cdent takes it to #openstack-sdks 17:00:37 i'm more on your side with that one cdent 17:00:42 cdent: yeah it's about getting better at API design, yep yep 17:00:45 and yea, we're out of time 17:00:51 take it top -sdks 17:00:56 #endmeeting