16:01:31 #startmeeting defcore 16:01:33 Meeting started Wed May 11 16:01:31 2016 UTC and is due to finish in 60 minutes. The chair is hogepodge. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:01:34 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:01:36 The meeting name has been set to 'defcore' 16:01:43 #topic roll call 16:01:48 o/ 16:01:49 o/ 16:01:54 o/ 16:02:01 o/ 16:02:09 o/ 16:02:27 * docaedo can be here for the first half at least :) 16:02:50 o/ 16:03:10 This week and the next our esteemed co-chairs, eglute and markvoelker 16:03:16 won't be here 16:03:33 Ok, let's start 16:03:39 #topic cycle naming 16:04:04 Lunar won by one vote in our polling 16:04:26 Thanks for everyone's suggestions and votes. 16:04:39 I finally won something! :) 16:04:49 #link results https://www.surveymonkey.com/results/SM-7JQKSYLR/ 16:04:55 sounds like a good opportunity to volunteer them for many tasks? 16:05:17 docaedo: definitely, they have a lot of +2's piling up :-D 16:05:30 dwalleck: \o/ 16:05:53 Interesting I also vote for Lunar... How come there is only one vote? 16:06:14 catherineD: Hm, I'm wondering if it's a cache issue. 16:06:19 lunar has 8 votes, no? 16:06:21 nvm 16:06:25 at least on my screen 16:06:29 oh won by 1 vote 16:06:36 catherineD: I think it caches the results if you've voted. 16:06:38 not only one vote my mistake 16:06:43 catherineD: ah, ok. 16:07:11 moving on 16:07:29 #topic Interoperability, APIs, Branding, and Beyond. 16:07:34 nikhil: this was your topic 16:07:36 o/ 16:07:40 Sorry for being late. 16:07:45 There are some services that are using internal only APIs (meant for admin) to manipulate image data. 16:08:01 This results in loosely coupling of the image information with the image data. 16:08:02 questions: 16:08:11 1) How can we come up with definition of an Image in openstack? 16:08:40 2) Do I need to invest time to research what all projects comsuming glance are not assuming incorrect constructs and possibly improve documentation? 16:08:56 interop if our #1 priority so I want to ensure we do all the right things 16:08:58 * nikhil done 16:10:35 or do I need to start a ML thread on this? 16:10:56 nikhil: I think the docs should be clear on what is the "correct" way to deal with an image, especially if there are multiple ways to do so 16:10:59 It seems like 1 should be easy. An image is a set of files (in the case where kernel and ramdisk are provided seperately) that can successfully boot on a given hypervisor. Naturally, an image that can boot in one cloud may not be able to boot in another given implementation details. 16:11:18 So, it would be nice for that information to be discoverable. 16:12:11 VanL: noted 16:12:23 hogepodge: hmm, I think there are a few gotchas to it. 16:12:48 for example, a full template like vmware instance template or ovf may not be an image if we think in terms of glance 16:12:57 Since image files have a type, could that be used to determine if an image works for a given cloud? 16:13:07 for but industry standards it should be to support say some sort of certification 16:13:38 dwalleck: we want to support that as pluggable way in the import mechanism so that more info can be given to a user 16:13:56 but there are some cases when the volumes (stored as images) in glance are 16:14:01 a pluggable import mechanism needs to be discoverable 16:14:04 I wonder if we are getting ahead of ourselves here. I'm not sure we want to be so heavyweight at the moment. 16:14:09 Nikhil: ++ to the idea of discoverability 16:14:32 stored as meta information and sometimes the flatenning happesn behind the scenes as there are assumptions on how the glance_store utility driver is supported 16:15:07 more so, plugins needs to pass the defined test suite 16:15:25 hogepodge: the API will be fine 16:15:34 RIght now, I think the first course would be to look at the APIs and their usage (both by projects, by implementations, and how you consider to be "correct" going forward) and try to come up with a clear way of communicating about what is expected by clients. 16:15:43 hogepodge: what you get out of an image may or may not be. I may need to do more research on this. 16:15:59 It's not ok to let plugins be defined to do whatever they want with arbitrarily defined inputs and outputs. That works directly against api interoperability. 16:16:24 hogepodge: the plugins would be simplistic and will not impact the API. 16:16:34 We face an uphill battle with differences in image formats 16:16:38 I think any discussions around images should be driven by identified differences and how they affect the API (VIO/Xen/KVM differences) 16:16:52 nikhil: it will if the plugins can be arbitrary. 16:17:01 VanL: I need to evaluate right now if we need to work hard on supporting a API for services like Nova so that they can use the right way of doing things. 16:17:14 nikhil: using cinder as an example, the user-facing api doesn't change when the driver plugin changes 16:17:21 (or wait another 3 years to get that support added) 16:17:41 hogepodge: no, the API won 16:17:46 won't* 16:18:05 hogepodge: I can go over the import spec on why not later if needed. 16:18:14 Fron interop point of view, image should be something that customers can bring to an OpenStack cloud. Which may be a package like ovf a a file in different format ... more important is what does OpenStack core APIs support 16:18:25 hodgepodge: As long as every backend supports every API facing action, then that's true 16:18:27 nikhil: to be honest, I'd like for us to stop using the nova proxy api, and we're in a place where we can start moving in that direction by requiring the direct apis and deprecating the compute-image apis (of which a large number are flagged because of proxy issues) 16:19:11 hogepodge: ++ 16:19:17 hogepodge: This discussion is separate from the Nova proxy API which are read only (user facing). Write (snapshots) are handled at the virt level. 16:19:56 I am talking about the right way the different virt drivers use to upload data to glance -- which is drastically different for each driver. 16:20:34 nikhil: different way means different APIs? 16:20:36 catherineD: glance cannot support ovf completely today. (Technical limitations) 16:20:43 catherineD: for example: 16:20:49 nikhil: isn't it in the scope of glance and the cross project work to make sure the backends work? I'm confused about what you're asking for 16:21:02 some drivers are using location_add to "directly set" a location on an image 16:21:25 nikhil: understand that ovf is not support .. we need to concentrate on the format that OpenStack supports today 16:21:44 as our concentration is trailing ... 16:22:06 nikhil: who writes the drivers? 16:22:08 hogepodge: at the summit a issue of "0 sized" images came up; this means that the location magic is used to make the copy on write work for volume snapshots. 16:22:33 catherineD: ok, thanks for that clarification. 16:22:49 hogepodge: the virt drivers are in Nova. 16:23:08 So it's a cross project issue? 16:23:34 hogepodge: on 0 sized images, I need more research on what a end user will get back when they do image-download. We have to rely on individual drivers to do the right thing. 16:23:54 nikhil: can you write up a summary of these issues and how they impact the api and interoperability and sent it to the mailing list? It seems like this is going to be a deeper discussion than we might have time for. 16:24:34 hogepodge: if a image is uploaded through nova virt driver in a way it becomes unusable to end user and there are assumptions on it being usable to just internal deployment, I need to do more research on it. Personally, I DO NOT want this to be GLance 16:24:34 To me volume snapshot won;t be a enduser support unless it can be downloaded ... and upload to a different OpenStack clouds. 16:24:41 GLance's responsibility. 16:25:09 hogepodge: ok, thanks. So, I take it we need initial research. 16:25:26 (meaning we do care about interop beyond the APIs) 16:25:28 nikhil: so it the upshot that snapshot shouldn't be required as a capability? 16:26:10 hogepodge: I am just asking for standardizing/documenting best practices and something that won't make glance responsible (as it may appear so). 16:26:51 (snapshot seems like a important capability) 16:27:07 snapshot should be a required feature but the image of the image of the snapshot maynot be 16:27:15 standardizing/documenting best practices for what? 16:27:27 snapshot 16:27:37 nikhil: Regardless of it being important, is it sufficiently widely supported, necessary, etc to be *required* at all times in all OpenStack clouds? 16:27:56 that means a particular env should support snapshot but the image of the snapshot may not be transfered to a diff environment 16:28:17 That is the question that needs to be answered. 16:28:19 that's very much a technical issue. If snapshot isn't available as a working and interoperable feature, that's guidance we need to use in determining the capabilities 16:28:43 catherineD: that's ok, because OpenStack will never have what I would call binary compatibility 16:29:16 hogepodge: but then people wont' be able to literally "move their workloads" 16:29:25 I can take more of this offline. 16:29:26 hogepodge: we need to test that an env can create and boot the created snapshot on same env 16:30:14 gema: workload is something predefine but snapshot is the capturing of something at the moment ... 16:30:38 gema: we lost that when we allowed for choice of hypervisor. Your application code should still be portable, but the images you boot aren't. Now, it's not as bad as you might think. Infra is using dib to create one image then transform it into the expected formats for different clouds 16:30:58 hogepodge, catherineD: ok, sounds reasonable 16:31:15 it would be nice to support transferable snapshot but that would be a more advance feature and shoud not be core 16:31:24 catherineD: +1 16:31:25 yep 16:32:37 nikhil: can I give you an action item to write a summary for the mailing list? 16:32:57 hogepodge: please do 16:33:02 #action nikhil write glance issues summary for defcore mailing list 16:33:23 Anything else on this topic? 16:33:41 none, I do have a bit of overlap in the next topic 16:33:50 but that can be no-op from the defcore team 16:34:15 sure, I kind of merged the two topics together, so we can move on to your next section 16:34:30 ok, thanks 16:34:35 I will ask next set of questions then 16:34:45 3. Do we need to account for all the internal usage of glance in services as interop is meant for end users? 16:35:10 4. As per Mark’s blog we _could_ have more than one API for same operation so for nova we want to keep the old API for POST + PUT that streams data to glance nodes and is not as complicated as newer import API 16:35:14 * nikhil done 16:37:03 So, I can chat a bit more on these things: 16:37:41 i) the RefStack testing will expect the interoperable API to exist in clouds. We can be clear on communicating that. 16:37:42 On 3, defcore is concerned with public facing, non-admin apis. One would hope the public-facing apis are durable to internal implementation details. If that durability isn't there it's a problem for choosing those apis for interoperability 16:37:59 To me , APIs that are end user facing should be accounted for interop ... anything internal should be cloud configuration options 16:39:20 hogepodge: are _all_ APIs expected to be so. (I'm assuming yes) 16:39:21 +1 catherineD -- and *should not be used* in any tests required for interop 16:39:22 ? 16:39:34 That mean we need to carefully choose the must-pass tests to matches the internal and external API used 16:39:45 VanL: yea 16:39:47 To 4, having more than one API is problematic, because if you offer n apis you'll have n choices. My opinion is that, under guidance from the TC, project leaders, and deployers, we pick APIs that we would expect every cloud to have. that creates an environment where app and sdk developers have guidance as to which API they can expect 16:40:22 hogepodge: what if it's a technical limitation (or something too complex)? 16:40:56 nikhil: what is the technical limitation? 16:41:30 VanL: the internal choice of a hypervisor bleeds through. You have to provide a bootable image, which is a config detail. Are you suggesting that DefCore just give up on image api completely? 16:41:46 hogepodge: it may take a few mins to discuss that. if we want to discuss it I can arrange for a more high bandwidth communication channel? 16:41:51 nikhil: would the technical limitation applied due to a certion configuration options? 16:42:24 nikhil: it is also a good candidate for the mailing list. We're at 15 minutes remaining and I wanted to make sure we had time for the next topic since we didn't get to it last week. 16:42:26 catherineD: It will be hard to type it all here without creating more confusion. 16:42:47 hogepodge: ok, thanks. I may take a week to get that done if it's ok? 16:42:59 (too much churn happening in glance atm) 16:43:02 hogepodge: If the end result would be "All clouds must be KVM" then that is not workable. 16:43:14 nikhil: sure, we want to make sure we have a thorough understanding of the issues 16:43:25 hogepodge: ok, thanks. 16:43:42 VanL: that isn't the case right now. the test fixtures are configurable to take that into account 16:43:54 nikhil: thx for bringing up the topics here ... 16:44:05 anytime! 16:44:29 nikhil: thank you. We really appreciate it. 16:44:35 (I'm just planning the cycle milestones & mine) 16:44:55 hogepodge: right back at ya. 16:44:58 Ok, are we ok to move on to the next topic and take the larger discussion to the mailing list? 16:45:07 yes, thx 16:45:28 #topic Test spec 16:45:38 gema: you've been the lead on this 16:45:46 I am working on a commit that I plan to submit on Friday and I have a couple of questions 16:46:05 dwalleck: do you have any extra input from your analysis that you'd like me to consider? 16:47:01 gema: Not at the moment. I think a lot of points ended up getting discussed in the etherpad over the atomicity point 16:47:18 * dwalleck deletes atomicity from his vocabulary 16:47:23 dwalleck: ok, sounds good 16:47:47 is the spec supposed to cover API design guidelines? 16:47:58 because some of the discussion in the etherpad seems to point towards that 16:48:11 #link https://etherpad.openstack.org/p/DefCoreSpec-draft 16:48:32 things like: Prefer Basic REST CRUD operations: Create, Read, Update, Delete? 16:48:39 gema: There is an API working group who puts out standards for OpenStack APIs 16:48:41 do we need to point that out on a test spec? 16:49:01 dwalleck: so we don't need to cover that 16:49:17 gema: I think the discussion is about API testing guidelines not API design 16:49:39 catherineD: cool, because I was about to leave that part blank, I don't feel qualified to write it :) 16:49:49 ok, I have all I need then 16:49:52 hogepodge: thanks 16:50:28 On this topic, does anyone want to add anything else? 16:50:38 gema: I think the functionality of many of the APIs kind of looks like CRUD, but there's often more 16:51:07 dwalleck: yeah, I figured as much that is why I didn't want to get into that 16:51:10 on a test spec 16:51:13 gotcha 16:51:49 * dwalleck didn't sleep so is a bit out of it 16:51:58 Ok, we'll move on. Please update the etherpad if you have more comments. 16:52:13 gema: thanks for working on this, and let us know if you need anything else to help out 16:52:20 hogepodge: will do, thanks 16:52:42 #topic TC Resolutions 16:52:57 There's been quite a bit of good discussion on the mailing list regarding the resolutions. 16:53:22 The TC will be voting on them no earlier than the May 24 meeting. 16:53:47 relevant links 16:53:49 #link https://review.openstack.org/#/c/312718/ 16:53:56 #link https://review.openstack.org/#/c/312719/ 16:54:11 #link Discussion http://lists.openstack.org/pipermail/defcore-committee/2016-May/001095.html 16:54:28 Any comments or feedback? 16:54:59 Moving on 16:55:11 #topic Interoperability Issues Report 16:55:17 #link https://etherpad.openstack.org/p/DefCoreInteropReport 16:55:36 Reminder to vote on issues, contribute thoughts, or add new issues. 16:56:22 Mark and Egle are looking to winnow that list down to 5. 16:56:37 #topic Work Items 16:56:51 The remainder of the agenda is largely work items from the previous meeting and Austin summit. 16:57:06 In the remaining minutes, if anyone has updates please feel free to let us know. 16:58:00 I've been buried since I came back from the summit, just starting to dig back out 16:58:44 I had burst of work, then also got delayed with other things. Turning my attention back to a number of items though, so expect new patches. 16:58:58 Ok, let's call it a meeting. 16:59:06 Thanks everyone! 16:59:12 thanks! 16:59:16 thanks! 16:59:24 #endmeeting