16:00:01 <smcginnis> #startmeeting Cinder
16:00:03 <openstack> Meeting started Wed Jun  1 16:00:01 2016 UTC and is due to finish in 60 minutes.  The chair is smcginnis. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:04 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:07 <Swanson> Hello.
16:00:08 <openstack> The meeting name has been set to 'cinder'
16:00:11 <smcginnis> ping: dulek duncant eharney geguileo winston-d e0ne jungleboyj jgriffith thingee smcginnis hemna xyang tbarron scottda erlon rhedlind jbernard _alastor_ vincent_hou kmartin patrickeast sheel dongwenjuan JaniceLee cFouts Thelo vivekd adrianofr mtanino yuriy_n17 karlamrhein diablo_rojo jay.xu jgregor baumann rajinir wilson-l
16:00:14 <patrickeast> hi
16:00:17 <eharney> hi
16:00:18 <e0ne> hi Team!
16:00:18 <kmartin> hello
16:00:19 <wilson-l> hi
16:00:19 <scottda> Hola
16:00:19 <cFouts> hi
16:00:20 <mtanino> hi
16:00:21 <geguileo> smcginnis: Thanks
16:00:23 <geguileo> Hi
16:00:26 <tbarron> hi
16:00:28 <ntpttr> hey all
16:00:28 <yuriy_n17> hi!
16:00:29 <DuncanT> Hi
16:00:31 <xyang1_> hi
16:00:33 <baumann> Hello!
16:00:33 <smcginnis> #link https://wiki.openstack.org/wiki/CinderMeetings Agenda
16:00:43 <smcginnis> Hey everyone!
16:01:00 <smcginnis> #topic Announcements
16:01:00 <Swanson> Hi Dr. Nick!
16:01:03 <smcginnis> Not too much today.
16:01:19 <smcginnis> #link https://etherpad.openstack.org/p/cinder-spec-review-tracking Review tracking
16:01:39 <smcginnis> Please use that link for tracking out priorities for this cycle.
16:01:58 <smcginnis> Reviewers please pay attention to areas we've identified as priorities for Newton.
16:02:16 <smcginnis> Gerrit dowtime this Friday for project renames.
16:02:22 <smcginnis> http://lists.openstack.org/pipermail/openstack-dev/2016-May/095691.html
16:02:32 <smcginnis> Friday 2016-06-03 at 20:00 UTC
16:02:35 <jungleboyj> Hello all.
16:02:42 <rajinir> o/
16:02:59 <smcginnis> #link https://etherpad.openstack.org/p/newton-cinder-midcycle Midcycle Etherpad
16:03:02 <karthikp> o/
16:03:15 <smcginnis> If you are planning on going to the midcycle please add your name to the list in the etherpad.
16:03:31 <smcginnis> And reserve your hotel room if you haven't already.
16:04:03 <smcginnis> And please start putting down ideas for topics to cover.
16:04:18 <smcginnis> #topic Migrate volume between backends in an async way
16:04:24 <smcginnis> wilson-l: Hi
16:04:31 <wilson-l> hello
16:04:41 <wilson-l> the main intention of the spec is trying to give the target backend a chance to perform the migration
16:04:55 <smcginnis> #link https://review.openstack.org/#/c/312853 Async migrate spec
16:05:14 <wilson-l> since now we always send the migration task to the source backend and the target backend will never have the chance to perform the migration
16:05:39 <smcginnis> wilson-l: How would the target backend be able to do this?
16:06:06 <hemna> hi
16:06:14 <diablo_rojo> Hello :)
16:06:25 <wilson-l> we assume one driver can call other driver's interface
16:06:32 <sheel> hi
16:06:49 <wilson-l> directly or undirectly
16:07:04 <DuncanT> smcginnis: Several backend can do transparent read-though to a different lun while copying the data from that lun
16:07:19 <e0ne> wilson-l: IMO, it should be done via RPC
16:07:27 <smcginnis> So only between two backends of the same driver type?
16:07:51 <DuncanT> smcginnis: In principle, any iSCSI source can be used
16:07:55 <wilson-l> yea this is what I'm thinking about
16:08:44 <wilson-l> For now only support the same driver type
16:08:44 <smcginnis> Seems... complicated.
16:08:47 <e0ne> DuncanT: what about FC and RBD-based drivers?
16:09:33 <scottda> wilson-l: And this is an optimization to make the volume available more quickly?
16:09:37 <wilson-l> If the backends are connected hrough FC
16:09:50 <DuncanT> e0ne: AFAIK no bankend currently supports that
16:10:05 <e0ne> :(
16:10:27 <kmartin> it would need to be the same driver type for us as the backend needs to have some additional configuration to support this
16:10:35 <smcginnis> I could conceivably do this with FC.
16:10:41 <DuncanT> e0ne: If a backend added support then it could be called.
16:10:42 <smcginnis> Not sure I would really want to though.
16:10:42 <kmartin> the 3PAR FC driver can support this
16:10:42 <wilson-l> scottda: can make the volume attachable more quickly
16:10:49 <xyang> wilson-l: are you going to call a storage backend API to achieve that?
16:10:57 <DuncanT> e0ne: Adding a migration-to-ceph gateway would be a fun thing to code
16:11:00 <mtanino> How user knows which combination of backend storages are avairable for this async migration?
16:11:11 <wilson-l> xyng: through RPC api
16:11:22 <e0ne> DuncanT: yes, we've already have something like that
16:11:36 <xyang> wilson-l: which method in the driver will be called?
16:11:39 <DuncanT> e0ne: Nice! Open code?
16:11:53 <e0ne> DuncanT: I mean migration to ceph code in cinder
16:12:11 <eharney> yes, this sounds like it overlaps w/ the generic migration code a bit
16:12:17 <DuncanT> e0ne: I mean transparent read-though live migration code
16:12:21 <xyang> wilson-l: migrate_volume?
16:12:28 <e0ne> DuncanT: oh, got it!
16:12:45 <smcginnis> xyang: New call, migrate_volume_target()
16:12:58 <xyang> smcginnis: thanks
16:13:02 <jungleboyj> Concern seems to be that this is very complicated for limited benefit?
16:13:06 <mtanino> eharney: I guess so.
16:13:10 <smcginnis> I think we would need a flag returned if it's able to be immediately attached or not.
16:13:16 <smcginnis> Some can probably do that.
16:13:26 <eharney> jungleboyj: i agree, but i'm thinking i need time to digest the spec
16:13:44 <smcginnis> But for others this would just be an optimized data migration to avoid the whole read/dd/write data movement.
16:13:49 <e0ne> DuncanT: we have to ask jbernard and eharney to help with it
16:14:19 * smcginnis will have to read through this spec in more detail
16:14:30 <mtanino> smcginnis: I think so too. Use backend functionality to migrate a volume.
16:14:55 <smcginnis> mtanino: Could be much more optimized that way, especially between like backends.
16:14:56 <jungleboyj> The good news is the design doesn't impact those that don't support it, but it isn't clear how easy it is for those who do support it to expose it.
16:14:59 <WenjunWang> i think it's better to let backend to implement the migrate function for the same backend,just like clone volume in one backend.
16:15:01 <e0ne> how it it different from current backend-specific migrations implementation?
16:15:11 <smcginnis> jungleboyj: I agree.
16:15:36 <smcginnis> e0ne: I think by being able to interact with the other backend driver. Is that right wilson-l?
16:16:27 <e0ne> smcginnis: sounds too complicated and very backend speciffic for me
16:16:44 <smcginnis> e0ne: I have the same concern.
16:16:44 <wilson-l> xyng: initialize_connection, terminate_connection
16:16:44 <wilson-l> xying: Mostly are these two methods
16:16:46 <wilson-l> For more details, we can comment directly on that patch
16:16:47 <wilson-l> I just rtse it up
16:16:47 <wilson-l> raise
16:16:49 <wilson-l> let the target backend to perform the migration can make it possible to attach the volume to server for real use when the migration is under going
16:16:50 <wilson-l> just need to give it a chance
16:16:51 <wilson-l> But currently it never have have the chnce
16:17:02 <wilson-l> s/chnce/chance
16:17:41 <DuncanT> My main concern with this is adding the  testing
16:18:01 <e0ne> DuncanT: +2
16:18:07 <xyang> wilson-l: I saw that  you listed VMAX.  I asked VMAX team to take a look of the spec but what they can do seems different from your description.  I think I need to re-read your spec again.
16:18:09 <DuncanT> We're once again showing with the live backup stuff that if it isn't in the gate tests it gets broken
16:18:15 <e0ne> we don't have migarions tests yet
16:18:47 <mtanino> yes...
16:18:50 <scottda> I'm working on migration tests now.
16:19:14 <mtanino> scottda: will be added Cinder internal tempest?
16:19:37 <wilson-l> xying: a new method named like migrate_volume_target will be called
16:19:47 <scottda> mtanino: No, we want to test migration with a volume attached , which calls Nova swap_volume...
16:19:48 <e0ne> my colleague works on snapshot backup tests
16:19:58 <xyang> wilson-l: ok, I’ll take a look again
16:20:02 <scottda> mtanino: And that should run for Nova and Cinder patches in the gate
16:20:13 <mtanino> scottda: I see.
16:20:39 <smcginnis> OK, let's review and comment on the spec patch.
16:20:48 <scottda> mtanino: I'd like to see it run in gate-tempest-dsvm-neutron-full, which both Nova and Cinder run.
16:20:49 <smcginnis> wilson-l: Anything else before we move on?
16:20:51 <wilson-l> smcginnis: thanck!
16:20:55 <xyang> e0ne: that is for internal tempest?
16:21:13 <smcginnis> wilson-l: Thank you.
16:21:16 <smcginnis> #topic Move nova/encryptors to os-brick
16:21:20 <smcginnis> lixiaoy1: Hey
16:21:20 <jungleboyj> smcginnis: I will get the storwize team to look as well.
16:21:21 <e0ne> xyang: not sure yet. it will be in tempest or in cinder code treee
16:21:29 <lixiaoy1> hey
16:21:37 <xyang> e0ne:  ok
16:21:38 <smcginnis> jungleboyj: Thanks, the more different backends we have input on the better.
16:21:45 <jungleboyj> smcginnis: ++
16:21:49 <lixiaoy1> Thank Duncan for the comments. And currently what I need is review
16:21:55 <e0ne> xyang: from my pov, it's ok to get them in cinder
16:22:03 <smcginnis> #link https://review.openstack.org/#/c/247372/ os-brick encryptors patch
16:22:10 <xyang> e0ne: is your co-working going to work on other tempest tests for backup too?
16:22:18 <e0ne> xyang: yes
16:22:25 <lixiaoy1> as this patch is prerequisite of encrypted volume, it is in in os-brick, all patches for encrypted volume are failed without the patch
16:22:26 <xyang> e0ne: cool
16:22:31 <e0ne> xyang: we've got some covegare results
16:22:38 <e0ne> xyang: I'm going to share it tomorrow
16:22:56 <smcginnis> lixiaoy1: Looks like nova has some issues yet?
16:23:01 <xyang> e0ne: great,  I’d like to see what else we need so we add enought coverage
16:23:16 <lixiaoy1> danpb added comments, and he agreed to do it.
16:23:54 <e0ne> xyang: afair, we've got a good enough coverage, I don't have the link for doc right now:(
16:24:07 <e0ne> xyang: I'll ping you tomorrow with update
16:24:14 <xyang> e0ne: sure, thanks!
16:24:39 <hemna> lixiaoy1, so nova is ok with the encryptors in os-brick now ?
16:25:44 <lixiaoy1> I think so.
16:26:08 <hemna> ok I'll take a look at the patch again
16:26:17 <lixiaoy1> hemna: thank you!
16:26:43 <lixiaoy1> hemna: I also added a test script based on your diediedie test
16:28:56 <smcginnis> lixiaoy1: Great, anything else on this topic?
16:29:07 <lixiaoy1> no, thanks
16:29:12 <smcginnis> lixiaoy1: Thank you.
16:29:27 <lixiaoy1> my pleasure
16:29:31 <smcginnis> #topic Open Discussion
16:29:40 <smcginnis> The floor is open...
16:29:49 <scottda> smcginnis: I put something on the agenda (late)...
16:30:00 <scottda> To discuss whether extensions should be version-able
16:30:02 <hemna> lixiaoy1, so, I would also like to see a CI tests reporting on os-brick patches for the encryptors
16:30:04 <smcginnis> scottda: Weird, I just refreshed.
16:30:08 <scottda> It came up with this review https://review.openstack.org/#/c/312063/
16:30:10 <hemna> lixiaoy1, if possible
16:30:35 <scottda> We can move many extensions to core (I'd like to do this work, but have not yet started)
16:30:45 <smcginnis> #link https://review.openstack.org/#/c/312063/
16:30:52 <scottda> But many extensions are not supported by all backends, and therefore should not be moved to core.
16:31:05 <scottda> Should extensions be versionable? Discuss....
16:31:10 <e0ne> scottda: do we have a list of them?
16:31:13 <smcginnis> scottda: Any idea how nova and others are handling this?
16:31:20 <geguileo> scottda: I think any extension that is going to be moved to core should be versionable
16:31:21 <smcginnis> I would think they have the same issue.
16:31:25 <e0ne> scottda: yes, they should be done as microversions
16:31:32 <xyang> I don’t think nova has extensions any more
16:31:40 <scottda> e0ne: No, but should be possbible to come up with from support matrix
16:31:45 <hemna> e0ne, why?  there is no api change
16:31:52 <lixiaoy1> hemna: About CI, I will investigate.
16:31:57 <smcginnis> xyang: So how do they handle non-core functionality? Just try it and see if it works?
16:31:58 <hemna> this is just an internal change that doesn't affect the cinder API
16:32:05 <scottda> geguileo: But what about extensions that cannot be moved to core?
16:32:08 <hemna> lixiaoy1, ok thank you.
16:32:22 <e0ne> #link https://etherpad.openstack.org/p/move-cinder-extensions-to-core
16:32:22 <xyang> I don’t know about nova.  I remember they have a support matrix about hypervisors
16:32:35 <DuncanT> hemna: The linked review is an API change
16:32:40 <geguileo> scottda: Good question  :-)
16:33:09 <dulek> Nova is also suffering a little from capabilities problems. See http://lists.openstack.org/pipermail/openstack-dev/2016-March/090927.html
16:33:10 <geguileo> scottda: I would let them do versioning as well, but that's just my opinion
16:33:14 <DuncanT> Does anybody actually turn off any of the extensions?
16:33:20 <hemna> DuncanT, ok that's a description and name change.  I'm replying to the discussion of moving extensions to core
16:33:25 <dulek> They plan to report possible actions per vm it seems.
16:33:37 <DuncanT> hemna: Ah, yes, they should be transparent
16:34:04 <hemna> I don't think it's a need for microversion change
16:34:07 <hemna> just move them.
16:34:22 <e0ne> AFAIK, we decided to move them as microversion and redirect from old urls should work
16:34:23 <scottda> This is not about moving extensions to core
16:34:28 <dulek> We should probably just report capabilities per volume type. We've started that discussion at the summit. That way we can just move all the extensions to core and keep things discoverable.
16:34:37 <smcginnis> Yeah, if it doesn't change the API itself, just how it's implemented internally, I think we can just move them.
16:34:39 <scottda> It is about extensions that remain extensions, because they are not core features
16:34:42 <scottda> i.e. CGs
16:34:48 <smcginnis> But if the public API changes, then we need to microversion them.
16:34:58 <hemna> the non moveable extensions stay
16:35:04 <hemna> I'm not sure what the issue is then
16:35:05 <xyang> another thing is we are going with generic groups, and CG will be deprecated eventually
16:35:16 <DuncanT> As far as I can tell, everybody ships with all of the standard extensions turned on, so we gain nothing by them not being in core
16:35:34 <hemna> xyang, but since CGs have been in, we may deprecate them, but I don't think we can ever remove them
16:35:39 <smcginnis> Anyone here run a cloud where you disable any of the extensions?
16:35:50 <scottda> hemna: The issue came up because xyang pointed out that in manila (and other services?) extensions were not version-able, by tradition.
16:35:50 <DuncanT> xyang: We just change the internal implementation of the current CG calls but keep them
16:35:54 <e0ne> DuncanT: fair enough
16:35:59 <smcginnis> It anyone reads the logs and has input, please ping me.
16:36:21 <xyang> hemna: DuncanT: we can do that too.  not sure if microversion on top of that will make it better or worse
16:36:26 <hemna> scottda, ok that's a different problem :)
16:36:27 <scottda> IMO, we should be able to version extensions.
16:36:35 <e0ne> scottda: +1
16:36:39 <smcginnis> scottda: Is there a reason we can't?
16:36:50 <hemna> well
16:36:51 <scottda> also IMO, things that are not supported by everyone should remain extensions.
16:36:59 <DuncanT> Just get rid of extensions IMO
16:37:02 <hemna> so, I guess it depends on what you mean by version
16:37:07 <sheel> DuncanT: +1
16:37:07 <scottda> smcginnis: No, there is not any reason we cannot
16:37:08 <hemna> microversions are changes to the Cinder API
16:37:09 <xyang> scottda: I just checked with bswartz and and cknight and they said there is no real reason not to version extension
16:37:18 <hemna> extensions aren't part of the cinder API proper
16:37:22 <DuncanT> You can' tell from the outside if they're extensions or not
16:37:25 <hemna> it's a grey area
16:37:34 <xyang> it’s just that all extension APIs in Manila have been moved to core
16:37:35 <hemna> DuncanT, true
16:37:38 <geguileo> hemna: But they are part of the Cinder API from the user's perspective
16:37:46 <DuncanT> geguileo: +1
16:38:06 <DuncanT> THis internal split between core and extension is entirely crazy and meaningless
16:38:16 <hemna> DuncanT, +1
16:38:22 <hemna> I don't see the point of the extensions really
16:38:23 <smcginnis> DuncanT: I remember conversations in the past about getting rid of them.
16:38:24 <geguileo> DuncanT: +1
16:38:33 <DuncanT> smcginnis Yup
16:38:35 <hemna> the cinderclient has to expose them
16:38:50 <hemna> and yet, they can theoretically get disabled on the cinder side
16:38:54 <hemna> confusion ensues
16:38:59 <sheel> for reference https://etherpad.openstack.org/p/move-cinder-extensions-to-core
16:39:08 <dulek> We have extensions call, which actually lists what is an extension and what's not. Only names, not endpoints however…
16:39:11 <scottda> Are there deployers who disable extensions? I would think some might...
16:39:25 <guitarzan> scottda: yes, but you can do the same thing with policy
16:39:32 <DuncanT> We used to in public cloud, but not HOS
16:39:36 <bswartz> oh crap I forgot about meeting
16:39:46 * bswartz sneaks in VERY late
16:39:46 <dulek> guitarzan: Oh, right, CGs are disabled also by policies…
16:39:58 <guitarzan> it doesn't really matter if internally they're extensions or not
16:40:01 <xyang> bswartz: your name was called:)
16:40:13 <guitarzan> on the other hand, we also have some of our own admin-y extensions
16:40:15 <DuncanT> bswartz: The most interesting bit is still happening
16:40:19 <guitarzan> so extension functionality is super important
16:40:22 <sheel> smcginnis:  conversation link for movement of extensions https://etherpad.openstack.org/p/move-cinder-extensions-to-core
16:40:34 <scottda> OK, so one think might be getting that review in, and allowing a microversion...another would be getting rid of extensions (which is a large bit of work).\
16:40:46 <geguileo> scottda: +1
16:40:51 <DuncanT> guitarzan: Allowing vendors to add extensions seems worth keeping, but getting rid of all the in-tree ones makes sense
16:40:55 <sheel> for now we should microversion them and later on we can move to core
16:41:06 <guitarzan> DuncanT: sure, I don't care at all where the in tree code lives
16:41:12 <smcginnis> sheel: Thanks, that was above. :)
16:41:16 <bswartz> DuncanT: you just said was I was going to say
16:41:19 <scottda> If we are going to move to core (later), we should definately microversion now.
16:41:27 <guitarzan> extension vs "core api" doesn't matter at all
16:41:32 <hemna> I think it's useful to have the extensions plumbing in place, so others can add their custom stuffs to Cinder for their deployment.  But stuff that gets checked into the cinder codebase seems to make sense to make part of core.
16:41:47 <smcginnis> guitarzan: Very good points.
16:41:56 <geguileo> hemna: I agree
16:42:01 <e0ne> hemna: +1
16:42:05 <smcginnis> hemna: Yeah, that makes sense to me.
16:42:12 <guitarzan> that being said, changing the routes for all of these calls might not make sense
16:42:36 <guitarzan> because you can't *not* support the old routes anyway
16:42:42 <DuncanT> We should keep a dummy extension in tree to keep the plumbing tested and testable
16:42:43 <hemna> personally, all of the routing and the API+extensions structure is horribly confusing and painful to navigate
16:42:52 <smcginnis> DuncanT: Yes!
16:43:07 <scottda> OK. Is there any controversy here? Current reviews/code that is in an extension could/should be microversioned....Moving extensions to core would come later (with a lot of work to code/review/test)
16:43:08 <hemna> DuncanT, yah that's basically what I think too.
16:43:36 <DuncanT> scottda: I'm fine with that plan in either order.
16:43:41 <jungleboyj> DuncanT: That is a good idea.
16:43:42 <geguileo> scottda: No controversy on my part  :-)
16:44:27 <DuncanT> I think bswartz was of the opinion/experience that moving into core was fairly easy and mechanical (which is what I'd expect)
16:44:36 <jungleboyj> scottda: I think that makes sense.
16:44:41 <scottda> OK, thanks. The main goal was to get past that CG API review..
16:44:55 <scottda> DuncanT:  I've looked at it. Simple, but not easy. It's a lot of code.
16:44:58 <DuncanT> But adding the version stuff before they're moved has no downsides I can see
16:45:03 <bswartz> yeah once you have microversions, you can safely move extensions into core -- old clients can continue to use extensions but new clients can use core APIs
16:46:33 <smcginnis> Anything else...
16:46:44 <scottda> smcginnis: Nope
16:47:00 <diablo_rojo> don't think so
16:47:04 <smcginnis> Thanks.
16:47:08 <smcginnis> GOing once...
16:47:14 <smcginnis> Going twice...
16:47:20 <smcginnis> Thanks everyone!
16:47:26 <geguileo> Thanks
16:47:28 <diablo_rojo> Thanks smcginnis :)
16:47:29 <e0ne> see you next week
16:47:34 <wilson-l> thanks
16:47:40 <smcginnis> #endmeeting