14:00:20 <rosmaita> #startmeeting cinder
14:00:20 <rosmaita> #topic roll call
14:00:20 <rosmaita> #link https://etherpad.openstack.org/p/cinder-ussuri-meetings
14:00:24 <openstack> Meeting started Wed Apr  8 14:00:20 2020 UTC and is due to finish in 60 minutes.  The chair is rosmaita. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:25 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:27 <openstack> The meeting name has been set to 'cinder'
14:00:37 <sfernand> hi
14:00:38 <eharney> hi
14:00:39 <lseki> o/
14:00:40 <enriquetaso> hi
14:00:45 <Liang__> hi
14:00:47 <e0ne> hi
14:00:56 <m5z> hi =]
14:00:58 <smcginnis> o/
14:00:59 <LiangFang> o/
14:01:08 <rosmaita> wow, nice turnout
14:01:10 <whoami-rajat> Hi
14:01:19 <rosmaita> #topic announcements
14:01:29 <jungleboyj> o/
14:01:31 <tosky> o/
14:01:40 <rosmaita> Friday and/or Monday are holidays in a lot of countries
14:01:57 <rosmaita> just be aware when requesting reviews, will be a bit slow this weekend
14:02:12 <rosmaita> TC election (look in your email for a ballot) ... voting ends 2020-04-14 23:45 UTC
14:02:14 <walshh_> hi
14:02:23 <jungleboyj> Please vote!
14:02:40 <rosmaita> i guess this is the stuff-happening-around-openstack edition of the announcements
14:02:49 <rosmaita> ironic declaration of independence: http://lists.openstack.org/pipermail/openstack-discuss/2020-April/013757.html
14:02:51 <smcginnis> :)
14:02:53 <rosmaita> really long thread
14:03:01 <rosmaita> just in case you are interested in that kind of thing
14:03:22 <rosmaita> ok, on to cinder project news
14:03:33 <rosmaita> os-brick 3.0.1 "ussuri-official" released last week
14:03:33 <e0ne> Ironic Independence Day - sounds good
14:03:41 <rosmaita> :)
14:03:53 <rosmaita> python-cinderclient 7.0.0 almost ready to go
14:04:09 <rosmaita> just need the release note approved: https://review.opendev.org/#/c/718234/
14:04:22 * jungleboyj clicks
14:04:45 <rosmaita> reminder: the stable/ussuri branch for both of those are cut upon release
14:05:02 <rosmaita> note to self: did we cut a branch for the brick-cinderclient-ext ?
14:05:18 <rosmaita> so from now on, bugfixes in master and backport to stable/ussuri
14:05:38 <rosmaita> tomorrow is milestone-3 release and FEATURE FREEZE for cinder
14:05:55 <rosmaita> so let's take a quick look at the feature situation
14:06:05 <rosmaita> #link https://blueprints.launchpad.net/cinder/ussuri
14:06:34 <rosmaita> these are the ussuri features that i'm aware of
14:07:15 <rosmaita> quick question for driver maintainers: are there driver features you have patches up for that aren't represented here?
14:07:30 <rosmaita> because review priority for the rest of the week is features
14:07:52 <lseki> I think sfernand has one
14:08:07 <rosmaita> is that for netapp active-active?
14:08:12 <lseki> yes!
14:08:13 <sfernand> rosmaita: We added support for active/active to the SolidFire recently
14:08:15 <smcginnis> I think some of the Dell teams have some patches up for new driver features.
14:08:22 <sfernand> we are getting reviews this week
14:08:34 <rosmaita> great
14:08:47 <sfernand> Do you think this change would require a spec or something? https://review.opendev.org/#/c/712799/5
14:09:15 <rosmaita> sfernand: no, a bp in launchpad will be fine
14:09:37 <rosmaita> i will create one and put it into ussuri after the meeting so we can track
14:09:57 <sfernand> ok! Thanks
14:10:33 <rosmaita> as far as the Dell/EMC drivers go, does someone have a list of feature patches?
14:11:15 <jungleboyj> I was getting pinged by walshh_  about their features.  We merged a couple.
14:11:49 <rosmaita> ok, i will take a quick look at the open reviews after the meeting and make bps for any open ones
14:11:50 <walshh_> all our features are merged.  Thanks to all who reviewed
14:12:00 <rosmaita> well, that makes it easy!
14:12:16 <jungleboyj> Yay!  Go team!
14:12:33 <rosmaita> ok, so reviewers: please concentrate on the feature patches for the rest of the week
14:12:48 <rosmaita> which is basically today and tomorrow for all intents and purposed
14:12:53 <rosmaita> *purposes
14:13:01 <rajinir> We have some VXFlexOS patches and several other Dell EMC driver patches pending
14:13:32 <rosmaita> rajinir: can you get a list together of the feature-oriented patches?
14:13:39 <jungleboyj> ++
14:13:41 <rosmaita> because as you know, features are not backportable
14:13:55 <rajinir> <rosmaita>  will do
14:14:03 <rosmaita> ok, great
14:15:18 <rosmaita> smcginnis: jungleboyj: do we need to do an M-3 release? i believe it's optional these days
14:15:50 <jungleboyj> rosmaita:  I believe it is optional.
14:15:53 <smcginnis> rosmaita: Correct, it's optional.
14:16:03 <smcginnis> We just need to be ready for the RC deadline.
14:16:09 <rosmaita> ok, i don't see a need to do one
14:16:38 <jungleboyj> ++
14:16:40 <rosmaita> so since smcginnis mentioned it, RC-1 is the week of 20 April
14:16:49 <ganso> o/
14:16:51 <rosmaita> so two weeks away
14:17:10 <rosmaita> i'll be working on the third-party CI compliance check early next week
14:17:27 <rosmaita> checking to see the they are responding, keeping logs, etc
14:17:43 <rosmaita> if any problems come up, they'll need to be addressed before RC-1
14:17:58 <jungleboyj> rosmaita: ++
14:18:10 <rosmaita> and of course, any driver maintainers can check their own CI before i do it if you want to get a head start
14:18:38 <jungleboyj> :-)
14:18:41 <rosmaita> otherwise, drivers may be subject to being marked as not supported
14:18:57 <rosmaita> ok, i think that's everything
14:19:04 <whoami-rajat> rosmaita, is the feature freeze also the deadline to mark drivers as supported?
14:19:05 <rosmaita> rajinir: don't forget to get me that list of patches
14:19:30 <rosmaita> whoami-rajat: no, they have until RC-1
14:19:32 <rajinir> rosmaita>  compiling now will share soon
14:19:40 <rosmaita> but new drivers must be merged before FF
14:19:47 <rosmaita> or request an FFE
14:20:06 <rosmaita> rajinir: ty
14:20:15 <whoami-rajat> rosmaita, okay. i also see macroSAN encapsulated some features with marking it as supported https://review.opendev.org/#/c/711388/
14:21:11 <whoami-rajat> they've a functional CI with one issue that we discussed the other day on cinder channel
14:21:39 <rosmaita> ok, thanks for bringing that up
14:21:47 <whoami-rajat> i couldn't contact them after that but it seems good to get this in as well?
14:22:11 <jungleboyj> Ugh.  Ok.  Guess we need to get some eyes on that.
14:22:20 <rosmaita> yes, we should prioritize this one
14:22:28 <rosmaita> i'll add a bp so we can track it
14:22:36 <whoami-rajat> great, thanks!
14:22:51 <rosmaita> let's not let the supported=true hold this up
14:23:09 <rosmaita> we can un-support it if we have to at RC-1 time
14:23:23 <rosmaita> anything else?
14:23:58 <whoami-rajat> not from me
14:24:22 <rosmaita> #topic Bug: Cinder Fail to extend attached volume using generic NFS driver
14:24:27 <rosmaita> throne82: that's you
14:24:47 <throne82> Helo
14:25:53 <throne82> While I was enabling the extend attach tests for NetApp drivers we had some errors on qemu-img
14:25:59 <rosmaita> #link https://bugs.launchpad.net/cinder/+bug/1870367
14:26:00 <openstack> Launchpad bug 1870367 in Cinder "Fail to extend attached volume using generic NFS driver" [Undecided,New]
14:26:23 <throne82> I tested with the generic NFS driver and it fails the same (since the implementation is the same)
14:27:16 <throne82> so the extend fails due to the qemu-img cant lock the volume to write anything
14:27:34 <rosmaita> yes, thanks for the detailed bug report
14:27:36 <eharney> the failure here makes sense, i think this will need some work in the generic NFS driver
14:28:09 <lseki> netapp ontap nfs drivers relies on image_utils, which is used by generic nfs driver as well
14:28:13 <eharney> qemu-img can't get a lock to resize the file because it's in use by nova's qemu
14:28:37 <lseki> I wonder if generic nfs had ever supported online extend in the past
14:29:00 <lseki> I think it did, because ontap nfs driver also did some time ago
14:29:56 <lseki> not sure what changed, the image_utils resize_image code didn't change for years
14:30:17 <eharney> older nfs configs may have let it succeed if they didn't have the same nfs lock support configured
14:30:26 <eharney> whether that is/was safe or not is another question
14:32:02 <lseki> seems that currently nfs driver is skipping extend attached volume tests
14:32:17 <lseki> #link https://b4a949e5f6fdf3010036-e4f20cff14b59b3a1c5b0d28b2b173f9.ssl.cf2.rackcdn.com/696626/2/check/devstack-plugin-nfs-tempest-full/098763c/testr_results.html
14:32:43 <rosmaita> so i guess the question is, can this be made safe or do we not allow extending an attached volume for nfs?
14:33:10 <eharney> i think we should block it in the driver for now, there should be some ways to do it safely with enough work
14:33:26 <rosmaita> that makes sense to me
14:33:31 <eharney> the problem is that extending attached volumes was added as a general cinder function w/o strict checking into what happens in each driver etc
14:33:59 <rosmaita> that is definitely a problem
14:34:07 <rosmaita> so we may see this again
14:34:32 <eharney> it was quite a while ago, so not a huge worry, but maybe
14:34:57 <rosmaita> well, any drivers currently skipping the tests should definitely take a look
14:35:09 <rosmaita> i mean, their maintainers should take a look
14:35:20 <smcginnis> I could have sworn we had required drivers to report if they supported extending attached volumes.
14:35:45 <whoami-rajat> the support matrix says we support it for nfs
14:36:02 <rosmaita> guess that will need an update
14:36:07 <kaisers_> There's a tempest flag for that test:  @testtools.skipUnless(CONF.volume_feature_enabled.extend_attached_volume,
14:36:07 <kaisers_> "Attached volume extend is disabled.")
14:36:35 <eharney> smcginnis: i was thinking the same thing, but i haven't found code for it
14:36:42 <rosmaita> throne82: has your question been answered?
14:36:54 <kaisers_> (source: https://github.com/openstack/tempest/blob/348fa311fe031ff7d04f41aa9e6ac65f6f6391fe/tempest/api/volume/test_volumes_extend.py)
14:37:28 <whoami-rajat> https://review.opendev.org/#/c/454287/ this is a generic implementation of the feature
14:38:06 <LiangFang> I remember lots of drivers in os-brick not support extend volume feature
14:39:17 <eharney> there's a difference between the os-brick extend_volume support and the cinder driver extend_volume
14:39:30 <LiangFang> ok
14:39:30 <eharney> i think the os-brick part is for updating Nova after the actual extend was done.  the latter being what fails here
14:39:54 <throne82> rosmaita: can we have a discussion later to further discuss if there's a safe way to do this?
14:40:08 <rosmaita> sure
14:40:22 <rosmaita> it's a bug, so we have a bit of time
14:40:43 <rosmaita> ok, throne82, thank you for bringing this up and working on it
14:41:01 <throne82> thanks!
14:41:01 <rosmaita> #topic Review needed: NFS encrypted volume support
14:41:07 <rosmaita> request from enriquetaso
14:41:12 <enriquetaso> Hi
14:41:17 <rosmaita> hello!
14:41:25 <enriquetaso> Hello cinder team, so this is a really old spec that we have "Support Cinder volume encryption with the NFS driver."
14:41:26 <enriquetaso> #link https://blueprints.launchpad.net/cinder/+spec/nfs-volume-encryption
14:41:52 <enriquetaso> I've been working on this : https://review.opendev.org/#/c/597148/ and I need some reviews :D
14:41:52 <enriquetaso> I know it's a long patch but It's ready for opinions.
14:42:21 <rosmaita> i've been seeing enriquetaso's reviews on several other patches lately
14:42:30 <rosmaita> so it would be good to do her a solid and review her patch
14:42:40 <enriquetaso> :D thanks rosmaita
14:42:42 <whoami-rajat> rosmaita++
14:43:33 <rosmaita> this is in the list of cycle features, btw, so it is important to review it soon
14:43:40 <rosmaita> anything else?
14:43:45 <enriquetaso> nop
14:43:46 <eharney> i think this one is pretty close (kind of a biased opinion), i mostly want to ensure it's being tested thoroughly
14:44:04 <enriquetaso> eharney++
14:44:11 <rosmaita> ok, great
14:44:20 <rosmaita> #topic cinder-tempest-plugin
14:44:26 <rosmaita> tosky: that's you
14:44:27 <tosky> so
14:45:00 <tosky> as you can see in the list, there are a few cinder-tempest-plugin reviews which are ready or almost-ready but in a reviewable state, and that would be nice to have
14:45:14 <jungleboyj> I was going to say that we needed to make sure eharney looked at her patch, but then I see the owner.  ;-)
14:45:18 <tosky> I can quickly say a few words about each of them
14:46:20 <tosky> - Enable c-bak and switch to the storage blacklist: https://review.opendev.org/#/c/717379/ -> this is mostly complete IMHO, and the subject says it all
14:46:52 <tosky> c-bak testing also with the cinder backend for cinder-tempest-plugin changes; it unblocks a few other tests
14:47:09 <tosky> same for Enable volume_revert tests https://review.opendev.org/#/c/717379/ , which unblocks revert tests on LVM
14:47:52 <tosky> - Add Snapshot data integrity test https://review.opendev.org/#/c/702495/  - this is IMHO logically fine, I only suggested to move some common code in a separate function
14:48:48 <tosky> - Add LVM+tgt tempest job https://review.opendev.org/537658 - originally proposed long time ago by Eric, it has been refactored; it looks fine now, even though it may conflict with https://review.opendev.org/#/c/717379/, we may need some reordering
14:49:09 <tosky> and finally, two other real tests about backups which are waiting on c-bak support:
14:49:13 <tosky> - Extending testing scope of Incremental Backup https://review.opendev.org/#/c/652817/
14:49:23 <tosky> - Add test for check dependencies between incr backups https://review.opendev.org/#/c/652771/
14:49:42 <rosmaita> ok, thanks ... looks like you have them listed in order of priority
14:49:45 <tosky> their results should be rechecked now that c-bak is active
14:49:50 <eharney> awesome to see all this work being done
14:49:58 <enriquetaso> tosky++
14:50:25 <tosky> I'm only nagging people, enriquetaso and whoami-rajat did most of the work here
14:50:38 <rosmaita> tosky: is it ok to hold off on the multipath scenario questions until next week?
14:50:47 <tosky> finally, I have a few questions about this patch, which was abandoned for a while, but it is interesting:
14:50:52 <tosky> oh, sure
14:50:55 <tosky> that was the one
14:51:09 <tosky> but you can read the question there and answer anytime - I won't stop you!
14:51:19 <rosmaita> thanks, just want to make sure we get to the other items
14:51:32 <tosky> sure, EOF for now
14:51:33 <rosmaita> #topic Allow removing NFS snapshots in error status is stuck.
14:51:41 <whoami-rajat> thanks tosky
14:51:47 <enriquetaso> Hello again ... So, I have 3 diff opinions in the patch:
14:51:48 <enriquetaso> #link https://review.opendev.org/#/c/679138/
14:51:57 <enriquetaso> I'm not sure what to do, maybe we can discuss it here and get a conclusion. i'll do my best to summarize them:
14:52:25 <enriquetaso> 1. Make the patch a partial bug fix because the root issue here is that the snapshot entries were created in the first place. But  If snapshots were created, then the decision was made to disable snapshots, we should still allow those available snapshots to be deleted. Otherwise there is no way to get rid of them without re configuring the system.
14:52:33 <enriquetaso> ^ smcginnis
14:52:53 <enriquetaso> 2. Don't make it a partial bug and still call _check_snapshot_support() in delete, but catch the expected exception.
14:52:53 <enriquetaso> Then log that exception, so admins have some trace of it, and then allow the delete to go through.
14:52:58 <enriquetaso> ^ hemna_
14:53:19 <enriquetaso> 3. Replace the current check because the introduction of this check dates back to Ubuntu 14 issue with libvirt < 1.2.7 so not sure how useful is the check in current deployments (maybe used for other purposes). Since the volume is only created in db, my recommendation would still be to just delete it from db during create.
14:53:19 <enriquetaso> It might be easily done by defining a new exception, raise it from from nfs driver and catch it in manager. This way, the snapshot is never created (as intended), the exception is raised stating the operation is not supported, there is no extra work to remove snapshot
14:53:23 <enriquetaso> ^ whoami-rajat
14:53:46 <enriquetaso> sorry for bothering you, but I'm confused
14:54:45 <hemna_> mep
14:55:07 <eharney> both #1 and #3 point to an idea that it would be useful for a failed snapshot creation to be recorded as "nothing actually happened on the backend that needs to be deleted later"
14:55:21 <eharney> but i think that's a larger scope thing to take on than the bug fix at hand here
14:57:55 <whoami-rajat> my idea is just to remove the conflict of whether we can delete a snapshot if it's disabled, also manual cleanup would be required but it's ok if it's implemented any other way as far as it solves the issue
14:58:13 <whoami-rajat> s/it's disabled/if support is disabled
14:58:36 <eharney> there doesn't have to be a conflict there, the driver could do a check as to whether it needs to delete anything when delete_snapshot is called and if not, return success before it queries whether snapshots are enabled
14:59:16 <rosmaita> we're down to 1 minute
14:59:27 <rajinir> https://www.irccloud.com/pastebin/yPQfJRXw/
14:59:27 <rosmaita> ganso: will need to postpone until next meeting
14:59:32 <rosmaita> rajinir: ty
14:59:37 <ganso> rosmaita: it may be quick, we could chat in the channel
14:59:37 <rajinir> rosmaita: https://etherpad.openstack.org/p/Dell_EMC_Driver_Feature_Patches_Ussuri
14:59:48 <rosmaita> ok
15:00:13 <rosmaita> thanks everyone, we have another meeting happening here in a minute
15:00:18 <rosmaita> #endmeeting