14:00:26 <rosmaita> #startmeeting cinder
14:00:26 <opendevmeet> Meeting started Wed Feb 16 14:00:26 2022 UTC and is due to finish in 60 minutes.  The chair is rosmaita. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:26 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:27 <opendevmeet> The meeting name has been set to 'cinder'
14:00:36 <rosmaita> #topic roll call
14:00:46 <jungleboyj> o/  Kind-of.  :-)
14:00:47 <eharney> hi
14:01:13 <whoami-rajat__> Hi
14:02:01 <felipe_rodrigues> hi
14:02:05 <yuval> o/
14:02:24 <fabiooliveira> o/
14:02:57 <TusharTgite> hi
14:03:11 <rosmaita> ok, let's get started
14:03:16 <e0ne> hi
14:03:19 <geguileo> hi
14:03:27 <rosmaita> #link https://etherpad.opendev.org/p/cinder-yoga-meetings
14:03:39 <rosmaita> #topic announcements
14:03:41 <caiquemello[m]> hi
14:03:50 <rosmaita> Festival of XS Reviews on Friday, 1400-1600 UTC
14:04:13 <rosmaita> you can find links and info in the top of the agenda etherpad
14:04:21 <rosmaita> in the "Spotlight Links"
14:04:45 <rosmaita> upcoming stuff:
14:04:51 <rosmaita> cinderclient release next week
14:05:41 <rafaelweingartner> rosmaita: do we have a open questions moment in the Cinder meetings?
14:05:55 <rosmaita> rafaelweingartner: sure, at the end of the meeting
14:06:48 <rosmaita> there are a few open patches, not much, though: project:openstack/python-cinderclient status:open branch:master
14:07:10 <rosmaita> if you want your patch to be considered, make sure you quickly address any negative votes, and that it's passing zuul
14:07:44 <rosmaita> we're adding at least one new mv to cinder this cycle, so look for the patch about that
14:07:59 <rosmaita> on that topic, btw, the cinder-side patch I'm talking about is:
14:08:25 <yuval> mv stand for?
14:08:33 <rosmaita> microversions
14:08:38 <rosmaita> i have it open somewhere
14:09:03 <rosmaita> #link https://review.opendev.org/c/openstack/cinder/+/822040
14:09:26 <yuval> ok, thanks
14:10:01 <whoami-rajat> I'm also working on the reimage feature, will work on getting the client patch passing zuul
14:10:18 <rosmaita> whoami-rajat: remind me, did that entail a new mv?
14:10:26 <whoami-rajat> yes
14:10:34 <whoami-rajat> it's a new volume action
14:10:45 <rosmaita> ok, then let's start an etherpad for cinderclient release
14:10:57 <rosmaita> we need those 2 cinder patches on there, because they need to merge first
14:11:36 <rosmaita> #link https://etherpad.opendev.org/p/yoga-cinderclient
14:11:59 <enriquetaso_> hi
14:12:40 <rosmaita> whoami-rajat: do you have a link to your cinder change?
14:12:53 <whoami-rajat> sure, 1 sec
14:13:21 <whoami-rajat> rosmaita,  https://review.opendev.org/c/openstack/cinder/+/606346
14:13:40 <rosmaita> thanks
14:16:08 <rosmaita> ok, we also have the cinder FEATURE FREEZE next week
14:16:21 <rosmaita> it applies to cinder and drivers
14:16:32 <rosmaita> we are working from these blueprints:
14:17:15 <rosmaita> #link https://blueprints.launchpad.net/cinder/yoga
14:17:40 <rosmaita> please update your BP with links to the code patches and update your status
14:18:49 <rosmaita> if your feature is *not* ready for review now, it's unlikely it will be reviewed, revised, and merged by next thursday
14:19:12 <rosmaita> so ping me in irc and i will move it to the 'zed' series
14:19:28 <rosmaita> #action rosmaita - set up zed series in launchpad
14:20:01 <rosmaita> if you have a driver feature, please help us out in the reviews
14:20:16 <rosmaita> make sure that your patch has passed your CI
14:20:29 <rosmaita> do a fresh run if you need to, we need to logs to be there
14:20:59 <rosmaita> would be helpful if you link into your logs to show that your feature is working
14:21:21 <rosmaita> reviewers can do that themselves, but it takes extra time, which means slower reviews
14:21:36 <rosmaita> so if you want to speed things up, do what you can to help us out
14:22:09 <rosmaita> so the review priorities over the next week are:
14:22:22 <rosmaita> #link https://etherpad.opendev.org/p/yoga-cinderclient
14:22:25 <rosmaita> and
14:22:33 <rosmaita> #link https://blueprints.launchpad.net/cinder/yoga
14:22:58 <rosmaita> and of course, critical bugs and security bugs, but those can be fixed after feature freeze
14:23:07 <rosmaita> so please concentrate on feature reviews
14:23:35 <rosmaita> finally, next week's meeting will be in videoconference
14:23:51 <rosmaita> usual time & place, i will make sure connection info is on the agenda
14:24:04 <rosmaita> #topic os-brick release status
14:24:16 <rosmaita> you may have seen the email i sent about this yesterday
14:24:24 <rosmaita> #link http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027296.html
14:25:12 <rosmaita> i think yuval's patch should be just about ready, i think he pushed a new patch set this morning
14:25:15 <rosmaita> #link https://review.opendev.org/c/openstack/os-brick/+/823654
14:25:33 <rosmaita> the other one is https://review.opendev.org/c/openstack/os-brick/+/811718
14:25:52 <chuckpiercey_> What is the remaining issue?
14:25:54 <rosmaita> geguileo was looking at it and discovered a different but related bug
14:26:21 <chuckpiercey_> We have no gerrit notice on that
14:26:24 <rosmaita> geguileo: what are your thoughts on https://review.opendev.org/c/openstack/os-brick/+/811718
14:26:26 <geguileo> the problem I found is that we removed the call to nvme disconnect
14:26:44 <geguileo> which is wrong for some cases
14:27:03 <geguileo> we have 2 variables depending on the driver/storage
14:27:16 <geguileo> 1. whether the targets are shared (subsystem)
14:27:29 <geguileo> 2. whether the storage system support namespace AER
14:28:04 <geguileo> for the LVM driver that doesn't do shared targets and doesn't support NS AER, the removal of the disconnect creates a problem
14:28:33 <geguileo> after disconnect the nvme kernel driver keeps trying to connect to the namespace
14:28:39 <chuckpiercey_> we did not remove a disconnect in that patch...
14:28:41 <geguileo> and it only gives up after 600 seconds
14:28:47 <geguileo> it was on another one
14:28:57 <geguileo> and I was trying to find out if we could have race conditions
14:29:46 <chuckpiercey_> if that condition exists it is upstream of this patch.
14:29:50 <geguileo> and during those 600 seconds we cannot reconnect that volume to that same host
14:30:08 <geguileo> chuckpiercey_: let me look for the patch where this was introduced
14:30:48 <geguileo> https://review.opendev.org/c/openstack/os-brick/+/800014
14:30:59 <geguileo> in this patch that is being proposed there is a rescan added
14:31:12 <geguileo> and if the backend doesn't support NS AER, then we have a race condition
14:31:58 <geguileo> and for 600 seconds after the rescan the previously removed device (that got back in due to the rescan) will not be able to connect (I think)
14:32:50 <geguileo> so basically with all the changes to the nvme driver what we are doing is supporting just storage systems that support NS AER
14:33:08 <geguileo> and not supporting those that don't support it
14:33:08 <chuckpiercey_> this isn't our patch - we'll take a look and respond with our thoughts in that patch. That one is a big patch.
14:33:34 <geguileo> these are 2 different things, but they are both related
14:33:46 <geguileo> both are related to NS AER
14:34:04 <rosmaita> OK, we have to release yoga os-brick tomorrow
14:34:10 <geguileo> due to that older patch we cannot connect recently removed volumes to the sam host
14:34:22 <chuckpiercey_> 811718 is small and is intended to verify current connections before reconnect
14:34:28 <rosmaita> i guess we need a "known issues" note
14:34:46 <geguileo> in this newer patch we would be introducing a new race condition for storage systems that share targets and don't support NS AER
14:35:43 <rosmaita> ok, but we already have the disconnect problem for systems that don't support NS AER
14:35:51 <geguileo> rosmaita: that would be an acceptable compromise from my perspective
14:36:21 <rosmaita> ok, i will put up a release notes patch
14:36:41 <rosmaita> basic idea is, if you are using nmve that doesn't support NS AER, you are SOL
14:37:19 <geguileo> rosmaita: yup
14:37:25 <rosmaita> and geguileo, please file a bug about the disconnect issue, i can link to that
14:37:43 <rosmaita> so where does that leave us with https://review.opendev.org/c/openstack/os-brick/+/811718
14:38:44 <geguileo> rosmaita: I would say that the code is fine
14:39:08 <rosmaita> ok, if you can drop your -1 from that patch, that will be helpful
14:39:10 <geguileo> because rescan is the recommended mechanism by the kernel nvme driver instead of a restart
14:40:02 <geguileo> rosmaita: done
14:40:23 <rosmaita> i have got to say, i do not feel comfortable +2ing any of these nvme patches any more
14:40:50 <rosmaita> i look back at these patches that have caused regressions and bugs, and i do not have a good track record here
14:41:13 <rosmaita> so: my question for cinder-core members right now is:
14:41:24 <rosmaita> who will review https://review.opendev.org/c/openstack/os-brick/+/811718 ?
14:41:45 <rosmaita> we have to release tomorrow
14:42:19 <geguileo> rosmaita: I will review it, and I'll most likely give the +2
14:42:46 <chuckpiercey_> thank you
14:43:07 <rosmaita> we need a second person ...
14:43:15 <geguileo> rosmaita: after all theirs is the only driver using multiple namespaces for a subsystem
14:43:15 <e0ne> rosmaita: let me try to run our CI for it
14:43:26 <rosmaita> e0ne: that would be very helpful
14:43:34 <geguileo> e0ne: what nvme storage are you using?
14:46:25 <rosmaita> hmmm, e0ne may have been disconnected
14:46:44 <e0ne> it seems we don't have CI now :(
14:46:50 <geguileo> ouch :-(
14:46:54 <rosmaita> it's the mellanox SPDK CI, is that right?
14:47:16 <e0ne> I'll try to catch my colleagues tomorrow
14:47:19 <e0ne> rosmaita: yes
14:47:42 <rosmaita> ok, thanks
14:47:55 <rosmaita> we are sort of relying on that CI to prevent regressions in the nvmeof connector
14:48:14 <rosmaita> we need to have a discussion about nvme at the PTG
14:48:58 <rosmaita> we have kioxia, lightbits, dell/emc, and pure all wanting to implement stuff
14:49:01 <geguileo> rosmaita: I'm currently looking into making a new nvmet target for the LVM driver
14:49:16 <yuval> If I can help somehow please let me know
14:49:18 <geguileo> one that uses shared targets and returns the new connection information type
14:49:33 <rosmaita> geguileo: so we could use regular Zuul CI to detect regressions, is that correct?
14:49:47 <geguileo> rosmaita: some of them at least
14:50:18 <geguileo> and then we could use the LVM driver for both kind of connections: sharing subsystem and not sharing
14:51:58 <rosmaita> ok, i will let the next PTL figure out a strategy for coordinating and testing nvme-of efforts
14:52:01 <e0ne> geguileo how can we test it without nvme storage?
14:52:14 <chuckpiercey_> Yaron would be happy to help on that @brian
14:52:20 <geguileo> e0ne: right now we can test NVMe with RDMA and TCP using the LVM driver
14:52:44 <geguileo> I even have a patch up for review to do this automatically with devstack
14:52:57 <e0ne> yes, but t require NVMe SSD doesn't it?
14:52:58 <jungleboyj> Nice.
14:53:07 <rosmaita> chuckpiercey_: ack
14:53:47 <whoami-rajat> ack
14:53:53 <geguileo> e0ne: I don't think it does
14:54:09 <rosmaita> ok, we will discuss this more at the PTG
14:54:11 <geguileo> I'm running it in side a VM
14:54:37 <rosmaita> ok, moving on
14:54:47 <rosmaita> #topic release cadence proposal
14:55:20 <rosmaita> proposal is to keep 6 month cycles, but go to "tick-tock" releases, where you can upgrade from tick to tick
14:55:29 <rosmaita> look at the proposal to make sense of what i just said
14:55:37 <rosmaita> #link https://review.opendev.org/c/openstack/governance/+/828777
14:56:49 <rosmaita> the idea is that tick-to-tick could require downtime; our rolling upgrades would only have to support tick-to-tock or tock-to-tick upgrades
14:57:02 <rosmaita> that is the idea, i don't know that it's that simple
14:57:29 <rosmaita> hopefully there will be a ptg discussion about this before the TC approves it
14:57:49 <rosmaita> #topic Call for Outreachy OpenStack mentors - May 2022 round
14:57:57 <rosmaita> #link http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027239.html
14:58:14 <rosmaita> that email has some useful links about being a mentor and what the commitment is
14:58:27 <enriquetaso_> :D
14:58:40 <rosmaita> i think there is some cinder stuff here: https://wiki.openstack.org/wiki/Internship_ideas
14:58:48 <rosmaita> probably needs to be updated
14:59:04 <rosmaita> enriquetaso_: can you talk a little more about this next week?
14:59:17 <enriquetaso_> yes!
14:59:20 <rosmaita> (because we are running out of time)
14:59:25 <enriquetaso_> sure
14:59:29 <rosmaita> #topic stable releases
14:59:38 <rosmaita> looks like everything has merged?
14:59:43 <whoami-rajat> yes
14:59:52 <rosmaita> cool, so nothing to do here
14:59:57 <whoami-rajat> apart from the dell patches in Xena which doesn't has a reply since past 2-3 weeks
15:00:03 <rosmaita> well, nothing for us, whoami-rajat has to post the patches
15:00:05 <whoami-rajat> so we will be skipping those in this release
15:00:15 <rosmaita> seems fair enough
15:00:26 <rosmaita> we are out of time
15:00:29 <whoami-rajat> will post the patches and update in the etherpad
15:00:29 <rafaelweingartner> Abount Xena, we would like to ask you guys about the patch: https://review.opendev.org/c/openstack/cinder/+/812685. Would you guys ind reviewing it?
15:00:30 <rosmaita> thanks everyone!
15:00:38 <whoami-rajat> also thanks for updating the etherpad rosmaita
15:00:41 <rafaelweingartner> mind*
15:01:04 <jungleboyj> Thank you!
15:01:52 <rosmaita> rafaelweingartner: ack, we have several issues around that, so i will be interested to see what you have come up with
15:02:07 <rosmaita> #endmeeting