16:00:10 #startmeeting cinder 16:00:11 Meeting started Wed Jun 6 16:00:10 2018 UTC and is due to finish in 60 minutes. The chair is smcginnis. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:12 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:00:14 The meeting name has been set to 'cinder' 16:00:16 #topic Rollcall 16:00:19 ping DuncanT diablo_rojo, diablo_rojo_phon, rajinir tbarron xyang xyang1 e0ne gouthamr thingee erlon tpsilva patrickeast tommylikehu eharney geguileo smcginnis lhx_ lhx__ aspiers jgriffith moshele hwalsh felipemonteiro lpetrut lseki _alastor_ 16:00:30 hi 16:00:47 hey 16:00:48 hello 16:00:52 hi 16:01:03 #link https://etherpad.openstack.org/p/cinder-rocky-meeting-agendas Agenda 16:01:21 Hey everyone. I will be your jungleboyj for today's festivities. 16:01:32 :P 16:01:45 hi 16:01:53 smcginnis: will you argue with yourself about Thursday? 16:02:01 e0ne_: Hehe :) 16:02:05 hi 16:02:34 OK, guess we can get going... 16:02:37 #topic Announcements 16:02:47 Rocky-2 milestone deadline is tomorrow. 16:02:50 #link https://releases.openstack.org/rocky/schedule.html 16:03:05 I need to actually check with jungleboyj on that, but I think we are in good shape. 16:03:37 Jay has also put up a set of patches marking drivers as unsupported for lack of CI. 16:03:45 #link https://review.openstack.org/#/q/status:open+project:openstack/cinder+branch:master+topic:ci_unsupported 16:04:23 I probably should add it as another agenda item to discuss, but I wanted to point out that one of the drivers in that set is the Brocade FCZM driver. 16:04:35 smcginnis, jungleboyj: please, don't forget about releasing python-brick-cinderclient-ext for Rocky 16:04:53 Personally, I would like to hold off on that one for awhile since we did give Cisco a lot of time for theirs. 16:04:55 smcginnis: yes we are good. I will tag today. 16:05:04 So if you have any opinions on that, please comment on the review. 16:05:07 jungleboyj: Great. 16:05:07 smcginnis: it means, we'll drop zomenanager... 16:05:22 o/ 16:05:28 e0ne_: Well, ironically, the Cisco one is now reporting regularly. 16:05:42 smcginnis: oh, it's a good news 16:05:44 That was my fear as well, but now we at least have one. 16:05:50 e0ne_: will do. 16:05:56 jungleboyj: thanks! 16:06:40 python-brick-cinderclient-ext is cycle-with-intermediary, not cycle-with-milestones, so at least process wise that could wait. 16:06:48 But still good to get the recent fixes released. 16:07:31 smcginnis: +1 16:08:14 That's all I had for announcements. Anyone know of any important announcements to bring up? 16:09:01 #topic Rocky Priorities 16:09:09 #link https://etherpad.openstack.org/p/cinder-spec-review-tracking 16:09:34 hi 16:09:36 I haven't reviewed this list too closely lately, but please take a look to see if anything needs review focus. 16:09:48 kien-ha: Welcome 16:10:02 There are a few new driver patches outstanding, but tomorrow is the deadline to have those merged. 16:10:16 And last I looked, most of the outstanding ones did not have CI up and stable yet. 16:10:32 :( 16:11:14 On the plus side, there were a few that were added, so that's nice to see. 16:11:54 +1 16:12:20 #topic Check-in on HA development progress 16:12:30 geguileo: I believe this is the weekly prod. :) 16:12:47 smcginnis: lol 16:13:00 smcginnis: I only have 1 comment from tommylikehu in the devref 16:13:13 geguileo, SHOULD HAVE SOME MINE TOO 16:13:14 I'm still waiting for other reviews 16:13:18 opss 16:13:18 erlon add more 16:13:26 lol 16:13:28 erlon_: oooops, I must have missed it 16:13:30 my bad 16:13:32 #link https://review.openstack.org/#/c/571242/ 16:13:54 erlon_: Oh, you added them today 16:14:04 sorry, it's been a crazy day 16:14:06 geguileo: Should it be un-WIPd and doc build failures fixed? 16:14:13 Just thinking that might get more reviews then. 16:14:25 doc failures are my todo notes 16:14:26 usually does 16:14:41 I'd like to get some more input 16:14:49 and I'll go over current comments to fix things 16:14:58 so a couple more eyes would be great to have 16:14:59 OK, please take a look and give geguileo feedback. 16:15:30 geguileo: Anything else we should discuss on the subject? 16:15:39 not at the moment 16:15:48 geguileo: OK, thanks! 16:16:00 #topic Follow-up Topics from the Forum 16:16:05 First, placement... 16:16:15 #link https://etherpad.openstack.org/p/YVR-cinder-placement 16:16:34 still didnt have time to spent on that :/ 16:16:38 Really wish this would have been recorded since there weren't a lot of Cinder folks there. 16:17:01 But I think most of the concerns raised were addressed and answered. 16:17:28 The biggest concern for me being the ability to design its use (if we choose to) such that we can run without it. 16:17:31 In a clean way. 16:17:53 what's the big win for us? 16:18:20 we don't have that many things to track... 16:18:27 Having a common external data source rather than reimplementing what they have done into local DB access. 16:18:29 geguileo, can you recap why you -2 on the placement patch? 16:18:31 and they are going to do the tracking like we do, in the DB 16:18:38 erlon_: sure 16:18:41 Anyone have a link to that patch handy? 16:19:17 https://review.openstack.org/#/c/559718/ 16:19:17 https://review.openstack.org/#/c/559718/ 16:19:17 #link https://review.openstack.org/#/c/559718/ 16:19:20 :) 16:19:24 Hah! 16:19:31 lol 16:19:34 it's easy to find out the -2 one from my lists 16:19:37 smcginnis: please, choose only one 16:19:44 Thanks. :D 16:20:19 Well, I don't want to spend too much time on it in the meeting, but just reminder that we should continue that discussion. 16:20:21 My concerns are that we are going to introduce a new component that its basically doing what we were going to do in the schedulers to fix this 16:20:28 And probably address some of the comments on the patch. 16:20:44 They are using the DB to avoid races and to have a single source of truth 16:20:48 Same thing we were going to do 16:20:59 but now we'll have an extra service (or N on HA) 16:21:10 Which uses more resources and introduces additional delays 16:21:14 geguileo: But rather than needing to implement all of the logic they already have in our own scheduler, when running in HA we can defer that to placement. 16:21:28 geguileo, I think the same 16:21:43 I would assume large deployments that are going to care about HA are going to be running it anyway for Nova, so it's one more config setting. 16:22:11 CERN has 20 instances running, for instance. 16:22:24 and one more interconnection source of problems for us 16:22:52 Vs more complexity in our scheduler. 16:23:06 Anyway, a lot of tradeoffs to consider here. 16:23:07 smcginnis: +1 16:23:11 e0ne_, you sent a spec for implementing that in the DB, how much change do you thing would be needed? 16:23:25 it does not seem too much 16:23:36 erlon_: it was just an idea before I read about placement 16:23:43 e0ne_: Do you have a link to that patch too? 16:23:50 Would be good for folks to get the full picture. 16:23:54 I thinks we shouldn't re-invent the wheel 16:24:05 #link https://review.openstack.org/556529 16:24:10 https://review.openstack.org/#/c/556529/ 16:24:11 Thank you 16:24:23 I'll abandon my patch in flavor of placement support 16:24:57 * geguileo fears this is going to be as good for Cinder as privsep was 16:25:24 e0ne_, Im just trying to measure the impact/size of both changes. If we are doing something Nova already doing in placement, but that is minimal, duplicating code it wouldnt be that much a concern 16:25:31 geguileo: Do you view privsep as a bad thing? 16:25:54 smcginnis: privsep for OS-Brick has been the worst thing that has ever happened to us 16:26:03 it serializes our requests 16:26:13 as it can only run 1 thing at a time 16:26:25 geguileo: I don't think those involved in the lock-setup upgrade of rootwrap changes would agree. 16:26:56 smcginnis: but anyone doing multipathing in the real world would 16:26:57 Anyway, let's move on since that was just one item in the list. :D 16:27:03 ok 16:27:08 ok 16:27:10 HA Active/Active work 16:27:15 #link https://etherpad.openstack.org/p/YVR18-cinder-ha-forum 16:27:23 Not sure if there was too much here to discuss. 16:27:31 The need for docs was raised. 16:27:47 One interesting question to me was someone asking about HA A/A for the backup service. 16:27:55 I couldn't agree more with the docs comment XD 16:28:00 ;) 16:28:05 Backup already supports HA A/A 16:28:15 Nothing to do there, job done 16:28:17 rofl 16:28:20 :) 16:28:30 Hah, I guess that points back to the need for more docs then. 16:29:14 The other interesting/concerning thing in the session was an assumption by someone that HA A/A would provide performance scaling. 16:29:31 I think we clarified that, but I was surprised by it. 16:30:18 it could, since you have now multiple privseps 16:30:31 geguileo: Haha, OK, fair enough. :) 16:30:37 };-) 16:30:41 Cinder Documentation Work 16:30:43 hehe, nice 16:30:51 #link https://etherpad.openstack.org/p/YVR18-cinder-documentation-forum 16:31:05 I believe Jay is planning on devoting some time to this. 16:31:32 We also had someone from Red Hat there that said their doc team was switching how they do things and would be getting more involved upstream. 16:31:47 Which would be really nice. I forget the number, but they had a good number of people. 16:32:24 One of the biggest issues raised was that the current docs have a lot of technical detail (when we actually have docs on something) but doesn't really guide folks on what they need and why. 16:32:56 So I think they were going to work on making things more task driven rather than just a dump of config options and CLI examples. 16:33:24 There was also the need for more standalone Cinder documentation. 16:33:34 And how/where/why you would want to use that. 16:34:59 I think that's all from the Forum. 16:35:24 Would have been useful to have more Cinder folks present, but we'll see what happens now with the talk about events. 16:35:42 I will say, this one seemed a lot more Design Summit-ish than the last few. 16:36:02 But maybe that was partly flashbacks from the last Summit in Vancouver. :) 16:36:27 #topic Continued discussion on Reusing Cinder Drivers/cinder-lib 16:36:50 I was hoping we would have jgriffith and jungleboyj here. 16:36:54 lol, that is a fase in for the return of the joint summits/ptg 16:37:05 erlon_: Hah, maybe. 16:37:14 geguileo: The floor is yours. 16:37:35 * jaypipes shakes fist at other jungleboyj for having same first name. 16:37:44 I know that Hellen tested cinderlib 16:37:52 jaypipes: sorry. 16:37:54 and it worked for VMAX on Rocky 16:37:58 jungleboyj: :) 16:38:02 walshh? 16:38:11 smcginnis: yup 16:38:12 I am kind of peripherally here now. 16:38:21 yes, very happy with it 16:38:23 anybody else had time to look at it? 16:38:37 unfortunately, not :( 16:38:39 Seems people are all over on supporting it. 16:38:41 #chair jungleboyj 16:38:42 Current chairs: jungleboyj smcginnis 16:38:46 walshh, did it worked cleanly? 16:38:52 I hope to have some time to play with it next week 16:39:00 e0ne_: awesome! 16:39:08 e0ne_, +1 16:39:10 pretty much...since we have a rest interface to the VMAX we have no external dependencies 16:39:17 me too 16:39:29 walshh: what did you do? 16:40:03 geguileo: have you updated your doc on what drivers are tested? 16:40:30 it was very easy to plug it in....very little intervention, just follow the readme 16:40:43 xyang: no, I have to do that 16:41:10 xyang: I'll also try to create instructions for each backend 16:41:33 geguileo, is there changes from BE to BE? 16:41:36 walshh: could you send me the config you used (masking passwords and ips) so I can add it? 16:41:52 sure 16:41:54 geguileo: sounds good 16:42:07 erlon_: what's BE? r:-??? 16:42:12 backend 16:42:18 rofl 16:42:34 only the configuration passed to the init changes 16:42:44 ok 16:42:53 depending on the backend I may need to fix something 16:43:01 like I had to do for Solidfire 16:43:10 that required project_id and user_id to be filled in 16:43:35 erlon_: anything that doesn't work an expected, just let me know and I'll fix it 16:43:53 geguileo, hmm, I think some drivers also use volume type fields 16:43:58 how that is handled? 16:44:08 sure 16:44:23 erlon_: I'd have to check, I think it's a silly volume type with no extra specs 16:44:55 right now it's only covering the most basic case 16:45:02 ok 16:45:05 later I'll add support for fancy stuff through the extra specs 16:46:19 geguileo: are you the only one supporting this? 16:46:28 jungleboyj: right now yes 16:46:33 geguileo: has anyone tested Unity? 16:46:35 it's a POC right now 16:46:47 xyang: I don't think so 16:46:53 basically I've tested 16:47:08 xtremio, qnap, rbd, lvm, kaminario 16:47:15 geguileo: ok. If this starts having impacts on drivers this gets more complicated. 16:47:21 then solidfire, and vmax have been tested 16:47:36 and I did some basic testing on 3PAR 16:47:46 I'll try to test 3PAR and Nimble soon 16:47:56 jungleboyj: the idea is not to impact the drivers 16:48:04 drivers do their thing like they do for Cinder 16:48:11 geguileo: thanks 16:48:28 I don't have access to other storage, so... 16:48:30 lol 16:48:32 So that fixes you're working on are all in Cinder lib. 16:48:44 jungleboyj: what fixes? the driver ones? 16:48:48 yes 16:49:01 Ok. That is good. 16:49:06 if there's a problem using cinderlib with a driver, what I fix is cinderlib 16:49:11 because it's the one at fault 16:49:41 geguileo, that is, if the driver works on Cinder :p 16:49:46 geguileo: cool. 16:49:51 erlon_: precisely!!! 16:50:10 erlon_: it's a good reference for me to know that it's my fault ;-) 16:50:25 New third party CI requirement? 16:50:31 lol 16:51:04 Half joking, but half serious. The only way to know it would work right with all the drivers is if all the drivers are tested with it. 16:51:32 smcginnis: True. 16:51:34 I think we could add it to the tempest run 16:51:46 and have it log the results but never fail even if it fails 16:51:52 But I think we have a ways to go for that yet. 16:51:53 it would only take 1 minute or so more 16:51:54 smcginnis, if that still is a PoC does not make sense to even think about that now 16:51:58 since everything is already deployed 16:52:00 make it non-voting 16:52:38 erlon_: Not now, but we are talking about making this a real CI'd thing in gate. 16:52:44 The problem if we add a new job is that it requires deploying things, whereas running it in the same tempest job would skip the deployment time 16:53:08 but we don't need tempest for that, do we? 16:53:16 geguileo: That is a good thing. 16:53:17 tpsilva: we don't 16:53:32 smcginnis, and we would test against LVM? 16:53:33 tpsilva: but if you are running tempest, then you already have everything you need in the controller node 16:53:45 smcginnis: we could test agains LVM and Ceph 16:54:34 geguileo: that would be a minimum starting point. 16:54:50 jungleboyj: OK, will start looking into that 16:55:06 geguileo: Thanks. 16:55:18 thank you all :-) 16:55:26 smcginnis: I know you have concerns ... let thoughts? 16:55:34 *other 16:56:37 Nah, nothing right now. 16:56:59 * erlon_ has a quick topic 16:57:06 One more topic on the agenda and ~3 minutes left. 16:57:20 #topic Enabling online extend tests by default on devstack 16:57:24 Ok. So plan going forward is geguileo will keep working on bugs and look into getting testing set up? 16:57:52 jungleboyj: +1 16:57:55 jungleboyj: sounds reasonable 16:58:07 so, we are considering adding the online extend tests to run by default on tempest 16:58:07 erlon_: Type quick! :) 16:58:10 :) 16:58:16 should be quick 16:58:24 #link https://review.openstack.org/#/c/572188 16:58:40 That makes sense to me. Anyone know of drivers that do not support this? There shouldn't really be. 16:58:46 we support online extend by default, but does not test it 16:59:00 Makes sense to test it. 16:59:05 smcginnis, some NetApp 16:59:14 we are working on the fix 16:59:29 erlon_: will it cause issues for them? 16:59:32 erlon_: Oh, right. 16:59:47 will it work for nfs? 16:59:57 e0ne_, yes 16:59:58 It would at least be good to know who needs to explicitly disable this. 17:00:07 erlon_: great 17:00:13 smcginnis: ++ 17:00:15 Should maybe send to the ML to get attention to it. 17:00:15 smcginnis: +1 17:00:18 Times up. 17:00:31 smcginnis: thanks for running things. 17:00:38 Thanks everyone. Sorry we didn't have more time to discuss erlon_, but I think we can continue later. 17:00:45 #endmeeting