16:07:49 #startmeeting cinder 16:07:50 Meeting started Wed Dec 5 16:07:49 2018 UTC and is due to finish in 60 minutes. The chair is jungleboyj. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:07:51 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:07:53 The meeting name has been set to 'cinder' 16:08:08 Hi again 16:08:12 o/ 16:08:16 hi 16:08:31 courtesy ping: jungleboyj diablo_rojo, diablo_rojo_phon, rajinir tbarron xyang xyang1 e0ne gouthamr thingee erlon tpsilva ganso patrickeast tommylikehu eharney geguileo smcginnis lhx_ lhx__ aspiers jgriffith moshele hwalsh felipemonteiro lpetrut lseki _alastor_ whoami-rajat yikun rosmaita 16:08:48 o/ 16:08:53 #topic announcements 16:08:55 <_alastor_> o/ 16:08:56 hi 16:09:51 So, the only announcement I have is that I have sent the responses to the User Survey Feedback that we put together to Superuser. 16:10:13 #link https://docs.google.com/document/d/1d4_nUfuZacGABG2hqqCz3hC238uoVqdloi_qVGRNoRY/edit?usp=sharing 16:10:50 Nicole made it sound like they have quite a few things to process yet so it will probably take them a bit to get to it. So, if you want to take another look and/or make comments, please do so. 16:11:33 o/ 16:11:37 Any questions or comments there? 16:12:13 nope, looks good, thanks for putting it together 16:12:26 rosmaita: No problem. Thanks for looking at it. 16:12:38 One other announcement I suppose. 16:12:46 We are coming up on milestone 2 for the release. 16:13:11 We have a couple of driver submissions so we should be trying to get them some bandwidth to review those in time. 16:13:21 Would greatly appreciate any assistance doing reviews there. 16:14:03 A list of the ones I know about can be seen here: 16:14:07 #link https://etherpad.openstack.org/p/cinder-spec-review-tracking 16:15:24 Ok. I think that is it for announcements. 16:15:55 #topic shared_targets_online_data_migration fails when cinder-volume service not running 16:16:05 imacdonn: You here? 16:16:52 Guess not. 16:17:15 #link https://bugs.launchpad.net/cinder/+bug/1806156 16:17:16 Launchpad bug 1806156 in Cinder "shared_targets_online_data_migration fails when cinder-volume service not running" [Undecided,New] 16:17:26 jungleboyj: I mentioned that issue at the PTG 16:17:36 when complaining about the issues on the shared_targets implementation 16:17:53 geguileo: Ok, I thought it at least sounded familiar 16:17:56 and I added the comment from the first link 16:18:15 to our code, to make sure we don't do it again: https://github.com/openstack/cinder/commit/2cd5957c5e891f0bc5cf57253c0f5e18b330e954 16:19:23 Ok. So what do we need to do to fix the existing problem? 16:19:41 it's a pita 16:20:03 because there is no way to do this offline 16:20:28 Ugh. Ok. 16:21:17 well, there may be a way 16:21:40 Can we fix shared targets as suggested in the bug to avoid the limitation? 16:22:00 Given that it looks like we didn't follow the spec. 16:23:07 well, that part of the spec is also against the whole rolling upgrades/online migration concept 16:23:32 :-( 16:23:40 in theory the volumes should have been migrated as they are accessed 16:23:59 Ok. 16:24:04 and there should have been a mechanism to do the online migrations without the services being online 16:24:56 I feel like this is going to be a problem that we are going to keep hitting in other places in the future. 16:25:40 well, if we implement things right it won't be aproblem 16:25:47 So, how should we move forward? 16:25:49 He he. 16:25:55 Isn't that always the case? 16:26:22 lol 16:26:48 if we don't care about overloading the DB on start, we can do the proposed solution 16:27:12 So, the root of the problem on the upgrade here is the fact that we are trying to query the shared_targets setting during the upgrade? 16:27:31 Accessing the volumes through the volume service? 16:27:58 yup 16:28:29 Ok. So, the proposed solution isn't unprecedented. Right? We query volumes for other things at volume service startup. 16:28:39 Checking if they are in deleting status and stuff like that. 16:28:47 well, it's against what should be done XD 16:28:52 I don't know if there's precendent 16:29:08 geguileo: Why is it against what should be done? 16:29:12 the solution is basically implement a data migration on start 16:29:29 on EVERY start of the service 16:29:47 FOREVER 16:29:56 Ok. I wonder how Nova has dealt with this? 16:30:05 * jungleboyj is scared by the all caps forever 16:30:30 unless we use a config option or store something in the DB to disable it once it's done 16:30:41 geguileo: That was what I was going to ask. 16:30:48 because you can upgrade from N to N+2 and will still want that field to be set correctly 16:30:57 We have access to the DB obviously during the migration. 16:31:23 the proper solution would have been to make the migration online when the volumes were loaded 16:31:33 Seems like we should be able to indicate to the DB that a migration has happened and that the volume service needs to do something on the next start. 16:31:48 and store in the service row if the backend required shared or not 16:32:00 and then on the next release use that field from the DB to do it 16:32:32 But we can't go back and fix that at this point. 16:32:39 nop 16:32:57 somebody has to go and figure out a decent solution 16:33:48 Ok. 16:34:23 I need to understand the urgency of this problem. Guess that isn't clear yet. 16:34:41 we should ask in the bug who/what is affected by this 16:34:56 Do we need to solve it today or can we put this on the agenda for the PTG and try to work it out when some of us are in a room. 16:35:08 Ok, yeah, hard to understand that without imacdonn here. 16:36:07 Lets start there. I will update the bug after the meeting and see if we can get more input. Follow up in next week's meeting when hopefully Iain can be here. 16:36:14 Sound like a plan? 16:36:32 +1 16:36:35 Cool. 16:36:51 #action jungleboyj to update the bug and we will follow up in next week's meeting 16:37:36 #topic remove policy checks at DB layer? 16:37:40 rosmaita: 16:37:45 i'll be quick. i thought removing the policy checks at the db layer was a no-brainer, but i am having second thoughts 16:37:53 :-) 16:37:56 anyway, since it's already on the agenda, let me explain what's up 16:38:05 here's the use case: an operator wants to have a "read-only" admin who can do audits but not make changes 16:38:15 if you try to do this in the policy file: 16:38:15 "some-delete-call": "rule:admin_api" 16:38:15 "some-get-call": "rule:admin_api or role:observer-admin" 16:38:23 the some-get-call fails when a non-admin user with role:observer-admin makes the call 16:38:35 it's traceable to the db layer where we have a decorator @require_admin_context 16:38:45 so we could eliminate that decorator ... but it decorates 97 functions in db/sqlalchemy/api.py 16:38:54 that seems pretty risky 16:39:04 plus, i've been thinking about this for a while, and it turns out that doing that won't completely address the use case anyway 16:39:22 the reason why not is that if you want your read-only admin to see stuff like the admin metadata on a volume, that person has to be a "real" admin (in the sense of having a property defined in context_is_admin in the policy file that will make is_admin:True hold) 16:39:35 so to handle that situation, you need to do an unsafe workaround that i was hoping to fix 16:39:46 it's unsafe because if you want a read-only admin, you have to give that person a role that fits into how context_is_admin is defined, which makes that person a serious admin 16:39:58 and then you have to plug up all the holes by adding something like "not role:observer-admin and ..." to each of the policies you *don't* want them to have 16:40:13 i don't see any way around that 16:40:22 anyway, i wrote up an etherpad while i was trying to figure this out 16:40:33 #link https://etherpad.openstack.org/p/different-types-of-admin 16:40:41 i'd appreciate it if anyone interested could read through and see if there's something i missed or if i'm being stupid 16:40:47 Yikes. That all sounds kind of scary for what would be a limited use case. 16:40:53 but i think my proposal at this point would be to "fix" this via documentation 16:41:02 and not touch the code at all 16:41:07 rosmaita: does the context store the roles of the caller? 16:41:25 geguileo: yes, pretty sure it does 16:41:38 rosmaita: At this point in time that sounds like the safest solution. 16:41:41 is it really a limited use case? i think the use case is "anyone who wants to adjust policy to give users access to certain things they don't normally have" 16:41:41 rosmaita: "fix" by documentation means that it's not possible? or what does it mean? 16:42:11 the workaround works, i could just write up an example of how to do this 16:42:24 eharney: I hadn't interpreted it that broadly. 16:42:30 the key thing is that the operator's responsibility to test carefully 16:42:35 rosmaita: the workaround is to add all those "not role:observer" rules? 16:42:48 geguileo: yes 16:42:58 and possibly more, depending on how fine-grained you want it 16:43:02 rosmaita: what about setting a Cinder conf for the admin-observer role name? 16:43:10 i think it is that broad unless i missed something here? 16:43:12 rosmaita: then check it in Cinder? 16:43:30 i think that wouldn't work in the long run 16:43:40 because you might also want a creator-only-admin 16:43:48 (i mention this in the ehterpad) 16:44:02 i think what we have now is flexible enough 16:44:09 I think that's different 16:44:10 operators just have to be careful 16:44:34 or do you mean you want to allow 1 person to do an admin create call that allows to create volumes like an admin? 16:45:11 geguileo: what i mean is an admin who can create everything an admin can, but can't delete anything 16:45:26 sort of a role for interns :) 16:45:40 He he he. 16:45:52 rosmaita: is that a real use case somebody has asked for? 16:46:07 yes 16:46:09 because I know the admin observer role is requested by many people 16:46:37 yeah, the observer came up this morning in the glance channel 16:46:38 the admin create only role seems less needed (from my point of view) 16:47:08 rackspace has the admin/observer/creator defined 16:47:17 even for normal user accounts 16:47:27 I think the admin can create but cannot delete should be resolved as you discussed, with the "not role:" rules 16:47:57 create is one thing, another is create as an admin 16:48:01 rosmaita: Could we start by taking something like rackspace has and document it in our documentation and see how people feel about that? 16:48:23 Point people to that if they are asking. 16:48:32 jungleboyj: i don't know if we want to go that far, but i have a writeup for observer-admin 16:48:44 don't want to go beyond that without a lot of tests! 16:48:47 I think we should have the observer role without so much work... 16:48:48 Then if there is enough demand readdress the risk/reward of changing Cinder. 16:48:54 although there's always the "no warranty" disclaimer 16:49:00 :-) 16:50:00 rosmaita: So, start by formalizing that documentation?> 16:50:03 we can do all those now, it's just that it's a little error-prone 16:50:20 jungleboyj: ok, i can put up a patch 16:50:39 rosmaita: Lets start there. 16:50:43 Any objections? 16:51:15 Ok. Cool. 16:51:20 here's that etherpad again: https://etherpad.openstack.org/p/different-types-of-admin 16:51:29 rosmaita: Thanks for bringing this up. 16:51:36 np 16:51:43 #action rosmaita to push up a patch with documentation 16:52:32 #topic update on possible mid-cycle 16:52:50 #link https://etherpad.openstack.org/p/cinder-stein-mid-cycle-planning 16:53:05 So, I brought this up with my management and we have budget to do this. 16:53:12 So, Lenovo can host. 16:53:29 nice! 16:53:42 The caveat is that our campus is undergoing a lot of construction so we would have to deal with that and possible parking challenges. 16:54:09 Is that a big enough deal to make anyone else in RTP volunteer to host? 16:55:07 Beuhler .... Beuhler 16:55:20 i am up for parking challenges 16:55:24 Guess that is a no. 16:55:53 Ok, like I said last time this would probably be pretty bare bones but we would have a place to meet and internet. 16:56:54 And the week proposed works for those people in RTP? 16:57:09 eharney: rosmaita _hemna jbernard 16:57:31 I know that smcginnis and I can get there as we planned the date together. 16:57:32 yes for me 16:58:04 Ok. I will keep moving forward with the process at Lenovo then. 16:58:24 Ok. Finaly topic. 16:58:43 3rd Party CI requirements for connectors 16:58:49 mszwed: You here? 16:58:52 Hi 16:58:55 I'm currently working with Mellanox on continuos integration setup for new SPDK NVMe-oF volume and target drivers. I want to know which tests should I run on this setup. First approach was to run same tests as for current nvmet driver, but there are also different opinions. 16:59:23 #link https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#Volume_.26_Connector_Drivers 16:59:56 So, I think the answer is: tox -e all -- volume 16:59:58 i'm not sure that faq is correct... 17:00:14 Ugh. We are out of time already. 17:00:15 and we clearly need to sort this out since there have been numerous questions lately about what tests are needed 17:00:25 eharney: Agreed. 17:00:35 ok 17:00:38 it needs to instruct people to run the cinder tempest plugin tests too :/ 17:00:52 Lets make this the first topic for next week. 17:01:13 eharney: Right, but I don't think we can require that right now as we haven't spread the word on that yet. 17:01:45 mszwed: Can you shoot for what I shared above and we will discuss more next week? 17:01:52 sure 17:01:58 Great. Thank you! 17:02:03 :) 17:02:08 Thanks everyone for joining. 17:02:14 Sorry for the network issues I had. 17:02:20 Thanks. 17:02:20 #endmeeting