14:00:33 <pdeore> #startmeeting glance
14:00:33 <opendevmeet> Meeting started Thu May 19 14:00:33 2022 UTC and is due to finish in 60 minutes.  The chair is pdeore. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:33 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:33 <opendevmeet> The meeting name has been set to 'glance'
14:00:33 <pdeore> #topic roll call
14:00:34 <pdeore> #link https://etherpad.openstack.org/p/glance-team-meeting-agenda
14:00:43 <pdeore> o/
14:00:47 <dansmith> o/
14:01:23 <mrjoshi> o/
14:01:35 <pdeore> lets wait few minutes for everyone to join
14:01:35 <jokke_> o/
14:02:46 <pdeore> abhishekk, and rosmaita are busy in resolving the broken gate issue
14:03:03 <rosmaita> well, "resolving" is a bit strong
14:03:13 <rosmaita> more like trying to figure out WTF
14:03:52 <abhishekk> yeah :/
14:04:07 <pdeore> yeah, of course..
14:04:09 <abhishekk> unfortunately last couple of days we are hitting with very strange errors
14:04:55 <abhishekk> if anyone interested to have a look, this is the patch which reported the error, https://review.opendev.org/c/openstack/glance/+/842400
14:05:14 <pdeore> ack
14:05:26 <pdeore> can we start ?
14:05:34 <croelandt> yes!
14:05:38 <pdeore> #topic release/periodic jobs
14:05:57 <pdeore> It's  M1 Release Week and we have released glance-store 4.0.0 yesterday
14:06:08 <pdeore> #link https://review.opendev.org/c/openstack/glance/+/842400 is causing failure in glance gate
14:06:25 <pdeore> so, we might skip glance M1 release if gate is not fixed today
14:07:53 <abhishekk> correction: above patch is not causing failure, above patch has reported the failure
14:08:28 <pdeore> ack
14:09:11 <pdeore> Periodic jobs all green except some oslo tips jobs failure for python3.6
14:09:19 <pdeore> #link https://review.opendev.org/c/openstack/glance/+/841350
14:09:19 <pdeore> #link https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/841368
14:09:30 <pdeore> Still waiting to get these patches merged
14:10:03 <pdeore> I have requested the infra people to look after the openstack-zuul-jobs patch as I'm not much familiar with those job changes and the comments received on the patch
14:10:45 <pdeore> moving ahead
14:10:50 <pdeore> #topic cache-update response code
14:11:32 <pdeore> #link https://review.opendev.org/c/openstack/glance/+/841278/1#message-456096e48b28e5b866deb8bf53e9258ee08219a0
14:11:39 <abhishekk> jokke_, there are some suggestions on the above backport, so can you reconsider this vote
14:12:41 <jokke_> abhishekk: I noticed
14:13:07 <abhishekk> ack
14:13:22 <jokke_> rosmaita: just a point to what you said, the spec you linked indeed called the 202, the _documentation_ is changed to that with the patch and stated 200
14:13:40 <rosmaita> yeah, but the original doc, the spec, said 202
14:13:56 <rosmaita> which is documentation
14:14:24 <rosmaita> and actually is supposed to be the design doc
14:16:35 <jokke_> yes I totally agree that there's been oversight on implementation and reviews. Just a note that we've been blocking these fixes even on master in past ... Like if we want to break the stable that discussion needs to happen in the mailing list
14:16:57 <jokke_> it's not something we should just go and do no matter how convenient it is for us
14:19:12 <dansmith> I have no problem expressing my support for this route to resolution on the mailing list, if it will help
14:19:38 <dansmith> not sure it's really necessary, but if it helps to just have the same conversation we've had on the ML that's fine with me, but I think we should try to do it quick
14:19:51 <dansmith> as the longer the wait the worse it is
14:19:55 <abhishekk> +1
14:21:31 <rosmaita> exactly
14:21:38 <jokke_> so if the stable maint team greenlights that api change, we also should bump the api version and add reno for it as I mentioned in my original review. just silently changing it is really bad practise
14:23:51 <rosmaita> you mean bump in yoga?
14:24:07 <dansmith> I thought it was just bump in master,
14:24:12 <jokke_> well bump in master and squash them together for backporting
14:24:15 <dansmith> but still backport to yoga
14:24:28 <dansmith> so bump the api in yoga too?
14:24:49 <jokke_> well if we backport the change, then yes
14:25:04 <dansmith> if we were making some real change that people were likely to struggle with I'd agree with much more caution, but I think we can be rational about this specific change and need not overreact, IMHO
14:25:15 <jokke_> the api version glance is reporting is there to indicate changes in the api
14:26:40 <jokke_> dansmith: I just think it's quite slippery slope to not treat every API breaking change at same severity
14:27:31 <dansmith> it is a slippery slope, which is why we should be cautious and critical about exceptions.. however, this seems like a good candidate for an exception to me
14:28:22 <jokke_> agreed, and there is documented process for it ;)
14:28:54 <dansmith> rosmaita: you wanna send an email to the list detailing what we discussed, ask for comment and we can go from there?
14:29:06 <rosmaita> i can do that
14:29:49 <jokke_> cheers
14:30:37 <pdeore> can we move ahead to next topic?
14:30:43 <jokke_> ++
14:30:47 <pdeore> #topic new location APIs
14:30:55 <pdeore> #link  https://review.opendev.org/c/openstack/glance-specs/+/840882
14:31:09 <pdeore> the spec is submitted by whoami-rajat
14:31:09 <whoami-rajat> hey
14:31:34 <whoami-rajat> so we discussed this idea at the PTG at glance cinder cross project session
14:31:39 * jokke_ is piling up for today's meeting ;)
14:31:52 <whoami-rajat> and as discussed i proposed a spec implementing the same idea
14:32:04 <whoami-rajat> but there are some concerns by jokke_ , which I've replied to
14:32:15 <whoami-rajat> just wanted to bring some attention to it
14:32:31 <whoami-rajat> I'm happy to continue the discussion either here or review but just want to get things moving
14:33:38 <jokke_> I expressed the concern for admin scoped api for this purpose already during the PTG discussion and after reading through the proposal I think it's very bad idea.
14:34:44 <rosmaita> can you explain that some more?
14:34:50 <jokke_> dansmith: I don't know if you have read through the spec yet. How do you feel about getting Nova storing adming credentials in configs and moving to use the new API for the ceph stuff? Will it ever happen?
14:34:51 <whoami-rajat> we need this API for both service-service interaction and also for operators, I'm not sure what a good alternative is
14:35:20 <dansmith> jokke_: I haven't read this spec, but will after the meeting
14:35:47 <whoami-rajat> we've kept the credentials for nova cinder interaction since forever and it doesn't have any side effects, as far as I've seen
14:35:56 <dansmith> sounds like this might be a thing for the service role, but yeah currently nova has no special creds for glance and obviously it'd be a change to start doing that
14:36:07 <whoami-rajat> and the credentials can be removed once service role is in place
14:36:29 <dansmith> right, service role would be ideal for this sort of thing
14:36:49 <dansmith> (just based on a quick skim)
14:36:53 <whoami-rajat> but again, I'm not sure what the timeline of that being available is, I already have a feature dependent on it stuck since yoga ...
14:37:03 <whoami-rajat> dependent on the service role
14:37:58 <dansmith> I'm not sure either as we just had some keystone discussion about it in the last week
14:38:32 <jokke_> rosmaita: my main concerns are storing the admin credentials all over the place as we very well know OS has no limitation for them. We're expecting to use them for API that's main purpose is to go around gapi on image data handling and thus we also loose any authorization in glance side for the location operations
14:38:40 <dansmith> adding admin credentials in the meantime would definitely be a big (and unfortunate) hammer, but I really haven't read the spec so ...
14:39:01 <whoami-rajat> that's my concern, we can't halt all work for keystone to make it available and can try to adapt current alternatives until then, but that's just my thinking
14:39:38 <jokke_> And we're aiming this thing to replace the current locations calls, meaning that every change on them needs to be coordinated between Nova (Never easy to get changes in), Cinder and Glance
14:39:46 <rosmaita> well, we could implement it for a 'service' role, and if people want to use it, they'll have to add it themselves
14:40:08 <dansmith> yeah, we really don't have to wait for keystone for service role stuff I think,
14:40:21 <dansmith> especially for specific configs like shared ceph
14:40:46 <rosmaita> so the locations API will be admin or service
14:40:53 <rosmaita> because actual admins will want to use it
14:40:59 <jokke_> Well the service role would still mean that we need to have some level of double authentication from Keystone side
14:41:18 <rosmaita> why double auth?
14:41:22 <whoami-rajat> jokke_, we will be keeping compatibility until all consumers have moved to new APIs, but the current code is not easy to follow and understand as of now
14:41:36 <dansmith> it means nova has a separate set of credentials for using that api, yes, but it does for pretty much every other downstream service already
14:42:30 <jokke_> rosmaita: isn't the whole idea that there is 2 tokens (or double purpose token) issued which effectively states "Service X is doing this operation on behalf of user Y"
14:42:38 <dansmith> no
14:43:09 <whoami-rajat> we've custom commands for locations in glanceclient which calls image-update and we've bunch of nested checks in image update, one of such check is ``show_multiple_locations``
14:43:28 <dansmith> for things like this (again, haven't read spec, but interpolating) nova would use the user's token for things on behalf of the user, but then use its own creds to peek behind the curtain for things like locations
14:43:59 <dansmith> glance wouldn't change other than its policy, and nova would just use a differnent ksclient to make the location calls,
14:44:06 <dansmith> like we do for neutron port bindings and such
14:44:16 <jokke_> dansmith: that's unfortunate
14:44:29 <whoami-rajat> yes, similar to what cinder does to call nova for certain operations
14:44:35 <dansmith> jokke_: unfortunate that this is how it works for every other service?
14:44:57 <dansmith> (genuine question)
14:45:07 <jokke_> dansmith: yes
14:45:28 <dansmith> the unfortunate thing is that right now those are all all-powerful admin users, which is very very unfortunate
14:45:40 <whoami-rajat> just an example of code implementation, priviledged_user is the key parameter here https://github.com/openstack/cinder/blob/master/cinder/compute/nova.py#L84
14:45:42 <jokke_> cause if the intention is not to expose the originating user, it still breaks the audit chain
14:45:53 <dansmith> with the service role being the thing endowed to do specific things, I don't think it's that unfortunate and I'm not sure what the realistic alternative is, based on how our stuff works
14:46:04 <jokke_> and removes the authorisation from the owning service
14:47:01 <dansmith> yeah, which is the purpose of the dual-token thing I think you were referring to earlier (although pejoratively, I thought), but AFAIK, that's not how we do most things today
14:47:49 <jokke_> Cause I think the locations here is pretty darn good example of why you would want the double authentication. You want to confirm that user is not poking the internals on their own, but you're after all changing the project owned resource and want to ensure who ever is requesting that having the rights to do so
14:48:03 <dansmith> anyway, apologies for not having read the spec, but perhaps we can move on and I'll comment on it after this and maybe we can circle back?
14:48:40 <whoami-rajat> I'm ok with it unless the spec gets stuck ...
14:48:46 <whoami-rajat> but i can always bring back the topic next week
14:48:56 <dansmith> jokke_: I dunno, if that api is restricted to admins and services, the audit chain doesn't seem terribly concerning to me.. audit chains are always nice, but nova is really the thing servicing the user and using glance's locations api to do things
14:49:22 <abhishekk> whoami-rajat, also I have concern related to spec
14:49:33 <jokke_> I think this might warrant wider discussion as well and sorry dansmith for throwing you right in. I just thought you might have better insight of how posible it would be to even get nova adobting this
14:49:38 <abhishekk> for new location APIs you are going to add new policies,right?
14:49:50 <whoami-rajat> yes
14:50:15 <dansmith> jokke_: I don't think nova adoption will be a problem because the spec for that will be "do for glance as we do for everything else" but still worth careful consideration, no argument there
14:50:15 <abhishekk> ok, I will also comment my thoughts on the spec
14:50:57 <dansmith> jokke_: I really like that nova->glance is all user-token at the moment :)
14:51:08 <jokke_> dansmith: agreed
14:51:24 <rosmaita> well, that does prompt a question from me
14:51:32 <rosmaita> is OSSN-0065 still an issue?
14:51:45 <jokke_> and the easy way to not expose it to the und users is to deploy service facing gapi and user facing gapi with separate configs and policies
14:51:53 * abhishekk apologies for me being not so active in today's meeting
14:52:06 <whoami-rajat> sure, but if your concern is regarding the policies can be modified for non-admins, the catch here is that these policies won't be shared unlike the current ones which are shared at other places hence the default is allowing members
14:52:23 <whoami-rajat> but we can continue discussion in review, don't want to take up all meeting time
14:52:43 <rosmaita> my question is whether in these modern times, is it still a security risk to expose locations?
14:53:03 <whoami-rajat> rosmaita, as discussed during the PTG, yes and it will be when we starting adding more features that allows more glance cinder cross project dependencies
14:53:14 <whoami-rajat> like rbd cow cloning when uploading volume to image
14:53:38 <jokke_> I just think requiring admin crdentials saved somewhere to be used for API that is by far mostly used for service-service interactions and expecting the consuming service to handle the authorisation what the uer should or shoul not be able to do is very very bad idea
14:53:46 <dansmith> rosmaita: well as jokke_ noted, you currently should be only exposing them to users via your internal endpoint,
14:53:58 <whoami-rajat> but my answer is based on glance team's points ^
14:54:02 <dansmith> so unless you've done your networking poorly...
14:54:15 <dansmith> it's still "host-based auth" which is not great either, but..
14:55:57 <pdeore> I think we can continue this discussion in review, as just 5 mins left
14:56:15 <jokke_> And I have hard time understanding how this would fit to the RBAC model, like what scope it should be
14:56:23 <jokke_> pdeore: ++ ... thanks
14:56:26 <dansmith> project
14:56:27 <whoami-rajat> sure
14:56:39 <abhishekk> Just an update, next week I will not be able to join the meeting as I have conflict which I can not avoid
14:57:00 <pdeore> Thanks, let's move to open discussion
14:57:01 <pdeore> #topic Open Discussion
14:57:01 <abhishekk> also pdeore will be going on vacation and will be back after next week
14:57:04 <jokke_> dansmith: but the back-end details are for sure system matter, not project matter and if it's project admin, it's not solving 0065
14:57:26 <abhishekk> so is there any volunteer to chair the meeting or we can skip it for next week?
14:57:28 <pdeore> yeah, I will also not be around for entire week
14:57:51 <croelandt> It's a holiday next Thursday here
14:58:00 <dansmith> maybe just punt then?
14:58:03 <jokke_> if both of you are away, should we postpone at least the locations discussion for the following week ... I'd say just cancel the meeting for next week all togeher
14:58:26 <abhishekk> sounds good to me, meantime we can discuss on the spec as well
14:58:29 <rosmaita> cancel works for me
14:58:41 <whoami-rajat> i think this is sort of the same problem when a volume is a project level resource but host attribute is a system level thing -- image (project level) , locations (system level)
14:58:52 <jokke_> if 3 of ye are away I'm sure we can coordinate any possibly urgent stuff in normal glance channel between myself dansmith and rosmaita
14:58:56 <dansmith> whoami-rajat: agree
14:59:09 <abhishekk> cool
14:59:41 <pdeore> great!
15:00:15 <abhishekk> pdeore, plz send the mail to ML saying next weeks meeting is canceled and we will be meeting a week after
15:00:23 <jokke_> as all of us are off tomorrow, I'll keep my eyes on ML next week for the backporting matter
15:00:32 <abhishekk> we can conclude now, thank you all
15:00:35 <pdeore> abhishekk, sure, I will do that
15:00:38 <whoami-rajat> thanks for the discussion everyone!
15:00:40 <jokke_> if agreed with stable maint team, will lift my -2 then
15:00:49 <pdeore> Thanks everyone for joining !!
15:00:54 <jokke_> thanks all!
15:01:05 <pdeore> #endmeeting