| *** akekane_ is now known as abhishekk | 06:46 | |
| *** dasm|off is now known as dasm | 13:21 | |
| pdeore | #startmeeting glance | 14:00 |
|---|---|---|
| opendevmeet | Meeting started Thu May 19 14:00:33 2022 UTC and is due to finish in 60 minutes. The chair is pdeore. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'glance' | 14:00 |
| pdeore | #topic roll call | 14:00 |
| pdeore | #link https://etherpad.openstack.org/p/glance-team-meeting-agenda | 14:00 |
| pdeore | o/ | 14:00 |
| dansmith | o/ | 14:00 |
| mrjoshi | o/ | 14:01 |
| pdeore | lets wait few minutes for everyone to join | 14:01 |
| jokke_ | o/ | 14:01 |
| pdeore | abhishekk, and rosmaita are busy in resolving the broken gate issue | 14:02 |
| rosmaita | well, "resolving" is a bit strong | 14:03 |
| rosmaita | more like trying to figure out WTF | 14:03 |
| abhishekk | yeah :/ | 14:03 |
| pdeore | yeah, of course.. | 14:04 |
| abhishekk | unfortunately last couple of days we are hitting with very strange errors | 14:04 |
| abhishekk | if anyone interested to have a look, this is the patch which reported the error, https://review.opendev.org/c/openstack/glance/+/842400 | 14:04 |
| pdeore | ack | 14:05 |
| pdeore | can we start ? | 14:05 |
| croelandt | yes! | 14:05 |
| pdeore | #topic release/periodic jobs | 14:05 |
| pdeore | It's M1 Release Week and we have released glance-store 4.0.0 yesterday | 14:05 |
| pdeore | #link https://review.opendev.org/c/openstack/glance/+/842400 is causing failure in glance gate | 14:06 |
| pdeore | so, we might skip glance M1 release if gate is not fixed today | 14:06 |
| abhishekk | correction: above patch is not causing failure, above patch has reported the failure | 14:07 |
| pdeore | ack | 14:08 |
| pdeore | Periodic jobs all green except some oslo tips jobs failure for python3.6 | 14:09 |
| pdeore | #link https://review.opendev.org/c/openstack/glance/+/841350 | 14:09 |
| pdeore | #link https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/841368 | 14:09 |
| pdeore | Still waiting to get these patches merged | 14:09 |
| pdeore | I have requested the infra people to look after the openstack-zuul-jobs patch as I'm not much familiar with those job changes and the comments received on the patch | 14:10 |
| pdeore | moving ahead | 14:10 |
| pdeore | #topic cache-update response code | 14:10 |
| pdeore | #link https://review.opendev.org/c/openstack/glance/+/841278/1#message-456096e48b28e5b866deb8bf53e9258ee08219a0 | 14:11 |
| abhishekk | jokke_, there are some suggestions on the above backport, so can you reconsider this vote | 14:11 |
| jokke_ | abhishekk: I noticed | 14:12 |
| abhishekk | ack | 14:13 |
| jokke_ | rosmaita: just a point to what you said, the spec you linked indeed called the 202, the _documentation_ is changed to that with the patch and stated 200 | 14:13 |
| rosmaita | yeah, but the original doc, the spec, said 202 | 14:13 |
| rosmaita | which is documentation | 14:13 |
| rosmaita | and actually is supposed to be the design doc | 14:14 |
| jokke_ | yes I totally agree that there's been oversight on implementation and reviews. Just a note that we've been blocking these fixes even on master in past ... Like if we want to break the stable that discussion needs to happen in the mailing list | 14:16 |
| jokke_ | it's not something we should just go and do no matter how convenient it is for us | 14:16 |
| dansmith | I have no problem expressing my support for this route to resolution on the mailing list, if it will help | 14:19 |
| dansmith | not sure it's really necessary, but if it helps to just have the same conversation we've had on the ML that's fine with me, but I think we should try to do it quick | 14:19 |
| dansmith | as the longer the wait the worse it is | 14:19 |
| abhishekk | +1 | 14:19 |
| rosmaita | exactly | 14:21 |
| jokke_ | so if the stable maint team greenlights that api change, we also should bump the api version and add reno for it as I mentioned in my original review. just silently changing it is really bad practise | 14:21 |
| rosmaita | you mean bump in yoga? | 14:23 |
| dansmith | I thought it was just bump in master, | 14:24 |
| jokke_ | well bump in master and squash them together for backporting | 14:24 |
| dansmith | but still backport to yoga | 14:24 |
| dansmith | so bump the api in yoga too? | 14:24 |
| jokke_ | well if we backport the change, then yes | 14:24 |
| dansmith | if we were making some real change that people were likely to struggle with I'd agree with much more caution, but I think we can be rational about this specific change and need not overreact, IMHO | 14:25 |
| jokke_ | the api version glance is reporting is there to indicate changes in the api | 14:25 |
| jokke_ | dansmith: I just think it's quite slippery slope to not treat every API breaking change at same severity | 14:26 |
| dansmith | it is a slippery slope, which is why we should be cautious and critical about exceptions.. however, this seems like a good candidate for an exception to me | 14:27 |
| jokke_ | agreed, and there is documented process for it ;) | 14:28 |
| dansmith | rosmaita: you wanna send an email to the list detailing what we discussed, ask for comment and we can go from there? | 14:28 |
| rosmaita | i can do that | 14:29 |
| jokke_ | cheers | 14:29 |
| pdeore | can we move ahead to next topic? | 14:30 |
| jokke_ | ++ | 14:30 |
| pdeore | #topic new location APIs | 14:30 |
| pdeore | #link https://review.opendev.org/c/openstack/glance-specs/+/840882 | 14:30 |
| pdeore | the spec is submitted by whoami-rajat | 14:31 |
| whoami-rajat | hey | 14:31 |
| whoami-rajat | so we discussed this idea at the PTG at glance cinder cross project session | 14:31 |
| * jokke_ is piling up for today's meeting ;) | 14:31 | |
| whoami-rajat | and as discussed i proposed a spec implementing the same idea | 14:31 |
| whoami-rajat | but there are some concerns by jokke_ , which I've replied to | 14:32 |
| whoami-rajat | just wanted to bring some attention to it | 14:32 |
| whoami-rajat | I'm happy to continue the discussion either here or review but just want to get things moving | 14:32 |
| jokke_ | I expressed the concern for admin scoped api for this purpose already during the PTG discussion and after reading through the proposal I think it's very bad idea. | 14:33 |
| rosmaita | can you explain that some more? | 14:34 |
| jokke_ | dansmith: I don't know if you have read through the spec yet. How do you feel about getting Nova storing adming credentials in configs and moving to use the new API for the ceph stuff? Will it ever happen? | 14:34 |
| whoami-rajat | we need this API for both service-service interaction and also for operators, I'm not sure what a good alternative is | 14:34 |
| dansmith | jokke_: I haven't read this spec, but will after the meeting | 14:35 |
| whoami-rajat | we've kept the credentials for nova cinder interaction since forever and it doesn't have any side effects, as far as I've seen | 14:35 |
| dansmith | sounds like this might be a thing for the service role, but yeah currently nova has no special creds for glance and obviously it'd be a change to start doing that | 14:35 |
| whoami-rajat | and the credentials can be removed once service role is in place | 14:36 |
| dansmith | right, service role would be ideal for this sort of thing | 14:36 |
| dansmith | (just based on a quick skim) | 14:36 |
| whoami-rajat | but again, I'm not sure what the timeline of that being available is, I already have a feature dependent on it stuck since yoga ... | 14:36 |
| whoami-rajat | dependent on the service role | 14:37 |
| dansmith | I'm not sure either as we just had some keystone discussion about it in the last week | 14:37 |
| jokke_ | rosmaita: my main concerns are storing the admin credentials all over the place as we very well know OS has no limitation for them. We're expecting to use them for API that's main purpose is to go around gapi on image data handling and thus we also loose any authorization in glance side for the location operations | 14:38 |
| dansmith | adding admin credentials in the meantime would definitely be a big (and unfortunate) hammer, but I really haven't read the spec so ... | 14:38 |
| whoami-rajat | that's my concern, we can't halt all work for keystone to make it available and can try to adapt current alternatives until then, but that's just my thinking | 14:39 |
| jokke_ | And we're aiming this thing to replace the current locations calls, meaning that every change on them needs to be coordinated between Nova (Never easy to get changes in), Cinder and Glance | 14:39 |
| rosmaita | well, we could implement it for a 'service' role, and if people want to use it, they'll have to add it themselves | 14:39 |
| dansmith | yeah, we really don't have to wait for keystone for service role stuff I think, | 14:40 |
| dansmith | especially for specific configs like shared ceph | 14:40 |
| rosmaita | so the locations API will be admin or service | 14:40 |
| rosmaita | because actual admins will want to use it | 14:40 |
| jokke_ | Well the service role would still mean that we need to have some level of double authentication from Keystone side | 14:40 |
| rosmaita | why double auth? | 14:41 |
| whoami-rajat | jokke_, we will be keeping compatibility until all consumers have moved to new APIs, but the current code is not easy to follow and understand as of now | 14:41 |
| dansmith | it means nova has a separate set of credentials for using that api, yes, but it does for pretty much every other downstream service already | 14:41 |
| jokke_ | rosmaita: isn't the whole idea that there is 2 tokens (or double purpose token) issued which effectively states "Service X is doing this operation on behalf of user Y" | 14:42 |
| dansmith | no | 14:42 |
| whoami-rajat | we've custom commands for locations in glanceclient which calls image-update and we've bunch of nested checks in image update, one of such check is ``show_multiple_locations`` | 14:43 |
| dansmith | for things like this (again, haven't read spec, but interpolating) nova would use the user's token for things on behalf of the user, but then use its own creds to peek behind the curtain for things like locations | 14:43 |
| dansmith | glance wouldn't change other than its policy, and nova would just use a differnent ksclient to make the location calls, | 14:43 |
| dansmith | like we do for neutron port bindings and such | 14:44 |
| jokke_ | dansmith: that's unfortunate | 14:44 |
| whoami-rajat | yes, similar to what cinder does to call nova for certain operations | 14:44 |
| dansmith | jokke_: unfortunate that this is how it works for every other service? | 14:44 |
| dansmith | (genuine question) | 14:44 |
| jokke_ | dansmith: yes | 14:45 |
| dansmith | the unfortunate thing is that right now those are all all-powerful admin users, which is very very unfortunate | 14:45 |
| whoami-rajat | just an example of code implementation, priviledged_user is the key parameter here https://github.com/openstack/cinder/blob/master/cinder/compute/nova.py#L84 | 14:45 |
| jokke_ | cause if the intention is not to expose the originating user, it still breaks the audit chain | 14:45 |
| dansmith | with the service role being the thing endowed to do specific things, I don't think it's that unfortunate and I'm not sure what the realistic alternative is, based on how our stuff works | 14:45 |
| jokke_ | and removes the authorisation from the owning service | 14:46 |
| dansmith | yeah, which is the purpose of the dual-token thing I think you were referring to earlier (although pejoratively, I thought), but AFAIK, that's not how we do most things today | 14:47 |
| jokke_ | Cause I think the locations here is pretty darn good example of why you would want the double authentication. You want to confirm that user is not poking the internals on their own, but you're after all changing the project owned resource and want to ensure who ever is requesting that having the rights to do so | 14:47 |
| dansmith | anyway, apologies for not having read the spec, but perhaps we can move on and I'll comment on it after this and maybe we can circle back? | 14:48 |
| whoami-rajat | I'm ok with it unless the spec gets stuck ... | 14:48 |
| whoami-rajat | but i can always bring back the topic next week | 14:48 |
| dansmith | jokke_: I dunno, if that api is restricted to admins and services, the audit chain doesn't seem terribly concerning to me.. audit chains are always nice, but nova is really the thing servicing the user and using glance's locations api to do things | 14:48 |
| abhishekk | whoami-rajat, also I have concern related to spec | 14:49 |
| jokke_ | I think this might warrant wider discussion as well and sorry dansmith for throwing you right in. I just thought you might have better insight of how posible it would be to even get nova adobting this | 14:49 |
| abhishekk | for new location APIs you are going to add new policies,right? | 14:49 |
| whoami-rajat | yes | 14:49 |
| dansmith | jokke_: I don't think nova adoption will be a problem because the spec for that will be "do for glance as we do for everything else" but still worth careful consideration, no argument there | 14:50 |
| abhishekk | ok, I will also comment my thoughts on the spec | 14:50 |
| dansmith | jokke_: I really like that nova->glance is all user-token at the moment :) | 14:50 |
| jokke_ | dansmith: agreed | 14:51 |
| rosmaita | well, that does prompt a question from me | 14:51 |
| rosmaita | is OSSN-0065 still an issue? | 14:51 |
| jokke_ | and the easy way to not expose it to the und users is to deploy service facing gapi and user facing gapi with separate configs and policies | 14:51 |
| * abhishekk apologies for me being not so active in today's meeting | 14:51 | |
| whoami-rajat | sure, but if your concern is regarding the policies can be modified for non-admins, the catch here is that these policies won't be shared unlike the current ones which are shared at other places hence the default is allowing members | 14:52 |
| whoami-rajat | but we can continue discussion in review, don't want to take up all meeting time | 14:52 |
| rosmaita | my question is whether in these modern times, is it still a security risk to expose locations? | 14:52 |
| whoami-rajat | rosmaita, as discussed during the PTG, yes and it will be when we starting adding more features that allows more glance cinder cross project dependencies | 14:53 |
| whoami-rajat | like rbd cow cloning when uploading volume to image | 14:53 |
| jokke_ | I just think requiring admin crdentials saved somewhere to be used for API that is by far mostly used for service-service interactions and expecting the consuming service to handle the authorisation what the uer should or shoul not be able to do is very very bad idea | 14:53 |
| dansmith | rosmaita: well as jokke_ noted, you currently should be only exposing them to users via your internal endpoint, | 14:53 |
| whoami-rajat | but my answer is based on glance team's points ^ | 14:53 |
| dansmith | so unless you've done your networking poorly... | 14:54 |
| dansmith | it's still "host-based auth" which is not great either, but.. | 14:54 |
| pdeore | I think we can continue this discussion in review, as just 5 mins left | 14:55 |
| jokke_ | And I have hard time understanding how this would fit to the RBAC model, like what scope it should be | 14:56 |
| jokke_ | pdeore: ++ ... thanks | 14:56 |
| dansmith | project | 14:56 |
| whoami-rajat | sure | 14:56 |
| abhishekk | Just an update, next week I will not be able to join the meeting as I have conflict which I can not avoid | 14:56 |
| pdeore | Thanks, let's move to open discussion | 14:57 |
| pdeore | #topic Open Discussion | 14:57 |
| abhishekk | also pdeore will be going on vacation and will be back after next week | 14:57 |
| jokke_ | dansmith: but the back-end details are for sure system matter, not project matter and if it's project admin, it's not solving 0065 | 14:57 |
| abhishekk | so is there any volunteer to chair the meeting or we can skip it for next week? | 14:57 |
| pdeore | yeah, I will also not be around for entire week | 14:57 |
| croelandt | It's a holiday next Thursday here | 14:57 |
| dansmith | maybe just punt then? | 14:58 |
| jokke_ | if both of you are away, should we postpone at least the locations discussion for the following week ... I'd say just cancel the meeting for next week all togeher | 14:58 |
| abhishekk | sounds good to me, meantime we can discuss on the spec as well | 14:58 |
| rosmaita | cancel works for me | 14:58 |
| whoami-rajat | i think this is sort of the same problem when a volume is a project level resource but host attribute is a system level thing -- image (project level) , locations (system level) | 14:58 |
| jokke_ | if 3 of ye are away I'm sure we can coordinate any possibly urgent stuff in normal glance channel between myself dansmith and rosmaita | 14:58 |
| dansmith | whoami-rajat: agree | 14:58 |
| abhishekk | cool | 14:59 |
| pdeore | great! | 14:59 |
| abhishekk | pdeore, plz send the mail to ML saying next weeks meeting is canceled and we will be meeting a week after | 15:00 |
| jokke_ | as all of us are off tomorrow, I'll keep my eyes on ML next week for the backporting matter | 15:00 |
| abhishekk | we can conclude now, thank you all | 15:00 |
| pdeore | abhishekk, sure, I will do that | 15:00 |
| whoami-rajat | thanks for the discussion everyone! | 15:00 |
| jokke_ | if agreed with stable maint team, will lift my -2 then | 15:00 |
| pdeore | Thanks everyone for joining !! | 15:00 |
| jokke_ | thanks all! | 15:00 |
| pdeore | #endmeeting | 15:01 |
| opendevmeet | Meeting ended Thu May 19 15:01:05 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:01 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/glance/2022/glance.2022-05-19-14.00.html | 15:01 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/glance/2022/glance.2022-05-19-14.00.txt | 15:01 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/glance/2022/glance.2022-05-19-14.00.log.html | 15:01 |
| *** dasm is now known as dasm|off | 21:26 | |
| shelrk | We're closed. You guys have to leave. | 23:43 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!