21:00:33 <timburke> #startmeeting swift
21:00:33 <opendevmeet> Meeting started Wed May 31 21:00:33 2023 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:33 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:33 <opendevmeet> The meeting name has been set to 'swift'
21:00:41 <timburke> who's here for the swift meeting?
21:00:45 <zaitcev> o/
21:01:28 <acoles> o/
21:01:52 <timburke> as usual, the agenda's at https://wiki.openstack.org/wiki/Meetings/Swift
21:02:03 <timburke> i only had a couple items i wanted to bring up
21:02:11 <zaitcev> I don't know why, but I'm here. I guess I'll just try and ask acoles to look at https://review.opendev.org/c/openstack/swift/+/787656
21:02:58 <timburke> i keep meaning to circle back on that one, too :-(
21:03:49 <timburke> #topic py3 ssync metadata bug
21:04:07 <timburke> and the fix at
21:04:08 <timburke> #link https://review.opendev.org/c/openstack/swift/+/884240
21:04:27 <acoles> zaitcev: darn, sorry!
21:05:38 <timburke> i haven't left my review yet, but i should later today. i've got a couple small changes to tests i'd like to see, and i'm part way into updating a probe test -- but it's a little tricky since swiftclient can't parse the non-ascii meta :-(
21:06:05 <timburke> core change looks correct, though
21:07:07 <timburke> i'll probably push over with my unit test changes but leave the probe test as a follow-up
21:07:31 <timburke> #topic logging/metrics for account/container info requests
21:08:04 <timburke> i realized this morning that we no longer emit subrequest logging/metrics for get_account/container_info requests
21:08:41 <timburke> it was an unintended consequence of https://review.opendev.org/c/openstack/swift/+/875819
21:09:12 <timburke> by going directly to the proxy-app, i bypassed logging :-(
21:09:50 <timburke> i've got a fix up -- it's still a little ugly, though
21:09:52 <timburke> #link https://review.opendev.org/c/openstack/swift/+/884931
21:11:28 <mattoliver> o/ (sorry im late)
21:11:36 <acoles> I'm reminded of some patch mattoliver was working on where we added a pipeline property that held a list of the apps...was it the internal client iter_shard_ranges patch?
21:12:26 <timburke> i'm torn about whether to stick with what i've got there (take the second-to-last item in the pipeline, assuming it's a ProxyLoggingMiddleware, otherwise go to the proxy app), or go searching for the right-most logging filter
21:12:58 <timburke> acoles, https://review.opendev.org/c/openstack/swift/+/879128 i think
21:13:20 <mattoliver> Yeah we're was that. But yeah it stored them as a list, so it would be easy to search
21:13:29 <acoles> yeah, thats' where we added _pipeline
21:13:48 <timburke> it's definitely handy for this fix :-)
21:14:39 <acoles> timburke: are you concerned about the overhead of searching every time?
21:15:26 <timburke> no -- my concern is more that there could be some middleware with unexpected consequences in between the last logging mware and proxy-server
21:16:48 <timburke> something that could do some sysmeta manipulations, say. also, a little worried about the possibility of recursion errors
21:16:51 <acoles> oh, now I'm confused...can you elaborate? what advantage does _pipeline[-2] have?
21:17:30 <zaitcev> I gave up wrapping my mind around that too.
21:19:06 <acoles> I guess [-2] means logging is the last middleware, then there's the app
21:19:20 <timburke> yeah
21:19:47 <timburke> if it's proxy logging, great. we wrote that and can trust it. otherwise, the patch as-written goes to the proxy app; we lose logging/metrics again, but at least we know all sysmeta, acls, etc in the response is intact
21:20:52 <timburke> i don't think it's *likely* that anyone would have configured their pipelines with a problematic middleware in between last logging and proxy-server, but it made me a little nervous
21:20:57 <acoles> but documenting 'if you put *any* middleware after the final logger then there's some metric that may not be emitted' seems odd vs documenting 'if you put middleware after logging that does weird stuff then blah blah'
21:22:17 <acoles> what would have happened, with a weird middleware, before you added the cut-through
21:23:39 <timburke> you'd get weird behaviors. all the more so if it's non-deterministic at which point in the pipeline the get-info call happens
21:24:46 <timburke> getting rid of the possibility of weird behaviors was definitely on my mind when i did the cut-through -- i'm not thrilled about re-introducing them :-/ but maybe it's the most defensible path
21:25:18 <acoles> I think I need to study the patch and the weirdness potential when I'm more awake
21:25:24 <acoles> thanks for alerting us to it!
21:25:34 <mattoliver> +1
21:26:16 <timburke> seems fair. fwiw, one of the weird cases i was thinking of was something like the 1space shunt, where you're trying to merge namespaces from different accounts/clusters
21:26:32 <timburke> anyway, that's all i've got
21:26:37 <timburke> #topic open discussion
21:26:46 <timburke> what else should we bring up this week?
21:27:48 <mattoliver> I've been out this week, so I've got nothing I can think of atm
21:27:54 <zaitcev> Me neither.
21:28:28 <timburke> all right. i think i'll wrap it early, then
21:28:31 <acoles> we shipped this change this week https://review.opendev.org/c/openstack/swift/+/883367
21:28:47 <acoles> and it does seem to have eliminated some curious timeouts
21:30:07 <timburke> yes! it seemed so strange that we were bumping memcached timeouts at all, much less so long after the configured timeout
21:30:31 <acoles> I guess one takeaway is that the network could move bytes faster than the proxy could EC encode them, so network never would block and the EC put greenthread never would yield to another greenthread :O
21:32:05 <timburke> all right
21:32:09 <timburke> thank you all for coming, and thank you for working on swift!
21:32:14 <timburke> #endmeeting