21:00:22 <timburke> #startmeeting swift
21:00:22 <opendevmeet> Meeting started Wed Jul  5 21:00:22 2023 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:22 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:22 <opendevmeet> The meeting name has been set to 'swift'
21:00:30 <timburke> who's here for the swift meeting?
21:00:58 <zaitcev> o7
21:01:44 <kota> o/
21:02:22 <clayg> kota: it was so good to see you in SF!
21:02:46 <kota> clayg: absolutely!
21:02:50 <clayg> zaitcev: I'm thinking about going to hang in Austin next - you wanna meet up and drive go karts or show me where you park your plane?!
21:04:02 <timburke> i didn't get around to updating the agenda, but there's a handful of patches worth mentioning
21:04:14 <timburke> #topic py311 testing
21:04:49 <timburke> https://review.opendev.org/c/openstack/swift/+/886541 merged, giving us a voting py311 unit test job!
21:05:07 <timburke> thanks clayg and mattoliver for reviewing
21:06:19 <timburke> next step (imo) is to get a func test job going, too, or at least switching my dev env :-)
21:06:56 <timburke> #topic ssync non-ascii metadata corruption
21:07:00 <timburke> #link https://review.opendev.org/c/openstack/swift/+/884240
21:07:47 <timburke> could still use some reviews. i think i got unit test coverage for all the changes now, though
21:10:30 <timburke> i think acoles is out this week, though -- might have to wait until next week
21:11:48 <clayg> heh, I think i still dev on py3.8 - do we do a lot of different python versions with functests currently?
21:12:53 <clayg> ssync metadata corruption!  I thought the annoying metadata corruption bug you were dealing with had something to do with py3 pickles?!
21:13:21 <timburke> 3.6, 3.8, and 3.9 iirc -- though i'll run 3.10 in dev (and at home)
21:13:40 <timburke> clayg, it's another bug :-(
21:14:31 <timburke> the first one was about reading py2 metadata after upgrading to py3; this one happens even in clusters that have *only ever been* py3
21:14:38 <clayg> oh you're right - i'm on 3.10 in dev too - py3.8 in prod, which is EOL after Q124!
21:15:10 <clayg> yikes, sounds nasty - and we lucked out with encryption.  it worked under py2 tho?!
21:15:57 <timburke> clayg, no, we're py3.7 in prod, we're already in trouble ;-)
21:16:09 <clayg> 😭
21:16:47 <timburke> ok, one more patch to draw attention to
21:17:01 <timburke> #topic recoverable timeouts and SLO manifests
21:18:01 <timburke> we noticed a (rare) issue where we'd get EC decode errors trying to do some ranged GETs into an SLO
21:18:31 <clayg> zaitcev: kota: did y'all see https://bugs.launchpad.net/swift/+bug/2020667 when it came in?  seems legit bad
21:19:26 <kota> not yet. looking
21:21:39 <timburke> i eventually tracked it down to an optimization i'd done to try to reduce the backend round-trips: previously, we'd try to get the range from the manifest, likely get back a 417, then need to make a new request to get the whole thing
21:22:37 <timburke> following https://github.com/openstack/swift/commit/e8b654f3 though, the proxy would tell the object server to ignore any Range and send the whole object if it looked like an SLO manifest
21:23:57 <timburke> the trouble is that when we hit a recoverable node timeout and try to pick up the stream with another node, we never *unset* that flag, so we get the whole object (er, frag archive) when we really *did* just want part of it
21:24:31 <timburke> #link https://review.opendev.org/c/openstack/swift/+/887235
21:24:36 <timburke> should fix it, though
21:25:16 <clayg> oh yeah, i remember that one!
21:25:25 <clayg> nice find 👍
21:27:06 <timburke> i think that's all i've got for this week -- if anyone's got review bandwidth for the ssync bug or the recoverable timeout bug, i'd appreciate it
21:27:10 <timburke> #topic open discussion
21:27:19 <timburke> what else should we bring up this week?
21:30:34 <timburke> all right, i'll call it then
21:30:45 <timburke> thank you all for coming, and thank you for working on swift!
21:30:48 <timburke> #endmeeting