21:00:04 <timburke> #startmeeting swift
21:00:04 <opendevmeet> Meeting started Wed Mar  6 21:00:04 2024 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:04 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:04 <opendevmeet> The meeting name has been set to 'swift'
21:00:11 <timburke> who's here for the swift meeting?
21:00:23 <kota> hi
21:01:23 <mattoliver> o/
21:01:43 <timburke> as usual, the agenda's at
21:01:44 <timburke> #link https://wiki.openstack.org/wiki/Meetings/Swift
21:02:04 <timburke> though i forgot to add the first thing i ought to talk about ;-)
21:02:10 <timburke> #topic swift release
21:02:35 <timburke> it's time to put together a release for caracal!
21:02:36 <mattoliver> lol
21:03:14 <timburke> they need cycle highlights this week, and i can never come up with them unless i'm doing a release
21:03:46 <timburke> i think we might technically have until next week to make the release, but sooner tends to be better
21:03:53 <mattoliver> I assume we can use the priority reviews page for patches we want to target to get into the release?
21:04:01 <acoles> o/ sorry I'm late
21:04:04 <timburke> absolutely!
21:04:09 <jianjian> o/ me too
21:04:20 <mattoliver> but, kk. Try and put my reviewer had on some more this week.
21:04:21 <timburke> #link https://wiki.openstack.org/wiki/Swift/PriorityReviews
21:04:38 <timburke> i need to update it; at least a few of those have merged :D
21:04:52 <mattoliver> but thats a good problem to have :)
21:06:13 <timburke> if anyone has other patches to add, please let me know! i should probably kick some of the ones currently listed off until after the release, as they're unlikely to make it (aws-chunked, i'm looking at you)
21:07:00 <jianjian> from patch list, looks like PriorityReviews means patches are almost done and need landing upstream soon
21:07:16 <mattoliver> maybe we just add a sub heading for non release priority reviews so they don't fall off our radar.
21:08:07 <mattoliver> jianjian: yeah, there are so many patches out there. Priority reviews is a way to tell reviewers which are a higher priority for reasons. so when you don't know what to review its a good place to go.
21:08:09 <timburke> i'll often stuff them into an HTML comment, just to avoid distracting from what matters *this week* ;-)
21:08:30 <mattoliver> yeah, thats ok too :)
21:08:47 <timburke> next up (and somewhat related)
21:09:01 <timburke> #topic expirer grace period
21:09:11 <timburke> #link https://review.opendev.org/c/openstack/swift/+/874806
21:09:11 <patch-bot> patch 874806 - swift - expirer: per account and container grace period - 12 patch sets
21:09:17 <timburke> #link https://review.opendev.org/c/openstack/swift/+/874710
21:09:17 <patch-bot> patch 874710 - swift - add x-open-expired to access expired objects - 30 patch sets
21:09:36 <timburke> the follow-ups have been squashed in so we've got just these two patches
21:09:53 <mattoliver> oh cool! I was about to ask if that had happened!
21:10:04 <timburke> there's a pep8 issue on the second, but otherwise i think they're ready for reviews
21:10:26 <mattoliver> kk!
21:10:54 <acoles> oh noce, Anish has stacked them in a chain too
21:10:57 <timburke> biggest question on my mind (with the release on the horizon) is whether we want to wait for both to be ready before merging the first one
21:10:59 <acoles> nice*
21:12:41 <mattoliver> I guess their independent.. but would it be annoying to have a release between the 2? possibly
21:12:51 <acoles> I think it makes sense, grace period is not much use without being able to access the expired object
21:13:03 <mattoliver> +1
21:13:12 <timburke> it may also be somewhat moot -- this week might not be quite enough time
21:13:31 <timburke> but i think i *will* add it to the priority reviews page
21:13:33 <jianjian> makes sense, +1
21:13:46 <timburke> next up
21:13:53 <timburke> #topic aws-chunked
21:13:55 <indianwhocodes> o/
21:14:01 <mattoliver> well lets get some +2/+1s on them, but only land the first when the other is ready.. ie both or nothing?
21:14:17 <timburke> 👍
21:14:25 <zaitcev> "Access expired objects"? Really, guys. Just how far deep the rabbit hole leads?
21:14:41 <mattoliver> zaitcev: lol, don't ask :P
21:15:42 <timburke> i've got the broad strokes of the aws-chunked chain laid out
21:15:49 <timburke> #link https://review.opendev.org/c/openstack/swift/+/909049
21:15:50 <patch-bot> patch 909049 - swift - s3api: Improve checksum-mismatch detection - 4 patch sets
21:15:56 <timburke> #link https://review.opendev.org/c/openstack/swift/+/909800
21:15:57 <patch-bot> patch 909800 - swift - utils: Add crc32c function - 4 patch sets
21:16:03 <timburke> #link https://review.opendev.org/c/openstack/swift/+/909801
21:16:03 <patch-bot> patch 909801 - swift - s3api: Add support for additional checksums - 5 patch sets
21:16:09 <timburke> #link https://review.opendev.org/c/openstack/swift/+/909802
21:16:09 <patch-bot> patch 909802 - swift - WIP: s3api: Additional checksums for MPUs - 5 patch sets
21:16:15 <timburke> #link https://review.opendev.org/c/openstack/swift/+/836755
21:16:16 <patch-bot> patch 836755 - swift - Add support of Sigv4-streaming - 14 patch sets
21:16:36 <mattoliver> oh wow, becoming quite a chain!
21:16:49 <mattoliver> oh and you have a crc32c one now. Nice
21:17:27 <indianwhocodes> that sure is a lot!
21:17:27 <zaitcev> timburke: re.909049, you see that BaseException.args is absent for me. I added the console log. What version were you at?
21:17:44 <zaitcev> timburke: I mean the version of python. Mine was 3.12.
21:17:58 <timburke> those middle three might all be able to get squashed into one, but i think they're already reasonably large
21:18:24 <mattoliver> oh maybe another 3.12 issue maybe?
21:18:34 <indianwhocodes> ^^
21:18:35 <zaitcev> I suppose I can +2 it as it is and wait until it blows up on py312, and feel smug?
21:18:48 <mattoliver> lol
21:19:06 <timburke> zaitcev, .args, or .arg? the snippet you had at https://review.opendev.org/c/openstack/swift/+/909049/4/swift/common/middleware/s3api/s3request.py#873 had .arg, which would be missing
21:19:06 <patch-bot> patch 909049 - swift - s3api: Improve checksum-mismatch detection - 4 patch sets
21:19:26 <zaitcev> Just use that str() dammit, what are we even discussing. I'm not inventing a new algorithm for the sharder here.
21:19:52 <zaitcev> Oh
21:19:54 <zaitcev> Hmm.
21:20:31 <timburke> but yeah, str() could work fine. i mainly was thinking i'd be explicit about how/where it came from
21:20:35 <jianjian> on patch "909800: utils: Add crc32c function", is this where the fast isa-l library will be used potentially?
21:20:45 <zaitcev> No, no, I was clearly wrong.
21:22:06 <timburke> jianjian, yup -- isa-l is preferred; if not available, fall back to kernel sockets on py3, or a simple pure-python implementation on py2
21:23:12 <timburke> my current stumbling block is some seemingly unrelated probe tests failing starting at p 909801 -- i haven't been able to reproduce them locally, and they don't seem to make much sense to me
21:23:12 <patch-bot> https://review.opendev.org/c/openstack/swift/+/909801 - swift - s3api: Add support for additional checksums - 5 patch sets
21:23:20 <jianjian> 👍
21:24:11 <timburke> they're all about signals and reload handling, but the patch is only touching things way over in s3api
21:25:03 <timburke> i'll keep on it, but if anyone has spare cycles to look at it, maybe a fresh set of eyes will see it more quickly
21:25:23 <timburke> #topic drive-full checker
21:25:36 <mattoliver> yeah that seems weird, I'll see if it fails for me
21:26:00 <timburke> i wonder if maybe i need to try under centos 8...
21:26:54 <mattoliver> oh thats an idea. I do have the centos PR for vsaio floating around.
21:27:11 <timburke> i've been reviewing zigo's patch! it's looking fairly good. i probably should have looked at how we (NVIDIA) deal with shutting down rsync earlier, tho :P
21:27:13 <mattoliver> let me try and recreate
21:27:48 <mattoliver> I was hoping I'd trick^Wconvince our SRE to look into it
21:28:19 <timburke> oh, boo! i can't even ping csmart directly here ;-)
21:28:44 <mattoliver> csmart: I can ;)
21:29:12 <csmart> 👋
21:29:13 <mattoliver> wow it worked!
21:29:19 <mattoliver> 😆
21:29:26 <timburke> ✨magic✨
21:29:44 <timburke> if anyone else wants to take a look, i'm sure zigo would appreciate it
21:29:47 <jianjian> csmart, welcome! :-)
21:29:59 <mattoliver> csmart: I was just says I'm trying to trick SRE into looking at the drive-full patch
21:30:17 <timburke> #link https://review.opendev.org/c/openstack/swift/+/907523
21:30:18 <patch-bot> patch 907523 - swift - drive-full-checker - 32 patch sets
21:30:24 <csmart> Sounds good to me!
21:30:38 <timburke> thanks!
21:30:55 <mattoliver> nice!
21:30:58 <acoles> csmart: welcome to the party
21:31:07 <timburke> zaitcev, out of curiosity, how do you guys deal with shutting down rsync when drives start to get full?
21:31:33 <timburke> might be good to have more than just the two perspectives on how to deal with it
21:31:44 <mattoliver> +1
21:32:11 <zaitcev> I don't think we deal with it at all. The field people know that Swift cannot survive a full filesystem.
21:33:03 <jianjian> btw, how do your field people define a full FS? what percentage of used?
21:33:16 <timburke> interesting. might be all the more reason to be interested -- there'd be a new cron job you could run to try to defend against it
21:33:42 <timburke> the good news is, i trust that csmart wouldn't hold back in criticizing NVIDIA's approach if he disagrees ;-)
21:33:58 <zaitcev> In addition, our 2nd day story is fairly weak until OSP 19 or so. It's an upcoming release based on Caracal IIRC, and it uses k8s. Previously, there was no built-in way to monitor or resolve filesystems creeping to full. Just Zabbix and that's it.
21:34:35 <zaitcev> jianjian: 100% is full. When new files are all created with 0 size.
21:35:56 <zaitcev> In legacy Linux we had a root-only reserve, but I suspect rsync defeats it somehow. I see cases where filesystems are totally full.
21:36:18 <mattoliver> sounds like having this checking could be helping redhat too then, thats a win
21:38:11 <timburke> all right, next up
21:38:20 <timburke> #topic part-number support
21:38:23 <timburke> #link https://review.opendev.org/c/openstack/swift/+/894570
21:38:23 <patch-bot> patch 894570 - swift - slo: part-number=N query parameter support - 90 patch sets
21:38:33 <timburke> #link https://review.opendev.org/c/openstack/swift/+/894580
21:38:34 <patch-bot> patch 894580 - swift - s3api: Support GET/HEAD request with ?partNumber - 99 patch sets
21:39:11 <mattoliver> woah 99, just one more indianwhocodes :P
21:39:30 <timburke> indianwhocodes, how's it going? anything we should watch out for as we review?
21:39:44 <jianjian> zaitcev: wow, you are dealing with 100% full, good to know
21:39:52 <mattoliver> yeah, ready for a review. Might be nice to get into the release if its ready
21:39:57 <jianjian> has to be 99 lol
21:40:10 <timburke> mattoliver, +1
21:41:05 <timburke> and that set, i feel pretty good about releasing just the first if that's how far we get
21:43:40 <timburke> all right, one last topic i had
21:44:03 <timburke> #topic drop support for libec<1.4
21:44:06 <timburke> #link https://review.opendev.org/c/openstack/pyeclib/+/839643
21:44:07 <patch-bot> patch 839643 - pyeclib - Drop support for liberasurecode<1.4.0 - 10 patch sets
21:45:09 <timburke> it's been more than seven years since 1.4.0 came out, i think we can feel pretty comfortable doing it
21:45:53 <mattoliver> yeah, sounds good to me.
21:45:56 <timburke> just wanted to give everyone one last chance to object
21:46:21 <timburke> i might still wait another month or so before merging, just to avoid it looking like it was supposed to go out in caracal
21:46:53 <zaitcev> I have no objection. We used to ship with 1.0.9, but that was a very long time ago.
21:47:08 <zaitcev> Before the zlib CRC problme, IIRC
21:47:42 <timburke> all right, sounds good. i'll self-approve if needed ;-)
21:47:47 <timburke> #topic open discussion
21:47:54 <timburke> anything else we ought to bring up this week?
21:48:21 <kota> Just FYI, I'll be at SanJose at GTC week.
21:48:22 <mattoliver> I have nothing
21:49:01 <timburke> kota, hurrah! want to try to meet up?
21:49:07 <acoles> timburke: any progress towards a feature branch for mpu? I have code I want to push somewhere
21:49:33 <acoles> nobody ever comes to the UK :'(
21:49:57 <timburke> acoles, thanks for the nudge! i hadn't done anything with it yet; if you've got code already, i'll try to do that in the next day or two
21:49:58 <kota> timburke: maybe, will check free time slot in the week evening. it might be fully busy week.
21:50:19 <acoles> timburke: thanks, let me know if I can do anything
21:51:45 <indianwhocodes> @timburke sorry lost connection there
21:51:54 <indianwhocodes> yes part-num patchg is ready to be merged upstream
21:53:55 <timburke> excellent -- i'll try to take a look this week
21:54:34 <indianwhocodes> kota will be nice to meet you in-person
21:55:16 <kota> indianwhocodes: oh, you'll be at there.
21:55:28 <indianwhocodes> yes.
21:56:05 <mattoliver> jianjian:  is in the area too
21:56:05 <kota> good to know, let's plan to meet in person
21:56:33 <mattoliver> Wish acoles and I could go to GTC now :(
21:56:50 <timburke> i can check in with notmyname, too
21:57:12 <kota> awesome
21:58:03 <timburke> all right, we're about at time
21:58:13 <timburke> thank you all for coming, and thank you for working on swift!
21:58:18 <timburke> #endmeeting