| kota | good morning | 20:55 |
|---|---|---|
| mattoliver | morning | 21:00 |
| indianwhocodes | afternoon | 21:02 |
| acoles | o/ | 21:02 |
| acoles | kota: good to have you back in the channel! | 21:03 |
| kota | acoles: thanks! | 21:04 |
| timburke | sorry to be so late! | 21:06 |
| timburke | #startmeeting swift | 21:06 |
| opendevmeet | Meeting started Wed Feb 8 21:06:34 2023 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. | 21:06 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 21:06 |
| opendevmeet | The meeting name has been set to 'swift' | 21:06 |
| timburke | who's here for the swift meeting? | 21:06 |
| kota | o/ | 21:06 |
| mattoliver | o/ | 21:06 |
| indianwhocodes | o/ | 21:07 |
| timburke | i've been a bit distracted by hardware issues, so i haven't update the agenda | 21:08 |
| mattoliver | nps, hows your laptop going? | 21:08 |
| timburke | but it couldn't hurt to close the loop on the s3api CVE | 21:08 |
| timburke | mattoliver, still dead-ish, but at least now i've got my desktop booting off the old hard drive -- feels much more like my work computer than before (when i was trying to do everything off my personal OS installs) | 21:10 |
| timburke | #topic s3api CVE | 21:10 |
| opendevreview | ASHWIN A NAIR proposed openstack/swift master: proxy-server exception logging shows replication_ip/port https://review.opendev.org/c/openstack/swift/+/860866 | 21:11 |
| timburke | just to close this out -- a tag is now up (2.28.1) for stable/xena | 21:11 |
| mattoliver | cool | 21:11 |
| timburke | since that's the last non-extended-maintenance branch, no more tags are expected | 21:11 |
| timburke | all the backports have landed (iirc that was already true last week, but never hurts to reiterate) | 21:12 |
| mattoliver | So now no excuse, update your clusters peoples :) | 21:12 |
| kota | great | 21:12 |
| indianwhocodes | done already :wink | 21:13 |
| mattoliver | Thanks for getting that all backported timburke | 21:13 |
| acoles | +1 | 21:13 |
| timburke | ...and that's all there is to it! CVE resolved! | 21:13 |
| mattoliver | \o/ | 21:14 |
| timburke | #topic March vPTG | 21:14 |
| timburke | again, the etherpad to collect topics is up | 21:14 |
| timburke | #link https://etherpad.opendev.org/p/swift-ptg-bobcat | 21:14 |
| timburke | and acoles has already added some to it :-) | 21:14 |
| mattoliver | nice! I'm still to dump whatever I can think of there... I'll try and get to that before next meeting | 21:15 |
| indianwhocodes | same here, wanna have something more precisefirst | 21:15 |
| timburke | i still need to add ring v2, and maybe a link to an ideas-style doc | 21:15 |
| timburke | never hurts to add a placeholder, though, and fill in more info as you get there :D | 21:16 |
| mattoliver | If ring v2 isn't there when I get to adding some topics I'll add it, but you can always update it | 21:17 |
| timburke | 👍 | 21:17 |
| indianwhocodes | imo, the last etherpad link is useful too, ref: https://etherpad.opendev.org/p/swift-ptg-antelope | 21:17 |
| mattoliver | (as I go through open patches from days old) | 21:17 |
| mattoliver | what, you mean those are stil things :P | 21:17 |
| timburke | i still haven't gotten to the time-slots poll, sorry -- seems likely that we'll settle on similar slots to the last few vPTGs | 21:18 |
| timburke | oh yeah, if anybody's looking for even *older* etherpads, there's also https://wiki.openstack.org/wiki/Swift/Etherpads | 21:18 |
| mattoliver | I've already added our news one to that | 21:19 |
| mattoliver | *new ones | 21:19 |
| timburke | i noticed someone had done it -- thanks mattoliver! | 21:20 |
| indianwhocodes | Thanks Tim, only found the ops doc now, seems very relevant! | 21:20 |
| timburke | that's all i've got right now -- who's got topics they'd like to bring up? | 21:20 |
| timburke | #topic open discussion | 21:20 |
| mattoliver | yeah, ops feedback is always really useful for thinking about pain points and future work | 21:21 |
| mattoliver | I've been working on some more stuck shard ranges | 21:21 |
| mattoliver | It's a weird edgecase. but something like | 21:21 |
| mattoliver | #link https://review.opendev.org/c/openstack/swift/+/872852 | 21:21 |
| mattoliver | should stop them from getting into failing a sharding_enabled check and get to the audit code to pull in new shard ranges from root. | 21:22 |
| indianwhocodes | been working on how i could increase the delay in between when the object is marked for expiry and when the expirer actually issues the delete | 21:23 |
| indianwhocodes | by been working (I mean just started) | 21:23 |
| mattoliver | The other fix was to add a db_state check in sharding_Enabled, but that would mean an extra directory listing check on every broker the sharder looks at (all of them). So this approach is a noop (well no extra io checks) | 21:23 |
| mattoliver | indianwhocodes: lol, and cool | 21:24 |
| indianwhocodes | Also if there is a way or an API we can introduce to recover expired data if the data they put under was incorrect | 21:24 |
| mattoliver | why would we increase the delay between expiry deletes? is there a bug? | 21:25 |
| mattoliver | oh I think you just answered that question | 21:25 |
| mattoliver | can't you just put new or remove the expiry metadata, so a post? | 21:25 |
| indianwhocodes | ya something like that | 21:26 |
| mattoliver | or do you mean post delete | 21:26 |
| timburke | can have things like a client that wants to extend the expiry, but there's a bunch of failures and they exhaust their retries | 21:26 |
| indianwhocodes | "x-delete-at:" -XPOST | 21:26 |
| indianwhocodes | but does it really work ? because i tired doing it and a head on the object still worked! | 21:27 |
| indianwhocodes | needs further investigation | 21:27 |
| timburke | pretty sure indianwhocodes is looking to introduce a recovery option in between the x-delete-at lapsing and the expirer actually laying down a tombstone | 21:27 |
| indianwhocodes | yup | 21:27 |
| indianwhocodes | many possible scenarios need to explain it better too, lol | 21:28 |
| mattoliver | ok, so a POST after the x-delete-at date, the obj server would consider it already deleted | 21:28 |
| mattoliver | even if it isn't, i guess | 21:28 |
| indianwhocodes | ya! | 21:28 |
| mattoliver | and it's a recover from that POV. | 21:28 |
| mattoliver | I get it! | 21:28 |
| mattoliver | well, you'd need to know it was in that state, maybe some kind of force. Or allow posts with x-delete-at's through the obj server barrier, the obj server is already looking at the x-delete-at date set on the object.. . | 21:30 |
| timburke | so down in the diskfile, we check for past-expiry -- and the object-server will 404 any POST to avoid exactly this scenario | 21:30 |
| mattoliver | anyway we can take this offline | 21:30 |
| timburke | #link https://github.com/openstack/swift/blob/2.31.0/swift/obj/diskfile.py#L2736-L2750 | 21:30 |
| mattoliver | yeah | 21:30 |
| timburke | but we could plumb in some (likely reseller-admin-gated) header to twiddle _open_expired | 21:31 |
| mattoliver | yeah ok, I look forward to reviewing some options. Ping me if we want to chat more | 21:32 |
| timburke | mattoliver, the sharding patch looks pretty clean -- did we already audit for other sharding-related sysmeta we should also add? | 21:32 |
| mattoliver | Good thinking, let me double check, just in case before we land anything | 21:33 |
| timburke | cleave context meta, maybe? | 21:35 |
| mattoliver | Oh only other thing of note, I've been looking into otel metrics again, so going to have aplay with the api some in some downstream tests here at work, and I hope to turn that learning into how we'd go about things in Swift... just a heads up, and probably a topic for the ptg | 21:35 |
| indianwhocodes | exciting! | 21:35 |
| timburke | nice! | 21:35 |
| timburke | as a heads-up, it's openstack election season again -- i'll try to get my self-nomination up by friday | 21:36 |
| kota | nice, and thanks for continuing it | 21:37 |
| mattoliver | Yeah timburke for PTL! | 21:37 |
| timburke | all right, i'll let y'all go then | 21:38 |
| timburke | thanks for coming, and thank you for working on swift! | 21:39 |
| timburke | #endmeeting | 21:39 |
| opendevmeet | Meeting ended Wed Feb 8 21:39:10 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 21:39 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/swift/2023/swift.2023-02-08-21.06.html | 21:39 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/swift/2023/swift.2023-02-08-21.06.txt | 21:39 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/swift/2023/swift.2023-02-08-21.06.log.html | 21:39 |
| opendevreview | Tim Burke proposed openstack/swift master: Don't clear x-container-sysmeta-sharding on delete_db https://review.opendev.org/c/openstack/swift/+/872852 | 22:23 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!