| opendevreview | Merged openstack/swift master: s3api: return 503 if mpu complete gets 409 deleting marker https://review.opendev.org/c/openstack/swift/+/901133 | 16:35 |
|---|---|---|
| opendevreview | Alistair Coles proposed openstack/swift master: DNM: check partnum==1 for non-slo https://review.opendev.org/c/openstack/swift/+/902172 | 16:44 |
| timburke | #startmeeting swift | 21:02 |
| opendevmeet | Meeting started Wed Nov 29 21:02:00 2023 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. | 21:02 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 21:02 |
| opendevmeet | The meeting name has been set to 'swift' | 21:02 |
| timburke | who's here for the swift team meeting? | 21:02 |
| acoles | o/ | 21:02 |
| mattoliver | o/ | 21:03 |
| timburke | first off, sorry for the last-minute cancellation a couple weeks ago, and the (not explicitly mentioned) cancellation last week for the US holiday | 21:04 |
| timburke | things have been a little hectic for me | 21:05 |
| timburke | which also means i don't have much of an agenda for this week -- though i have the sense that there's been a lot of effort concentrated around the SLO/MPU part-number changes | 21:05 |
| acoles | no worries | 21:05 |
| mattoliver | yeah no worries Tim | 21:06 |
| mattoliver | I've been really distracted the last 2 weeks, so unfortuantly I haven't done much myself anyway. | 21:07 |
| timburke | is there anything that we'd like to draw extra attention to, or that has questions that need answering that we ought to bring up? | 21:07 |
| acoles | we've merged the slo refactor that Clay worked so hard on :) | 21:07 |
| timburke | 🎉 | 21:07 |
| mattoliver | \o/ | 21:07 |
| acoles | so our review effort is now focussed on the new partnumber support in SLO, and then s3api | 21:08 |
| acoles | those are the indianwhocodes patches | 21:08 |
| timburke | oh, and gerrit's down for now... i think the SLO one was https://review.opendev.org/c/openstack/swift/+/894570 tho | 21:09 |
| acoles | I think we've all been distracted by holidays and family stuff in last week or so | 21:09 |
| -opendevstatus- NOTICE: The Gerrit service on review.opendev.org will be restarting momentarily for a patch update to address a recently observed regression preventing some changes from merging | 21:10 | |
| timburke | speaking of distractions, there's a chance i'll be out for jury duty next week... | 21:10 |
| timburke | i'll set a reminder for myself to get a message out one way or the other my tuesday evening, though | 21:11 |
| mattoliver | kk, or just let me know and I'll pass it along | 21:11 |
| timburke | so i've just got one other thing i've been thinking about the last few weeks: | 21:13 |
| timburke | i noticed that our "seamless reloads" aren't always so seamless | 21:13 |
| timburke | it's pretty easy to see: run swift-bench, then reload the proxy part way through -- there'll be a bunch of connection resets | 21:14 |
| mattoliver | oh good test | 21:15 |
| timburke | i'm pretty sure the issue comes down to incoming connections sitting in the old workers' accept queues; when we close the old listen socket, our accept() call immediately starts returning EBADF and all the queued connections get reset | 21:16 |
| timburke | so i'm going down an ebpf rabbit hole to figure out how we might be able to prioritize the new worker sockets over the old ones | 21:16 |
| mattoliver | ahh yeah. We could test that by turning the accept queue down to something tiny | 21:17 |
| timburke | it'll be an adventure! | 21:17 |
| mattoliver | yay | 21:17 |
| acoles | so we wait for greenthreads to complete in flight requests but those in the accept queue are dropped? | 21:17 |
| timburke | yup, as best i can tell | 21:17 |
| mattoliver | the accept queue have been accepted by the OS but we haven't accepted them from our side of the socket. So if we're not pulling requests of the socket anymore, those clients miss out | 21:18 |
| timburke | eh, the queue doesn't need to be tiny -- in fact, if we make it *too* small, you'll see the bad behavior regardless, as the kernel doesn't have anywhere to put the connection (so it'll just drop it) | 21:19 |
| acoles | right | 21:19 |
| mattoliver | yeah, I just meant to test if its the accept queue | 21:20 |
| mattoliver | I wonder if there is a way to put the socket into close_wait but still deal with the requests (if that is even possible) not sure of the socket state machine. | 21:21 |
| timburke | i think something like https://paste.opendev.org/show/bcMWJVwkAbAlhpt6xQY0/ shows the bad pretty well, and doesn't even need swift or eventlet | 21:22 |
| mattoliver | Anyway, I super interesting research spike, good thinking Tim. | 21:22 |
| timburke | but i think that's all i've got for this week | 21:24 |
| mattoliver | kk, I don't have anything either. A big chuck of downstream work is now completed for me, so hopefully I'll get a bunch of swift time now, so maybe next week I'll have been more productive :) | 21:25 |
| timburke | all right, i think i'll call it early | 21:27 |
| timburke | thank you all for coming, and thank you for working on swift! | 21:27 |
| timburke | #endmeeting | 21:27 |
| opendevmeet | Meeting ended Wed Nov 29 21:27:35 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 21:27 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/swift/2023/swift.2023-11-29-21.02.html | 21:27 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/swift/2023/swift.2023-11-29-21.02.txt | 21:27 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/swift/2023/swift.2023-11-29-21.02.log.html | 21:27 |
| acoles | ok \o | 21:27 |
| opendevreview | Clay Gerrard proposed openstack/swift master: proxy: always use namespaces for updating and listing https://review.opendev.org/c/openstack/swift/+/900350 | 21:57 |
| opendevreview | Clay Gerrard proposed openstack/swift master: sq: do not require auto_shard https://review.opendev.org/c/openstack/swift/+/902205 | 21:57 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!