21:00:32 <timburke> #startmeeting swift
21:00:32 <opendevmeet> Meeting started Wed Nov  8 21:00:32 2023 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:32 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:32 <opendevmeet> The meeting name has been set to 'swift'
21:00:41 <timburke> who's here for the swift meeting?
21:01:52 <mattoliver> o/
21:02:37 <mattoliver> Although I had a very distracted week and didn't get much Swift work done.. but I'm here :)
21:03:45 <kota> o/
21:03:49 <opendevreview> Tim Burke proposed openstack/swift master: slo: refactor GET/HEAD response handling  https://review.opendev.org/c/openstack/swift/+/893578
21:04:10 <timburke> so i neglected to update the agenda again (sorry)
21:05:07 <timburke> but the main things i've been thinking about are (1) the SLO refactors that are stacked in front of the part-number APIs
21:05:37 <timburke> and (2) how we manage our processes (especially WSGI servers)
21:06:06 <mattoliver> yeah, the chain seems to be getting long. And a bit daunting. How are they looking?
21:06:32 <timburke> #topic SLO refactor
21:06:50 <timburke> the first part has now merged, just refactoring tests
21:06:59 <timburke> #link https://review.opendev.org/c/openstack/swift/+/896466
21:07:59 <timburke> the second part (which *does* impact how SLO behaves) is coming along; i've been pushing up a few fixups today
21:08:02 <timburke> #link https://review.opendev.org/c/openstack/swift/+/893578
21:08:48 <timburke> i think at this point i'm happy with it; if anyone else wants to take a look, you might want to do that soon
21:09:43 <timburke> as it is, clayg, acoles, and indianwhocodes are all listed as authors, and i'm likely to put a +2 on it once we get a clean test run
21:10:21 <mattoliver> OK. I'll be more dilegent and make sure a corner off parts of my day to reviews. Probably healther for my mind to take a break an conext switch anyway :)
21:11:28 <timburke> thanks! don't worry too much if you don't find time; we'll likely get it merged by next week, i think
21:11:34 <timburke> #topic part-number APIs
21:12:13 <timburke> on top of that refactor, we've got a fairly classic new-slo-feature, new-s3api-feature chain
21:12:25 <timburke> #link https://review.opendev.org/c/openstack/swift/+/894570
21:12:35 <timburke> #link https://review.opendev.org/c/openstack/swift/+/894580
21:13:31 <mattoliver> yeah, kk so the ones that continue on the SLO refactor chain (for those playing along at home)
21:14:03 <timburke> these are also coming along, though there's a a bit more i'd like to see before they merge, specifically around making sure we send back X-Parts-Count and Content-Range when appropriate
21:15:00 <mattoliver> Cool, I'll work up the chain, as getting the refactor squared away is need first.
21:17:11 <mattoliver> Might also do a local refactor of tracing on top and see how SLO actions look in a trace in comparision I assume I sholdn't see any difference.. but might be nice to check :hmm:
21:18:09 <timburke> oh, nice idea! yeah, i'd expect they should all look more or less the same (at least, until you start exercising the new api)
21:18:21 <timburke> #topic wsgi server management
21:18:39 <mattoliver> yeah, 100% expect so. Just want to get used to doing it.. in case we see differences
21:18:50 <timburke> i've been trying to browbeat clayg into reviewing some patches of mine that we've been carrying for a while
21:19:23 <timburke> one's landed already
21:19:26 <timburke> #link https://review.opendev.org/c/openstack/swift/+/837633
21:20:19 <timburke> the last in that chain still needs some review, though
21:20:21 <timburke> #link https://review.opendev.org/c/openstack/swift/+/837641
21:21:07 <timburke> that will remove the process-tree polling from `swift-reload` in favor of a more explicit signal that the reload has completed
21:21:14 <timburke> but there were some other patches going, too
21:21:27 <timburke> #link https://review.opendev.org/c/openstack/swift/+/789035
21:22:49 <timburke> ensures old worker processes don't hang around indefinitely following a reload (we've occasionally had ops notice old workers continuing to process in-flight requests for many hours after a reload)
21:23:54 <timburke> which reminded clay that he didn't remember how reloads work
21:24:01 <timburke> so i wrote up some docs about it!
21:24:06 <timburke> #link https://review.opendev.org/c/openstack/swift/+/900256
21:24:30 <mattoliver> oh yeah. I think remembering the process polling is only a intermediate change.. I wonder if I should review the new approach first. If we like that we could skip the ps polling patch altogether (unless its too big a patch to merge).
21:24:36 <mattoliver> \o/ docs! awesome
21:25:44 <timburke> too late on the process-tree polling! sorry. https://github.com/openstack/swift/blob/master/swift/cli/reload.py#L109
21:25:55 <mattoliver> yeah, I needed to read up on systemd stuff when I wanted to review. nice one man! I just wish gerrit could display SVGs.
21:26:33 <mattoliver> oh well, I probably merged that so it's my bad :P
21:26:39 <timburke> the docs preview site is pretty good: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_34c/900256/2/check/openstack-tox-docs/34c02b1/docs/overview_wsgi_reload.html
21:27:38 <mattoliver> oh that is cool. I'd have just looked locally, but that's much easier!
21:29:15 <timburke> i'm realizing that it maybe should be a more-general "WSGI Server Process Management" page, though, with a starting section about how graceful shutdowns work
21:29:36 <timburke> working through all of these, clayg and i have been debating about finding a way to get rid of the socket-closer process, so i might spend some time looking into that next
21:30:39 <timburke> oh! and there was one other patch i was reminded of:
21:30:46 <timburke> #link https://review.opendev.org/c/openstack/swift/+/747496
21:32:23 <timburke> currently, if you send two USR1 signals in quick succession, the second one can land in between the execv call and installing the new signal handler -- in that case, we've still got the defaul handler, which just kills the process
21:33:08 <mattoliver> oh, yeah opps.
21:33:56 <mattoliver> +1 to a wsgi server process page too.
21:34:10 <timburke> that patch sets a signal mask before execv, then clears it after we've got the handler installed. unfortunately, it had to dip into ctypes to work on py2 -- but maybe i could let the bug linger on py2?
21:35:18 <mattoliver> yeah, I think we're getting ready to drop py2, so don't go to too much effort. Like atm tracing is a noop in py2. Move to py3 if you want it.
21:35:44 <timburke> all right, i think that's all i've got
21:35:48 <timburke> #topic open discussion
21:35:57 <timburke> anything else we ought to bring up this week?
21:36:36 <mattoliver> nope, I mean I do want to work on some sharding improvements. Maybe play with high water marks in shard replication.. but just haven't had time.. maybe soon.
21:38:11 <timburke> all right then -- we'll stop a little early
21:38:23 <timburke> thank you all for coming, and thank you for working on swift!
21:38:27 <timburke> #endmeeting