21:00:32 #startmeeting swift 21:00:32 Meeting started Wed Nov 8 21:00:32 2023 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:32 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:32 The meeting name has been set to 'swift' 21:00:41 who's here for the swift meeting? 21:01:52 o/ 21:02:37 Although I had a very distracted week and didn't get much Swift work done.. but I'm here :) 21:03:45 o/ 21:03:49 Tim Burke proposed openstack/swift master: slo: refactor GET/HEAD response handling https://review.opendev.org/c/openstack/swift/+/893578 21:04:10 so i neglected to update the agenda again (sorry) 21:05:07 but the main things i've been thinking about are (1) the SLO refactors that are stacked in front of the part-number APIs 21:05:37 and (2) how we manage our processes (especially WSGI servers) 21:06:06 yeah, the chain seems to be getting long. And a bit daunting. How are they looking? 21:06:32 #topic SLO refactor 21:06:50 the first part has now merged, just refactoring tests 21:06:59 #link https://review.opendev.org/c/openstack/swift/+/896466 21:07:59 the second part (which *does* impact how SLO behaves) is coming along; i've been pushing up a few fixups today 21:08:02 #link https://review.opendev.org/c/openstack/swift/+/893578 21:08:48 i think at this point i'm happy with it; if anyone else wants to take a look, you might want to do that soon 21:09:43 as it is, clayg, acoles, and indianwhocodes are all listed as authors, and i'm likely to put a +2 on it once we get a clean test run 21:10:21 OK. I'll be more dilegent and make sure a corner off parts of my day to reviews. Probably healther for my mind to take a break an conext switch anyway :) 21:11:28 thanks! don't worry too much if you don't find time; we'll likely get it merged by next week, i think 21:11:34 #topic part-number APIs 21:12:13 on top of that refactor, we've got a fairly classic new-slo-feature, new-s3api-feature chain 21:12:25 #link https://review.opendev.org/c/openstack/swift/+/894570 21:12:35 #link https://review.opendev.org/c/openstack/swift/+/894580 21:13:31 yeah, kk so the ones that continue on the SLO refactor chain (for those playing along at home) 21:14:03 these are also coming along, though there's a a bit more i'd like to see before they merge, specifically around making sure we send back X-Parts-Count and Content-Range when appropriate 21:15:00 Cool, I'll work up the chain, as getting the refactor squared away is need first. 21:17:11 Might also do a local refactor of tracing on top and see how SLO actions look in a trace in comparision I assume I sholdn't see any difference.. but might be nice to check :hmm: 21:18:09 oh, nice idea! yeah, i'd expect they should all look more or less the same (at least, until you start exercising the new api) 21:18:21 #topic wsgi server management 21:18:39 yeah, 100% expect so. Just want to get used to doing it.. in case we see differences 21:18:50 i've been trying to browbeat clayg into reviewing some patches of mine that we've been carrying for a while 21:19:23 one's landed already 21:19:26 #link https://review.opendev.org/c/openstack/swift/+/837633 21:20:19 the last in that chain still needs some review, though 21:20:21 #link https://review.opendev.org/c/openstack/swift/+/837641 21:21:07 that will remove the process-tree polling from `swift-reload` in favor of a more explicit signal that the reload has completed 21:21:14 but there were some other patches going, too 21:21:27 #link https://review.opendev.org/c/openstack/swift/+/789035 21:22:49 ensures old worker processes don't hang around indefinitely following a reload (we've occasionally had ops notice old workers continuing to process in-flight requests for many hours after a reload) 21:23:54 which reminded clay that he didn't remember how reloads work 21:24:01 so i wrote up some docs about it! 21:24:06 #link https://review.opendev.org/c/openstack/swift/+/900256 21:24:30 oh yeah. I think remembering the process polling is only a intermediate change.. I wonder if I should review the new approach first. If we like that we could skip the ps polling patch altogether (unless its too big a patch to merge). 21:24:36 \o/ docs! awesome 21:25:44 too late on the process-tree polling! sorry. https://github.com/openstack/swift/blob/master/swift/cli/reload.py#L109 21:25:55 yeah, I needed to read up on systemd stuff when I wanted to review. nice one man! I just wish gerrit could display SVGs. 21:26:33 oh well, I probably merged that so it's my bad :P 21:26:39 the docs preview site is pretty good: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_34c/900256/2/check/openstack-tox-docs/34c02b1/docs/overview_wsgi_reload.html 21:27:38 oh that is cool. I'd have just looked locally, but that's much easier! 21:29:15 i'm realizing that it maybe should be a more-general "WSGI Server Process Management" page, though, with a starting section about how graceful shutdowns work 21:29:36 working through all of these, clayg and i have been debating about finding a way to get rid of the socket-closer process, so i might spend some time looking into that next 21:30:39 oh! and there was one other patch i was reminded of: 21:30:46 #link https://review.opendev.org/c/openstack/swift/+/747496 21:32:23 currently, if you send two USR1 signals in quick succession, the second one can land in between the execv call and installing the new signal handler -- in that case, we've still got the defaul handler, which just kills the process 21:33:08 oh, yeah opps. 21:33:56 +1 to a wsgi server process page too. 21:34:10 that patch sets a signal mask before execv, then clears it after we've got the handler installed. unfortunately, it had to dip into ctypes to work on py2 -- but maybe i could let the bug linger on py2? 21:35:18 yeah, I think we're getting ready to drop py2, so don't go to too much effort. Like atm tracing is a noop in py2. Move to py3 if you want it. 21:35:44 all right, i think that's all i've got 21:35:48 #topic open discussion 21:35:57 anything else we ought to bring up this week? 21:36:36 nope, I mean I do want to work on some sharding improvements. Maybe play with high water marks in shard replication.. but just haven't had time.. maybe soon. 21:38:11 all right then -- we'll stop a little early 21:38:23 thank you all for coming, and thank you for working on swift! 21:38:27 #endmeeting