| opendevreview | Merged openstack/swift master: memcached: handle failed incr+add+incr better https://review.opendev.org/c/openstack/swift/+/880677 | 02:09 |
|---|---|---|
| opendevreview | Romain de Joux proposed openstack/swift master: Encode header in latin-1 with wsgi_to_bytes https://review.opendev.org/c/openstack/swift/+/884240 | 11:06 |
| opendevreview | Merged openstack/swift master: memcached: add method and time_spent to exception logs https://review.opendev.org/c/openstack/swift/+/884516 | 15:10 |
| opendevreview | Tim Burke proposed openstack/swift master: proxy: Bring back logging for get_*_info requests https://review.opendev.org/c/openstack/swift/+/884931 | 18:59 |
| opendevreview | Tim Burke proposed openstack/swift master: proxy: Bring back logging/metrics for get_*_info requests https://review.opendev.org/c/openstack/swift/+/884931 | 19:21 |
| timburke | #startmeeting swift | 21:00 |
| opendevmeet | Meeting started Wed May 31 21:00:33 2023 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. | 21:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 21:00 |
| opendevmeet | The meeting name has been set to 'swift' | 21:00 |
| timburke | who's here for the swift meeting? | 21:00 |
| zaitcev | o/ | 21:00 |
| acoles | o/ | 21:01 |
| timburke | as usual, the agenda's at https://wiki.openstack.org/wiki/Meetings/Swift | 21:01 |
| timburke | i only had a couple items i wanted to bring up | 21:02 |
| zaitcev | I don't know why, but I'm here. I guess I'll just try and ask acoles to look at https://review.opendev.org/c/openstack/swift/+/787656 | 21:02 |
| timburke | i keep meaning to circle back on that one, too :-( | 21:02 |
| timburke | #topic py3 ssync metadata bug | 21:03 |
| timburke | and the fix at | 21:04 |
| timburke | #link https://review.opendev.org/c/openstack/swift/+/884240 | 21:04 |
| acoles | zaitcev: darn, sorry! | 21:04 |
| timburke | i haven't left my review yet, but i should later today. i've got a couple small changes to tests i'd like to see, and i'm part way into updating a probe test -- but it's a little tricky since swiftclient can't parse the non-ascii meta :-( | 21:05 |
| timburke | core change looks correct, though | 21:06 |
| timburke | i'll probably push over with my unit test changes but leave the probe test as a follow-up | 21:07 |
| timburke | #topic logging/metrics for account/container info requests | 21:07 |
| timburke | i realized this morning that we no longer emit subrequest logging/metrics for get_account/container_info requests | 21:08 |
| timburke | it was an unintended consequence of https://review.opendev.org/c/openstack/swift/+/875819 | 21:08 |
| timburke | by going directly to the proxy-app, i bypassed logging :-( | 21:09 |
| timburke | i've got a fix up -- it's still a little ugly, though | 21:09 |
| timburke | #link https://review.opendev.org/c/openstack/swift/+/884931 | 21:09 |
| mattoliver | o/ (sorry im late) | 21:11 |
| acoles | I'm reminded of some patch mattoliver was working on where we added a pipeline property that held a list of the apps...was it the internal client iter_shard_ranges patch? | 21:11 |
| timburke | i'm torn about whether to stick with what i've got there (take the second-to-last item in the pipeline, assuming it's a ProxyLoggingMiddleware, otherwise go to the proxy app), or go searching for the right-most logging filter | 21:12 |
| timburke | acoles, https://review.opendev.org/c/openstack/swift/+/879128 i think | 21:12 |
| mattoliver | Yeah we're was that. But yeah it stored them as a list, so it would be easy to search | 21:13 |
| acoles | yeah, thats' where we added _pipeline | 21:13 |
| timburke | it's definitely handy for this fix :-) | 21:13 |
| acoles | timburke: are you concerned about the overhead of searching every time? | 21:14 |
| timburke | no -- my concern is more that there could be some middleware with unexpected consequences in between the last logging mware and proxy-server | 21:15 |
| timburke | something that could do some sysmeta manipulations, say. also, a little worried about the possibility of recursion errors | 21:16 |
| acoles | oh, now I'm confused...can you elaborate? what advantage does _pipeline[-2] have? | 21:16 |
| zaitcev | I gave up wrapping my mind around that too. | 21:17 |
| acoles | I guess [-2] means logging is the last middleware, then there's the app | 21:19 |
| timburke | yeah | 21:19 |
| timburke | if it's proxy logging, great. we wrote that and can trust it. otherwise, the patch as-written goes to the proxy app; we lose logging/metrics again, but at least we know all sysmeta, acls, etc in the response is intact | 21:19 |
| timburke | i don't think it's *likely* that anyone would have configured their pipelines with a problematic middleware in between last logging and proxy-server, but it made me a little nervous | 21:20 |
| acoles | but documenting 'if you put *any* middleware after the final logger then there's some metric that may not be emitted' seems odd vs documenting 'if you put middleware after logging that does weird stuff then blah blah' | 21:20 |
| acoles | what would have happened, with a weird middleware, before you added the cut-through | 21:22 |
| timburke | you'd get weird behaviors. all the more so if it's non-deterministic at which point in the pipeline the get-info call happens | 21:23 |
| timburke | getting rid of the possibility of weird behaviors was definitely on my mind when i did the cut-through -- i'm not thrilled about re-introducing them :-/ but maybe it's the most defensible path | 21:24 |
| acoles | I think I need to study the patch and the weirdness potential when I'm more awake | 21:25 |
| acoles | thanks for alerting us to it! | 21:25 |
| mattoliver | +1 | 21:25 |
| timburke | seems fair. fwiw, one of the weird cases i was thinking of was something like the 1space shunt, where you're trying to merge namespaces from different accounts/clusters | 21:26 |
| timburke | anyway, that's all i've got | 21:26 |
| timburke | #topic open discussion | 21:26 |
| timburke | what else should we bring up this week? | 21:26 |
| mattoliver | I've been out this week, so I've got nothing I can think of atm | 21:27 |
| zaitcev | Me neither. | 21:27 |
| timburke | all right. i think i'll wrap it early, then | 21:28 |
| acoles | we shipped this change this week https://review.opendev.org/c/openstack/swift/+/883367 | 21:28 |
| acoles | and it does seem to have eliminated some curious timeouts | 21:28 |
| timburke | yes! it seemed so strange that we were bumping memcached timeouts at all, much less so long after the configured timeout | 21:30 |
| acoles | I guess one takeaway is that the network could move bytes faster than the proxy could EC encode them, so network never would block and the EC put greenthread never would yield to another greenthread :O | 21:30 |
| timburke | all right | 21:32 |
| timburke | thank you all for coming, and thank you for working on swift! | 21:32 |
| timburke | #endmeeting | 21:32 |
| opendevmeet | Meeting ended Wed May 31 21:32:14 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 21:32 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/swift/2023/swift.2023-05-31-21.00.html | 21:32 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/swift/2023/swift.2023-05-31-21.00.txt | 21:32 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/swift/2023/swift.2023-05-31-21.00.log.html | 21:32 |
| acoles | o/ | 21:32 |
| opendevreview | Tim Burke proposed openstack/swift master: Encode header in latin-1 with wsgi_to_bytes https://review.opendev.org/c/openstack/swift/+/884240 | 23:07 |
| opendevreview | Tim Burke proposed openstack/swift master: Add non-ascii meta to ssync probe test https://review.opendev.org/c/openstack/swift/+/884954 | 23:07 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!