Wednesday, 2021-12-01

opendevreviewTim Burke proposed openstack/swift master: updater: Add timing stats  https://review.opendev.org/c/openstack/swift/+/81973201:11
opendevreviewTim Burke proposed openstack/swift master: updater: Skip updates to overloaded nodes  https://review.opendev.org/c/openstack/swift/+/81993101:11
opendevreviewTim Burke proposed openstack/swift master: updater: Skip updates to overloaded nodes  https://review.opendev.org/c/openstack/swift/+/81993101:34
opendevreviewMatthew Oliver proposed openstack/swift master: WIP: First attempt at a ArributeRateLimit Iterator  https://review.opendev.org/c/openstack/swift/+/81769906:25
opendevreviewAlistair Coles proposed openstack/swift master: internal-client: pass global_conf to loadapp  https://review.opendev.org/c/openstack/swift/+/82002913:59
mattoliverunfortuantly, the wife needs to head into the office this morning, so I'm on kid drop off which will happen right on meeting time :( 20:31
timburke_👍 no worries20:53
*** timburke_ is now known as timburke20:53
zaitcevI edited the agenda a little.20:53
kotagood morning20:56
timburkezaitcev, was it the "small tasks suitable for students" item? i can make sure we bring it up (as it is, i'm not sure we'll get to all of what's on there... though maybe some of the items will largely amount to "nope, still haven't looked at it")20:57
timburke#startmeeting swift21:00
opendevmeetMeeting started Wed Dec  1 21:00:08 2021 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.21:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:00
opendevmeetThe meeting name has been set to 'swift'21:00
timburkewho's here for the swift meeting?21:00
zaitcevo/21:00
kotao/21:00
acoleso/21:01
timburkei know mattoliver is unavailable right now, but clayg and acoles may still be around21:01
timburkefirst up, a couple discussions i saw on the ML that i thought were worth pointing out21:01
timburke#topic release cadence21:01
timburke#link http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025684.html21:02
clayg👋21:02
timburkei don't think this will really impact us much, but thought it was worth highlighting that the larger openstack community seems interested in adjusting to a longer release cycle21:02
timburkewe'll continue to release as often as we figure is necessary, though i might need to remember to actually take advantage of that "with intermediary" more often ;-)21:03
timburke#topic minimum python version21:04
timburke#link http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025988.html21:04
timburkefor the yoga cycle, openstack as a whole is committed to testing under py38 and py3921:05
timburke#link https://governance.openstack.org/tc/reference/runtimes/yoga.html21:05
claygRC cadence was interesting; I hadn’t seen that.21:06
timburkepeople are understandably skittish of trying to support untested pythoin versions -- i wouldn't be surprised if there were constructions in py38 that would SyntaxError on py36. so they're debating about proposing python_requires='>=3.8' all over the place21:07
timburkeit doesn't immediately impact us -- i figure as long as we're supporting py27, keeping support for py36 seems trivial21:07
timburkebut it got me wonder: can we finally commit to dropping py27 support in some forseeable time frame?21:08
timburkethis might largely be a question for acoles, clayg, and i -- i *think* we're the main stragglers that haven't made the jump yet21:09
timburke(though it'd probably be worth checking in with someone at OVH to see how they're doing)21:09
timburkemore directly: is anyone actively *opposed* to dropping py27 support following the yoga release (~end of March)?21:11
claygtimburke: you know me - as soon as we're off py27 I don't care how fast we drop it - I think we could even make the case that it should be dropped in spite of us (and we may not even find that to agreeable if it becomes too difficult to support py27 on zuul/openstack)21:11
claygtimburke: that might be a reasonable timeline to put down as the plan of record; but if we agree here we could ask the same question on the ML21:12
timburkeyes -- i'll definitely do so. send out something like "we're finally getting off py2, yoga will be the last to support it, and backports to yoga may become difficult -- is there anyone who actively still *wants* py2? speak now or forever hold your peace"21:14
acoles+121:15
timburkeso -- assuming we drop py2 -- what *should* our minimum supported python be at that point?21:15
kotait sounds reasonable, i think my older costumers is still running on py2 but they won't to bump Swift version as well as python version...21:16
zaitcevI'm okay with anything. I'll have to find issues when I backport to py36, but I can live with it.21:16
kotas/is/are/21:16
kotaso i can recommend to use py3 definately21:16
zaitcevIf we drop py2, we can finally merge Romain's parting gift, the elimination of queues.21:17
timburkeoh yeah -- i'd forgotten that only had trouble on py2...21:17
timburkefwiw, py36 eol is scheduled for around the end of the year -- but i know redhat will continue to service it for a while after21:19
zaitcevFor many years.21:20
timburkezaitcev, fwiw i'm not opposed to keeping py36 testing -- might be the sort of thing where we drop py2 first, then see how much we're actually itching to use the newest features21:20
zaitcevtimburke: Sounds good to me.21:21
timburkeall right, i'll plan on sending out a message to the ML about finally dropping py2 support then21:21
timburke#topic ptg action items21:22
timburkeit's been a bit since we last checked in -- let's see how we're doing on these ;-)21:22
timburkei never did give feedback on the interop change :-(21:23
zaitcevWhat's "(timburke) abandon old patches"21:23
timburkethough now it's abandoned, so... *shrug*21:23
timburkezaitcev, i haven't done much with it recently, but earlier i'd abandoned all the outstanding hummingbird and losf patches, for example21:24
zaitcevI see, thanks.21:24
zaitcevEthercalc throws 50321:25
timburkewowza -- we've still got a patch open from 2013... https://review.opendev.org/q/p:openstack/swift+is:open,50021:26
timburkezaitcev, i noticed that, too -- unfortunate that mattoliver isn't around to update us on that21:27
timburkebut i did at least make some progress on dropping logging translations!21:28
timburkeacoles, ever get a chance to think more about the dark data watcher? https://review.opendev.org/c/openstack/swift/+/78765621:28
acoles:( to my shame no. I mean, I have thought but not acted.21:29
acolesit is on my radar though. apologies zaitcev , we've had a busy time internally recently21:29
timburkeno worries -- we've had some other fires to deal with :-)21:29
zaitcevwell I worry... a little bit... :-)21:29
timburkeall right -- there's been *some* progress on these, and we'll continue to try to do better ;-)21:30
timburkenext up, i noticed a few items that have mostyly already been resolved, but seem worth calling out21:31
timburke#topic reconstructor not cleaning up handoffs21:31
timburkewe saw an issue in our big cluster recently when we were doing a (relatively small) expansion21:32
timburkeeven though we were running with handoffs_only, a bunch of handoff partitions wouldn't get cleaned up even after several reconstructor cycles21:32
timburkethe crux seemed to be that we weren't cleaning up non-durable data properly, even though we *did* revert it -- which should be addressed by https://review.opendev.org/c/openstack/swift/+/81855821:33
acolesmy bad, we fixed it once then broke it so now we fixed it again21:34
timburkei think i may want to backport it to xena...21:34
acolesyes21:35
timburkehaving the partition hang around led to us noticing another bug: .meta files weren't getting cleaned up as part of a revert21:36
timburkewhich should be fixed by https://review.opendev.org/c/openstack/swift/+/81854421:36
timburke(i'm less certain that one needs to be backported)21:36
acolesI think we concluded that in the absence of the lingering non-durable, the meta files would be cleaned up eventually. The patch just does it 'deliberately and properly'.21:38
timburkebut one thing *that* investigation led to was the discovery that a lone .meta doesn't get reflected in hashes.pkl and no attempt is made to send that bit of information to peers (at least, for ec policies)21:38
timburkei'm not sure whether that ought to be considered a bug or not, but it seemed like something worth calling out21:39
timburkeacoles, yeah -- i think i'm leaning toward *not* backporting the second one21:40
acoles+121:40
timburkefwiw, we just upgraded production with those patches today, and our handoff counts are looking *way* better :-)21:40
timburke#topic arm jobs now voting21:41
timburkeacoles fixed something that had been bugging me for a bit -- when the arm jobs would fail, zuul would still send a comment saying the build succeeded21:42
acoles...and worse...I had been believing zuul! :)21:42
timburketrouble was that i'd marked them non-voting in the definition, because i was worried that it'd make zuul vote -1 when they failed. but the pipeline wasn't set up to leave +1/-1, so my concern was unwarranted21:43
timburke#topic tempurl signature logging21:43
acolesto be fair, clarkb shared the wisdom and I merely pressed keys21:43
timburke:-)21:44
timburke#link https://bugs.launchpad.net/swift/+bug/168579821:44
timburke#link https://review.opendev.org/c/openstack/swift/+/81747621:44
timburkehas anyone had a chance to review this?21:44
timburkei'm not seeing much activity since the patch was proposed ~3 weeks ago, just want to make sure it doesn't fall off our radars21:45
zaitcevI'm looking.21:47
timburkei'll try to do another pass on it this week, maybe experiment with moving the redaction into proxy-logging (which already does something similar with auth tokens)21:47
acolestimburke: where did your thinking end up re moving the fix to the proxy-logging?21:48
zaitcevenv.setdefault('obsucre_map', {}) -- mattoliver: I do think the tests would catch this :-)21:48
timburkeacoles, mainly that i wanted to try it out and see how it looked before committing to it ;-)21:49
acoleskk21:49
timburkewe'll check in on it again next week21:49
timburke#topic small tasks suitable for a term project21:49
acoleszaitcev: lol. "obsucre" may be some kind of hidden sugar?! (a terrible play on my school boy french)21:50
timburkezaitcev, i think this was your topic?21:50
acoleszaitcev: we do have https://bugs.launchpad.net/swift/+bugs?field.tag=low-hanging-fruit21:51
zaitcevThey are dsariel's students. I forgot to ask what year or level they are at.21:51
zaitcevacoles: thanks, I'll pass that along and then we'll review the list.21:52
acolesbe sure to alert us if/when a student joins the community so we can do our best to help21:53
timburkei think my main question is: what do we expect the student to want to get out of it? just looking to work on their python? an opportunity to interact with an open source community? a chance to think hard about distributed systems? there's probably a bit of a spectrum21:53
timburkeand different tasks would suit different goals21:54
zaitcevThinking hard may be a little early. Or, rather, it's never too early, but we cannot expect anything to come out.21:54
zaitcevSo, the first two.21:54
timburke👍21:54
zaitcevAllright, this is good enough for now. I was hoping if you had a pet project or fix, but the list is good.21:55
timburkethose low-hanging-fruit bugs seem like good choices, then. i'll try to write up a few more of them if i think of anything new21:56
timburkeall right -- we're pretty close to time. i'll save the other topics for next week21:56
timburke#topic open discussion21:56
timburkeit's been a long time since our last meeting -- is there anything else we should bring up before the end?21:57
timburkeall right then. thank you all for coming, and thank you for working on swift!22:01
timburke#endmeeting22:01
opendevmeetMeeting ended Wed Dec  1 22:01:09 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)22:01
opendevmeetMinutes:        https://meetings.opendev.org/meetings/swift/2021/swift.2021-12-01-21.00.html22:01
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/swift/2021/swift.2021-12-01-21.00.txt22:01
opendevmeetLog:            https://meetings.opendev.org/meetings/swift/2021/swift.2021-12-01-21.00.log.html22:01

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!