Wednesday, 2021-03-03

*** johanssone has quit IRC00:16
*** jmasud has quit IRC00:16
*** tosky has quit IRC00:17
*** martial has quit IRC00:19
*** johanssone has joined #openstack-meeting00:19
*** jmasud has joined #openstack-meeting00:24
*** yasufum has joined #openstack-meeting00:36
*** macz_ has quit IRC00:49
*** SotK has quit IRC00:58
*** SotK has joined #openstack-meeting00:58
*** mlavalle has quit IRC01:02
*** gyee has quit IRC01:17
*** jmasud has quit IRC01:30
*** jmasud has joined #openstack-meeting01:30
*** jmasud has quit IRC01:33
*** jmasud has joined #openstack-meeting01:34
*** rcernin has quit IRC02:00
*** rcernin has joined #openstack-meeting02:29
*** rcernin has quit IRC02:29
*** gagehugo has quit IRC02:29
*** rcernin has joined #openstack-meeting02:30
*** jmasud has quit IRC02:31
*** gagehugo has joined #openstack-meeting02:33
*** rcernin has quit IRC02:44
*** macz_ has joined #openstack-meeting02:46
*** macz_ has quit IRC02:50
*** jamesdenton has quit IRC02:58
*** jamesdenton has joined #openstack-meeting02:58
*** yasufum has quit IRC03:01
*** rcernin has joined #openstack-meeting03:16
*** jmasud has joined #openstack-meeting03:27
*** dasp has quit IRC03:34
*** yasufum has joined #openstack-meeting03:43
*** dasp has joined #openstack-meeting03:45
*** psachin has joined #openstack-meeting03:50
*** jmasud has quit IRC04:11
*** jmasud has joined #openstack-meeting04:12
*** udesale has joined #openstack-meeting04:35
*** vishalmanchanda has joined #openstack-meeting05:27
*** evrardjp has quit IRC05:33
*** evrardjp has joined #openstack-meeting05:33
*** jamesdenton has quit IRC06:27
*** jamesdenton has joined #openstack-meeting06:27
*** bnemec has quit IRC06:41
*** bnemec has joined #openstack-meeting06:49
*** ralonsoh has joined #openstack-meeting07:37
*** belmoreira has joined #openstack-meeting07:44
*** rcernin has quit IRC07:44
*** rpittau|afk is now known as rpittau07:52
*** jmasud has quit IRC07:53
*** yasufum has quit IRC08:01
*** dklyle has quit IRC08:03
*** jmasud has joined #openstack-meeting08:07
*** ociuhandu has joined #openstack-meeting08:13
*** yasufum has joined #openstack-meeting08:18
*** browny has quit IRC08:24
*** ociuhandu has quit IRC08:30
*** tosky has joined #openstack-meeting08:34
*** ociuhandu has joined #openstack-meeting08:35
*** jmasud has quit IRC08:39
*** browny has joined #openstack-meeting08:42
*** ociuhandu has quit IRC09:39
*** ociuhandu has joined #openstack-meeting10:04
*** lpetrut has joined #openstack-meeting10:22
*** ociuhandu has quit IRC10:28
*** ociuhandu has joined #openstack-meeting10:37
*** jmasud has joined #openstack-meeting10:38
*** whoami-rajat__ has joined #openstack-meeting11:11
*** yasufum has quit IRC11:17
*** ociuhandu has quit IRC11:41
*** dsariel has joined #openstack-meeting11:51
*** udesale_ has joined #openstack-meeting11:57
*** jmasud has quit IRC11:59
*** udesale has quit IRC12:00
*** ociuhandu has joined #openstack-meeting12:12
*** ociuhandu has quit IRC12:17
*** ociuhandu has joined #openstack-meeting12:37
*** psachin has quit IRC13:01
*** rh-jlabarre has quit IRC13:01
*** rh-jelabarre has joined #openstack-meeting13:11
*** rh-jelabarre has quit IRC13:11
*** rh-jelabarre has joined #openstack-meeting13:12
*** e0ne has joined #openstack-meeting13:45
*** paulbisaacs has quit IRC13:51
*** ociuhandu has quit IRC13:58
*** ociuhandu has joined #openstack-meeting13:59
*** thgcorrea has joined #openstack-meeting14:02
*** ociuhandu has quit IRC14:04
*** ociuhandu has joined #openstack-meeting14:07
*** takamatsu has joined #openstack-meeting14:17
*** ociuhandu has quit IRC14:27
*** ociuhandu has joined #openstack-meeting14:28
*** ociuhandu has quit IRC14:33
*** ociuhandu has joined #openstack-meeting14:40
*** lpetrut has quit IRC15:09
*** hemna has quit IRC15:39
*** hemna has joined #openstack-meeting15:40
*** dklyle has joined #openstack-meeting15:40
*** takamatsu has quit IRC16:11
*** ociuhandu has quit IRC16:15
*** yasufum has joined #openstack-meeting16:18
*** smcginnis has quit IRC16:25
*** macz_ has joined #openstack-meeting16:25
*** ociuhandu has joined #openstack-meeting16:31
*** smcginnis has joined #openstack-meeting16:34
*** e0ne has quit IRC16:37
*** macz_ has quit IRC16:46
*** macz_ has joined #openstack-meeting16:47
*** mlavalle has joined #openstack-meeting16:51
*** udesale_ has quit IRC16:52
*** hemna has quit IRC17:01
*** browny has quit IRC17:01
*** ralonsoh has quit IRC17:01
*** stand has quit IRC17:01
*** etp has quit IRC17:01
*** redrobot has quit IRC17:01
*** radez has quit IRC17:01
*** moguimar has quit IRC17:01
*** imcsk8 has quit IRC17:01
*** lifeless_ has quit IRC17:01
*** mahatic has quit IRC17:01
*** ralonsoh has joined #openstack-meeting17:05
*** stand has joined #openstack-meeting17:05
*** hemna has joined #openstack-meeting17:07
*** browny has joined #openstack-meeting17:07
*** radez has joined #openstack-meeting17:07
*** etp has joined #openstack-meeting17:07
*** redrobot has joined #openstack-meeting17:07
*** mahatic has joined #openstack-meeting17:07
*** moguimar has joined #openstack-meeting17:07
*** imcsk8 has joined #openstack-meeting17:07
*** lifeless_ has joined #openstack-meeting17:07
*** e0ne has joined #openstack-meeting17:13
*** yasufum has quit IRC17:18
*** ociuhandu_ has joined #openstack-meeting17:21
*** ociuhandu has quit IRC17:25
*** ociuhandu_ has quit IRC17:26
*** e0ne has quit IRC17:26
*** macz_ has quit IRC17:27
*** whoami-rajat__ has quit IRC17:29
*** jmasud has joined #openstack-meeting17:31
*** ociuhandu has joined #openstack-meeting17:32
*** ociuhandu has quit IRC17:37
*** e0ne has joined #openstack-meeting17:43
*** e0ne has quit IRC17:43
*** jeremyfreudberg has left #openstack-meeting17:48
*** jmasud has quit IRC17:54
*** jmasud has joined #openstack-meeting17:57
*** e0ne has joined #openstack-meeting17:59
*** yasufum has joined #openstack-meeting18:08
*** rpittau is now known as rpittau|afk18:11
*** ralonsoh has quit IRC18:23
*** lpetrut has joined #openstack-meeting18:38
*** lpetrut has quit IRC18:49
*** yishihara has quit IRC18:51
*** yasufum has quit IRC19:14
*** jmasud has quit IRC19:19
*** jmasud has joined #openstack-meeting19:22
*** jmasud has quit IRC19:37
*** lifeless_ is now known as lifeless19:56
*** jmasud has joined #openstack-meeting20:01
*** slaweq has quit IRC20:04
*** rcernin has joined #openstack-meeting20:46
*** slaweq has joined #openstack-meeting20:57
timburke__#startmeeting swift21:00
openstackMeeting started Wed Mar  3 21:00:02 2021 UTC and is due to finish in 60 minutes.  The chair is timburke__. Information about MeetBot at http://wiki.debian.org/MeetBot.21:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:00
*** openstack changes topic to " (Meeting topic: swift)"21:00
*** acoles has joined #openstack-meeting21:00
openstackThe meeting name has been set to 'swift'21:00
timburke__who's here for the swift meeting?21:00
acoleso/21:00
mattoliverauo/21:00
kota_hi21:00
seongsoochoo/21:00
rledisezo/21:00
*** gyee has joined #openstack-meeting21:01
*** zaitcev has joined #openstack-meeting21:01
timburke__as usual, the agenda's at https://wiki.openstack.org/wiki/Meetings/Swift21:02
timburke__first up21:02
timburke__#topic sharding backports21:02
*** openstack changes topic to "sharding backports (Meeting topic: swift)"21:02
timburke__they're all merged!21:02
mattoliverau\o/ nice21:02
kota_great21:03
timburke__unfortunately, zaitcev doesn't seem to be here right now, or i'd ask him if there was anything else he needed (either more patches backported, since we've had some more sharding-related patches on master, or stable releases/tags requested)21:03
timburke__so i don't know that there's much to discuss, really. just keeping people posted21:04
timburke__but speaking of...21:04
timburke__#topic releases21:04
*** openstack changes topic to "releases (Meeting topic: swift)"21:04
zaitcevtimburke__: I think it's enough, because it's going to be auto-sharding and stuff from there on.21:04
timburke__ok, sounds good. fwiw, we still haven't touched the autosharding algorithm or switched it to default to on or anything21:05
timburke__mainly just putting more pieces in place to make it easier to recover if sharding goes sideways and you end up with overlaps21:05
timburke__we're getting to the end of the cycle! we should do some releases (both client and server)21:06
mattoliverauwhich we need for autosharding :)21:06
mattoliverau+121:06
timburke__i've got a draft of some 2.27.0 release notes at https://review.opendev.org/c/openstack/swift/+/77770821:06
timburke__there's a lot of great stuff!21:06
timburke__please take a look and let me know if anything needs more details or better wording21:07
*** slaweq has quit IRC21:07
timburke__on the client side, the gate's fixed! i still need to write up some proper release notes, though (and it's been a while since i have...)21:08
timburke__i don't think we've got any outstanding issues that we'd want to hold a release on; be sure to let me know if i'm wrong there :-)21:09
timburke__#topic CORS tests21:09
*** openstack changes topic to "CORS tests (Meeting topic: swift)"21:09
timburke__so i wanted to bring this to the community one more time -- acoles clayg and mattoliverau have all at least taken a look at the new tests in https://review.opendev.org/c/openstack/swift/+/533028 and found them more or less useful/readable21:11
timburke__...but the four of us are all at nvidia, and i want to give people a chance to raise any lingering concerns before we go pulling a bunch of javascript tests into our tree21:12
mattoliverauyeah, I know theres some js in there, but once you take a look at the framework its not to complicated and really tests CORS functionality. So I think it's a big win.21:12
kota_README.rst may be the entry point?21:13
timburke__so: kota_, rledisez, zaitcev, seongsoocho -- any worries or doubts that make you think we *shouldn't* add this test suite?21:13
timburke__yeah, that's a good starting point21:13
timburke__fwiw, it's got a (voting!) gate job21:14
rledisezI'm all for it. Better to have JS tests than no test at all. And the JS part doesn't look that terrible21:14
kota_Cool stuff. I know selenium a little sounds nice to test of browser based real CORS21:14
timburke__acoles put up a dnm patch to see what failures look like: https://12f1c315a5e3b29b1269-6e0919d6274382461a92af7573f772ff.ssl.cf5.rackcdn.com/777405/1/check/swift-func-cors/7cfac1e/cors-test-results.txt21:15
acolesFWIW even though I'm at nvidia I did initially push back on timburke__ bringing in the js/browser based tests, but it didn't take me too long to get up to speed on the framework, the README is pretty good, and it does give us robust test capability, so I'm now +221:15
acolesoh, and timburke__ even has the test failures enumerated in an artifact published by zuul :)21:16
timburke__the biggest reason (in my mind) to go ahead and do the in-browser tests is that there's no chance the tests aren't representative of what a client would do21:17
*** rcernin has quit IRC21:17
timburke__i'm not hearing any immediate opposition -- is there anyone that would like to do a review in the next couple days and would rather we not merge it until you have?21:19
acolestick21:21
acolestock21:21
timburke__all right. let's do it then!21:21
acoles:)21:21
timburke__#topic relinker21:21
*** openstack changes topic to "relinker (Meeting topic: swift)"21:21
timburke__so we (nvidia) started a part power increase yesterday!21:22
timburke__and we hit some bugs :-(21:22
timburke__acoles has been great about getting patches up to address some of the races we observed21:22
kota_:(21:22
acoles:'(21:23
*** jmasud has quit IRC21:23
*** e0ne has quit IRC21:23
seongsoocho:(  I'm still doing a relink job .. (almost 2 weeks per node )21:23
timburke__the relinker itself seemed to do well, but the object-server would occassionally fail to link into the new partition, due to one of three errors21:23
acolespatch chain starts here https://review.opendev.org/c/openstack/swift/+/77847421:24
timburke__seongsoocho, ouch! remind me: how full are the nodes? you might want to adjust the ionice to let it go faster -- i feel like 2 weeks is getting to be a long time21:24
seongsoochoeach node has 12 disk (used 3TB)21:25
seongsoocho80% full per node21:26
timburke__do you have an estimate of how many diskfiles are on each disk? iirc we had just under 1,000,000 per disk21:27
*** gary_perkins has joined #openstack-meeting21:27
seongsoochoAnd The log says 1million diskfiles are relinked21:28
timburke__ok. and you're relinking on all nodes at once, yeah?21:29
timburke__or one at a time?21:29
seongsoochoyes  I did it all nodes at once21:30
mattoliveraurunning it concurrently on each device (ie multiple reliners, at least until Tims other patch lands), or one at a time.21:30
timburke__one disk at a time, irrc -- limitation of the stable version seongsoocho is on21:31
timburke__so actually -- i guess 2 weeks doesn't sound so crazy :-(21:31
seongsoochoYes.... too long..21:32
mattoliverauwell good news all that will get quicker, so long as we fix obj race conditions21:32
timburke__ok, i don't know how much else i really want or need to say on this -- i just wanted to give a little update, and a heads-up that there are more part-power-increase patches coming21:32
timburke__#topic shrinking21:33
*** openstack changes topic to "shrinking (Meeting topic: swift)"21:33
timburke__acoles, mattoliverau how's it going?21:33
*** irclogbot_2 has quit IRC21:33
*** bauzas has quit IRC21:33
*** ianw has quit IRC21:33
*** kopecmartin has quit IRC21:33
*** DinaBelova has quit IRC21:33
*** freerunner has quit IRC21:33
*** amotoki has quit IRC21:33
*** janders has quit IRC21:33
*** johnthetubaguy has quit IRC21:33
timburke__(i'm assuming not much progress the last day or two, but figure there may have been some work toward the end of last week ;-)21:33
acolesI can't remember what we reported in last week's meeting, so...21:34
mattoliverauGood, got a little distracted. I'm working on not detecting new ACTIVE shards as shrinkible when there is a small shard container at a primary. An edge case that could be problematic21:34
*** irclogbot_2 has joined #openstack-meeting21:34
*** bauzas has joined #openstack-meeting21:34
*** ianw has joined #openstack-meeting21:34
*** johnthetubaguy has joined #openstack-meeting21:34
*** kopecmartin has joined #openstack-meeting21:34
*** DinaBelova has joined #openstack-meeting21:34
*** freerunner has joined #openstack-meeting21:34
*** amotoki has joined #openstack-meeting21:34
*** janders has joined #openstack-meeting21:34
*** tepper.freenode.net changes topic to "relinker (Meeting topic: swift)"21:34
acolesmattoliverau: has added shrinking candidates to the recon dump21:34
mattoliverauThe recon shrinking_candidates patch landed.21:34
acolesI fixed a couple of issues with the tool to find shrinkable shards21:35
mattoliverauAlso working on a patch for the s-m-s-r tool to use the sharder config, so the tool is easier to use as it'll default to your configuration.21:35
acoles^^ that's almost done IIRC21:36
timburke__cool, sounds good21:36
acolesI'm working on being able to include an estimate of tombstone row count when deciding to shrink - I'm worried about shrinking an apparently empty shard but actually causing large number of tombstone rows to be moved :(21:36
mattoliveraunice21:37
mattoliverauconfig patch: https://review.opendev.org/c/openstack/swift/+/77458421:37
mattoliveraualso been playing with mean and std dev in relation to shards to see if it'll give us a good glance at how distributed the shard ranges are: https://review.opendev.org/c/openstack/swift/+/77792221:38
*** e0ne has joined #openstack-meeting21:38
mattoliverauBut that's just a wip while I play around with displaying some kind of variance number to see distribution at a glance.21:39
timburke__all those sound super useful. keep up the good work!21:39
timburke__#topic open discussion21:39
*** openstack changes topic to "open discussion (Meeting topic: swift)"21:39
timburke__anytyhing else we should discuss this week?21:39
timburke__all right21:42
timburke__thank you all for coming, and thank you for working on swift!21:42
timburke__#endmeeting21:42
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"21:42
openstackMeeting ended Wed Mar  3 21:42:15 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)21:42
openstackMinutes:        http://eavesdrop.openstack.org/meetings/swift/2021/swift.2021-03-03-21.00.html21:42
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/swift/2021/swift.2021-03-03-21.00.txt21:42
openstackLog:            http://eavesdrop.openstack.org/meetings/swift/2021/swift.2021-03-03-21.00.log.html21:42
*** acoles has left #openstack-meeting21:42
*** jmasud has joined #openstack-meeting22:03
*** rcernin has joined #openstack-meeting22:05
*** rcernin has quit IRC22:11
*** timburke_ has joined #openstack-meeting22:11
*** rcernin has joined #openstack-meeting22:11
*** timburke__ has quit IRC22:13
*** jamesdenton has quit IRC22:15
*** jamesdenton has joined #openstack-meeting22:15
*** smekala has joined #openstack-meeting22:29
*** belmoreira has quit IRC22:35
*** e0ne has quit IRC22:42
*** smekala has quit IRC22:55
*** rcernin has quit IRC22:56
*** zaitcev has left #openstack-meeting23:00
*** rcernin has joined #openstack-meeting23:01
*** timburke__ has joined #openstack-meeting23:01
*** thgcorrea has quit IRC23:01
*** timburke_ has quit IRC23:04
*** rcernin has quit IRC23:26
*** vishalmanchanda has quit IRC23:44

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!