Thursday, 2020-08-06

*** gtema has quit IRC00:04
*** zaitcev has quit IRC01:49
*** gtema has joined #openstack-swift02:00
*** zaitcev has joined #openstack-swift02:03
*** ChanServ sets mode: +v zaitcev02:03
*** gtema has quit IRC02:04
*** zamba has quit IRC03:03
*** zamba has joined #openstack-swift03:24
*** psachin has joined #openstack-swift03:31
*** djhankb has quit IRC03:32
*** m75abrams has joined #openstack-swift03:34
*** djhankb has joined #openstack-swift03:35
*** gyee has quit IRC03:50
*** gtema has joined #openstack-swift04:01
*** rcernin has quit IRC04:03
*** gtema has quit IRC04:06
*** rcernin has joined #openstack-swift04:24
*** evrardjp has quit IRC04:33
*** evrardjp has joined #openstack-swift04:33
*** gtema has joined #openstack-swift06:02
*** gtema has quit IRC06:07
*** gtema has joined #openstack-swift06:34
*** gtema has quit IRC06:49
*** gtema has joined #openstack-swift06:50
*** rcernin has quit IRC06:54
*** takamatsu has quit IRC08:02
*** dsariel has quit IRC08:36
*** takamatsu has joined #openstack-swift08:58
*** takamatsu has quit IRC09:07
*** takamatsu has joined #openstack-swift09:07
*** dsariel has joined #openstack-swift09:10
*** mikecmpbll has joined #openstack-swift09:29
*** takamatsu has quit IRC09:53
*** tkajinam has quit IRC10:33
*** takamatsu has joined #openstack-swift11:06
*** rcernin has joined #openstack-swift12:08
*** mikecmpbll has quit IRC12:31
*** mikecmpbll has joined #openstack-swift12:34
*** m75abrams has quit IRC13:00
*** rcernin has quit IRC13:40
*** gtema has quit IRC14:42
*** psachin has quit IRC14:43
*** gtema has joined #openstack-swift15:13
*** gtema has quit IRC15:18
*** gtema has joined #openstack-swift15:40
*** gtema_ has joined #openstack-swift15:52
*** gtema has quit IRC15:55
*** dsariel has quit IRC16:17
*** dsariel has joined #openstack-swift16:17
*** patchbot has joined #openstack-swift16:34
*** gyee has joined #openstack-swift16:51
DHEdoes this sound like a good idea? a cron job that, hourly, runs swift-object-replicator -o /etc/swift/object-server-handoffsonly.conf    # where said file obviously specifies handoffs only mode, and kept separate from the normal replicator job16:55
claygmaybe! do you have a lot of handoffs sometimes?17:17
*** gtema_ has quit IRC17:23
clayghey if anyone else is sharding containers watch your memcache misses: https://bugs.launchpad.net/swift/+bug/189064317:28
openstackLaunchpad bug 1890643 in OpenStack Object Storage (swift) "shard range cache can get too big for memcache" [Undecided,New]17:28
claygI was trolling some code the other day and this smelled for me: https://github.com/openstack/swift/blob/master/swift/proxy/controllers/base.py#L218717:31
claygShould we be consuming response.body for 503 on GET to container listing?17:31
claygWhen my container servers were getting overwhelmed I noticed a bunch of connections in CLOSE_WAIT; but don't necessarily know if that's because the proxy wasn't reading/closing error connections 🤷‍♂️17:32
*** dsariel has quit IRC17:33
*** mikecmpbll has quit IRC17:36
*** mikecmpbll has joined #openstack-swift17:40
claygI kinda feel like a healthy connection when closed ends up in TIME_WAIT 🤔17:46
DHEclayg: I've been put in the unpleasant position of having a lot of data to move around, some big honking machines with nice hardware, but an inadequate network connection to support them. I'm raising the weights of these new machines in small increments and watching them peg their 1gig ports as they fill over time.17:54
DHEhonestly I'm pegging them by manually running such an object-replicator command manually in order to get the data moved as fast as possible. it sucks.17:55
*** patchbot has quit IRC18:01
*** patchbot has joined #openstack-swift18:01
claygwe often kick things into handoff mode during a rebalance; sorry it's not easy going for you 😥18:04
*** timburke has quit IRC18:30
*** timburke has joined #openstack-swift18:30
*** ChanServ sets mode: +v timburke18:30
*** gmann is now known as gmann_afk18:35
DHEwell it's not your fault. I blame the switches for only having 1gig ports.18:42
DHEand even the ones that do have 10gig+ speeds I get messages asking if it's expected that we hit 80% utilization on it18:43
DHE:/18:43
*** gtema has joined #openstack-swift19:23
*** gtema has quit IRC19:28
*** dsariel has joined #openstack-swift20:18
*** mikecmpbll has quit IRC20:41
*** mikecmpbll has joined #openstack-swift20:47
*** dsariel has quit IRC21:01
*** rcernin has joined #openstack-swift21:23
*** rcernin has quit IRC21:27
*** rcernin has joined #openstack-swift21:34
*** rcernin has quit IRC21:42
*** rcernin has joined #openstack-swift22:12
*** hoonetorg has quit IRC22:15
*** rcernin has quit IRC22:17
*** gmann_afk is now known as gmann22:35
*** rcernin has joined #openstack-swift22:47
*** rcernin has quit IRC22:52
*** tkajinam has joined #openstack-swift22:55
*** rcernin has joined #openstack-swift23:20
mattoliveraumorning23:33

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!