Friday, 2019-06-14

*** henriqueof has quit IRC00:00
*** ianychoi has joined #openstack-swift00:19
*** gyee has quit IRC00:38
*** rcernin has quit IRC01:27
*** rcernin has joined #openstack-swift01:27
*** BjoernT has joined #openstack-swift03:14
*** BjoernT has quit IRC03:19
*** BjoernT has joined #openstack-swift03:20
*** mugsie has quit IRC03:22
*** dosaboy has quit IRC03:22
*** szaher has quit IRC03:22
*** mvkr has quit IRC03:22
*** mgagne has quit IRC03:22
*** dasp has quit IRC03:22
*** guimaluf has quit IRC03:22
*** d34dh0r53 has quit IRC03:22
*** psachin has joined #openstack-swift03:31
openstackgerritzhufl proposed openstack/swift master: Fix :param: in docstring  https://review.opendev.org/66528403:33
*** mugsie has joined #openstack-swift03:37
*** dosaboy has joined #openstack-swift03:37
*** szaher has joined #openstack-swift03:38
*** mvkr has joined #openstack-swift03:38
*** mgagne has joined #openstack-swift03:38
*** dasp has joined #openstack-swift03:38
*** guimaluf has joined #openstack-swift03:38
*** d34dh0r53 has joined #openstack-swift03:38
*** irclogbot_2 has quit IRC03:41
openstackgerritMerged openstack/swift master: Update mailmap for Kota Tsuyuzaki  https://review.opendev.org/66420003:41
*** irclogbot_2 has joined #openstack-swift03:42
*** dasp has quit IRC03:58
*** dasp has joined #openstack-swift04:21
openstackgerritMerged openstack/swift feature/losf: Merge branch 'master' into feature/losf  https://review.opendev.org/66517004:23
*** pcaruana has joined #openstack-swift05:09
*** e0ne has joined #openstack-swift05:21
*** BjoernT has quit IRC05:33
*** takamatsu has joined #openstack-swift06:34
*** viks__ has joined #openstack-swift06:57
*** rdejoux has joined #openstack-swift07:07
*** tesseract has joined #openstack-swift07:08
viks__I see the following in my swift storage node logs at times and fails gets automatically resolved:07:10
viks__```07:10
viks__object-replicator: 229243 successes, 1 failures07:10
viks__object-replicator: 212547 successes, 15 failures07:10
viks__object-replicator: 229225 successes, 19 failures07:10
viks__object-replicator: 211476 successes, 24 failures07:10
viks__object-replicator: 229219 successes, 25 failures07:10
viks__object-replicator: 211190 successes, 2 failures07:10
viks__object-replicator: 229241 successes, 3 failures07:10
viks__```07:10
viks__But i'm unable to find what is causing this failure. Is it expected. Can someone help me how to troubleshoot this issue?07:10
*** rcernin has quit IRC07:22
*** e0ne has quit IRC07:35
*** ccamacho has joined #openstack-swift08:30
*** tkajinam has quit IRC08:38
*** e0ne has joined #openstack-swift09:08
*** tdasilva has quit IRC10:04
*** psachin has quit IRC10:37
*** psachin has joined #openstack-swift10:39
*** tdasilva has joined #openstack-swift12:46
*** ChanServ sets mode: +v tdasilva12:46
*** BjoernT has joined #openstack-swift13:25
*** tdasilva has quit IRC13:32
*** hseipp has joined #openstack-swift13:34
*** mrjk has joined #openstack-swift14:02
*** tdasilva has joined #openstack-swift14:06
*** ChanServ sets mode: +v tdasilva14:06
*** BjoernT has quit IRC14:08
*** e0ne has quit IRC14:13
*** edausq has quit IRC14:16
*** baojg has joined #openstack-swift14:18
*** hseipp has quit IRC14:18
*** hseipp has joined #openstack-swift14:19
openstackgerritThiago da Silva proposed openstack/swift master: Enable s3api in saio docker container  https://review.opendev.org/66540314:20
*** tdasilva has quit IRC14:22
*** tdasilva has joined #openstack-swift14:22
*** ChanServ sets mode: +v tdasilva14:22
timburkeviks__, most likely it's just that the remote nodes were overloaded and replication hit a timeout. to confirm that, look for messages in the replicator log like "Error syncing handoff partition" or "Error syncing partition"14:52
timburkesomething like "Exception in top-level replication loop" would be more troubling14:52
*** e0ne has joined #openstack-swift14:57
*** e0ne has quit IRC15:00
timburkebasically, as long as there don't seem to be persistent failures, it's normal operations and nothing to worry about15:00
*** mrjk has quit IRC15:02
*** mrjk has joined #openstack-swift15:02
timburke(man, now i want to track time-since-last-successful-sync on a per-partition basis -- looking at outliers would probably be interesting)15:03
viks__timburke:  I'm not seeing any of the above mentioned error15:03
*** baojg has quit IRC15:06
viks__but it resolves on it's own  after many iterations15:06
viks__when checked with  `/recon/replication/object` endpoint, it says rsync error.. but rsync log also does  not show any error.15:13
*** e0ne has joined #openstack-swift15:15
*** e0ne has quit IRC15:21
openstackgerritTim Burke proposed openstack/swift master: Fix up how we memcache on py3  https://review.opendev.org/66211515:21
openstackgerritTim Burke proposed openstack/swift master: py3: Port the tempurl middleware  https://review.opendev.org/65292815:21
*** e0ne has joined #openstack-swift15:24
*** tdasilva has quit IRC15:26
*** e0ne has quit IRC15:27
*** e0ne has joined #openstack-swift15:29
*** e0ne has quit IRC15:33
*** takamatsu has quit IRC16:07
*** hseipp has quit IRC16:23
*** hseipp has joined #openstack-swift16:43
*** hseipp has quit IRC17:08
*** e0ne has joined #openstack-swift17:41
openstackgerritClay Gerrard proposed openstack/swift master: Allow "harder" symlinks  https://review.opendev.org/63309417:42
*** guimaluf has quit IRC17:42
*** e0ne has quit IRC17:46
*** rdejoux has quit IRC18:08
*** psachin has quit IRC18:08
*** tesseract has quit IRC18:18
*** e0ne has joined #openstack-swift18:42
*** e0ne has quit IRC19:16
*** e0ne has joined #openstack-swift19:16
*** e0ne has quit IRC19:21
*** e0ne has joined #openstack-swift19:23
*** e0ne has quit IRC19:33
*** viks__ has quit IRC19:36
*** pcaruana has quit IRC19:39
openstackgerritTim Burke proposed openstack/swift master: py3: port the test of locale  https://review.opendev.org/66504220:31
timburke🤮20:33
openstackgerritTim Burke proposed openstack/swift master: py3: Specify an encoding when loading db.pending pickles  https://review.opendev.org/66548622:44
openstackgerritTim Burke proposed openstack/swift master: Stop sending non-UTF-8 prefixes in func tests  https://review.opendev.org/66548722:45
timburkeon ^^^ there are probably others -- i think we'll be playing whack-a-mole for a while as we do dev work and receive bug reports against py322:46
timburkeer, vvv22:47
openstackgerritTim Burke proposed openstack/swift master: Move calls to self.app outside of error handling  https://review.opendev.org/66548822:47
openstackgerritTim Burke proposed openstack/swift master: Move call to global_conf_callback after loadapp()  https://review.opendev.org/66549423:20
timburkei feel like ^^^ needs some functional validation that replication_concurrency is still respected23:20
timburke(sorry for the flurry of activity -- i'm trying to break up https://review.opendev.org/#/c/663794/ into independent patches that each make sense on their own)23:22
patchbotpatch 663794 - swift - py3: get func tests passing - 1 patch set23:22

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!