| *** gyee has quit IRC | 00:24 | |
| seongsoocho | Does anyone use ssync in production? I try to move replication method from rsync to ssync and want to know if it is better performance(?) and reliability than rsync. | 00:28 |
|---|---|---|
| *** seongsoocho has quit IRC | 00:43 | |
| *** seongsoocho has joined #openstack-swift | 00:43 | |
| seongsoocho | timburke: The PTG etherpad of swift is different from what we are using. https://etherpad.opendev.org/p/wallaby-ptg-swift <- This is the link on the PTG page | 00:48 |
| timburke | seongsoocho, thanks for letting me know! i'll get it fixed right away | 00:52 |
| seongsoocho | :-) | 00:53 |
| timburke | iirc rledisez and alecuyer use ssync exclusively; clayg acoles and i use it just for ec data, and continue to use rsync for replicated | 00:53 |
| timburke | my understanding is that rsync is generally higher performance, but i've never run the benchmarks myself | 00:53 |
| timburke | http://ptg.openstack.org/etherpads.html is fixed :-) | 00:55 |
| seongsoocho | A-ha. what is the advantages of using ssync ? I try to find about ssync but can't find the docs. | 00:59 |
| clayg | It’s not well documented. acoles has snuck quite a few features into the protocol. It’s “just” an HTTP body 🤷♂️ | 01:04 |
| *** rcernin has quit IRC | 02:52 | |
| *** rcernin has joined #openstack-swift | 02:57 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-swift | 04:33 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: memcache: Refuse to error limit the last available server https://review.opendev.org/759183 | 05:05 |
| *** m75abrams has joined #openstack-swift | 05:26 | |
| *** mikecmpbll has quit IRC | 06:27 | |
| *** mikecmpbll has joined #openstack-swift | 06:29 | |
| *** noonedeadpunk has quit IRC | 06:31 | |
| *** ormandj has quit IRC | 06:34 | |
| *** ormandj has joined #openstack-swift | 06:35 | |
| *** noonedeadpunk has joined #openstack-swift | 06:39 | |
| *** rpittau|afk is now known as rpittau | 06:54 | |
| *** klamath_atx has quit IRC | 07:42 | |
| *** rcernin has quit IRC | 07:42 | |
| *** rcernin has joined #openstack-swift | 07:49 | |
| *** rcernin has quit IRC | 07:54 | |
| *** rcernin has joined #openstack-swift | 08:01 | |
| *** rcernin has quit IRC | 08:12 | |
| *** mikecmpbll has quit IRC | 08:31 | |
| *** mikecmpbll has joined #openstack-swift | 08:34 | |
| *** klamath_atx has joined #openstack-swift | 08:43 | |
| *** mikecmpbll has quit IRC | 10:54 | |
| *** takamatsu has joined #openstack-swift | 11:54 | |
| *** tdasilva has quit IRC | 13:36 | |
| *** tdasilva has joined #openstack-swift | 13:37 | |
| *** ChanServ sets mode: +v tdasilva | 13:37 | |
| *** m75abrams has quit IRC | 13:46 | |
| *** takamatsu is now known as mauro|call | 14:04 | |
| *** mauro|call is now known as takamatsu | 14:04 | |
| *** tdasilva_ has joined #openstack-swift | 14:29 | |
| *** ChanServ sets mode: +v tdasilva_ | 14:29 | |
| *** tdasilva has quit IRC | 14:32 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: memcache: Refuse to error limit the last available server https://review.opendev.org/759183 | 15:04 |
| *** gregwork has joined #openstack-swift | 15:26 | |
| *** rpittau is now known as rpittau|afk | 15:57 | |
| timburke | acoles, https://review.opendev.org/#/c/759183/ | 16:53 |
| patchbot | patch 759183 - swift - memcache: Refuse to error limit the last available... - 2 patch sets | 16:53 |
| acoles | got it | 16:56 |
| openstackgerrit | Tim Burke proposed openstack/swift master: Add option to REPLICATE to just invalidate hashes https://review.opendev.org/758636 | 17:04 |
| ormandj | does object-updater also look at what's on disk vs. what's on container db? we've got some 'interesting' situations in which our containerdb references an object that we can't find on any of the three primaries or handoffs | 17:05 |
| openstackgerrit | Tim Burke proposed openstack/swift master: Add tasks_per_second option to expirer https://review.opendev.org/755894 | 17:10 |
| timburke | ormandj, no, the object->container update is a one-time thing, not an ongoing check. object server will try one update to one container server; if it fails, it writes down an "async pending" to have the object updater retry to all container servers. updater will retry as needed, but once it gets successful responses back, it's done (for that object as of that timestamp) | 17:17 |
| timburke | you might be interested in https://review.opendev.org/706653; it starts to look at continued consistency checks, though more from the object-is-on-disk-but-not-in-listings perspective | 17:19 |
| patchbot | patch 706653 - swift - Let developers/operators add watchers to object au... - 37 patch sets | 17:19 |
| timburke | we've also got https://github.com/openstack/swift/blob/2.26.0/bin/swift-account-audit that tries to go the other way and validate that everything in listings actually is accessible | 17:19 |
| ormandj | timburke: we grepped in the async pending directories on the primary nodes for this objects for the object name | 17:22 |
| ormandj | nothing there | 17:22 |
| ormandj | but the object doesn't exist in any of the 6 locations it might | 17:23 |
| ormandj | and the container db still 'has it' on all of the container db copies we've checked | 17:23 |
| ormandj | confirmed, checked all three primary container dbs, no dice | 17:31 |
| timburke | any container servers been offline for an extended period then reintroduced? could be that the tombstone row got reaped then an old db that still knew about the since-deleted object sent the record back over | 17:37 |
| timburke | if you're certain the data doesn't exist (checked all primaries, replication's pretty well settled, maybe even checked all disks both under objects and quarantined dirs) it's gone -- and you should be able to just issue another delete to clean up the container listing | 17:39 |
| ormandj | yeah, we're positive, but we don't know if this widespread or not | 17:40 |
| ormandj | good call on the other checks | 17:40 |
| ormandj | we'll run through those exercises | 17:40 |
| *** whmcr has joined #openstack-swift | 17:44 | |
| ormandj | no re: off long periods of time | 17:45 |
| ormandj | but we did have the massive replication delay | 17:45 |
| ormandj | no quarentined directories | 17:46 |
| timburke | oh! and when you said the primaries didn't have it -- did they have tombstones? if you were doing HEADs, a 404 is for a tombstone if it has a X-Backend-Timestamp header | 17:53 |
| *** fingo has quit IRC | 19:02 | |
| *** fingo has joined #openstack-swift | 19:05 | |
| *** gyee has joined #openstack-swift | 19:26 | |
| mattoliverau | morning | 22:19 |
| *** rcernin has joined #openstack-swift | 22:31 | |
| *** rcernin has quit IRC | 22:49 | |
| *** rcernin has joined #openstack-swift | 22:52 | |
| *** benj_ has quit IRC | 23:21 | |
| *** benj_ has joined #openstack-swift | 23:26 | |
| openstackgerrit | Matthew Oliver proposed openstack/swift master: Push shardrange audit hacks into a patchset https://review.opendev.org/749614 | 23:56 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!