| *** jv|afk has quit IRC | 01:01 | |
| openstackgerrit | Matthew Oliver proposed openstack/swift master: Auto-sharding: Initial steps https://review.opendev.org/667030 | 01:03 |
|---|---|---|
| openstackgerrit | Matthew Oliver proposed openstack/swift master: sharding: first attempt at _elect_leader https://review.opendev.org/667579 | 01:03 |
| mattoliverau | ^ rebased | 01:03 |
| *** gyee has quit IRC | 01:26 | |
| *** rcernin has quit IRC | 01:27 | |
| *** rcernin has joined #openstack-swift | 01:30 | |
| openstackgerrit | Matthew Oliver proposed openstack/swift master: Add branch and tag tags to docker image https://review.opendev.org/732480 | 01:34 |
| mattoliverau | ^ just more playing | 01:34 |
| *** renich has joined #openstack-swift | 01:58 | |
| *** renich_ has joined #openstack-swift | 02:04 | |
| *** renich has quit IRC | 02:05 | |
| *** renich_ is now known as renich | 02:05 | |
| *** rcernin has quit IRC | 02:37 | |
| *** rcernin has joined #openstack-swift | 03:37 | |
| *** manuvakery has joined #openstack-swift | 04:28 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-swift | 04:33 | |
| manuvakery | Hi , the copy command aws s 3 cp "bucket1/obj.key" "bucket2/obj.key" failing when using aws cli version 2 . It tries to make a GET call with tagging so return 501 as we don't support object tagging. In the initial look i noted that the copy fails only for objects with segments . I will open a bug in launchpad ASAP | 04:35 |
| *** renich has quit IRC | 04:40 | |
| zaitcev | Thanks | 04:50 |
| *** dsariel has quit IRC | 06:41 | |
| *** ccamacho has joined #openstack-swift | 07:14 | |
| *** rcernin has quit IRC | 07:15 | |
| *** rcernin has joined #openstack-swift | 07:20 | |
| *** rcernin has quit IRC | 07:21 | |
| *** rcernin has joined #openstack-swift | 07:21 | |
| *** m75abrams has joined #openstack-swift | 07:25 | |
| mattoliverau | it's pretty hacky, but been playing with some potential gap and overlap detection code: https://gist.github.com/matthewoliver/ecd7e9a1d3fb74e2f119ff2c8a484f5d | 07:32 |
| mattoliverau | in case anyone else is interested in my horrible hacking. | 07:32 |
| mattoliverau | :P | 07:32 |
| *** rcernin has quit IRC | 07:41 | |
| *** rpittau|afk is now known as rpittau | 07:50 | |
| manuvakery | https://docs.aws.amazon.com/cli/latest/userguide/cliv2-migration.html | 08:06 |
| manuvakery | specifically https://docs.aws.amazon.com/cli/latest/userguide/cliv2-migration.html#cliv2-migration-s3-copy-metadata | 08:07 |
| manuvakery | its clear that the mulit part copies are going to fail with was cli v2 | 08:09 |
| *** dtantsur|afk is now known as dtantsur | 08:13 | |
| *** tkajinam has quit IRC | 08:48 | |
| *** rpittau is now known as rpittau|bbl | 10:02 | |
| *** ravsingh has joined #openstack-swift | 11:21 | |
| *** donnyd_ has quit IRC | 11:33 | |
| *** donnyd_ has joined #openstack-swift | 11:34 | |
| *** donnyd_ has quit IRC | 11:34 | |
| *** rpittau|bbl is now known as rpittau | 11:35 | |
| *** TobbeCN has joined #openstack-swift | 11:37 | |
| *** donnyd_ has joined #openstack-swift | 11:42 | |
| *** donnyd_ is now known as donnyd | 11:44 | |
| *** TobbeCN has quit IRC | 12:07 | |
| *** ianychoi_ has joined #openstack-swift | 12:32 | |
| *** benj_- has joined #openstack-swift | 12:36 | |
| *** rdejoux has quit IRC | 12:36 | |
| *** mvkr has quit IRC | 12:36 | |
| *** TViernion has quit IRC | 12:36 | |
| *** kukacz_ has quit IRC | 12:36 | |
| *** zigo has quit IRC | 12:36 | |
| *** benj_ has quit IRC | 12:36 | |
| *** tonyb has quit IRC | 12:36 | |
| *** paladox has quit IRC | 12:36 | |
| *** ianychoi has quit IRC | 12:36 | |
| *** benj_- is now known as benj_ | 12:36 | |
| *** mvkr has joined #openstack-swift | 12:37 | |
| *** paladox has joined #openstack-swift | 12:37 | |
| *** TViernion has joined #openstack-swift | 12:38 | |
| *** rdejoux has joined #openstack-swift | 12:45 | |
| *** zigo_ has joined #openstack-swift | 12:46 | |
| *** m75abrams has quit IRC | 12:47 | |
| *** dosaboy has quit IRC | 12:50 | |
| *** dosaboy has joined #openstack-swift | 12:51 | |
| *** tkajinam has joined #openstack-swift | 13:02 | |
| *** rpittau is now known as rpittau|brb | 13:24 | |
| *** cschwede has joined #openstack-swift | 13:30 | |
| *** ChanServ sets mode: +v cschwede | 13:30 | |
| *** cschwede has quit IRC | 13:53 | |
| *** rpittau|brb is now known as rpittau | 14:01 | |
| *** dsariel has joined #openstack-swift | 14:30 | |
| alecuyer | the other day I rambled about how I found it difficult to store a copy of an object's metadata so we can serve HEAD requests from an SSD. I tried to write a more coherent explanation here: https://etherpad.opendev.org/p/losf-fast-HEAD (also linked in the PTG doc) | 14:42 |
| alecuyer | but of course let's be focused on waterfall EC later. I'm eager to see clayg 's whiteboard :-) | 14:43 |
| clayg | alecuyer: hahah | 14:43 |
| alecuyer | taped your phone to that opposite wall yet? ;) | 14:44 |
| *** gyee has joined #openstack-swift | 14:46 | |
| clayg | i'm going to try to pre-prepare some images | 14:46 |
| DHE | I've been looking at SSD offload at the filesystem layer. Intel has their "CAS" product (gone free on github), and ZFS has this as a native feature... | 14:49 |
| alecuyer | clayg: nice, thanks! | 14:49 |
| alecuyer | DHE: yup we looked into that, CAS we were unable to get to work reliably. ZFS worked great, but still does require a fair amount of memory, and when you're over 80% usage to performance does drop significantly :/ Did you give it a try in your env ? | 14:51 |
| alecuyer | s/to/the | 14:51 |
| alecuyer | And I think for ZFS the thing we couldn't get to work is that, if you loose a top level vdev, you are forced to reboot the machine when replacing a disk | 14:52 |
| DHE | alecuyer: I have an existing pool I set it up with. I ran into an issue requiring linkat() features to be force disabled with a swift code mod | 14:52 |
| alecuyer | so unless you have a setup where ZFS had its own redundancy, that's not really possible | 14:52 |
| alecuyer | oh right | 14:53 |
| DHE | that can be changed in ZFS. the default behavior in the event of pool unrecoverable fault is to hang hoping the admin can rectify that. you can change that to force IO error responses | 14:53 |
| alecuyer | and then you're able to change your drive and recreate the pool? I thought we coudn't. maybe with another pool name? | 14:54 |
| DHE | well if a toplevel vdev dies the pool is up in smoke unless you can recover it (ddrescue, etc) | 14:54 |
| alecuyer | yeah but that's fine you can leave it to swift to rebuild | 14:55 |
| DHE | yeah you'd have to ditch the pool, recreate, and let swift recover it | 14:55 |
| DHE | I'm honestly working with an existing pool temporarily in this situation. it's not going to persist. | 14:56 |
| alecuyer | something else, it would be nice to disable the auditor and use ZFS scrub. Now it uses the sequential resilver, so you'd get much faster checks :) | 14:56 |
| zaitcev | wait wait wait, I just started looking into adding semantic checks to auditor :-) | 15:02 |
| alecuyer | zaitcev: Not to worry, I'm not really serious about it ;) | 15:03 |
| timburke | you guys reminded me of https://bugs.launchpad.net/swift/+bug/1823785 | 15:44 |
| openstack | Launchpad bug 1823785 in OpenStack Object Storage (swift) "Container replicator can propagate corrupt timestamps" [Undecided,New] | 15:44 |
| clayg | timburke: 🤮 | 15:46 |
| timburke | rledisez, i'd be curious about your opinion on https://review.opendev.org/#/c/733296/ | 15:55 |
| patchbot | patch 733296 - swift - relinker: Skip locked data dirs - 1 patch set | 15:55 |
| timburke | i think most of https://review.opendev.org/#/c/695344/ makes sense, but the block-waiting-for-lock felt a little off to me | 15:57 |
| patchbot | patch 695344 - swift - relinker: Improve performance by limiting I/O - 9 patch sets | 15:57 |
| zaitcev | dsariel: I'm okay with Tim's changes in most of 732441 but I'm leaving the probe changes to you. | 16:00 |
| DHE | alecuyer: the auditor is its own process and possibly init script/systemd service. you can probably just not run it | 16:02 |
| DHE | still, in the event of an unclean shutdown what is responsible for finding incomplete uploads and clearing them? | 16:02 |
| *** tkajinam has quit IRC | 16:05 | |
| timburke | DHE, replicator/reconstructor: https://github.com/openstack/swift/blob/master/swift/obj/replicator.py#L816-L818 for example | 16:11 |
| DHE | oh good | 16:23 |
| timburke | of course, if you've got O_TMPFILE/linkat support, you don't even have to worry about that -- we let the kernel/filesystem handle it | 16:29 |
| *** rpittau is now known as rpittau|afk | 16:31 | |
| *** dtantsur is now known as dtantsur|afk | 16:33 | |
| *** jv|afk has joined #openstack-swift | 16:40 | |
| *** ravsingh has quit IRC | 16:49 | |
| DHE | you say that, but ZFS has a bug (fixed in Git by me) where each call to linkat() on a O_TMPFILE file will trigger a `sync` operation on the whole pool. this kills performance | 17:01 |
| DHE | I guess ZFS doesn't honour sync operations on O_TMPFILE files and then when you call linkat() it suddenly panics and does a heavy-handed state sync | 17:02 |
| *** manuvakery has quit IRC | 17:28 | |
| *** jv|afk has quit IRC | 17:51 | |
| *** m75abrams has joined #openstack-swift | 19:04 | |
| *** jv|afk has joined #openstack-swift | 20:01 | |
| *** jv|afk has quit IRC | 20:25 | |
| clayg | oh no, i just realized if I'm going to share my screen like matt did I have to try and get zoom working on my laptop which I've had bad luck with... | 20:34 |
| clayg | I guess I'll try to reboot now, and if it doesn't work we'll just all have to look at the doc: https://docs.google.com/document/d/1kdy4lrroIS7bZ2PP4_dHbybNzVuEhIipx5JJsn4NxVo/edit?usp=sharing | 20:39 |
| mattoliverau | I like the zoom Flatpak, that's what I'm using (I think). Will confirm when I'm actually at my computer. | 20:39 |
| *** jv|afk has joined #openstack-swift | 20:48 | |
| *** gyee has quit IRC | 20:55 | |
| *** ccamacho has quit IRC | 20:56 | |
| clayg | I think i have two zooms installed now 🤔 | 20:56 |
| *** gyee has joined #openstack-swift | 20:57 | |
| mattoliverau | hopefully one works :P | 21:00 |
| clayg | thanks guys! | 22:17 |
| *** noonedeadpunk has quit IRC | 22:19 | |
| *** noonedeadpunk has joined #openstack-swift | 22:20 | |
| *** tdasilva has quit IRC | 22:35 | |
| *** samueldmq has quit IRC | 22:35 | |
| *** rpittau|afk has quit IRC | 22:36 | |
| *** donnyd has quit IRC | 22:36 | |
| *** clayg has quit IRC | 22:36 | |
| *** gregwork has quit IRC | 22:36 | |
| *** viks____ has quit IRC | 22:37 | |
| *** beisner has quit IRC | 22:37 | |
| *** tdasilva has joined #openstack-swift | 22:37 | |
| *** ChanServ sets mode: +v tdasilva | 22:37 | |
| *** jrosser has quit IRC | 22:37 | |
| *** fyx has quit IRC | 22:37 | |
| *** gmann has quit IRC | 22:37 | |
| *** seongsoocho has quit IRC | 22:37 | |
| *** seongsoocho_ has joined #openstack-swift | 22:37 | |
| *** gmann has joined #openstack-swift | 22:39 | |
| *** gregwork has joined #openstack-swift | 22:39 | |
| *** clayg has joined #openstack-swift | 22:40 | |
| *** ChanServ sets mode: +v clayg | 22:40 | |
| *** jrosser has joined #openstack-swift | 22:41 | |
| *** donnyd has joined #openstack-swift | 22:41 | |
| *** beisner has joined #openstack-swift | 22:42 | |
| *** seongsoocho_ has quit IRC | 22:46 | |
| *** seongsoocho has joined #openstack-swift | 22:48 | |
| *** rcernin has joined #openstack-swift | 22:48 | |
| *** m75abrams has quit IRC | 22:53 | |
| *** tkajinam has joined #openstack-swift | 22:56 | |
| *** samueldmq has joined #openstack-swift | 23:02 | |
| *** rpittau|afk has joined #openstack-swift | 23:04 | |
| *** fyx has joined #openstack-swift | 23:12 | |
| *** gyee has quit IRC | 23:46 | |
| seongsoocho | wow. virtual ptg also great. See you next ptg or summit and meet face to face | 23:47 |
| timburke | mattoliverau, if you get a chance, mind looking over https://review.opendev.org/#/c/731653/ again? i think i like where it wound up | 23:48 |
| patchbot | patch 731653 - swift - Don't auto-create shard containers - 7 patch sets | 23:48 |
| *** rcernin has quit IRC | 23:49 | |
| *** dsariel has quit IRC | 23:51 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!