| *** jd__ has quit IRC | 00:02 | |
| *** annegent_ has joined #openstack-swift | 00:03 | |
| *** ho has joined #openstack-swift | 00:07 | |
| *** abhirc has quit IRC | 00:12 | |
| ho | good morning guys! | 00:13 |
|---|---|---|
| *** Nadeem has quit IRC | 00:17 | |
| *** lcurtis has quit IRC | 00:19 | |
| *** jd__ has joined #openstack-swift | 00:26 | |
| *** haomaiwa_ has joined #openstack-swift | 00:26 | |
| *** tellesnobrega_ has joined #openstack-swift | 00:31 | |
| *** annegent_ has quit IRC | 00:32 | |
| zaitcev | ohayou gozaimasu, ho-san | 00:43 |
| ho | zaitcev: ohayou gozaimasu. zaitcev-san :) | 00:54 |
| *** addnull has joined #openstack-swift | 00:59 | |
| *** jasondotstar has joined #openstack-swift | 01:07 | |
| *** lpabon has quit IRC | 01:18 | |
| openstackgerrit | Parantap Roy proposed openstack/python-swiftclient: SwiftClient object beginning with / or ./ https://review.openstack.org/148791 | 01:20 |
| *** addnull has quit IRC | 01:22 | |
| openstackgerrit | Clay Gerrard proposed openstack/python-swiftclient: Fix environment sanitization for TestServiceUtils https://review.openstack.org/149108 | 01:25 |
| *** tellesnobrega_ has quit IRC | 01:29 | |
| *** addnull has joined #openstack-swift | 01:29 | |
| *** addnull has quit IRC | 01:33 | |
| *** addnull has joined #openstack-swift | 01:34 | |
| *** d0ugal has quit IRC | 01:55 | |
| *** d0ugal has joined #openstack-swift | 01:55 | |
| *** d0ugal is now known as Guest85104 | 01:56 | |
| *** dfg_ has joined #openstack-swift | 01:57 | |
| *** ho__ has joined #openstack-swift | 01:58 | |
| *** briancline_ has quit IRC | 01:58 | |
| *** tristanC_ has joined #openstack-swift | 01:58 | |
| *** briancline_ has joined #openstack-swift | 01:58 | |
| *** kragniz_ has joined #openstack-swift | 01:59 | |
| *** mordred_ has joined #openstack-swift | 02:04 | |
| *** amandap_ has joined #openstack-swift | 02:05 | |
| *** ho has quit IRC | 02:06 | |
| *** dfg has quit IRC | 02:06 | |
| *** addnull has quit IRC | 02:06 | |
| *** tristanC has quit IRC | 02:06 | |
| *** amandap has quit IRC | 02:06 | |
| *** AbyssOne_ has quit IRC | 02:06 | |
| *** kragniz has quit IRC | 02:06 | |
| *** mordred has quit IRC | 02:06 | |
| *** dvorkbjel has quit IRC | 02:06 | |
| *** mordred_ is now known as mordred | 02:06 | |
| *** AbyssOne_ has joined #openstack-swift | 02:06 | |
| *** dvorkbjel has joined #openstack-swift | 02:07 | |
| *** addnull has joined #openstack-swift | 02:08 | |
| *** addnull has quit IRC | 02:12 | |
| *** tellesnobrega_ has joined #openstack-swift | 02:20 | |
| *** addnull has joined #openstack-swift | 02:22 | |
| *** addnull has quit IRC | 02:26 | |
| *** abhirc has joined #openstack-swift | 02:31 | |
| *** amandap_ is now known as amandap | 02:34 | |
| *** lcurtis has joined #openstack-swift | 02:43 | |
| *** addnull has joined #openstack-swift | 02:44 | |
| *** addnull has quit IRC | 02:49 | |
| *** addnull has joined #openstack-swift | 02:54 | |
| *** tsg has joined #openstack-swift | 02:54 | |
| *** tsg_ has joined #openstack-swift | 02:54 | |
| *** lpabon has joined #openstack-swift | 02:55 | |
| *** lpabon has quit IRC | 02:55 | |
| *** tsg_ has quit IRC | 02:56 | |
| *** addnull has quit IRC | 02:59 | |
| *** kei_yama has joined #openstack-swift | 03:02 | |
| *** kei_yama has quit IRC | 03:09 | |
| *** kei_yama has joined #openstack-swift | 03:12 | |
| *** bill_az has quit IRC | 03:18 | |
| *** addnull has joined #openstack-swift | 03:23 | |
| *** kei_yama has quit IRC | 03:24 | |
| *** kei_yama has joined #openstack-swift | 03:25 | |
| *** addnull has quit IRC | 03:28 | |
| *** fandi has quit IRC | 03:28 | |
| *** bpap has quit IRC | 03:30 | |
| *** bkopilov has quit IRC | 03:32 | |
| *** jasondotstar has quit IRC | 03:47 | |
| *** zhiyan has quit IRC | 03:48 | |
| *** zhiyan has joined #openstack-swift | 03:49 | |
| *** addnull has joined #openstack-swift | 03:49 | |
| *** annegent_ has joined #openstack-swift | 03:50 | |
| *** addnull has quit IRC | 03:53 | |
| *** tellesnobrega_ has quit IRC | 04:01 | |
| *** addnull has joined #openstack-swift | 04:34 | |
| *** bkopilov has joined #openstack-swift | 04:34 | |
| *** addnull has quit IRC | 04:38 | |
| *** bkopilov has quit IRC | 04:39 | |
| *** addnull has joined #openstack-swift | 04:51 | |
| *** dmsimard_away is now known as dmsimard | 04:53 | |
| *** addnull has quit IRC | 04:55 | |
| *** ppai has joined #openstack-swift | 04:55 | |
| *** annegent_ has quit IRC | 05:02 | |
| *** annegent_ has joined #openstack-swift | 05:03 | |
| *** addnull has joined #openstack-swift | 05:04 | |
| *** dmsimard is now known as dmsimard_away | 05:07 | |
| *** addnull has quit IRC | 05:09 | |
| *** addnull has joined #openstack-swift | 05:16 | |
| *** addnull has quit IRC | 05:21 | |
| *** addnull has joined #openstack-swift | 05:28 | |
| *** tsg has quit IRC | 05:33 | |
| *** addnull has quit IRC | 05:39 | |
| *** annegent_ has quit IRC | 05:43 | |
| *** addnull has joined #openstack-swift | 05:48 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:54 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:54 | |
| *** lcurtis has quit IRC | 05:55 | |
| *** addnull has quit IRC | 06:04 | |
| *** addnull has joined #openstack-swift | 06:10 | |
| *** gvernik has joined #openstack-swift | 06:12 | |
| *** annegent_ has joined #openstack-swift | 06:13 | |
| *** addnull has quit IRC | 06:14 | |
| *** annegent_ has quit IRC | 06:20 | |
| *** addnull has joined #openstack-swift | 06:22 | |
| *** addnull has quit IRC | 06:25 | |
| *** addnull has joined #openstack-swift | 06:27 | |
| *** gvernik has quit IRC | 06:30 | |
| *** addnull has quit IRC | 06:51 | |
| *** addnull has joined #openstack-swift | 06:58 | |
| *** silor has joined #openstack-swift | 07:04 | |
| *** zaitcev has quit IRC | 07:07 | |
| *** addnull has quit IRC | 07:14 | |
| *** nshaikh has joined #openstack-swift | 07:26 | |
| *** addnull has joined #openstack-swift | 07:30 | |
| *** addnull has quit IRC | 07:34 | |
| *** chlong has quit IRC | 07:38 | |
| *** Guest85104 is now known as d0ugal | 07:42 | |
| *** d0ugal is now known as Guest79195 | 07:42 | |
| *** Guest79195 is now known as d0ugal | 07:48 | |
| *** d0ugal has joined #openstack-swift | 07:48 | |
| *** addnull has joined #openstack-swift | 07:55 | |
| *** addnull has quit IRC | 07:57 | |
| *** rledisez has joined #openstack-swift | 08:09 | |
| *** geaaru has joined #openstack-swift | 08:24 | |
| *** oomichi_ has quit IRC | 08:40 | |
| *** goodes has quit IRC | 08:50 | |
| *** nottrobin has quit IRC | 08:50 | |
| *** ipolyzos has quit IRC | 08:51 | |
| *** zhiyan has quit IRC | 08:55 | |
| *** goodes has joined #openstack-swift | 09:00 | |
| *** ipolyzos has joined #openstack-swift | 09:01 | |
| *** nottrobin_ has joined #openstack-swift | 09:01 | |
| *** zhiyan has joined #openstack-swift | 09:02 | |
| *** jordanP has joined #openstack-swift | 09:21 | |
| *** jistr has joined #openstack-swift | 09:24 | |
| *** annegent_ has joined #openstack-swift | 09:27 | |
| *** nshaikh has left #openstack-swift | 09:30 | |
| *** annegent_ has quit IRC | 09:32 | |
| *** acoles_away is now known as acoles | 09:32 | |
| *** ahonda has quit IRC | 09:44 | |
| *** nellysmitt has joined #openstack-swift | 09:50 | |
| *** bkopilov has joined #openstack-swift | 09:57 | |
| *** haomaiwa_ has quit IRC | 10:01 | |
| *** fandi has joined #openstack-swift | 10:02 | |
| *** chlong has joined #openstack-swift | 10:15 | |
| *** aix has joined #openstack-swift | 10:20 | |
| *** chlong has quit IRC | 10:20 | |
| *** jasondotstar has joined #openstack-swift | 10:30 | |
| *** jasondotstar has quit IRC | 10:30 | |
| *** chlong has joined #openstack-swift | 10:33 | |
| *** chlong has quit IRC | 10:37 | |
| *** chlong has joined #openstack-swift | 10:49 | |
| *** mahatic has joined #openstack-swift | 10:55 | |
| *** annegent_ has joined #openstack-swift | 11:16 | |
| *** dmorita has quit IRC | 11:20 | |
| *** dmorita has joined #openstack-swift | 11:21 | |
| *** annegent_ has quit IRC | 11:21 | |
| *** nshaikh has joined #openstack-swift | 11:21 | |
| *** kei_yama has quit IRC | 11:37 | |
| *** aix has quit IRC | 11:38 | |
| *** kragniz_ is now known as kragniz | 11:45 | |
| *** dmorita has quit IRC | 11:46 | |
| openstackgerrit | Merged openstack/swift: Improve object-replicator startup time. https://review.openstack.org/140178 | 11:54 |
| *** jyoti-ranjan has joined #openstack-swift | 11:57 | |
| jyoti-ranjan | Hi, I am getting following error: | 11:58 |
| jyoti-ranjan | ERROR with Object server 192.168.0.10:6000/d1 re: Trying to get final status of PUT to /v1/AUTH_5674fc4e041c4855aba39537ce24beff/glance/255606aa-5ffc-4440-b67d-619665802ab8: Timeout (10s | 11:58 |
| jyoti-ranjan | Any input to fix will be of great help. | 11:58 |
| jyoti-ranjan | Error occurs occassionally. | 11:58 |
| *** wolsen_ has quit IRC | 12:00 | |
| *** dosaboy has quit IRC | 12:02 | |
| *** xianghui has quit IRC | 12:02 | |
| *** xianghui has joined #openstack-swift | 12:04 | |
| *** dosaboy has joined #openstack-swift | 12:04 | |
| *** wolsen has joined #openstack-swift | 12:05 | |
| *** echevemaster has quit IRC | 12:14 | |
| *** haomaiwang has joined #openstack-swift | 12:28 | |
| *** haomai___ has joined #openstack-swift | 12:32 | |
| *** haomaiwang has quit IRC | 12:35 | |
| *** chlong has quit IRC | 12:44 | |
| *** ho__ has quit IRC | 12:54 | |
| openstackgerrit | Prashanth Pai proposed openstack/swift: Allow rsync to use compression https://review.openstack.org/148454 | 12:55 |
| *** km has quit IRC | 12:56 | |
| *** jyoti-ranjan2 has joined #openstack-swift | 13:14 | |
| *** annegent_ has joined #openstack-swift | 13:16 | |
| *** annegent_ has quit IRC | 13:21 | |
| *** tdasilva has joined #openstack-swift | 13:26 | |
| *** marcusvrn_ has joined #openstack-swift | 13:33 | |
| *** bkopilov has quit IRC | 13:39 | |
| *** aix has joined #openstack-swift | 13:40 | |
| *** annegent_ has joined #openstack-swift | 13:43 | |
| *** tristanC_ is now known as tristanC | 13:47 | |
| *** miqui has joined #openstack-swift | 13:54 | |
| *** annegent_ has quit IRC | 13:57 | |
| *** fandi has quit IRC | 14:02 | |
| *** annegent_ has joined #openstack-swift | 14:04 | |
| openstackgerrit | Prashanth Pai proposed openstack/swift: Allow rsync to use compression https://review.openstack.org/148454 | 14:14 |
| *** annegent_ has quit IRC | 14:15 | |
| ppai | Hi all, is the "id" member in ring structure unique in a machine or across the cluster ? | 14:19 |
| *** jyoti-ranjan has quit IRC | 14:19 | |
| *** jyoti-ranjan2 has quit IRC | 14:19 | |
| *** tsg has joined #openstack-swift | 14:20 | |
| tdasilva | I think the id is for a device | 14:33 |
| *** nshaikh has left #openstack-swift | 14:34 | |
| rledisez | AFAIK, id is unique and cannot be reused | 14:34 |
| rledisez | the ring is a python structure, the devices are in an array, the id is the index of the array | 14:34 |
| rledisez | when a device is remove, the value is set to None | 14:34 |
| rledisez | so, i think you can considere it is unique in a cluster and not reused | 14:35 |
| rledisez | sorry for the nickname | 14:35 |
| *** haomai___ has quit IRC | 14:36 | |
| rledisez | clayg: hello | 14:38 |
| rledisez | clayg: did you have time to look at my review since i updated the patch following your recommendations? https://review.openstack.org/#/c/137659/ | 14:39 |
| *** SkyRocknRoll has quit IRC | 14:46 | |
| *** fandi has joined #openstack-swift | 14:51 | |
| *** fandi has quit IRC | 14:53 | |
| *** fandi has joined #openstack-swift | 14:53 | |
| *** fandi has quit IRC | 14:53 | |
| ppai | rledisez, thank you | 14:54 |
| *** ppai has quit IRC | 14:59 | |
| *** fandi has joined #openstack-swift | 15:05 | |
| *** nellysmitt has quit IRC | 15:05 | |
| *** dmsimard_away is now known as dmsimard | 15:12 | |
| *** lpabon has joined #openstack-swift | 15:13 | |
| *** annegent_ has joined #openstack-swift | 15:18 | |
| *** fandi has quit IRC | 15:19 | |
| *** annegent_ has quit IRC | 15:20 | |
| openstackgerrit | Takashi Kajinami proposed openstack/swift: Remove redundant container updating after rsync https://review.openstack.org/149308 | 15:21 |
| *** fandi has joined #openstack-swift | 15:21 | |
| *** fandi has quit IRC | 15:22 | |
| *** bkopilov has joined #openstack-swift | 15:23 | |
| openstackgerrit | Prashanth Pai proposed openstack/swift: Skip nodes that does not support REPLICATE verb https://review.openstack.org/149315 | 15:29 |
| *** CrackerJackMack has quit IRC | 15:32 | |
| openstackgerrit | Takashi Kajinami proposed openstack/swift: Prevent redundant commenting by drive-audit https://review.openstack.org/149317 | 15:33 |
| *** CrackerJackMack has joined #openstack-swift | 15:33 | |
| openstackgerrit | Takashi Kajinami proposed openstack/swift: Remove redundant container updating after rsync https://review.openstack.org/149308 | 15:36 |
| *** annegent_ has joined #openstack-swift | 15:41 | |
| *** annegent_ has quit IRC | 15:56 | |
| *** briancurtin has quit IRC | 15:57 | |
| *** lcurtis has joined #openstack-swift | 15:59 | |
| *** gvernik has joined #openstack-swift | 15:59 | |
| *** briancurtin has joined #openstack-swift | 16:02 | |
| *** lpabon has quit IRC | 16:04 | |
| openstackgerrit | Thiago da Silva proposed openstack/swift: WIP: single-process https://review.openstack.org/149329 | 16:09 |
| *** bill_az has joined #openstack-swift | 16:13 | |
| *** abhirc has quit IRC | 16:13 | |
| *** nshaikh has joined #openstack-swift | 16:16 | |
| *** hugespoon has joined #openstack-swift | 16:17 | |
| *** annegent_ has joined #openstack-swift | 16:20 | |
| *** fandi has joined #openstack-swift | 16:36 | |
| *** jrichli has joined #openstack-swift | 16:36 | |
| *** annegent_ has quit IRC | 16:37 | |
| *** tsg has quit IRC | 16:46 | |
| *** annegent_ has joined #openstack-swift | 16:53 | |
| *** nshaikh has quit IRC | 16:54 | |
| *** portante has quit IRC | 16:56 | |
| *** ndk_ has quit IRC | 16:56 | |
| *** marcusvrn_ is now known as marcusvrn | 16:57 | |
| *** lcurtis has quit IRC | 16:58 | |
| *** annegent_ has quit IRC | 17:01 | |
| *** IRTermite has joined #openstack-swift | 17:02 | |
| *** portante has joined #openstack-swift | 17:03 | |
| *** ChanServ sets mode: +v portante | 17:03 | |
| *** ndk has joined #openstack-swift | 17:11 | |
| *** abhirc has joined #openstack-swift | 17:14 | |
| *** rledisez has quit IRC | 17:20 | |
| *** MooingLemur has quit IRC | 17:23 | |
| *** tsg has joined #openstack-swift | 17:25 | |
| *** aix has quit IRC | 17:27 | |
| *** abhirc has quit IRC | 17:30 | |
| *** abhirc has joined #openstack-swift | 17:31 | |
| *** annegent_ has joined #openstack-swift | 17:32 | |
| *** MooingLemur has joined #openstack-swift | 17:33 | |
| *** jordanP has quit IRC | 17:44 | |
| *** annegent_ has quit IRC | 17:44 | |
| *** david-lyle has joined #openstack-swift | 17:50 | |
| *** lpabon has joined #openstack-swift | 17:56 | |
| *** nellysmitt has joined #openstack-swift | 18:00 | |
| *** gvernik has quit IRC | 18:01 | |
| *** nellysmitt has quit IRC | 18:04 | |
| *** lcurtis has joined #openstack-swift | 18:11 | |
| *** marcusvrn has quit IRC | 18:12 | |
| *** nellysmitt has joined #openstack-swift | 18:17 | |
| abhirc | raies: I don;t have any other leads , but if you poke around the #openstack channel u might get some info , also check the openstack dev mailing list | 18:19 |
| *** zaitcev has joined #openstack-swift | 18:22 | |
| *** ChanServ sets mode: +v zaitcev | 18:22 | |
| *** annegent_ has joined #openstack-swift | 18:31 | |
| *** annegent_ has quit IRC | 18:33 | |
| *** annegent_ has joined #openstack-swift | 18:46 | |
| *** nellysmitt has quit IRC | 18:47 | |
| *** gyee has joined #openstack-swift | 18:54 | |
| *** jistr has quit IRC | 18:58 | |
| *** mahatic has quit IRC | 18:59 | |
| *** annegent_ has quit IRC | 19:10 | |
| *** acoles is now known as acoles_away | 19:18 | |
| clayg | rledisez - you left! :'( | 19:19 |
| clayg | I did look at it, got distracted getting vagrant-swift-all-in-one to support trusty (which i'm not even sure if I pushed up yet...) | 19:19 |
| clayg | ... but I needed the newer kernel to functionally test the patch, so trusty seemed like it would be good enough and then didn't get back to review - I'll add it to the list | 19:20 |
| * clayg yells into to abyss | 19:21 | |
| *** annegent_ has joined #openstack-swift | 19:22 | |
| *** tsg has quit IRC | 19:23 | |
| swifterdarrell | clayg: careful... they say it'll yell back | 19:23 |
| *** silor1 has joined #openstack-swift | 19:25 | |
| *** silor has quit IRC | 19:27 | |
| *** geaaru has quit IRC | 19:28 | |
| *** nellysmitt has joined #openstack-swift | 19:42 | |
| *** annegent_ has quit IRC | 19:43 | |
| *** panbalag has joined #openstack-swift | 19:58 | |
| openstackgerrit | Samuel Merritt proposed openstack/swift: Optimize replication of targeted devices/partitions. https://review.openstack.org/149384 | 20:07 |
| *** wasmum has joined #openstack-swift | 20:21 | |
| *** tsg has joined #openstack-swift | 20:48 | |
| clayg | wtf is this? -> https://bugs.launchpad.net/swift/+bug/1413711 | 20:50 |
| clayg | opening a grammer bug does *not* fix the typos | 20:50 |
| tdasilva | clayg: similar to this one: https://bugs.launchpad.net/swift/+bug/1365084 | 21:00 |
| mattoliverau | Morning | 21:07 |
| openstackgerrit | Merged openstack/swift: Implement OPTIONS verb for storage nodes. https://review.openstack.org/140103 | 21:19 |
| *** tellesnobrega_ has joined #openstack-swift | 21:24 | |
| tdasilva | mattoliverau: morning :-) | 21:29 |
| dmsimard | FYI.. Nasty glance bug we've hit with Swift as backend: https://bugs.launchpad.net/glance/+bug/1412802 | 21:34 |
| *** fifieldt__ has quit IRC | 21:37 | |
| torgomatic | dmsimard: busted uploader, eh? doesn' | 21:38 |
| torgomatic | t sound fun | 21:38 |
| *** fifieldt__ has joined #openstack-swift | 21:38 | |
| dmsimard | My understanding was that glance was chunking files, uploading every chunks | 21:38 |
| dmsimard | but didn't upload the last chunk | 21:39 |
| dmsimard | or something | 21:39 |
| *** jodah has joined #openstack-swift | 21:43 | |
| jodah | Is there a way to ensure that an object PUT to swift is placed on a particular node? | 21:44 |
| ahale | if you have access to the rings and the config secrets you can just keep renaming until you get what you want | 21:45 |
| jodah | hahaha - yea | 21:45 |
| jodah | Does the ring take physical locality of nodes into account - or just ring locality (the docs mention swift attempts to place replicas far apart on the ring)? | 21:46 |
| jodah | By physical locality I mean something such as placing replicas on different racks | 21:46 |
| ahale | by zone, so splitting zones by rack would do that | 21:47 |
| jodah | but you may (likely?) suffer higher latency by replicating across zones? | 21:47 |
| ahale | or regions, but i dont really know much about that, someone else will though | 21:47 |
| ahale | maybe, it depends how you network really | 21:48 |
| jodah | indeed. i think you're probably hitting a few more switches though | 21:48 |
| ahale | you can have a separate replication interface and network if it all gets really crazy | 21:48 |
| jodah | but swift would need to know about those things | 21:48 |
| ahale | yep thats all in the rings | 21:48 |
| jodah | does it replicate across AZs now? or can it? | 21:48 |
| *** lcurtis has quit IRC | 21:49 | |
| ahale | it depends how you define az | 21:49 |
| ahale | but yeah it can replicate across continents - though i've only done within datacenters | 21:49 |
| zaitcev | Back it up, guys. Let's first identify who jodah wants "to ensure that an object PUT to swift is placed on a particular node". Is this knowledge visible to client and then you have API to force placement? Or, you want cluster administrator configure ring for locality? | 21:49 |
| jodah | For an administrator | 21:50 |
| jodah | For another OpenStack service, really, in my case Trove/Sahara | 21:50 |
| ahale | ok :) i was just going with the questions | 21:50 |
| zaitcev | Okay. Are you willing to take the durability hit that's inevitably associated with locality? | 21:50 |
| zaitcev | Oh, it's Sahara | 21:50 |
| zaitcev | I knew it :-) | 21:51 |
| *** silor1 has quit IRC | 21:51 | |
| jodah | ;) | 21:51 |
| jodah | well ideally we'd want 1 copy on the same rack as something, like say, the same rack/host that our VM is on. but of course we'd want Swift to place other replicas in other places | 21:51 |
| jodah | ....as it does | 21:51 |
| zaitcev | yeah. well, you can't do that with a DHT, because the hashing is the key | 21:51 |
| jodah | but still, having 1-2 copies on the same rack would cut down on network saturation. | 21:51 |
| jodah | could hashing potentially take location into account - like, to be biased towards nodes on the same rack (affinity) or diff racks (anti-affinity)? | 21:52 |
| jodah | like to only select from nodes that are or are not on the same rack, and hash from those | 21:52 |
| jodah | I'm asking from the perspective of Trove and Sahara, but really the ability to strategically place replicas behind different points of failure (racks) even within the same AZ could be beneficial to Swift as well | 21:53 |
| ahale | ah - I have been thinking around the same things with zerovm and data locality and keep thinking more about storage policies to limit where my data goes in a larger cluster | 21:53 |
| zaitcev | Maybe you should look at Ceph. They allow you to maneuver the placement groups like that. | 21:54 |
| jodah | right - the common thing with all of these is that they're datastores with partitioned/replicated data. and for various reasons - latency, fault tolerance, location matters. | 21:54 |
| jodah | even the concept of a placement group only gives you affinity - you still want anti-affinity for fault tolerance. | 21:55 |
| openstackgerrit | Samuel Merritt proposed openstack/swift: Make ThreadPools deallocatable. https://review.openstack.org/145647 | 21:55 |
| zaitcev | Anyway, I have a feeling that you don't know about the exceedingly ugly kludge that Sahara implemented to force locality into Swift. Not sure if I should point it out, you may sleep better without knowing. | 21:56 |
| jodah | I do | 21:56 |
| jodah | haha | 21:56 |
| jodah | It's unfortunate, as all agree | 21:56 |
| mattoliverau | jodah: long story short, no you can't. Swift will use the ring to place the data on different devices, and other then weights and your availablility zones you don't have that much control. But you could use storage policies to have specific sets of storage nodes, and have a little more control that way. | 21:56 |
| jodah | So I'm interested in looking for a more general solution, particularly since this would benefit Trove as well. | 21:57 |
| zaitcev | Ah, okay. So that's why you place the emphasis is on the administrator. | 21:57 |
| jodah | Yea | 21:57 |
| jodah | End users could benefit from something similar - placement group affinity and anti-affinity - but they don't necessarily need this. | 21:57 |
| zaitcev | Listen, we need to put this onto agenda for hackathon in February. | 21:57 |
| zaitcev | poke notmyname about it | 21:58 |
| jodah | mattoliverau: Is this something that could reasonably be suggested for the future? For the benefit of Sahara/Trove - it would be nice to be able to place objects onto specific nodes, such as those sharing a rack with a particular VM. For the benefit of Swift itself, it would be nice if the same underlying information could be used to influence where the ring places replicas. | 21:59 |
| tdasilva | mattoliverau, zaitcev: wouldn't the write_affinity setting help jodah ? | 22:14 |
| mattoliverau | jodah: you can play with the write affinity on the proxy to force writing to local nodes (or certain regions and or regions first), but if you have more then "replica" availablility zones then you can't garrentee a certian zone as it depends on the where the ring partition is being placed in the cluster which is decided at ring creation/rebalance not run time. If you have a storage policy containing only 3 (or replica | 22:15 |
| mattoliverau | count) availability zones and say these zones are racks, and are evenly balanced, then you can reasonably say one replica would be on each rack | 22:15 |
| zaitcev | tdasilva: I don't know, sorry. I need to know what we have now and what jodah needs overall. | 22:15 |
| mattoliverau | *certain regions or zones first | 22:15 |
| mattoliverau | Still pre coffee for me.. so not quite awake yet :P | 22:16 |
| jodah | Good info | 22:17 |
| mattoliverau | tdasilva: yeah it could a little, I was just writing something about that :P | 22:17 |
| tdasilva | mattoliverau: I think a while back you said something about "great minds" :P | 22:19 |
| mattoliverau | tdasilva: ;) | 22:19 |
| jodah | I understand that :) | 22:23 |
| jodah | zaitcev: The primary requirement is the need to be able to place an object on a particular host, or at least, onto a particular rack. | 22:23 |
| jodah | The secondary requirement is the need to instruct Swift to (for certain objects - perhaps) place some replicas on the same rack and others on a separate rack. This implies that Swift has knowledge of host->rack information, which of course it doesn't. But this ability to "place" resources into a specific location is needed by us from Nova as well - so perhaps a common solution for housing this information could make sense. | 22:23 |
| jodah | Both goals are about exploiting data locality, for minimizing read/write latency, and for fault tolerance by ensuring that replicas are placed behind separate points of failure. | 22:25 |
| zaitcev | You can create a policy with a more restricted ring for that today... Client has to select it though (presumably it's not the default one). I think ahale mentioned that above. | 22:25 |
| jodah | ok - i'll read up on policies | 22:26 |
| jodah | can a policy be selected per object or are they globally configured? | 22:26 |
| zaitcev | Policies are per container. | 22:26 |
| jodah | ok | 22:26 |
| *** bpap has joined #openstack-swift | 22:29 | |
| *** dmsimard is now known as dmsimard_away | 22:32 | |
| clayg | torgomatic: now someone is going to ask you to move the initialization code in a revivify method and make it so you can call that after youc all terminate to make it _alive again | 22:35 |
| torgomatic | clayg: yes; being wrong never seems to stop people from doing things ;) | 22:36 |
| jodah | Different question: When GETting an object from an endpoint given via the list endpoints call available in the middleware, presumably the object is just read from local storage on the host at the endpoint? | 22:37 |
| torgomatic | jodah: that's the theory | 22:37 |
| zaitcev | Good heavens, it's an actual word - http://www.dict.org/bin/Dict?Form=Dict2&Database=*&Query=vivify | 22:37 |
| jodah | lol - i'll assume yes then :) | 22:37 |
| clayg | zaitcev: sorry i picked it up from swifterdarrell - i had the same reaction | 22:37 |
| clayg | at the time I felt like there was another word i had that ment the same thing but poeple wouldn't freak out when you said it - now I feel like that's the exact word I want when that's what I mean | 22:38 |
| swifterdarrell | clayg: sure it wasn't reify/reified? | 22:38 |
| zaitcev | The _best_ misunderstanding _ever_. | 22:39 |
| clayg | swifterdarrell: jesus, probably | 22:39 |
| *** annegent_ has joined #openstack-swift | 22:49 | |
| *** abhirc has quit IRC | 22:50 | |
| *** annegent_ has quit IRC | 22:55 | |
| *** hugespoon has quit IRC | 22:56 | |
| *** hugespoon has joined #openstack-swift | 23:13 | |
| *** briancline_ has quit IRC | 23:14 | |
| *** jrichli has quit IRC | 23:15 | |
| *** zaitcev has quit IRC | 23:26 | |
| *** tellesnobrega_ has quit IRC | 23:31 | |
| *** jasondotstar has joined #openstack-swift | 23:34 | |
| *** briancurtin has quit IRC | 23:41 | |
| *** briancurtin has joined #openstack-swift | 23:41 | |
| *** echevemaster has joined #openstack-swift | 23:46 | |
| *** briancurtin has quit IRC | 23:49 | |
| *** david-lyle is now known as david-lyle_afk | 23:56 | |
| *** chlong has joined #openstack-swift | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!