| *** asettle has joined #openstack-swift | 00:04 | |
| *** asettle has quit IRC | 00:09 | |
| openstackgerrit | Janie Richling proposed openstack/swift: WIP: Set backend content length for fallocate - EC Policy https://review.openstack.org/282578 | 00:11 |
|---|---|---|
| *** chsc has quit IRC | 00:16 | |
| notmyname | mattoliverau: cool. so it's ready to land? | 00:27 |
| *** sgundur has left #openstack-swift | 00:32 | |
| *** logan- has quit IRC | 00:35 | |
| *** garthb__ has quit IRC | 00:35 | |
| *** logan- has joined #openstack-swift | 00:41 | |
| *** loth has joined #openstack-swift | 00:44 | |
| loth | Hey all, how generally is the recommended way to do geo-relication for swift? storage policies? | 00:45 |
| *** diogogmt has quit IRC | 00:46 | |
| notmyname | loth: depends on your goal | 00:49 |
| notmyname | loth: one way (and the most common way I see it) is setting up a storage policy that spans multiple regions | 00:49 |
| notmyname | then at that point, if you add an object to a container with the global storage policy, you end up with the data geo-replicated | 00:51 |
| loth | I did see this https://www.mirantis.com/blog/configuring-multi-region-cluster-openstack-swift/ which is called proxy affinity i guess? | 00:52 |
| notmyname | https://swiftstack.com/wp-content/uploads/2015/06/SwiftStackGlobalClusterDeploymentGuide.pdf is a swiftstack version of how to do that. it has config UI stuff particular to swiftstack's product, but all the settings are configurable by hand in the config files | 00:53 |
| notmyname | loth: and https://swiftstack.com/docs/admin/cluster_management/regions.html | 00:54 |
| loth | ah okay, this is a more recent version of the one i found from 2012 :) | 00:54 |
| loth | I'll check them out, thanks! | 00:55 |
| notmyname | loth: the reason I point you there is that setting up a global cluster is tricky. well, more tricky to do right than perhaps a casual IRC conversation allow for. it's good to have thorough docs to peruse | 00:55 |
| *** gyee has quit IRC | 00:56 | |
| *** trifon has quit IRC | 01:01 | |
| loth | Is Swift a good storage location for CDN services, or should that be handled with something else? I can see objects can be made accessible via HTTP | 01:01 |
| notmyname | IMO, swift is fantastic as a CDN origin server. and I think swift is likely overkill for CDN PoPs | 01:04 |
| loth | Found this as an origin server middleware: https://github.com/dpgoetz/sos | 01:05 |
| notmyname | right. that's one that's been used at both RAX and HP for CDN services | 01:06 |
| notmyname | at very large scale in public clouds | 01:06 |
| loth | notmyname: Thanks for the info! Do you know of any documentation regarding the recommended number of proxies vs storage boxes? Would that depend on the kind of usage that is expected? I can see that the recommended zone number is 5+, and those are largely flexible depending on cluster size. | 01:09 |
| notmyname | oh wow. 5+ zones is a *very* old recommendation | 01:10 |
| notmyname | the recommended number of proxies is very much dependent on your usage patterns | 01:10 |
| notmyname | and, again with the swiftstack docs, https://swiftstack.com/docs/admin/hardware.html | 01:11 |
| notmyname | (BTW, I work at swiftstack, so that's why I'm familiar with the docs there) | 01:11 |
| loth | Was working with these, which are linked in the openstack doc pages http://docs.openstack.org/developer/swift/deployment_guide.html | 01:11 |
| notmyname | loth: as a very very general rule of thumb, you could likely do 1 proxy per 1000 requests per second that you need. with at least 2 because HA | 01:12 |
| loth | And those numbers are per zone? | 01:12 |
| notmyname | no. per cluster | 01:12 |
| notmyname | loth: bah! yes, the 5+ zone requirement was true back in 2012. since then it hasn't been true | 01:13 |
| loth | suppose openstack doc team is a bit behind :) | 01:13 |
| notmyname | that's not on the docs team. that's on us | 01:15 |
| *** _JZ_ has quit IRC | 01:15 | |
| notmyname | I filed a bug for it | 01:15 |
| loth | Thats interesting, so in these examples there is one zone per rack? | 01:20 |
| notmyname | one zone per rack is generally what people do. that's a reasonable recommendation | 01:22 |
| loth | What was the reasoning for the 5 zone minimum in the past? | 01:22 |
| notmyname | basically, a zone is a physical failure domain in your deployment. and a rack normally has a single ToR switch that is the failure domain | 01:22 |
| notmyname | originally, swift would only place 1 replica per zone. so for 3 replicas, you wanted a couple of handoff locations, and that gives you 5 or more | 01:23 |
| notmyname | however, these days, the only requirement is that you have at least as many drives as your replica count | 01:23 |
| loth | Ah, makes sense | 01:25 |
| notmyname | interestingly, there's been several generations of ring placement algorithms in swift. the first was the replica-per-zone model. next was the unique-as-possible. lately, we've updated that to be a "distributed-as-balanceable" | 01:27 |
| loth | How does that work for clusters on the older algorithms, is there some sort of migration needed? | 01:27 |
| notmyname | the first lasted about a year. the unique as possible lasted about 3 or 4 years. distributed as balanceable has been the new hotness for about 6 months now | 01:28 |
| notmyname | nope. each one can understand the older formats and just makes things better as you adjust capacity and rebalance | 01:28 |
| notmyname | note that I've grossly trivialized some of the really hard and really interesting work that's gone in to all of that :-) | 01:29 |
| loth | Hehe it sounds awesome :) | 01:29 |
| loth | Currently I've got seperate keystone deployments in every region (which I relize now is probably a mistake). Is it best to have a centralized keystone deployment serving many regions for Swift? | 01:31 |
| notmyname | you can read a lot of stuff abotu it at http://docs.openstack.org/developer/swift/overview_ring.html | 01:31 |
| notmyname | yeah, a centralized identity system normally makes sense. otherwise you get into "interesting" coordination issues | 01:32 |
| loth | My other concern is applications already built around Swift wouldnt work without a centralized keystone server | 01:33 |
| notmyname | shouldn't matter | 01:34 |
| notmyname | apps that work with swift need to have some endpoint from which to get an auth token | 01:35 |
| notmyname | doesn't really matter where, as long as the auth token they get can be validated with the swift cluster they connect to | 01:35 |
| loth | I thought apps that specify a single server (keystone) while giving a region would be expecting endpoint to change for each region, i guess thats incorrect | 01:38 |
| *** arch-nemesis has quit IRC | 01:46 | |
| loth | Thanks again for the assistance! | 01:54 |
| *** lyrrad has quit IRC | 01:55 | |
| *** lpabon has joined #openstack-swift | 02:05 | |
| *** klrmn has quit IRC | 02:06 | |
| *** asettle has joined #openstack-swift | 02:10 | |
| *** asettle has quit IRC | 02:15 | |
| *** rickyrem has joined #openstack-swift | 02:50 | |
| *** lpabon has quit IRC | 03:01 | |
| *** mingdang1 has joined #openstack-swift | 03:03 | |
| *** trifon has joined #openstack-swift | 03:05 | |
| *** rickyrem has quit IRC | 03:17 | |
| *** trifon has quit IRC | 03:19 | |
| *** asettle has joined #openstack-swift | 03:20 | |
| *** asettle has quit IRC | 03:25 | |
| *** trifon has joined #openstack-swift | 03:38 | |
| *** asettle has joined #openstack-swift | 03:43 | |
| *** mingdang1 has quit IRC | 03:46 | |
| *** asettle has quit IRC | 03:48 | |
| *** klrmn has joined #openstack-swift | 03:49 | |
| *** nadeem has joined #openstack-swift | 03:51 | |
| *** asettle has joined #openstack-swift | 04:06 | |
| *** asettle has quit IRC | 04:11 | |
| *** trifon has quit IRC | 04:15 | |
| *** esker has quit IRC | 04:33 | |
| *** esker has joined #openstack-swift | 04:43 | |
| *** mingdang1 has joined #openstack-swift | 04:47 | |
| *** chlong_ has joined #openstack-swift | 05:21 | |
| *** trifon has joined #openstack-swift | 05:25 | |
| *** haomaiwang has joined #openstack-swift | 05:39 | |
| *** zaitcev has quit IRC | 05:50 | |
| *** diazjf1 has quit IRC | 05:56 | |
| *** diazjf has joined #openstack-swift | 05:56 | |
| *** diazjf has quit IRC | 05:57 | |
| *** mingyu has joined #openstack-swift | 05:59 | |
| *** haomaiwang has quit IRC | 06:01 | |
| *** haomaiwang has joined #openstack-swift | 06:01 | |
| *** asettle has joined #openstack-swift | 06:09 | |
| *** asettle has quit IRC | 06:14 | |
| *** resker has joined #openstack-swift | 06:18 | |
| *** esker has quit IRC | 06:18 | |
| *** mingyu has quit IRC | 06:20 | |
| *** mwheckmann has quit IRC | 06:38 | |
| *** nadeem has quit IRC | 06:47 | |
| *** arch-nemesis has joined #openstack-swift | 06:51 | |
| *** trifon has quit IRC | 06:52 | |
| *** nadeem has joined #openstack-swift | 06:54 | |
| *** haomaiwang has quit IRC | 07:01 | |
| *** haomaiwang has joined #openstack-swift | 07:01 | |
| *** mingdang1 has quit IRC | 07:37 | |
| *** forcebanana has joined #openstack-swift | 07:38 | |
| *** haomaiwang has quit IRC | 08:01 | |
| *** haomaiwa_ has joined #openstack-swift | 08:01 | |
| *** haomaiwa_ has quit IRC | 08:03 | |
| openstackgerrit | Tim Burke proposed openstack/swift: By default, disallow inbound X-Timestamp headers https://review.openstack.org/279256 | 08:05 |
| *** klrmn has quit IRC | 08:16 | |
| *** haomaiwang has joined #openstack-swift | 08:18 | |
| *** haomaiwang has quit IRC | 08:23 | |
| *** asettle has joined #openstack-swift | 08:31 | |
| *** resker has quit IRC | 08:32 | |
| *** asettle has quit IRC | 08:36 | |
| *** mingdang1 has joined #openstack-swift | 08:37 | |
| *** esker has joined #openstack-swift | 08:40 | |
| *** mingdang1 has quit IRC | 08:41 | |
| *** haomaiwang has joined #openstack-swift | 08:50 | |
| *** haomaiwang has quit IRC | 08:54 | |
| *** SkyRocknRoll has joined #openstack-swift | 08:59 | |
| *** haomaiwang has joined #openstack-swift | 09:05 | |
| *** haomaiwang has quit IRC | 09:10 | |
| *** haomaiwang has joined #openstack-swift | 09:34 | |
| *** haomaiwang has quit IRC | 09:47 | |
| *** haomaiwa_ has joined #openstack-swift | 09:50 | |
| *** trifon has joined #openstack-swift | 09:57 | |
| *** SkyRocknRoll has quit IRC | 10:00 | |
| *** haomaiwa_ has quit IRC | 10:01 | |
| *** haomaiwang has joined #openstack-swift | 10:01 | |
| *** haomaiwang has quit IRC | 10:02 | |
| *** stantonnet has quit IRC | 10:04 | |
| *** stantonnet has joined #openstack-swift | 10:05 | |
| *** mingdang1 has joined #openstack-swift | 10:25 | |
| *** esker has quit IRC | 10:32 | |
| *** SkyRocknRoll has joined #openstack-swift | 10:33 | |
| *** esker has joined #openstack-swift | 10:38 | |
| *** openstackgerrit_ has quit IRC | 10:40 | |
| *** openstackgerrit_ has joined #openstack-swift | 10:41 | |
| *** haomaiwa_ has joined #openstack-swift | 10:45 | |
| *** openstackgerrit_ has quit IRC | 10:48 | |
| *** haomaiwa_ has quit IRC | 10:49 | |
| *** haomaiwang has joined #openstack-swift | 11:00 | |
| *** asettle has joined #openstack-swift | 11:00 | |
| *** haomaiwang has quit IRC | 11:01 | |
| *** asettle has quit IRC | 11:06 | |
| *** Jeffrey4l has joined #openstack-swift | 11:17 | |
| *** biyiklioglu has joined #openstack-swift | 11:24 | |
| *** biyiklioglu has quit IRC | 11:29 | |
| *** biyiklioglu has joined #openstack-swift | 11:29 | |
| *** nadeem has quit IRC | 11:44 | |
| *** biyiklioglu has quit IRC | 11:54 | |
| *** ChubYann_ has quit IRC | 12:10 | |
| *** ChubYann_ has joined #openstack-swift | 12:23 | |
| *** Jeffrey4l has quit IRC | 12:41 | |
| *** asettle has joined #openstack-swift | 12:53 | |
| *** Jeffrey4l has joined #openstack-swift | 12:54 | |
| *** asettle has quit IRC | 12:58 | |
| *** haomaiwa_ has joined #openstack-swift | 13:09 | |
| *** urth has quit IRC | 13:31 | |
| *** urth has joined #openstack-swift | 13:34 | |
| *** haomaiwa_ has quit IRC | 14:01 | |
| *** haomaiwang has joined #openstack-swift | 14:01 | |
| *** asettle has joined #openstack-swift | 14:12 | |
| *** asettle has quit IRC | 14:18 | |
| *** baojg has joined #openstack-swift | 14:18 | |
| *** forcebanana has quit IRC | 14:25 | |
| *** mingdang1 has quit IRC | 14:34 | |
| *** haomaiwang has quit IRC | 15:01 | |
| *** haomaiwang has joined #openstack-swift | 15:01 | |
| *** sgundur has joined #openstack-swift | 15:41 | |
| *** sgundur has left #openstack-swift | 15:41 | |
| *** haomaiwang has quit IRC | 16:01 | |
| *** haomaiwang has joined #openstack-swift | 16:01 | |
| openstackgerrit | Tim Burke proposed openstack/swift: By default, disallow inbound X-Timestamp headers https://review.openstack.org/279256 | 16:24 |
| *** arch-nemesis has quit IRC | 16:25 | |
| *** Jeffrey4l has quit IRC | 16:32 | |
| *** arch-nemesis has joined #openstack-swift | 16:41 | |
| *** klrmn has joined #openstack-swift | 16:42 | |
| *** asettle has joined #openstack-swift | 16:52 | |
| *** asettle has quit IRC | 16:57 | |
| *** rcernin has joined #openstack-swift | 16:59 | |
| *** arch-nemesis has quit IRC | 17:00 | |
| *** haomaiwang has quit IRC | 17:01 | |
| *** haomaiwa_ has joined #openstack-swift | 17:01 | |
| *** asettle has joined #openstack-swift | 17:07 | |
| natarej | so i'm in a situation where i can repurpose some hardware for swift storage nodes for some excellent value capacity | 17:09 |
| natarej | 32 bays available with 12 core xeons | 17:09 |
| natarej | however it's limited to 1gbe which would be outward facing and 1x10gbe cluster facing | 17:10 |
| *** asettle has quit IRC | 17:12 | |
| *** arch-nemesis has joined #openstack-swift | 17:12 | |
| natarej | what would be the most storage you'd be comfortable running with such network limitations? could run up to 256TB for $7000USD but it would take over 2 days to rebalance with the single 10gbe | 17:14 |
| *** arch-nemesis has quit IRC | 17:22 | |
| openstackgerrit | Joel Wright proposed openstack/python-swiftclient: Do not reveal auth token in swiftclient log messages by default https://review.openstack.org/282363 | 17:46 |
| *** ChubYann_ has quit IRC | 17:54 | |
| *** rcernin has quit IRC | 17:55 | |
| *** rcernin has joined #openstack-swift | 17:59 | |
| *** haomaiwa_ has quit IRC | 18:01 | |
| *** haomaiwang has joined #openstack-swift | 18:01 | |
| *** ChubYann_ has joined #openstack-swift | 18:07 | |
| *** baojg has quit IRC | 18:27 | |
| *** asettle has joined #openstack-swift | 18:50 | |
| *** mwheckmann has joined #openstack-swift | 18:55 | |
| *** asettle has quit IRC | 18:55 | |
| *** mwheckmann has quit IRC | 18:59 | |
| *** haomaiwang has quit IRC | 19:01 | |
| *** haomaiwa_ has joined #openstack-swift | 19:01 | |
| *** mwheckmann has joined #openstack-swift | 19:02 | |
| *** SkyRocknRoll has quit IRC | 19:06 | |
| *** mwheckmann has quit IRC | 19:08 | |
| *** haomaiwa_ has quit IRC | 20:01 | |
| *** haomaiwa_ has joined #openstack-swift | 20:01 | |
| *** jmccarthy has quit IRC | 20:11 | |
| *** jmccarthy has joined #openstack-swift | 20:12 | |
| *** Gordoni has joined #openstack-swift | 20:32 | |
| *** proteusguy__ has quit IRC | 20:49 | |
| *** asettle has joined #openstack-swift | 20:51 | |
| *** rcernin has quit IRC | 20:56 | |
| *** asettle has quit IRC | 20:56 | |
| *** haomaiwa_ has quit IRC | 21:01 | |
| *** haomaiwang has joined #openstack-swift | 21:01 | |
| *** proteusguy__ has joined #openstack-swift | 21:02 | |
| *** forcebanana has joined #openstack-swift | 21:12 | |
| *** sgundur has joined #openstack-swift | 21:25 | |
| *** rcernin has joined #openstack-swift | 21:30 | |
| *** trifon has quit IRC | 21:30 | |
| *** klrmn has quit IRC | 21:46 | |
| *** klrmn has joined #openstack-swift | 21:47 | |
| *** haomaiwang has quit IRC | 22:01 | |
| *** haomaiwang has joined #openstack-swift | 22:01 | |
| *** sgundur has left #openstack-swift | 22:07 | |
| *** haomaiwang has quit IRC | 23:01 | |
| *** haomaiwang has joined #openstack-swift | 23:01 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!