| *** mingyu_ has joined #openstack-swift | 00:24 | |
| *** mingyu has quit IRC | 00:26 | |
| spsurya | hi....swifters can i add new storage node in SAIO installed node ? | 00:27 |
|---|---|---|
| clayg | probably... I mean... anything is possible. As with most things you'll find going from 1 -> 2 is harder than going from 2 -> 3 | 00:32 |
| clayg | you might want to spin up some vms or something and setup a multinode cluster so you know how that *works* - then come up with a plan to migrate whatever setup/tooling you have for this existing setup... | 00:33 |
| DHE | spsurya: well, yes. SAIO just means you are simulating multiple "nodes" by using different TCP ports rather than different IPs. | 00:43 |
| DHE | spsurya: are you still in a 'testing' mode or do you want to actually make this a real redundant installation? | 00:44 |
| spsurya | DHE: for now I am trying to install swift with SAIO and then create object with/without user then want add new node | 00:46 |
| spsurya | then want to access that object from new node | 00:46 |
| spsurya | So looking for the configuration regarding ip:port, new device mount points etc | 00:48 |
| spsurya | where i have to change | 00:49 |
| spsurya | thanks clayg for I wiil try | 00:49 |
| DHE | is that a bot? | 00:49 |
| spsurya | DHE: couldn't parse | 00:50 |
| DHE | n/m i'm wrong | 00:50 |
| DHE | adding new storage nodes is half running the services on them and half building/distributing the new ring files to all hosts so they agree it is a cluster member | 00:52 |
| spsurya | DHE: yes | 00:56 |
| openstackgerrit | Merged openstack/slogging master: This commit remove dead link in Readme https://review.openstack.org/566762 | 00:56 |
| DHE | yes.. and SAIO doesn't really change that. you're just running one node that sorta simulates many. | 00:57 |
| spsurya | DHE: yes | 01:01 |
| *** gholt has quit IRC | 01:04 | |
| *** bkopilov_ has quit IRC | 01:35 | |
| *** bkopilov__ has quit IRC | 01:35 | |
| *** m_kazuhiro has joined #openstack-swift | 01:35 | |
| m_kazuhiro | good morning | 01:35 |
| kota_ | notmyname: should i go patch 3-7 in the feature deep patch chain first? | 01:58 |
| patchbot | No data found for patch 3 | 01:58 |
| kota_ | thanks patchbot for the answer. | 01:59 |
| *** psachin` has joined #openstack-swift | 02:13 | |
| *** klamath has joined #openstack-swift | 02:30 | |
| *** klamath_ has quit IRC | 02:33 | |
| *** bkopilov_ has joined #openstack-swift | 02:55 | |
| *** bkopilov__ has joined #openstack-swift | 02:56 | |
| *** two_tired has joined #openstack-swift | 03:05 | |
| *** klamath has quit IRC | 03:18 | |
| *** two_tired has quit IRC | 03:23 | |
| notmyname | kota_: no, no. I was just using random numbers as examples | 03:26 |
| *** gyankum has joined #openstack-swift | 03:41 | |
| kota_ | notmyname: ok :-) | 03:47 |
| *** gyee has quit IRC | 03:48 | |
| *** m_kazuhiro has quit IRC | 03:53 | |
| *** mingyu_ has quit IRC | 04:06 | |
| *** m_kazuhiro has joined #openstack-swift | 04:09 | |
| *** gyee has joined #openstack-swift | 04:17 | |
| *** gyee has quit IRC | 04:24 | |
| *** cshastri has joined #openstack-swift | 04:31 | |
| *** links has joined #openstack-swift | 04:38 | |
| *** mingyu has joined #openstack-swift | 05:11 | |
| *** klamath has joined #openstack-swift | 05:18 | |
| *** klamath has quit IRC | 05:37 | |
| *** ccamacho has joined #openstack-swift | 05:41 | |
| *** pcaruana has joined #openstack-swift | 06:01 | |
| *** cshastri has quit IRC | 06:03 | |
| *** armaan has joined #openstack-swift | 06:07 | |
| *** klamath has joined #openstack-swift | 06:11 | |
| *** armaan has quit IRC | 06:11 | |
| *** armaan has joined #openstack-swift | 06:12 | |
| *** klamath has quit IRC | 06:15 | |
| *** cshastri has joined #openstack-swift | 06:19 | |
| *** armaan has quit IRC | 06:24 | |
| *** armaan has joined #openstack-swift | 06:25 | |
| *** cshastri has quit IRC | 06:26 | |
| *** mingyu_ has joined #openstack-swift | 06:37 | |
| *** mingyu has quit IRC | 06:37 | |
| *** cshastri has joined #openstack-swift | 06:41 | |
| *** mingyu_ has quit IRC | 06:42 | |
| *** hseipp has joined #openstack-swift | 06:43 | |
| *** rcernin has quit IRC | 07:08 | |
| *** m_kazuhiro has quit IRC | 07:11 | |
| *** armaan has quit IRC | 07:14 | |
| *** klamath has joined #openstack-swift | 07:25 | |
| *** klamath has quit IRC | 07:29 | |
| *** geaaru has joined #openstack-swift | 07:40 | |
| acoles | good morning | 08:01 |
| acoles | I think notmyname's comments in here yesterday are useful input for deep-review reviewers. | 08:08 |
| acoles | I'd add, if anyone core reviewer would like high bandwidth discussion with me (voice/video) then just ask | 08:08 |
| openstackgerrit | HCLTech-SSW proposed openstack/swift master: Implemented the fix to handle the HTTP request methods other than GET. https://review.openstack.org/566806 | 08:10 |
| *** neonpastor has joined #openstack-swift | 08:31 | |
| *** zigo_ has quit IRC | 08:40 | |
| *** linkmark has quit IRC | 08:41 | |
| *** linkmark has joined #openstack-swift | 08:42 | |
| *** zigo has joined #openstack-swift | 08:45 | |
| *** d0ugal__ has quit IRC | 08:57 | |
| *** d0ugal has joined #openstack-swift | 08:58 | |
| *** armaan has joined #openstack-swift | 09:01 | |
| *** armaan has quit IRC | 09:08 | |
| *** gkadam has joined #openstack-swift | 09:19 | |
| *** klamath has joined #openstack-swift | 09:26 | |
| *** klamath has quit IRC | 09:31 | |
| *** mingyu has joined #openstack-swift | 09:33 | |
| *** mingyu has quit IRC | 09:35 | |
| *** mingyu_ has joined #openstack-swift | 09:35 | |
| mattoliverau | acoles: I'm doing a diagram for sharding GET.. And just want to clarify. container listing while sharding. If ranges have sharded, then go visit the shard containers. If hitting beyond where cleaving is up to, then just grab the retiring db's records ignore the shards. | 09:39 |
| *** bkopilov__ has quit IRC | 09:39 | |
| *** bkopilov_ has quit IRC | 09:39 | |
| acoles | mattoliverau: correct. specifically, only shard ranges in one of the SHARD_LISTING_STATES https://github.com/openstack/swift/blob/d0424c2dd3e8f9a0fd54f87e9928b8f4cab11d61/swift/container/backend.py#L62 are used for listings - any 'gap' is filled by the retiring db. | 09:41 |
| acoles | mattoliverau: and thanks for working on the diagram! | 09:42 |
| mattoliverau | acoles: ta.. cool, that's what got from reading the code, just wanted to confirm. It's alright the diagram just points to where it looks. I haven't bothered with updating words yet. | 09:43 |
| *** armaan has joined #openstack-swift | 10:04 | |
| * kota_ is still half of the patch 1 yet :/ | 10:05 | |
| patchbot | https://review.openstack.org/#/c/1/ - openstack-infra/system-config - Add puppet module for ssh that installs an sshd_co... (MERGED) | 10:05 |
| kota_ | no patchbot! | 10:05 |
| acoles | lol! | 10:05 |
| kota_ | but it looks interesting that is the first patch in the openstack world. | 10:05 |
| acoles | kota_ is reviewing openstack from the beggining :) | 10:05 |
| acoles | ugh, beginning* | 10:06 |
| kota_ | lol | 10:07 |
| kota_ | hmm... it's over 7 pm my local time. | 10:07 |
| kota_ | let's push my current comments there. | 10:07 |
| acoles | kota_: thanks! | 10:08 |
| kota_ | acoles: added | 10:11 |
| *** kei_yama has quit IRC | 10:14 | |
| * kota_ is preparing to head for my home. | 10:17 | |
| *** mingyu_ has quit IRC | 10:19 | |
| *** rcernin has joined #openstack-swift | 10:35 | |
| mattoliverau | acoles: ok, I think I have all the diagrams updated, do you mind if I push up a new version of the docs? | 10:37 |
| acoles | mattoliverau: no, please do - did you base on the changes I pushed yesterday? | 10:38 |
| mattoliverau | I think so.. but let me double check :) | 10:38 |
| acoles | commit 65c46a | 10:39 |
| acoles | mattoliverau: I added list of ShardRange states in that version | 10:40 |
| mattoliverau | Just pulled it elsewhere, the head it 65c46a and when diffed against my new one only a bunch of svg (all xml) is added and some renames in the doc.. so looks good. | 10:44 |
| acoles | mattoliverau: cool, thanks for checking | 10:44 |
| openstackgerrit | Matthew Oliver proposed openstack/swift feature/deep-review: Add container sharding documentation https://review.openstack.org/565749 | 10:46 |
| mattoliverau | make all the digarms continue to build up and so all use the same example data | 10:47 |
| mattoliverau | and I've been editting them in inkscape, which is OSS, in case anyone wants to change anything... or just ask me too :) | 10:48 |
| mattoliverau | kk, time to go spend some time with the family. Night | 10:48 |
| mattoliverau | acoles: let me know if any of the digrams are incorrect | 10:49 |
| acoles | thank mattoliverau | 10:49 |
| acoles | you! | 10:49 |
| acoles | timburke: if you get chance to comment https://review.openstack.org/#/c/565742/1/swift/common/db.py@414 | 10:53 |
| patchbot | patch 565742 - swift (feature/deep-review) - Refactoring, test infrastructure changes and cleanup | 10:53 |
| *** mrjk_ has quit IRC | 11:27 | |
| *** mrjk_ has joined #openstack-swift | 11:28 | |
| *** mingyu has joined #openstack-swift | 11:46 | |
| acoles | clayg: could you handle timburke's comments here https://review.openstack.org/#/c/565742/1/swift/common/db.py | 11:50 |
| patchbot | patch 565742 - swift (feature/deep-review) - Refactoring, test infrastructure changes and cleanup | 11:50 |
| *** cbartz has joined #openstack-swift | 11:52 | |
| *** bkopilov_ has joined #openstack-swift | 12:25 | |
| *** bkopilov__ has joined #openstack-swift | 12:27 | |
| *** Supun has joined #openstack-swift | 12:41 | |
| remix_tj | tdasilva: hi, are you still maintaining this? https://github.com/thiagodasilva/ansible-saio i'm having some errors with ansible 2.4.3.0 which i'm fixing and i'd like to provide feedback if the tool is still alive | 12:47 |
| *** cbartz has quit IRC | 12:49 | |
| *** openstackgerrit has quit IRC | 12:49 | |
| *** gyankum has quit IRC | 12:53 | |
| *** NM has joined #openstack-swift | 13:03 | |
| *** armaan has quit IRC | 13:05 | |
| *** gkadam_ has joined #openstack-swift | 13:05 | |
| *** armaan has joined #openstack-swift | 13:06 | |
| *** gkadam has quit IRC | 13:07 | |
| *** cshastri has quit IRC | 13:09 | |
| *** vinsh_ has joined #openstack-swift | 13:12 | |
| *** vinsh has quit IRC | 13:12 | |
| *** mvenesio has joined #openstack-swift | 13:13 | |
| *** vinsh_ has quit IRC | 13:14 | |
| *** bswrchrd has joined #openstack-swift | 13:14 | |
| *** vinsh has joined #openstack-swift | 13:14 | |
| *** armaan has quit IRC | 13:23 | |
| *** mingyu has quit IRC | 13:23 | |
| *** armaan has joined #openstack-swift | 13:23 | |
| *** klamath has joined #openstack-swift | 13:28 | |
| *** bswrchrd has quit IRC | 13:32 | |
| *** klamath has quit IRC | 13:32 | |
| *** bswrchrd has joined #openstack-swift | 13:32 | |
| *** klamath has joined #openstack-swift | 13:39 | |
| *** NM has quit IRC | 13:54 | |
| *** NM has joined #openstack-swift | 13:55 | |
| *** ccamacho has quit IRC | 14:00 | |
| *** ccamacho1 has joined #openstack-swift | 14:00 | |
| *** ccamacho1 has quit IRC | 14:00 | |
| *** ccamacho has joined #openstack-swift | 14:00 | |
| *** armaan has quit IRC | 14:07 | |
| *** armaan has joined #openstack-swift | 14:07 | |
| *** psachin` has quit IRC | 14:07 | |
| *** mingyu has joined #openstack-swift | 14:14 | |
| *** armaan has quit IRC | 14:25 | |
| *** armaan has joined #openstack-swift | 14:26 | |
| *** geaaru has quit IRC | 14:31 | |
| *** rcernin has quit IRC | 14:47 | |
| *** gyankum has joined #openstack-swift | 15:00 | |
| *** NM has quit IRC | 15:02 | |
| acoles | @notmyname and others: I'm wondering how much 'under the hood' detail we want in the sharding docs. I think descriptions of processes and interactions are good but I am inclined to not include much description of code e.g. classes, methods, but rather leave that to docstrings where it is more likely to be maintained. That implies deleting some of the current doc rather than updating it. Any opinions? | 15:05 |
| *** links has quit IRC | 15:08 | |
| *** cbartz has joined #openstack-swift | 15:13 | |
| *** Supun has quit IRC | 15:16 | |
| *** ukaynar has joined #openstack-swift | 15:22 | |
| *** cbartz has quit IRC | 15:28 | |
| *** Supun has joined #openstack-swift | 15:33 | |
| *** gkadam__ has joined #openstack-swift | 15:43 | |
| *** ukaynar has quit IRC | 15:44 | |
| notmyname | good morning | 15:44 |
| notmyname | acoles: I think that sounds like a good idea (trimming the docs to not have the code docs in there | 15:44 |
| *** ukaynar has joined #openstack-swift | 15:46 | |
| *** gkadam_ has quit IRC | 15:46 | |
| *** zaitcev has joined #openstack-swift | 15:49 | |
| *** ChanServ sets mode: +v zaitcev | 15:49 | |
| *** ccamacho has quit IRC | 15:49 | |
| *** gyan_ has joined #openstack-swift | 15:51 | |
| notmyname | acoles: are you pushing up any more patch set revisions tonight? | 15:54 |
| *** gyankum has quit IRC | 15:54 | |
| acoles | notmyname: yes, within next hour | 15:54 |
| notmyname | ok, thanks | 15:54 |
| *** openstackgerrit has joined #openstack-swift | 15:57 | |
| openstackgerrit | Gage Hugo proposed openstack/python-swiftclient master: WIP - Migrate to using keystoneauth session https://review.openstack.org/560138 | 15:57 |
| *** gyee has joined #openstack-swift | 16:04 | |
| *** Supun has quit IRC | 16:06 | |
| *** Supun has joined #openstack-swift | 16:07 | |
| *** SkyRocknRoll has joined #openstack-swift | 16:25 | |
| *** ccamacho has joined #openstack-swift | 16:28 | |
| *** hseipp has quit IRC | 16:28 | |
| zaitcev | I was just shown a cluster where one container has sqlite db 23GB long. You've read that right: 23 Gigabytes. Apparently there's no constraint on the number of objects in a container. Should there be? Sharding is on its way, I know, but this is just unreasonable, don't you think? | 16:30 |
| *** SkyRocknRoll has quit IRC | 16:33 | |
| *** Supun has quit IRC | 16:35 | |
| *** gkadam__ has quit IRC | 16:42 | |
| notmyname | eventlet bug filed for the utf8 stuff that came up yesterday https://github.com/eventlet/eventlet/issues/488 | 16:43 |
| *** cshastri has joined #openstack-swift | 16:45 | |
| *** SkyRocknRoll has joined #openstack-swift | 16:45 | |
| *** silor has joined #openstack-swift | 16:46 | |
| *** NM has joined #openstack-swift | 16:48 | |
| *** mingyu has quit IRC | 16:49 | |
| *** mingyu has joined #openstack-swift | 16:49 | |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep-review: Refactoring, test infrastructure changes and cleanup https://review.openstack.org/565742 | 16:51 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep-review: Add ShardRange class https://review.openstack.org/565743 | 16:51 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep-review: Add support for sharding in ContainerBroker https://review.openstack.org/565744 | 16:51 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep-review: Add shard range support to container server https://review.openstack.org/565745 | 16:51 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep-review: Enable proxy to build listings from shards https://review.openstack.org/565746 | 16:51 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep-review: Redirect object updates to shard containers https://review.openstack.org/565747 | 16:51 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep-review: Add sharder daemon, manage_shard_ranges tool and probe tests https://review.openstack.org/565748 | 16:51 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep-review: Add container sharding documentation https://review.openstack.org/565749 | 16:51 |
| notmyname | zaitcev: quotas are the only way to control it. otherwise, encourage users to spread objects across containers. and get feature/deep-review landed ;-) | 16:52 |
| *** gyan_ has quit IRC | 16:52 | |
| *** NM has quit IRC | 16:53 | |
| *** NM has joined #openstack-swift | 16:55 | |
| *** ccamacho has quit IRC | 16:58 | |
| *** ccamacho has joined #openstack-swift | 16:58 | |
| *** geaaru has joined #openstack-swift | 17:08 | |
| *** gyee has quit IRC | 17:17 | |
| *** hseipp has joined #openstack-swift | 17:18 | |
| *** hseipp has quit IRC | 17:20 | |
| *** cshastri has quit IRC | 17:32 | |
| *** geaaru has quit IRC | 17:36 | |
| *** geaaru has joined #openstack-swift | 17:39 | |
| *** jlvillal is now known as jlvacation | 17:51 | |
| *** hseipp has joined #openstack-swift | 18:06 | |
| DHE | https://docs.openstack.org/swift/latest/deployment_guide.html says "Currently it is recommended to use 3 [replicas] (as this is the only value that has been tested)". Surely that can't be true, right? | 18:13 |
| notmyname | $ git show 9916adb | grep Date | 18:14 |
| notmyname | Date: Fri Jul 23 17:15:29 2010 -0500 | 18:14 |
| notmyname | yeah. that's a *bit* dated | 18:15 |
| DHE | good because I'm considering something like 4 to 8 (replicas, not related to EC) | 18:17 |
| *** gyee has joined #openstack-swift | 18:19 | |
| *** NM has quit IRC | 18:24 | |
| notmyname | as with nearly everything in swift, the answer is "it depends" | 18:26 |
| notmyname | replicas are great for smaller objects, objects that require lower time-to-first-byte latency, objects frequently accessed, and objects distributed across a wide geographic area | 18:27 |
| notmyname | EC is better for bigger objects that aren't accessed as frequently | 18:28 |
| notmyname | (note that "frequent access" in my mind is measured in req/sec, not "this super popular object is accessed a couple times an hour") | 18:28 |
| *** SkyRocknRoll has quit IRC | 18:29 | |
| notmyname | for objects that are bigger than about 1MB, EC can give you better efficiency in storage space. with bigger objects, it can give you better speed, if you're otherwise network constrained | 18:29 |
| notmyname | 4 replicas may be good for smaller objects stored in a couple of regions | 18:30 |
| notmyname | I'd seriously investigate some edge caching system before putting 8 replicas across 8 regions in a single cluster | 18:31 |
| notmyname | but again. it all depends | 18:31 |
| *** psachin` has joined #openstack-swift | 18:33 | |
| *** guimaluf has joined #openstack-swift | 18:34 | |
| *** ukaynar has quit IRC | 18:35 | |
| *** ukaynar has joined #openstack-swift | 18:35 | |
| DHE | The use case I'm looking at is multi-region and the most extensive abuse of composite rings I can possibly perform | 18:40 |
| DHE | so 1-2 replicas per-region | 18:40 |
| *** ukaynar has quit IRC | 18:40 | |
| *** ukaynar has joined #openstack-swift | 18:45 | |
| zaitcev | Guys, have swift-object-info ever worked on container objects? I seem to recall that it did, but now it says "Invalid metadata". | 18:47 |
| *** openstackgerrit has quit IRC | 18:49 | |
| acoles | zaitcev: swift-container-info ? | 18:57 |
| *** NM has joined #openstack-swift | 19:20 | |
| *** silor has quit IRC | 19:23 | |
| *** hseipp has quit IRC | 19:28 | |
| *** psachin` has quit IRC | 19:30 | |
| tdasilva | remix_tj: i'm still maintaining that repo and would definetely welcome any PRs you may have... | 20:05 |
| notmyname | tdasilva: what would you think of including more of that repo in swift upstream? | 20:16 |
| notmyname | (in general, just the idea. not necessarily "import it all in right now") | 20:17 |
| tdasilva | notmyname: absolutely, i'm all for it, althought not sure what parts you have in mind... | 20:18 |
| tdasilva | btw..the probe tests ansible scripts was largely a copy of it.... | 20:19 |
| *** pcaruana has quit IRC | 20:21 | |
| notmyname | after sharding lands, I've got some ansible stuff I'd like to propose upstream. mostly a reorg of the probe tests one, but slightly different. related to the swift-in-a-container stuff I was doing | 20:21 |
| tdasilva | awesome! looking forward to it.... | 20:23 |
| tdasilva | notmyname: it would be cool to do more with the ansible scripts...thinking of multi-node upgrade tests | 20:24 |
| notmyname | yep! | 20:28 |
| * tdasilva goes back on vacation mode, see you all on monday | 20:34 | |
| *** ianychoi has quit IRC | 21:15 | |
| *** NM has quit IRC | 21:27 | |
| mattoliverau | acoles: +1 re doc. All the extra under the hood was to help fellow swift devs understand the POCs.. we probably don't have to be as detailed, cause then we'd have to maintain it | 21:42 |
| notmyname | I just got through all the docs above the "beyond here be dragons" part. looks god | 21:43 |
| notmyname | *good | 21:43 |
| *** ukaynar has quit IRC | 21:50 | |
| *** rcernin has joined #openstack-swift | 22:25 | |
| *** kei_yama has joined #openstack-swift | 23:09 | |
| *** bkopilov_ has quit IRC | 23:31 | |
| *** bkopilov__ has quit IRC | 23:32 | |
| *** bkopilov__ has joined #openstack-swift | 23:44 | |
| *** bkopilov_ has joined #openstack-swift | 23:44 | |
| *** bkopilov__ has quit IRC | 23:52 | |
| *** bkopilov_ has quit IRC | 23:52 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!