| *** zul has quit IRC | 00:01 | |
| *** mingyu has joined #openstack-swift | 00:14 | |
| *** diogogmt has joined #openstack-swift | 00:35 | |
| *** tqtran has quit IRC | 00:39 | |
| *** cdelatte has quit IRC | 00:51 | |
| *** m_kazuhiro has joined #openstack-swift | 00:52 | |
| * kota_ is backt to online at office | 00:53 | |
| *** mingyu has quit IRC | 00:56 | |
| *** vint_bra has joined #openstack-swift | 01:00 | |
| *** chlong has quit IRC | 01:03 | |
| notmyname | tshirt shop... http://d.not.mn/babylon_burning.jpg | 01:03 |
|---|---|---|
| *** klrmn has quit IRC | 01:07 | |
| *** neonpastor has joined #openstack-swift | 01:09 | |
| *** vint_bra has quit IRC | 01:11 | |
| *** ukaynar has joined #openstack-swift | 01:23 | |
| *** wuhg has joined #openstack-swift | 01:31 | |
| *** tqtran has joined #openstack-swift | 01:36 | |
| *** arch-nemesis has joined #openstack-swift | 01:38 | |
| *** tqtran has quit IRC | 01:40 | |
| *** openstack has joined #openstack-swift | 01:44 | |
| *** asettle has joined #openstack-swift | 01:47 | |
| *** arnoldchen has quit IRC | 01:48 | |
| kota_ | notmyname: amazing, that doesn't look a shop but a factory | 01:49 |
| *** asettle has quit IRC | 01:51 | |
| *** vint_bra has joined #openstack-swift | 01:52 | |
| *** vint_bra has quit IRC | 01:53 | |
| jrichli | notmyname: I can tell they look great! :-) | 01:54 |
| *** _JZ_ has quit IRC | 02:08 | |
| *** bkopilov has quit IRC | 02:10 | |
| *** bkopilov_ has quit IRC | 02:10 | |
| mattoliverau | notmyname: wow, very industrial. | 02:15 |
| *** chlong has joined #openstack-swift | 02:19 | |
| openstackgerrit | zheng yin proposed openstack/swift: Add test cases in the common/test_db https://review.openstack.org/366689 | 02:36 |
| *** dmorita has quit IRC | 02:39 | |
| *** mingyu has joined #openstack-swift | 02:43 | |
| *** _JZ_ has joined #openstack-swift | 03:21 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/liberasurecode: Add get_by_desc ret val handling to get_fragment_size https://review.openstack.org/341016 | 03:23 |
| *** bkopilov_ has joined #openstack-swift | 03:28 | |
| *** bkopilov has joined #openstack-swift | 03:28 | |
| *** mingyu has quit IRC | 03:32 | |
| *** mingyu has joined #openstack-swift | 03:32 | |
| openstackgerrit | Merged openstack/liberasurecode: Add get_by_desc ret val handling to get_fragment_size https://review.openstack.org/341016 | 03:37 |
| *** tongli has quit IRC | 03:38 | |
| *** dmorita has joined #openstack-swift | 03:39 | |
| *** dmorita has quit IRC | 03:43 | |
| *** arch-nemesis has quit IRC | 04:01 | |
| *** suresh has joined #openstack-swift | 04:16 | |
| suresh | hii all, what are the alternative's of cloudberry in linux to access openstack swift | 04:16 |
| suresh | please someone help? | 04:16 |
| notmyname | backing up to swift? netbackup, commvault, rclone, zmanda, duplicity. probably others if you look. might be easier to find a backup system you like and then it probably supports using swift | 04:25 |
| suresh | notmyname: thanks | 04:27 |
| *** links has joined #openstack-swift | 04:28 | |
| suresh | notmyname: if i want to just access swift then which software (free version) is better? | 04:28 |
| notmyname | depends on what you want to do. the primary way to interact with swift is via the http API. but there's been lots of different client written. some with a GUI. some CLI. just depends on what you want to do | 04:30 |
| suresh | I want a client which has GUI | 04:32 |
| suresh | notmyname: yeah i want a client which has GUI to access swift | 04:35 |
| *** psachin has joined #openstack-swift | 04:36 | |
| openstackgerrit | Merged openstack/pyeclib: Correct fragment size handling https://review.openstack.org/341054 | 04:37 |
| *** can8dnSix has joined #openstack-swift | 04:40 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/pyeclib: Ref count for dict item should be Py_DECREF https://review.openstack.org/344066 | 04:41 |
| openstackgerrit | Merged openstack/pyeclib: Ref count for dict item should be Py_DECREF https://review.openstack.org/344066 | 04:47 |
| kota_ | ok, the fix on memory leak on EC policy landed successfully. | 04:53 |
| kota_ | s/on/for/ | 04:53 |
| kota_ | of/ | 04:53 |
| kota_ | ? | 04:53 |
| kota_ | my english is always bad :/ | 04:57 |
| *** chsc has joined #openstack-swift | 04:57 | |
| notmyname | kota_: "the fix for the memory leak..." is right | 04:58 |
| kota_ | notmyname: thanks my teacher! | 04:58 |
| *** takashi has joined #openstack-swift | 04:58 | |
| suresh | notmyname: i want a client to access swift which has GUI | 04:59 |
| suresh | not REST client and POSTMAN | 04:59 |
| notmyname | suresh: what have you found so far with your own searching? | 04:59 |
| *** dmorita has joined #openstack-swift | 04:59 | |
| *** mingyu has quit IRC | 04:59 | |
| *** Jeffrey4l__ has joined #openstack-swift | 05:00 | |
| *** Jeffrey4l_ has quit IRC | 05:03 | |
| *** dmorita has quit IRC | 05:04 | |
| suresh | cloudberry, cyberduck are avialable for windows | 05:04 |
| suresh | notmyname: i want alternative softwares for linux? | 05:04 |
| notmyname | not sure. there are some browser dashboards for swift. you might have luck searching for something like that | 05:06 |
| notmyname | http://www.619.io/swift-explorer might now. I can't vouch for it--i've never used it | 05:07 |
| notmyname | *might work | 05:07 |
| suresh | notmyname: thanks | 05:09 |
| *** can8dnSix has quit IRC | 05:10 | |
| *** chlong has quit IRC | 05:14 | |
| openstackgerrit | zheng yin proposed openstack/swift: use value to replace dict[key] https://review.openstack.org/367080 | 05:19 |
| patchbot | Error: You don't have the #openstack-swift,op capability. If you think that you should have this capability, be sure that you are identified before trying again. The 'whoami' command can tell you if you're identified. | 05:19 |
| *** ChubYann has quit IRC | 05:23 | |
| *** chsc has quit IRC | 05:26 | |
| *** chlong has joined #openstack-swift | 05:30 | |
| *** links has quit IRC | 05:31 | |
| *** cnf has quit IRC | 05:32 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:34 | |
| *** tqtran has joined #openstack-swift | 05:38 | |
| *** McMurlock1 has joined #openstack-swift | 05:40 | |
| *** tqtran has quit IRC | 05:42 | |
| *** ppai has joined #openstack-swift | 05:45 | |
| ahale_ | clayg: nope - not on that , idk if its a swift or a xfs thing, i dont think swift tbh | 05:46 |
| *** links has joined #openstack-swift | 05:46 | |
| ahale_ | oh , just seen email ... | 05:46 |
| *** ukaynar has quit IRC | 05:51 | |
| *** furlongm_ has joined #openstack-swift | 05:59 | |
| *** furlongm has quit IRC | 06:00 | |
| *** hseipp has joined #openstack-swift | 06:12 | |
| *** chlong has quit IRC | 06:13 | |
| *** trananhkma has joined #openstack-swift | 06:13 | |
| openstackgerrit | zheng yin proposed openstack/swift: use value to replace dict[key] https://review.openstack.org/367080 | 06:15 |
| patchbot | Error: You don't have the #openstack-swift,op capability. If you think that you should have this capability, be sure that you are identified before trying again. The 'whoami' command can tell you if you're identified. | 06:15 |
| *** furlongm_ has quit IRC | 06:15 | |
| *** hseipp has quit IRC | 06:23 | |
| *** furlongm has joined #openstack-swift | 06:23 | |
| *** hoonetorg has quit IRC | 06:25 | |
| *** chlong has joined #openstack-swift | 06:25 | |
| *** ttrumm_ has joined #openstack-swift | 06:27 | |
| clayg | kota_: yay tsg helped! so glad you got that fix merged! | 06:29 |
| kota_ | clayg: thank you! one of my priorities finished! | 06:29 |
| clayg | :) | 06:30 |
| clayg | two more to go! | 06:30 |
| kota_ | clayg: you know! I'm reviewing optimistic-GET right now ;-) | 06:30 |
| *** ppai has quit IRC | 06:33 | |
| *** mingyu has joined #openstack-swift | 06:39 | |
| *** ppai has joined #openstack-swift | 06:45 | |
| *** Jeffrey4l__ has quit IRC | 07:00 | |
| *** dmorita has joined #openstack-swift | 07:01 | |
| *** ttrumm_ has quit IRC | 07:01 | |
| *** geaaru has joined #openstack-swift | 07:01 | |
| *** Jeffrey4l__ has joined #openstack-swift | 07:02 | |
| *** tesseract- has joined #openstack-swift | 07:04 | |
| *** dmorita has quit IRC | 07:05 | |
| *** ppai has quit IRC | 07:08 | |
| *** ttrumm has joined #openstack-swift | 07:09 | |
| *** ttrumm_ has joined #openstack-swift | 07:10 | |
| clayg | gah, seriously - someone give me some new way to think about this clean imports series? | 07:11 |
| clayg | how did so many of these patches get merged w/o *anyone* asking - gee - if we don't enforce this and it doesn't seem to matter - should it be removed from the guideline instead? | 07:12 |
| clayg | why can't the guideline just be wrong? why am happy when people come in with typo fixes in docs - but annoyed when they come in with non-material changes to code? | 07:13 |
| *** ttrumm has quit IRC | 07:13 | |
| clayg | like you wanna change code - *awesome* - we have a huge backlog - you want fix typos - *awesome* - we have terrible linters for natural language - you wanna flake8 something *harder* MY-RAAAAGE | 07:13 |
| clayg | ... it doesn't make any sense? | 07:13 |
| *** abhitechie has joined #openstack-swift | 07:15 | |
| *** rledisez has joined #openstack-swift | 07:17 | |
| *** abhinavtechie has joined #openstack-swift | 07:18 | |
| *** abhitechie has quit IRC | 07:19 | |
| *** ppai has joined #openstack-swift | 07:20 | |
| *** maestropandy has joined #openstack-swift | 07:25 | |
| *** maestropandy has left #openstack-swift | 07:25 | |
| *** ttrumm has joined #openstack-swift | 07:32 | |
| *** openstackgerrit has quit IRC | 07:33 | |
| *** openstackgerrit has joined #openstack-swift | 07:34 | |
| *** sams-gleb has joined #openstack-swift | 07:35 | |
| *** ttrumm_ has quit IRC | 07:35 | |
| *** ttrumm has quit IRC | 07:37 | |
| *** tqtran has joined #openstack-swift | 07:39 | |
| *** cnf has joined #openstack-swift | 07:39 | |
| *** cnf has joined #openstack-swift | 07:39 | |
| *** tqtran has quit IRC | 07:43 | |
| *** abhinavtechie has quit IRC | 07:50 | |
| *** abhitechie has joined #openstack-swift | 07:50 | |
| *** ttrumm has joined #openstack-swift | 07:51 | |
| *** chlong has quit IRC | 07:52 | |
| *** SkyRocknRoll has quit IRC | 07:53 | |
| *** ttrumm has quit IRC | 07:56 | |
| *** takashi has quit IRC | 07:58 | |
| *** mingyu has quit IRC | 07:59 | |
| *** maestropandy has joined #openstack-swift | 08:00 | |
| *** ttrumm has joined #openstack-swift | 08:02 | |
| *** ttrumm_ has joined #openstack-swift | 08:03 | |
| *** ttrumm_ has quit IRC | 08:05 | |
| *** ttrumm has quit IRC | 08:05 | |
| *** SkyRocknRoll has joined #openstack-swift | 08:10 | |
| *** maestropandy has left #openstack-swift | 08:15 | |
| *** admin6 has joined #openstack-swift | 08:15 | |
| *** Jeffrey4l__ has quit IRC | 08:20 | |
| *** ttrumm has joined #openstack-swift | 08:20 | |
| *** cnf has quit IRC | 08:27 | |
| *** cnf has joined #openstack-swift | 08:28 | |
| *** jordanP has joined #openstack-swift | 08:32 | |
| *** asettle has joined #openstack-swift | 08:40 | |
| *** joeljwright has joined #openstack-swift | 08:43 | |
| *** ChanServ sets mode: +v joeljwright | 08:43 | |
| *** kei_yama has quit IRC | 08:52 | |
| clayg | jesus for the love of god I still can't get my GOPATH's happy - i mean it works - but go tool wants to take every chance it gets to do something new and weird that is not what I wanted | 08:59 |
| clayg | i've finally come to terms that despite the file-ish looking syntax the arguments to go get/test/build etc are *not* paths | 09:00 |
| *** dmorita has joined #openstack-swift | 09:01 | |
| *** maestropandy has joined #openstack-swift | 09:03 | |
| *** dmorita has quit IRC | 09:06 | |
| *** Jeffrey4l__ has joined #openstack-swift | 09:17 | |
| ahale_ | well i found an object that was in a .data that was a directory, had some partial, validish looking metadata and the two parts involved (where it is and where metadata says it should be) were on a common machine at one time, and that the machine it ended up on had the wrong partition at one point. i'll comment on that bug | 09:22 |
| *** acoles_ is now known as acoles | 09:23 | |
| *** asettle has quit IRC | 09:23 | |
| *** asettle has joined #openstack-swift | 09:23 | |
| *** asettle has quit IRC | 09:23 | |
| *** asettle has joined #openstack-swift | 09:41 | |
| *** mvk has quit IRC | 09:43 | |
| *** sams-gleb has quit IRC | 09:45 | |
| *** sams-gleb has joined #openstack-swift | 09:46 | |
| *** abhinavtechie has joined #openstack-swift | 09:50 | |
| *** abhitechie has quit IRC | 09:53 | |
| openstackgerrit | Donagh McCabe proposed openstack/swift: Corrections for the API specifications in api-ref https://review.openstack.org/354767 | 10:04 |
| openstackgerrit | Alistair Coles proposed openstack/python-swiftclient: Make object option dependent on container in usage strings https://review.openstack.org/367261 | 10:12 |
| *** mvk has joined #openstack-swift | 10:13 | |
| openstackgerrit | Merged openstack/swift: simple log information https://review.openstack.org/348724 | 10:25 |
| *** joeljwright has quit IRC | 10:34 | |
| kota_ | done for looking at all codes on the patch 215276 | 10:37 |
| patchbot | https://review.openstack.org/#/c/215276/ - swift - Enable object server to return non-durable data | 10:37 |
| kota_ | that was so hard but it's also joy to find great codes inside. | 10:37 |
| kota_ | I hope my comments could be helpful to improve for getting merged. | 10:38 |
| *** tqtran has joined #openstack-swift | 10:42 | |
| *** tqtran has quit IRC | 10:46 | |
| *** joeljwright has joined #openstack-swift | 10:48 | |
| *** ChanServ sets mode: +v joeljwright | 10:48 | |
| *** bkopilov has quit IRC | 11:01 | |
| *** bkopilov_ has quit IRC | 11:02 | |
| *** joeljwright has quit IRC | 11:10 | |
| *** maestropandy has quit IRC | 11:19 | |
| acoles | kota_: thanks for your work on that ^^ I will try to review/update soon | 11:19 |
| openstackgerrit | Merged openstack/python-swiftclient: Fixed inconsistent positional arguments naming https://review.openstack.org/366227 | 11:29 |
| *** links has quit IRC | 11:52 | |
| *** McMurlock1 has quit IRC | 11:56 | |
| *** McMurlock1 has joined #openstack-swift | 11:56 | |
| *** abhinavtechie has quit IRC | 12:00 | |
| *** psachin has quit IRC | 12:01 | |
| *** psachin has joined #openstack-swift | 12:01 | |
| *** dmorita has joined #openstack-swift | 12:10 | |
| *** dmorita has quit IRC | 12:15 | |
| *** joeljwright has joined #openstack-swift | 12:15 | |
| *** ChanServ sets mode: +v joeljwright | 12:15 | |
| *** wuhg has quit IRC | 12:21 | |
| *** trananhkma has quit IRC | 12:23 | |
| openstackgerrit | Mahati Chamarthy proposed openstack/swift: Delete old tombstones https://review.openstack.org/346865 | 12:26 |
| *** bkopilov_ has joined #openstack-swift | 12:27 | |
| *** bkopilov has joined #openstack-swift | 12:29 | |
| *** nikivi has joined #openstack-swift | 12:31 | |
| *** ppai has quit IRC | 12:41 | |
| *** cdelatte has joined #openstack-swift | 12:42 | |
| *** maestropandy has joined #openstack-swift | 12:46 | |
| *** sbelous__ has joined #openstack-swift | 12:50 | |
| *** sbelous__ has left #openstack-swift | 12:50 | |
| *** klamath has joined #openstack-swift | 12:51 | |
| *** m_kazuhiro has quit IRC | 12:54 | |
| *** maestropandy has quit IRC | 12:54 | |
| *** vint_bra has joined #openstack-swift | 12:55 | |
| *** geaaru has quit IRC | 12:59 | |
| *** vint_bra has quit IRC | 13:00 | |
| *** maestropandy has joined #openstack-swift | 13:00 | |
| *** mingyu has joined #openstack-swift | 13:06 | |
| *** mingyu has quit IRC | 13:09 | |
| *** StraubTW has joined #openstack-swift | 13:10 | |
| *** jraju has joined #openstack-swift | 13:12 | |
| *** mingyu has joined #openstack-swift | 13:13 | |
| openstackgerrit | Alistair Coles proposed openstack/python-swiftclient: Make tempurl command check for valid object path https://review.openstack.org/367377 | 13:26 |
| *** mingyu has quit IRC | 13:32 | |
| *** joeljwright has quit IRC | 13:37 | |
| *** mingyu has joined #openstack-swift | 13:38 | |
| *** ttrumm has quit IRC | 13:38 | |
| *** ttrumm has joined #openstack-swift | 13:38 | |
| *** nikivi has quit IRC | 13:42 | |
| *** ttrumm has quit IRC | 13:43 | |
| *** silor has joined #openstack-swift | 13:44 | |
| *** psachin has quit IRC | 13:49 | |
| *** Administrator_ has joined #openstack-swift | 13:51 | |
| *** zhugaoxiao has quit IRC | 13:54 | |
| tdasilva | good morning | 13:55 |
| *** diogogmt_ has joined #openstack-swift | 13:58 | |
| *** mingyu has quit IRC | 13:59 | |
| *** diogogmt has quit IRC | 14:00 | |
| *** diogogmt_ is now known as diogogmt | 14:00 | |
| *** vint_bra has joined #openstack-swift | 14:02 | |
| *** mingyu has joined #openstack-swift | 14:06 | |
| *** tongli has joined #openstack-swift | 14:09 | |
| *** baojg has joined #openstack-swift | 14:12 | |
| *** mingyu has quit IRC | 14:13 | |
| *** joeljwright has joined #openstack-swift | 14:17 | |
| *** ChanServ sets mode: +v joeljwright | 14:17 | |
| *** zul has joined #openstack-swift | 14:20 | |
| *** zul has quit IRC | 14:22 | |
| *** zul has joined #openstack-swift | 14:23 | |
| *** admin6_ has joined #openstack-swift | 14:32 | |
| *** _JZ_ has quit IRC | 14:32 | |
| openstackgerrit | Merged openstack/python-swiftclient: Make options arguments truely optional https://review.openstack.org/357063 | 14:33 |
| *** admin6 has quit IRC | 14:35 | |
| *** admin6_ is now known as admin6 | 14:35 | |
| *** scarab_ has joined #openstack-swift | 14:38 | |
| *** mvk has quit IRC | 14:42 | |
| *** takashi has joined #openstack-swift | 14:56 | |
| *** jraju has quit IRC | 14:59 | |
| acoles | jrichli: nice work fixing post-as-copy of a dlo manifest | 14:59 |
| *** chsc has joined #openstack-swift | 15:01 | |
| *** chsc has joined #openstack-swift | 15:01 | |
| jrichli | acoles: Thanks for the +2! That one has been a long time coming for what ended up being pretty simple. | 15:02 |
| openstackgerrit | Merged openstack/swift: Allow DLO manifest to be updated when using post-as-copy https://review.openstack.org/356239 | 15:09 |
| *** scarab_ has quit IRC | 15:10 | |
| *** _JZ_ has joined #openstack-swift | 15:14 | |
| *** joeljwright has quit IRC | 15:15 | |
| *** joeljwright has joined #openstack-swift | 15:19 | |
| *** ChanServ sets mode: +v joeljwright | 15:19 | |
| *** StraubTW has quit IRC | 15:27 | |
| *** StraubTW has joined #openstack-swift | 15:29 | |
| notmyname | good morning | 15:31 |
| *** joeljwright has quit IRC | 15:35 | |
| *** zul has quit IRC | 15:39 | |
| *** zul has joined #openstack-swift | 15:40 | |
| *** zul has quit IRC | 15:40 | |
| *** zul has joined #openstack-swift | 15:41 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: api-ref: Move repeated paragraph to an include file https://review.openstack.org/367471 | 15:43 |
| *** sams-gleb has quit IRC | 15:49 | |
| *** maestropandy has quit IRC | 15:55 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: api-ref: clean up account listing examples https://review.openstack.org/366211 | 15:55 |
| notmyname | there's another interesting thread starting on the ML, related to the "openstack principles" patch I referenced a couple of days ago. | 16:04 |
| notmyname | based on conversations I've had with many of you, I suspect the initial email will resonate http://lists.openstack.org/pipermail/openstack-dev/2016-September/103223.html | 16:05 |
| *** McMurlock1 has quit IRC | 16:07 | |
| *** SkyRocknRoll has quit IRC | 16:11 | |
| oxinabox | Does the swiftclient.service class mess around with the encoding of data? | 16:12 |
| oxinabox | Like change to Latin1 or perhaps do HTTP escaping? | 16:12 |
| *** joeljwright has joined #openstack-swift | 16:13 | |
| *** ChanServ sets mode: +v joeljwright | 16:13 | |
| *** nadeem has joined #openstack-swift | 16:13 | |
| *** nadeem has quit IRC | 16:13 | |
| notmyname | oxinabox: the object data itself? no. it may do some encoding on metadata to ensure it fits in the http headers | 16:14 |
| oxinabox | Thanls | 16:15 |
| oxinabox | ^thanks | 16:15 |
| oxinabox | Unrelated: I would like to put a text corpus in to Object storage (Eg the British National Corpus. Or the Brown Corpus, or heck< every single wikipedia article). These data tend to come in many many smallish files. | 16:20 |
| *** chsc has quit IRC | 16:20 | |
| oxinabox | What is the sensible way to store them? | 16:20 |
| oxinabox | My goal is to not have to put them on to my local disk. | 16:20 |
| oxinabox | (I have giant piles of RAM) | 16:21 |
| *** tesseract- has quit IRC | 16:21 | |
| notmyname | how to store it in swift is more related to how you'll be using it | 16:24 |
| oxinabox | So generally I'll want to read the files in one or two or maybe a few hundred at a time. Maybe all of them, if they will fit in to RAM. | 16:25 |
| notmyname | eg you could store each work/article as a separate object. or you could have a single giant file that you split(1) into smaller bits that are stored. or you could organize/group it any number of other ways | 16:25 |
| notmyname | in general, swift is very very good at dealing with lots of concurrent streams over a single stream. so you'll normally be able to download 1000 1MB objects faster than 1 1Gb object | 16:26 |
| oxinabox | Ah, ok, that is almost the opposite of how I thought it would be | 16:26 |
| oxinabox | My other user case is storing giant 5-12GB HDF5 files. | 16:27 |
| *** dmorita has joined #openstack-swift | 16:27 | |
| notmyname | a single replica of a single object is stored as a file on a single hard drive. so if you have 3x replias, then you've got--at most--3 hard drives that can serve the data. if you've got 1000 smaller objects, each of those can come from one of three drives, and you'll use more drives in your cluster. | 16:28 |
| oxinabox | So lowering my Segment size is a good thing | 16:29 |
| notmyname | the default for single object max size is 5GB. it's generally a very bad idea to change this, but you can use large objects (http://docs.openstack.org/developer/swift/overview_large_objects.html) to have arbitrarily sized logical objects in the cluster | 16:29 |
| oxinabox | Yeah, I've been using large objects. | 16:29 |
| oxinabox | with a segement size of 1GB. | 16:29 |
| oxinabox | Which gets me ~50MB upload speed, but only 20MB download speed. | 16:29 |
| timburke | it's worth noting, though, that there's some lower-bound. having a few PB worth of <1kB objects isn't great either | 16:30 |
| notmyname | yes. use a smaller segment size. but there's a balance. with the example of 1000 objects, if you only have 10 drives in your cluster, each drive will be doing a *lot* of random IO as it tries to serve a lot of requests | 16:31 |
| *** silor has quit IRC | 16:31 | |
| oxinabox | I have no idea how many drives are in the cluster. | 16:31 |
| oxinabox | The total OpenStack cluster has like 10K nodes. | 16:31 |
| notmyname | cool | 16:31 |
| *** ukaynar has joined #openstack-swift | 16:31 | |
| *** ametts has joined #openstack-swift | 16:32 | |
| notmyname | yeah. what timburke said, too. very small objects (less than 100kB? 512kB? 1MB?) can cause other, different problems | 16:32 |
| oxinabox | I think there is 1-2K nodes in my region. But potentially it might be 4K nodes. | 16:32 |
| notmyname | basically, if you've got a few PB of data and you'll be doing a lot of concurrent access to it, splitting it into 10MB or 100MB chunks would be a great starting point for testing the throughput curve | 16:32 |
| oxinabox | I've only got GB of data. | 16:33 |
| oxinabox | but still it applies, no? | 16:33 |
| *** miniman has quit IRC | 16:33 | |
| timburke | on the up/down speed with large objects: on download, we've only got the one connection and the proxy takes care of fetching the segments sequentially. if you're fetching multiple large objects at once, i'd expect you could reach parity, though | 16:34 |
| *** nadeem has joined #openstack-swift | 16:34 | |
| oxinabox | I don't generally need to fetch multiple large objects. at a time. | 16:34 |
| *** takashi has quit IRC | 16:35 | |
| oxinabox | Is the proxy configurable? | 16:35 |
| oxinabox | ooh wait, does that man that more segments = faster upload, but slower download? | 16:37 |
| oxinabox | s/man/mean | 16:37 |
| timburke | then we need to go add an option to do concurrent downloads :-) i know the idea's been tossed around before; i think https://github.com/FredHutch/swift-commander maybe does something like that? | 16:38 |
| timburke | download should be about the same, provided reasonable (>=1MB) segment sizes. the trouble is that no matter how many segments, you're only using the one connection | 16:40 |
| *** jordanP has quit IRC | 16:40 | |
| *** tongli_ has joined #openstack-swift | 16:41 | |
| *** baojg has quit IRC | 16:43 | |
| *** tongli has quit IRC | 16:44 | |
| timburke | on upload, more segments -> faster upload, but only up to a point. once you've got as many segments as you've got worker threads to upload them, you won't see much improvement unless you add more workers. and adding more workers will only help until you saturate your link (or your disk IO) | 16:44 |
| oxinabox | I guess I am really more interested in download that upload -- you upload once, but download many times. | 16:45 |
| *** tongli_ has quit IRC | 16:46 | |
| notmyname | the same principle applies for downloads | 16:47 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Make container sync copy SLO manifests https://review.openstack.org/348495 | 16:48 |
| notmyname | (but that's from the perspective of the API, not necessarily functionality that's been exposed via concurrent download threads in the CLI) | 16:50 |
| notmyname | oxinabox: are you writing your own client connections or are you using the CLI tool? | 16:50 |
| oxinabox | Right now, I am wrapping `swiftclient.service`, for Julia | 16:51 |
| oxinabox | I was wrapping `swiftclient.client` but that does not handle making segments for me. | 16:51 |
| notmyname | right. you're doing it right (with service) | 16:51 |
| oxinabox | Except that is kinda aweful, because service really intensly thinks about files on disk. | 16:52 |
| oxinabox | AFAICT. | 16:52 |
| oxinabox | so to write something from memory, I end up writing it to a temp file in `/dev/shm` | 16:52 |
| oxinabox | and then giving service that filename | 16:53 |
| timburke | SwiftService *can* work with python objects, but it's a bit more clunky. you'll want to pass in SwiftUploadObjects instead of paths, and use something like a BytesIO as the source | 16:57 |
| oxinabox | yeah, I was trying that originally, but I kept getting segfaults. (which I didn't get with swiftclient.client). I beleive that is probably related to issues passing file streams between Julia and Python. | 16:59 |
| oxinabox | though I cna't quiet work it out enough to file a bug report on it. | 16:59 |
| oxinabox | (against the Julia FFI to python library) | 16:59 |
| oxinabox | It maybe that it gets garbage collected and closed for some reason. anyway that is another game. | 17:00 |
| timburke | if RAM is no issue, what about just passing strings between them? Julia string -> Python string -> Python BytesIO? | 17:01 |
| oxinabox | I kinda put that afternoon of pain out of my mind. | 17:01 |
| timburke | fair enough :-) | 17:01 |
| oxinabox | Because SwiftService does not except a String unless it is a filename. so to make the conversion happen in python I would actually need to write some python code. | 17:02 |
| *** tongli has joined #openstack-swift | 17:04 | |
| *** tongli has quit IRC | 17:08 | |
| *** zul has quit IRC | 17:10 | |
| *** takashi has joined #openstack-swift | 17:10 | |
| *** zul has joined #openstack-swift | 17:14 | |
| *** joeljwright has quit IRC | 17:18 | |
| *** klrmn has joined #openstack-swift | 17:20 | |
| timburke | submitted https://bugs.launchpad.net/python-swiftclient/+bug/1621562 for concurrent downloads | 17:22 |
| openstack | Launchpad bug 1621562 in python-swiftclient "Download large objects concurrently" [Wishlist,Confirmed] | 17:22 |
| oxinabox | Thanks | 17:22 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-swiftclient: Updated from global requirements https://review.openstack.org/89250 | 17:24 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 17:25 |
| *** rledisez has quit IRC | 17:25 | |
| *** joeljwright has joined #openstack-swift | 17:25 | |
| *** ChanServ sets mode: +v joeljwright | 17:25 | |
| oxinabox | Here we go: my julia package for Swift https://github.com/oxinabox/SwiftObjectStores.jl | 17:29 |
| oxinabox | I really need to write it a readme :-P but it is like 2AM, so good night all | 17:30 |
| notmyname | oxinabox: good night | 17:30 |
| *** tongli has joined #openstack-swift | 17:30 | |
| *** tongli has quit IRC | 17:34 | |
| *** chsc has joined #openstack-swift | 17:35 | |
| *** chsc has joined #openstack-swift | 17:35 | |
| *** Suyi_ has joined #openstack-swift | 17:35 | |
| *** acoles is now known as acoles_ | 17:41 | |
| *** tqtran has joined #openstack-swift | 17:46 | |
| *** tongli has joined #openstack-swift | 17:48 | |
| *** tongli has quit IRC | 17:52 | |
| *** wasmum has quit IRC | 18:06 | |
| *** ChubYann has joined #openstack-swift | 18:14 | |
| *** dcourtois has quit IRC | 18:16 | |
| *** tongli has joined #openstack-swift | 18:25 | |
| *** joeljwright has quit IRC | 18:27 | |
| *** tongli has quit IRC | 18:28 | |
| *** tongli has joined #openstack-swift | 18:28 | |
| *** tongli has quit IRC | 18:30 | |
| *** tongli has joined #openstack-swift | 18:31 | |
| *** tongli has quit IRC | 18:32 | |
| *** tongli has joined #openstack-swift | 18:32 | |
| *** tongli has quit IRC | 18:33 | |
| *** tongli has joined #openstack-swift | 18:34 | |
| *** Jeffrey4l__ has quit IRC | 18:42 | |
| *** david-lyle has quit IRC | 18:53 | |
| *** david-lyle has joined #openstack-swift | 18:53 | |
| *** Jeffrey4l__ has joined #openstack-swift | 18:54 | |
| openstackgerrit | Thai Tran proposed openstack/swift: WIP Investigating Account ACL https://review.openstack.org/356715 | 19:06 |
| *** ukaynar has quit IRC | 19:19 | |
| *** McMurlock1 has joined #openstack-swift | 19:34 | |
| *** silor has joined #openstack-swift | 19:34 | |
| *** silor1 has joined #openstack-swift | 19:37 | |
| *** furlongm_ has joined #openstack-swift | 19:38 | |
| *** furlongm has quit IRC | 19:38 | |
| *** suresh has quit IRC | 19:38 | |
| *** silor has quit IRC | 19:39 | |
| *** silor1 is now known as silor | 19:39 | |
| *** tongli_ has joined #openstack-swift | 19:43 | |
| *** tongli has quit IRC | 19:46 | |
| *** Jeffrey4l__ has quit IRC | 19:50 | |
| *** takashi has quit IRC | 20:03 | |
| pdardeau | notmyname: acoles: nice job participating on patch 357260 | 20:03 |
| patchbot | https://review.openstack.org/#/c/357260/ - governance - Write down OpenStack principles | 20:03 |
| pdardeau | also clayg on same patch | 20:05 |
| notmyname | pdardeau: *please* comment on the patch or the new ML thread to share your thoughts. one of the underlying frustrations is that there's a lot of back-channel or secret conversations, and I'd like more of that to be brought out | 20:06 |
| pdardeau | notmyname: the big concern there would be expressing my true thoughts in a constructive manner. i'll try to think of something constructive. | 20:08 |
| *** cdelatte has quit IRC | 20:08 | |
| tdasilva | pdardeau: i know the feeling | 20:09 |
| clayg | sphthtth constructive - try *incendiary* - that's my motto | 20:10 |
| clayg | I can't say what I feel because I'm ineloquent and too passionate? Scoff. Maybe that's why we have a representative body - people that I can elect that share my sentiment - but are better at verbalizing and compromise - like notmyname and cdent :D | 20:13 |
| tdasilva | clayg incendiary??? but I find you so subtle :-) | 20:16 |
| pdardeau | ok, clayg, i'll give a shot at true feelings (probably incendiary on this). the whole of the text reads like a cult manifesto, but I don't say that to be mean or belittling - it just reads that way to me | 20:16 |
| notmyname | pdardeau: then it should be said. also, wow ;-) | 20:17 |
| *** asettle has quit IRC | 20:19 | |
| *** asettle has joined #openstack-swift | 20:21 | |
| *** asettle has quit IRC | 20:25 | |
| notmyname | I'd request that *anyone* who feels strongly about patch 357260 or the thread starting at http://lists.openstack.org/pipermail/openstack-dev/2016-September/103223.html should make a comment in gerrit or respond on the mailing list | 20:30 |
| patchbot | https://review.openstack.org/#/c/357260/ - governance - Write down OpenStack principles | 20:30 |
| notmyname | it's important that people pay attention and make sure their voices are heard | 20:30 |
| notmyname | and, yes, speaking out is scary (for me too) | 20:30 |
| clayg | it's not just *that* one - don't forget patch 349068 | 20:30 |
| patchbot | https://review.openstack.org/#/c/349068/ - governance - describe a process for managing community-wide goals (MERGED) | 20:31 |
| *** klamath has quit IRC | 20:37 | |
| *** _JZ_ has quit IRC | 20:46 | |
| *** tongli_ has quit IRC | 20:52 | |
| *** tongli has joined #openstack-swift | 20:53 | |
| *** tongli_ has joined #openstack-swift | 20:56 | |
| *** tongli has quit IRC | 20:57 | |
| *** mvk has joined #openstack-swift | 20:58 | |
| *** silor has quit IRC | 21:11 | |
| clayg | i *hate* this error message: go install: no install location for directory /vagrant/swift/go/bench outside GOPATH | 21:12 |
| clayg | what does that even mean? when you look it up no one even attempts to justify what that could possilby *mean* they just blinding throw out hacky workarounds about setting gopath or gobin or goroot until it works | 21:13 |
| clayg | but in my case it *does* work when I do it interactively and not when I try to script it - so I have to understand wtf it acctually *means* so that I can try to build up the difference in the runtime environment in which the command works and fails | 21:13 |
| clayg | even interactively if I run it from ~/swift/go - it fails but if I run it from ./go/src/github.com/openstack/swift/go (a symlink to ~/swift/go btw) it *does* work - but from the script - grrr - i bet it's something with how chef's cwd resource follows symlinks | 21:15 |
| clayg | OMG screw you chef and screw you GOPATH! Computers suck! | 21:17 |
| *** ametts has quit IRC | 21:19 | |
| openstackgerrit | Thai Tran proposed openstack/swift: Supporting JSON format for Account ACL in keystoneauth https://review.openstack.org/356715 | 21:24 |
| *** tongli_ has quit IRC | 21:26 | |
| *** ametts has joined #openstack-swift | 21:31 | |
| ntata | Hello all! Can someone please review this patch 353097 ? | 21:44 |
| patchbot | https://review.openstack.org/#/c/353097/ - swift - Add man page for swift.conf | 21:44 |
| notmyname | ntata: looking | 21:46 |
| ntata | Thank you! :) | 21:51 |
| openstackgerrit | John Dickinson proposed openstack/swift: Add man page for swift.conf https://review.openstack.org/353097 | 21:54 |
| notmyname | ntata: one small nit. I fixed it and pushed over | 21:55 |
| notmyname | and I'll land it | 21:55 |
| ntata | Thank you again! :) | 21:55 |
| *** ametts has quit IRC | 21:57 | |
| *** vint_bra has quit IRC | 22:12 | |
| *** tristanC has quit IRC | 22:12 | |
| *** tristanC has joined #openstack-swift | 22:13 | |
| *** nadeem has quit IRC | 22:18 | |
| mattoliverau | Morning | 22:20 |
| notmyname | hello mattoliverau | 22:20 |
| *** mvk has quit IRC | 22:23 | |
| *** mvk has joined #openstack-swift | 22:37 | |
| *** chsc has quit IRC | 22:40 | |
| *** McMurlock1 has quit IRC | 22:48 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: just testing out vim-go https://review.openstack.org/367685 | 22:57 |
| *** StraubTW has quit IRC | 22:59 | |
| clayg | interesting... http://104.239.166.47:8080/job/Swift%20Golang%20Checks/226/console | 23:03 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Apply go fmt to tempauth https://review.openstack.org/367685 | 23:04 |
| clayg | redbo: why don't nadeem and dfg hang out in channel anymore? http://104.239.166.47:8080/job/Swift%20Golang%20Checks/187/ is the first failure I can find - teh job was triggered by https://review.openstack.org/#/c/328508/ - but I don't think that's the change that broke it? | 23:19 |
| patchbot | patch 328508 - swift (feature/hummingbird) - Force /var/tmp as the test directory because of xattr | 23:19 |
| *** dmorita has quit IRC | 23:20 | |
| clayg | but that was *before* patch 363111 - i'm not really sure what's breaking down | 23:20 |
| patchbot | https://review.openstack.org/#/c/363111/ - swift (feature/hummingbird) - Merge remote-tracking branch 'origin/master' into ... (MERGED) | 23:20 |
| *** asettle has joined #openstack-swift | 23:22 | |
| *** kei_yama has joined #openstack-swift | 23:23 | |
| *** dmorita has joined #openstack-swift | 23:27 | |
| redbo | clayg: ew.. it was probably that merge from master, and I need to re-run setup.py on the docker container. | 23:28 |
| redbo | Oh, if it was happening before that, I don't know. | 23:28 |
| *** asettle has quit IRC | 23:30 | |
| openstackgerrit | Merged openstack/swift: Add man page for swift.conf https://review.openstack.org/353097 | 23:33 |
| redbo | I rebuilt the docker image and it seems to be working again. | 23:41 |
| clayg | redbo: k, let me know if there's anything I can do help | 23:42 |
| clayg | redbo: also tell dfg & nadeem to get in here | 23:46 |
| *** Suyi_ has quit IRC | 23:48 | |
| clayg | redbo: whoa | 23:51 |
| mattoliverau | nadeem was in earlier, but I don't think he bouncers. | 23:56 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
