| notmyname | https://etherpad.openstack.org/p/swift-ptg-pike has been reorganized into three big groups | 00:07 |
|---|---|---|
| notmyname | 1. stuff that just needs a few people in a corner to go work through and land | 00:07 |
| notmyname | 2. bigger stuff that needs more discussion and is currently being done or should be done soon | 00:08 |
| notmyname | 3. bigger stuff that is unlikely to be prioritized in the pike cycle | 00:08 |
| clayg | can we put everything in #3 and just chill next week? | 00:21 |
| notmyname | clayg: (tongue in cheek) that would take another tc resolution | 00:22 |
| kota_ | good morning | 00:24 |
| kota_ | at office | 00:24 |
| mattoliverau | kota_: morning | 00:24 |
| kota_ | mattoliverau: o/ | 00:24 |
| *** sams-gleb has joined #openstack-swift | 00:25 | |
| *** sams-gleb has quit IRC | 00:29 | |
| *** dmorita has joined #openstack-swift | 00:37 | |
| *** dmorita has quit IRC | 00:46 | |
| *** gatuus_ has quit IRC | 01:04 | |
| *** gatuus has quit IRC | 01:04 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift master: Global EC Under Development Documentation https://review.openstack.org/432513 | 01:10 |
| *** catintheroof has quit IRC | 01:13 | |
| *** jamielennox is now known as jamielennox|away | 01:18 | |
| *** jamielennox|away is now known as jamielennox | 01:25 | |
| clayg | has anyone ever used a "structural diff" tool for python before? | 01:31 |
| clayg | I'm trying to tease out any differences in test_server.py besides moving the tests to a base class in patch 219165 | 01:35 |
| patchbot | https://review.openstack.org/#/c/219165/ - swift - EC Fragment Duplication - Foundational Global EC C... | 01:35 |
| clayg | I'm trying https://github.com/yinwang0/psydiff | 01:35 |
| clayg | ... it's doing stuff - but its taking awhile. | 01:35 |
| *** dmorita has joined #openstack-swift | 01:37 | |
| *** dmorita has quit IRC | 01:41 | |
| clayg | i feel like this is probably one of those things pycharm would do for me really cheaply somehow? That or it would work most of time - and when it doesn't there's no hope of ever figuring out why | 01:45 |
| clayg | Total duration: 21.4 minutes !!! | 01:46 |
| clayg | better be freaking great! | 01:46 |
| clayg | nope, didn't really work at all | 01:51 |
| *** mmotiani_ has joined #openstack-swift | 01:51 | |
| *** winggundamth has joined #openstack-swift | 02:04 | |
| *** tonanhngo has quit IRC | 02:06 | |
| *** tonanhngo has joined #openstack-swift | 02:12 | |
| *** tonanhngo has quit IRC | 02:17 | |
| *** winggundamth has quit IRC | 02:23 | |
| *** tqtran has quit IRC | 02:24 | |
| *** winggundamth has joined #openstack-swift | 02:25 | |
| *** sams-gleb has joined #openstack-swift | 02:27 | |
| *** sams-gleb has quit IRC | 02:31 | |
| mahatic | good morning | 03:35 |
| *** deepbook5broo has joined #openstack-swift | 03:36 | |
| *** deepbook5broo has left #openstack-swift | 03:36 | |
| mahatic | clayg: ah, you pushed up a test already for https://bugs.launchpad.net/swift/+bug/1665141 - I was looking at it yesterday | 03:36 |
| openstack | Launchpad bug 1665141 in OpenStack Object Storage (swift) "Reconstructor should not hash suffixes after failure" [High,New] | 03:36 |
| *** zhugaoxiao has quit IRC | 03:49 | |
| *** david-lyle has quit IRC | 03:49 | |
| *** zhugaoxiao has joined #openstack-swift | 03:50 | |
| *** david-lyle has joined #openstack-swift | 03:50 | |
| *** links has joined #openstack-swift | 04:03 | |
| *** adu has joined #openstack-swift | 04:04 | |
| *** adu has left #openstack-swift | 04:04 | |
| *** mmotiani_ has quit IRC | 04:10 | |
| *** mmotiani_ has joined #openstack-swift | 04:11 | |
| *** psachin has joined #openstack-swift | 04:19 | |
| *** mmotiani_ has quit IRC | 04:27 | |
| *** mmotiani_ has joined #openstack-swift | 04:27 | |
| *** lowecase has joined #openstack-swift | 04:28 | |
| *** lowecase has quit IRC | 04:28 | |
| *** sams-gleb has joined #openstack-swift | 04:29 | |
| *** lowecase has joined #openstack-swift | 04:30 | |
| *** lowecase has quit IRC | 04:30 | |
| notmyname | good evening | 04:30 |
| *** sams-gleb has quit IRC | 04:34 | |
| mattoliverau | notmyname: evening | 04:39 |
| *** mmotiani_ has quit IRC | 04:41 | |
| *** mmotiani_ has joined #openstack-swift | 04:43 | |
| *** daidv has joined #openstack-swift | 04:46 | |
| *** tqtran has joined #openstack-swift | 04:51 | |
| *** tqtran has quit IRC | 04:55 | |
| *** mmotiani_ has quit IRC | 04:57 | |
| *** mmotiani_ has joined #openstack-swift | 04:58 | |
| *** rcernin has joined #openstack-swift | 05:08 | |
| *** dja has quit IRC | 05:11 | |
| *** mmotiani_ has quit IRC | 05:12 | |
| *** mmotiani_ has joined #openstack-swift | 05:13 | |
| *** maestropandy has joined #openstack-swift | 05:17 | |
| *** klrmn has quit IRC | 05:18 | |
| *** tqtran has joined #openstack-swift | 05:53 | |
| *** daidv has quit IRC | 05:54 | |
| *** tqtran has quit IRC | 05:58 | |
| *** daidv has joined #openstack-swift | 06:00 | |
| *** rcernin has quit IRC | 06:21 | |
| *** sams-gleb has joined #openstack-swift | 06:33 | |
| *** sams-gleb has quit IRC | 06:37 | |
| *** ukaynar has joined #openstack-swift | 06:38 | |
| *** jerrygb has joined #openstack-swift | 06:41 | |
| *** jerrygb has quit IRC | 06:46 | |
| *** geaaru has joined #openstack-swift | 06:46 | |
| *** sams-gleb has joined #openstack-swift | 07:00 | |
| *** jerrygb has joined #openstack-swift | 07:00 | |
| *** rcernin has joined #openstack-swift | 07:01 | |
| *** dja has joined #openstack-swift | 07:02 | |
| *** cshastri has joined #openstack-swift | 07:04 | |
| *** ukaynar has quit IRC | 07:05 | |
| *** jerrygb has quit IRC | 07:05 | |
| *** tesseract has joined #openstack-swift | 07:11 | |
| *** ChubYann has quit IRC | 07:16 | |
| *** dja has quit IRC | 07:20 | |
| *** maestropandy has quit IRC | 07:45 | |
| *** amoralej|off is now known as amoralej | 07:46 | |
| *** tqtran has joined #openstack-swift | 07:55 | |
| *** tqtran has quit IRC | 07:59 | |
| *** mmotiani_ has quit IRC | 08:10 | |
| *** pcaruana has joined #openstack-swift | 08:22 | |
| *** dmorita has joined #openstack-swift | 08:24 | |
| *** webmobi has joined #openstack-swift | 08:25 | |
| webmobi | Hi I just want to know cloudfuse supports openstack swift with tempauth just for testing | 08:26 |
| *** dmorita has quit IRC | 08:31 | |
| *** rledisez has joined #openstack-swift | 08:34 | |
| *** daidv has quit IRC | 08:46 | |
| *** bkopilov_ has quit IRC | 08:48 | |
| *** bkopilov has quit IRC | 08:48 | |
| *** dmorita has joined #openstack-swift | 08:49 | |
| *** jordanP has joined #openstack-swift | 08:49 | |
| *** dmorita has quit IRC | 08:53 | |
| *** cbartz has joined #openstack-swift | 09:01 | |
| *** silor has joined #openstack-swift | 09:17 | |
| *** maestropandy has joined #openstack-swift | 09:30 | |
| *** maestropandy has left #openstack-swift | 09:30 | |
| *** silor1 has joined #openstack-swift | 09:32 | |
| *** silor has quit IRC | 09:33 | |
| *** silor1 is now known as silor | 09:33 | |
| *** joeljwright has joined #openstack-swift | 09:34 | |
| *** ChanServ sets mode: +v joeljwright | 09:34 | |
| kota_ | I'm going to finish up my this week work, looking forward to meeting all swifters at Atlanta PTG | 09:34 |
| *** wer has quit IRC | 09:35 | |
| kota_ | see you | 09:35 |
| *** wer has joined #openstack-swift | 09:35 | |
| *** Jeffrey4l has quit IRC | 09:42 | |
| *** Jeffrey4l has joined #openstack-swift | 09:55 | |
| *** tqtran has joined #openstack-swift | 09:56 | |
| *** Jeffrey4l has quit IRC | 09:56 | |
| *** Jeffrey4l has joined #openstack-swift | 09:57 | |
| *** tqtran has quit IRC | 10:00 | |
| *** bob_cheesey has joined #openstack-swift | 10:07 | |
| bob_cheesey | morning all | 10:09 |
| bob_cheesey | i've recently taken over maintenance of aswift cluster and whilst i've increased capacity before, i've never reduced it (retiring old nodes). my question is this - obviously I should rebalance the rings first rather than just removing the old node, but once I've weighted the old node down to zero, do i need to keep it in the cluster? | 10:10 |
| bob_cheesey | i know the rest of cluster will handle the job of creating any missing replicas, so does the old node serve any purpose once it's devices are weighted out? | 10:11 |
| *** Jeffrey4l has quit IRC | 10:18 | |
| *** Jeffrey4l has joined #openstack-swift | 10:19 | |
| *** dmorita has joined #openstack-swift | 10:25 | |
| *** dmorita has quit IRC | 10:26 | |
| *** dmorita_ has joined #openstack-swift | 10:26 | |
| *** dmorita_ has quit IRC | 10:31 | |
| *** dmorita has joined #openstack-swift | 10:43 | |
| *** jerrygb has joined #openstack-swift | 10:46 | |
| *** sams-gleb has quit IRC | 10:47 | |
| *** sams-gleb has joined #openstack-swift | 10:47 | |
| *** dmorita has quit IRC | 10:48 | |
| *** jerrygb has quit IRC | 10:50 | |
| *** jerrygb has joined #openstack-swift | 10:51 | |
| *** sams-gleb has quit IRC | 10:51 | |
| *** jerrygb_ has joined #openstack-swift | 10:54 | |
| *** jerrygb has quit IRC | 10:55 | |
| *** winggundamth has quit IRC | 11:10 | |
| *** kei_yama has quit IRC | 11:14 | |
| cbartz | bob_cheesey: No, if you have a x-replica (x>1) ring, you can turn it off. Like you already thought, the other nodes will take care of the missing replicas. | 11:14 |
| bob_cheesey | cbartz: wonderful, thanks! we're running with 3 replicas so that makes my node replacement timeline a lot shorter | 11:15 |
| bob_cheesey | thanks! | 11:15 |
| cbartz | bob_cheesey: You are welcome. | 11:15 |
| *** sams-gleb has joined #openstack-swift | 11:51 | |
| *** dmorita has joined #openstack-swift | 12:09 | |
| *** jerrygb_ has quit IRC | 12:10 | |
| *** dmorita has quit IRC | 12:13 | |
| *** catintheroof has joined #openstack-swift | 12:23 | |
| *** vint_bra has joined #openstack-swift | 12:24 | |
| *** jerrygb has joined #openstack-swift | 12:29 | |
| *** NM has joined #openstack-swift | 12:32 | |
| *** jerrygb_ has joined #openstack-swift | 12:45 | |
| *** jerrygb has quit IRC | 12:46 | |
| *** dmorita has joined #openstack-swift | 12:49 | |
| *** dmorita_ has joined #openstack-swift | 12:58 | |
| *** jerrygb_ has quit IRC | 13:02 | |
| *** dmorita has quit IRC | 13:02 | |
| *** jerrygb has joined #openstack-swift | 13:02 | |
| *** kjz has quit IRC | 13:02 | |
| *** silor has quit IRC | 13:10 | |
| *** links has quit IRC | 13:36 | |
| *** NM has quit IRC | 13:59 | |
| *** vint_bra has quit IRC | 13:59 | |
| *** NM has joined #openstack-swift | 14:03 | |
| *** vint_bra has joined #openstack-swift | 14:05 | |
| openstackgerrit | Mahati Chamarthy proposed openstack/swift master: WIP - follow on for race issue fix https://review.openstack.org/435460 | 14:08 |
| *** chlong has joined #openstack-swift | 14:15 | |
| *** cshastri has quit IRC | 14:19 | |
| *** gatuus has joined #openstack-swift | 14:29 | |
| *** tnovacik has joined #openstack-swift | 14:34 | |
| *** gatuus has quit IRC | 14:37 | |
| *** acoles_ is now known as acoles | 14:40 | |
| *** amoralej is now known as amoralej|lunch | 14:41 | |
| *** joeljwright1 has joined #openstack-swift | 14:41 | |
| *** acoles is now known as acoles_ | 14:42 | |
| *** agarner_away has quit IRC | 14:42 | |
| *** aleph1 has joined #openstack-swift | 14:43 | |
| *** joeljwright has quit IRC | 14:43 | |
| *** mmotiani_ has joined #openstack-swift | 14:45 | |
| *** sams-gleb has quit IRC | 14:54 | |
| *** sams-gleb has joined #openstack-swift | 14:55 | |
| *** sams-gleb has quit IRC | 14:59 | |
| *** jerrygb has quit IRC | 14:59 | |
| *** jerrygb has joined #openstack-swift | 15:00 | |
| openstackgerrit | Mahati Chamarthy proposed openstack/swift master: WIP - follow on for race issue fix in reconstructor https://review.openstack.org/435460 | 15:02 |
| *** psachin has quit IRC | 15:04 | |
| *** jerrygb has quit IRC | 15:04 | |
| *** mmotiani_ has quit IRC | 15:05 | |
| *** mmotiani_ has joined #openstack-swift | 15:06 | |
| *** ukaynar has joined #openstack-swift | 15:10 | |
| *** mmotiani_ has quit IRC | 15:12 | |
| *** sams-gleb has joined #openstack-swift | 15:13 | |
| *** tnovacik has quit IRC | 15:14 | |
| *** psachin has joined #openstack-swift | 15:15 | |
| *** amoralej|lunch is now known as amoralej | 15:18 | |
| *** bkopilov has joined #openstack-swift | 15:26 | |
| *** bkopilov_ has joined #openstack-swift | 15:26 | |
| *** mmotiani_ has joined #openstack-swift | 15:35 | |
| *** chsc has joined #openstack-swift | 15:38 | |
| *** chsc has joined #openstack-swift | 15:38 | |
| *** mmotiani_ has quit IRC | 15:53 | |
| *** mmotiani_ has joined #openstack-swift | 15:55 | |
| *** psachin has quit IRC | 15:56 | |
| *** dmorita_ has quit IRC | 15:57 | |
| *** tqtran has joined #openstack-swift | 16:00 | |
| *** bob_cheesey has quit IRC | 16:04 | |
| *** tqtran has quit IRC | 16:04 | |
| *** ukaynar has quit IRC | 16:08 | |
| *** bkopilov_ has quit IRC | 16:13 | |
| *** bkopilov has quit IRC | 16:13 | |
| *** pcaruana has quit IRC | 16:21 | |
| notmyname | good morning | 16:22 |
| rledisez | good morning notmyname. i see this talk may happen in boston: https://www.openstack.org/summit/boston-2017/vote-for-speakers/#/17465 | 16:25 |
| rledisez | i'm wondering if you have (or can have) the contact info of these guys | 16:26 |
| rledisez | i would be very interested to talk with them. it seems we are doing the same thing | 16:26 |
| notmyname | rledisez: yeah, that looks pretty cool, doesn't it? unfortunately, the first I've heard of it is when someone pointed out the talk presentation | 16:28 |
| rledisez | ok. i guess the foundation has their emails as they have an account on openstack.org. but it's surely not ok to give them to anybody (aka me :)) | 16:30 |
| tdasilva | it doesn't look like they have contributed to openstack before, can't really find their names on stackalytics | 16:31 |
| *** caiobrentano has joined #openstack-swift | 16:35 | |
| *** rcernin has quit IRC | 16:35 | |
| *** ukaynar has joined #openstack-swift | 16:35 | |
| rledisez | i just saw there is a contact form on the profile page on openstack.org: https://www.openstack.org/community/speakers/profile/11024/jeff-li | 16:36 |
| rledisez | i'll try this way, we'll see | 16:37 |
| notmyname | yeah, that's interesting | 16:37 |
| notmyname | I'd love to see what they're doing | 16:37 |
| tdasilva | rledisez: yeah that works, also found them on linkedin | 16:38 |
| notmyname | in my mind, the small file optimization is great. also, the refactored object server and repl protocol (in golang) is also great | 16:38 |
| *** rledisez has left #openstack-swift | 16:38 | |
| *** rledisez has joined #openstack-swift | 16:38 | |
| notmyname | I think I'd prefer to see small file work build on top of new object servers instead of implementing it on old ones and reimplementing it on new ones | 16:39 |
| notmyname | but I don't want to slow down stuff that's going on either :-) | 16:39 |
| *** tonanhngo has joined #openstack-swift | 16:39 | |
| cbartz | cschwede_: Just watched your talk https://www.openstack.org/videos/vancouver-2015/monitoring-your-swift-cluster-health . At the end you say you upload the example config files somewhere. Can you please send me the location, if the resources are still online? You can send me via eMail, if I am not anymore in IRC. This would be great. | 16:39 |
| notmyname | that ordering and dependency is one thing I'm hoping to come to grips with next week | 16:39 |
| rledisez | i think golang should come first, but as it's not ready we are working on python to validate the solution (we will probably have production deployement of small files stuff before golang is prod ready) | 16:39 |
| rledisez | so let's think of our effort as a way to validate the idea | 16:40 |
| notmyname | rledisez: yep. there's the reality :-) | 16:40 |
| rledisez | if it work, we will re-do it in golang once it replaced the python version | 16:40 |
| *** gatuus has joined #openstack-swift | 16:40 | |
| rledisez | and it would be insane to merge that kind of stuff in hummingbird before it replaced python. one step at a time :) | 16:41 |
| timburke | good morning | 16:41 |
| tdasilva | notmyname: what do you mean by: "...build on top of new object servers instead of implementing it on old ones..." | 16:41 |
| *** tesseract has quit IRC | 16:41 | |
| timburke | golang object servers, rather than python | 16:41 |
| tdasilva | ok | 16:42 |
| notmyname | ya | 16:42 |
| tdasilva | what's the plan concerning golang conversations next week? | 16:42 |
| tdasilva | in terms of TC approval stuff, not internal "what we need to work on hummingbird" stuff | 16:43 |
| notmyname | tdasilva: good question. that probably will involve some more hallway conversation than just "in the swift room" stuff | 16:43 |
| notmyname | I'm plannign on reading up ont he tc rules and hopefully on mon/tues talk to the right not-swift people | 16:44 |
| notmyname | and have some answers/guidance for swift people on wed-fri | 16:45 |
| tdasilva | ok, sounds like a plan | 16:45 |
| notmyname | tdasilva: but I'd definitely appreciate your help on that too :-) | 16:45 |
| notmyname | I think it's a lot of simply writing down our answers to stuff like how to do docs, dependencies, etc | 16:46 |
| tdasilva | notmyname: yep, i talked to Flavio earlier today and mentioned we might reach out to him there | 16:46 |
| notmyname | cool | 16:47 |
| notmyname | irc or is he in westford right now? | 16:47 |
| tdasilva | irc | 16:47 |
| notmyname | k | 16:47 |
| tdasilva | he mentioned we should ping dtroyer too | 16:47 |
| cschwede_ | cbartz: i think you're looking for https://github.com/cschwede/fedora-swift-rdo | 16:48 |
| notmyname | yeah, dean was working on the golang testing rules | 16:48 |
| *** ukaynar has quit IRC | 16:48 | |
| cbartz | cschwede_: Thank you. ;-) | 16:49 |
| tdasilva | stepping out for lunch, be back later | 16:50 |
| *** dmorita has joined #openstack-swift | 16:58 | |
| *** cbartz has left #openstack-swift | 17:01 | |
| *** dmorita has quit IRC | 17:02 | |
| *** jordanP has quit IRC | 17:07 | |
| *** Guest36_ has joined #openstack-swift | 17:19 | |
| *** Guest36_ is now known as Darkpixl | 17:19 | |
| *** zaitcev has joined #openstack-swift | 17:23 | |
| *** ChanServ sets mode: +v zaitcev | 17:23 | |
| *** rledisez has quit IRC | 17:39 | |
| *** jerrygb_ has joined #openstack-swift | 17:46 | |
| *** Raymii has quit IRC | 17:47 | |
| *** jerrygb has joined #openstack-swift | 17:50 | |
| timburke | :-( libec's crc32 seems to produce different results on my mac vs my linux vm. which means i can't decode frags generated on one from the other, because the headers are invalid | 17:51 |
| *** jerrygb_ has quit IRC | 17:51 | |
| notmyname | that seems ... troubling? | 17:52 |
| notmyname | timburke: same data but different crc? | 17:54 |
| timburke | but at the same time, things are at least consistent on the same machine. mac can decode mac, linux can decode linux. only linux's crc matches python stdlib's, though | 17:55 |
| timburke | did we ever make the assertion that frags should be portable cross-platform? | 17:55 |
| *** klrmn has joined #openstack-swift | 17:56 | |
| timburke | and if we *fix* the implementation for my mac (i think it might have to do with 32 vs 64 bit ints, maybe?) did we just break some users' data on disk? | 17:58 |
| notmyname | timburke: do you have a repro script I can run? | 17:58 |
| *** bikmak has quit IRC | 17:59 | |
| notmyname | oh, so 64 vs 32 bit? meaning you might find similar issues on 32 bit linux? | 17:59 |
| timburke | and the more i look at https://github.com/openstack/liberasurecode/blob/master/src/utils/chksum/crc32.c the less i like it -- why is it declared with ints when those are only guaranteed to be at least 16 bits? | 17:59 |
| notmyname | int instead of explicit uing64_t or something? | 18:00 |
| timburke | notmyname: fwiw, i realized that my mac thinks the appropriate format string for uint64_t is %lu, while the linux vm thinks it's %llu | 18:00 |
| notmyname | (it's been a long time since I've done C types) | 18:00 |
| timburke | uint32_t -- it's right there in the name! | 18:01 |
| notmyname | so it got compiled as 32 bit on the mac and 64 bit on linux? or vice versa? | 18:04 |
| timburke | or did i have that the other way? maybe the mac wanted llu and linux wanted lu. either way, https://review.openstack.org/#/c/434162/5/src/erasurecode.c should maybe be using PRIu64 | 18:06 |
| patchbot | patch 434162 - liberasurecode - Fix valgrind-check and memory leak | 18:06 |
| *** Darkpixl has quit IRC | 18:18 | |
| *** joeljwright1 has quit IRC | 18:33 | |
| *** tqtran has joined #openstack-swift | 18:51 | |
| *** dmorita has joined #openstack-swift | 18:59 | |
| -openstackstatus- NOTICE: Restarting gerrit due to performance problems | 19:04 | |
| *** dmorita has quit IRC | 19:05 | |
| *** oshritf_ has joined #openstack-swift | 19:29 | |
| *** ChubYann has joined #openstack-swift | 19:32 | |
| *** tnovacik has joined #openstack-swift | 19:32 | |
| *** oshritf_ has quit IRC | 19:37 | |
| timburke | o_O so my go bindings behave the same on mac & linux vm. but...how could that even be an issue? the data's getting written down in C... | 19:38 |
| timburke | like, why are the python bindings doing different things on different platforms, but the go bindings *don't*? | 19:38 |
| *** dmorita has joined #openstack-swift | 19:48 | |
| *** dmorita has quit IRC | 19:53 | |
| pdardeau_ | timburke: is there a single standard for official crc algorithm? | 19:53 |
| *** tnovacik has quit IRC | 19:55 | |
| *** tnovacik has joined #openstack-swift | 19:56 | |
| timburke | i feel like https://github.com/torvalds/linux/blob/master/lib/crc32.c ought to be a pretty good reference implementation... | 19:57 |
| notmyname | timburke: so why doesn't libec use that one? | 19:58 |
| notmyname | a question for tsg I guess | 19:58 |
| notmyname | GPL maybe? | 19:58 |
| pdardeau_ | see 1st paragraph of "Standards and common use" https://en.wikipedia.org/wiki/Cyclic_redundancy_check | 19:58 |
| timburke | ...cross-platform goodness? idk | 19:58 |
| openstackgerrit | Thiago da Silva proposed openstack/swift master: Symlink implementation. https://review.openstack.org/232162 | 20:00 |
| *** dmorita has joined #openstack-swift | 20:04 | |
| *** mmotiani_ has quit IRC | 20:09 | |
| *** dmorita has quit IRC | 20:09 | |
| *** silor has joined #openstack-swift | 20:28 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift master: Do not sync suffixes when remote rejects reconstructor revert https://review.openstack.org/435152 | 20:30 |
| *** silor has quit IRC | 20:32 | |
| *** amoralej is now known as amoralej|off | 20:37 | |
| *** jerrygb has quit IRC | 20:44 | |
| *** dmorita has joined #openstack-swift | 20:52 | |
| *** dmorita has quit IRC | 20:57 | |
| *** mmotiani_ has joined #openstack-swift | 20:57 | |
| timburke | our table at https://github.com/openstack/liberasurecode/blob/master/src/utils/chksum/crc32.c#L69 seems to match the one from https://github.com/madler/zlib/blob/master/crc32.h -- which would imply CRC-32, rather than SSE4's CRC-32C (which seems to be what we meant to do https://github.com/openstack/liberasurecode/blob/master/src/utils/chksum/crc32.c#L115-L116) | 21:00 |
| timburke | bleh. thanks pdardeau_ -- it was definitely useful to be able to quickly look up names from polynomial hex | 21:02 |
| *** bkopilov has joined #openstack-swift | 21:04 | |
| *** bkopilov_ has joined #openstack-swift | 21:04 | |
| *** dmorita has joined #openstack-swift | 21:08 | |
| *** NM has quit IRC | 21:10 | |
| *** caiobrentano has quit IRC | 21:12 | |
| *** dmorita has quit IRC | 21:12 | |
| *** tnovacik has quit IRC | 21:28 | |
| *** Jeffrey4l_ has joined #openstack-swift | 21:34 | |
| *** Jeffrey4l has quit IRC | 21:34 | |
| *** tonanhngo has quit IRC | 21:38 | |
| notmyname | timburke: have you come across any "set the network chunk size" things in the requests library? | 21:45 |
| timburke | notmyname: how do you mean? like, the size for transfer-encoding: chunked? or the buffers involved for the socket? | 21:47 |
| notmyname | buffers | 21:48 |
| timburke | i've not played with that... how do we do it in swift? | 21:48 |
| notmyname | I'm helping someone diagnose a problem, and they're reporting that swiftclient is using a pretty small buffer size (but TBH, that's reported, not repro and confirmed). | 21:48 |
| notmyname | the ultimate problem here is two different clients sending to the same swift cluster and swiftclient is slow. and I'm told that swiftclient is using much smaller network buffers | 21:50 |
| notmyname | I was hoping to find some constant somewhere to hack a hard-coded change into and test it to see if there's a difference, but it seems like requests is full of magic wrapped in abstraction ;-) | 21:51 |
| *** dmorita has joined #openstack-swift | 21:55 | |
| *** jerrygb has joined #openstack-swift | 21:57 | |
| timburke | notmyname: might be able to try something like http://stackoverflow.com/questions/24569428/how-to-specify-socket-options-in-python-requests-lib-since-urllib3-v1-8-3-has#24570887 to set SO_SNDBUF/SO_RCVBUF sizes... | 21:58 |
| *** dmorita has quit IRC | 22:00 | |
| *** vint_bra has quit IRC | 22:00 | |
| *** mmotiani_ has quit IRC | 22:01 | |
| timburke | or if you're feeling adventurous, go poking requests.packages.urllib3.connection.HTTPConnection.default_socket_options directly | 22:01 |
| *** jerrygb has quit IRC | 22:03 | |
| *** tonanhngo has joined #openstack-swift | 22:16 | |
| *** sams-gleb has quit IRC | 22:41 | |
| *** sams-gleb has joined #openstack-swift | 22:41 | |
| *** sams-gleb has quit IRC | 22:41 | |
| *** dmorita has joined #openstack-swift | 22:43 | |
| *** dmorita has quit IRC | 22:47 | |
| *** jerrygb has joined #openstack-swift | 23:00 | |
| *** gatuus has quit IRC | 23:04 | |
| *** NM has joined #openstack-swift | 23:10 | |
| timburke | :-/so we don't *quite* use the zlib crc32 algo... while the table looks right, the fact that we're dealing with ints instead of unsigned ints means https://github.com/openstack/liberasurecode/blob/master/src/utils/chksum/crc32.c#L127 is technically off in undefined behavior | 23:11 |
| *** NM has quit IRC | 23:11 | |
| timburke | "When performed on a signed type, the result is technically undefined and compiler dependant,[5] however most compilers will perform an arithmetic shift, causing the blank to be filled with the sign bit of the left operand." | 23:11 |
| *** catintheroof has quit IRC | 23:21 | |
| *** catintheroof has joined #openstack-swift | 23:22 | |
| *** catintheroof has quit IRC | 23:26 | |
| *** dmorita has joined #openstack-swift | 23:30 | |
| timburke | second issue: i don't see where INTEL_SSE4 would get defined, so i think all of https://github.com/openstack/liberasurecode/blob/master/src/utils/chksum/crc32.c#L46-L66 is dead code | 23:32 |
| timburke | i *do* see a INTEL_SSE42, which seems to be what's called for anyway -- the crc32 instruction seems to have been added in 4.2 https://en.wikipedia.org/wiki/SSE4#SSE4.2 | 23:32 |
| timburke | but none of any of that explains why i get good, zlib-compatible crcs from go but busted crcs from python. that's just weird | 23:34 |
| *** dmorita has quit IRC | 23:34 | |
| notmyname | timburke: that sounds bad | 23:39 |
| notmyname | so it could be a difference between gcc on linux and clang/llvm on apple? | 23:39 |
| timburke | notmyname: it doesn't seem great, no. might be the compiler, but that would seem to imply that i've got some schizophrenia with my gcc compiler on my linux vm | 23:41 |
| notmyname | timburke: that sounds bad | 23:41 |
| timburke | also, that i'm not picking up the same liberasurecode at runtime between go and python? i guess? | 23:41 |
| notmyname | ;-) | 23:41 |
| notmyname | I think this conversation is you saying something and me <up arrow><enter> | 23:42 |
| notmyname | timburke: that sounds bad | 23:42 |
| *** sams-gleb has joined #openstack-swift | 23:42 | |
| timburke | yeah, yeah it does. the one redeaming point is that we don't actively have people yelling at us that all of they're EC data is unreadable because we think the header's gone bad | 23:43 |
| notmyname | timburke: that sounds bad^wgood | 23:43 |
| notmyname | ;-) | 23:43 |
| timburke | so hopefully it's actually user error...somehow. and the next libec upgrade *totally won't* flip that bit and make everything terrible, right? | 23:43 |
| notmyname | so the risk is what? when loading a different libec...yeah. that. the next libec (or even reinstalling the current==recompile) doesn't inadvertently change the bits | 23:44 |
| notmyname | wow. if ever there's a use case for reproducible builds, this is it :-) | 23:44 |
| notmyname | I learned some about that work at LCA this year | 23:44 |
| *** chlong has quit IRC | 23:46 | |
| notmyname | the goal being that a deliverable is bit-for-bit identical to every other deliverable (on a given platform, arch, etc) every single time it is built for every single person | 23:47 |
| *** sams-gleb has quit IRC | 23:47 | |
| notmyname | meaning that in this case, libec on a 64-bit box would be exactly the same as every other 64 bit box | 23:48 |
| notmyname | if I understand it correctly | 23:48 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!