| *** jamielennox is now known as jamielennox|away | 00:07 | |
| *** jamielennox|away is now known as jamielennox | 00:11 | |
| *** ogelbukh has quit IRC | 00:11 | |
| *** remix_tj has quit IRC | 00:12 | |
| *** ogelbukh has joined #openstack-swift | 00:13 | |
| *** jroll has quit IRC | 00:14 | |
| *** remix_tj has joined #openstack-swift | 00:14 | |
| mattoliverau | MooingLemur: wow, there is a patch hurricanerix is working on revamp the object expiry in Swift and remove the need for the expirer daemon | 00:17 |
|---|---|---|
| MooingLemur | oh, nice :) | 00:18 |
| MooingLemur | I have that split across 8 hosts now, myself. | 00:18 |
| *** jroll has joined #openstack-swift | 00:21 | |
| *** tqtran has quit IRC | 00:23 | |
| clayg | mattoliverau: is hurricanerix still working on that? I thought aerwin3 took that over but then got caught up on other stuff? | 00:41 |
| mattoliverau | clayg: yeah he is, it's one of his OSIC tasks | 00:46 |
| clayg | mattoliverau: awesome! | 00:59 |
| clayg | mattoliverau: gtk! | 00:59 |
| kota_ | good morning | 01:05 |
| * kota_ is now at office | 01:05 | |
| kota_ | mattoliverau, clayg: https://review.openstack.org/#/c/252085/ <- that one for object-expier? or new another one comming? | 01:08 |
| patchbot | patch 252085 - swift - Refactoring the expiring objects feature | 01:08 |
| *** _JZ__ has quit IRC | 01:17 | |
| *** _JZ_ has quit IRC | 01:17 | |
| *** sams-gleb has joined #openstack-swift | 01:22 | |
| *** sgundur_ has joined #openstack-swift | 01:26 | |
| *** sams-gleb has quit IRC | 01:26 | |
| *** cppforlife_ has joined #openstack-swift | 01:34 | |
| *** briancurtin has joined #openstack-swift | 01:35 | |
| *** cdelatte has quit IRC | 01:54 | |
| *** dmorita has quit IRC | 01:57 | |
| *** dmorita has joined #openstack-swift | 01:58 | |
| *** sgundur_ has quit IRC | 02:13 | |
| clayg | kota_: mattoliverau: I'm glad that patch 283946 merged to fix lp bug #1488608 - but I feel like I'm missing a test to demonstrate the change in stats output? all of the test changes look related the refactoring (why did the replicator need a router again?) | 02:15 |
| openstack | Launchpad bug 1488608 in OpenStack Object Storage (swift) "stats output in reconstructor.py gives wrong device count" [Low,Fix released] https://launchpad.net/bugs/1488608 - Assigned to Bill Huber (wbhuber) | 02:15 |
| patchbot | https://review.openstack.org/#/c/283946/ - swift - Fix stats calculation in object-reconstructor (MERGED) | 02:15 |
| *** asettle has joined #openstack-swift | 02:16 | |
| kota_ | clayg: lemme check... | 02:17 |
| clayg | it's maybe NBD - I have a branch where I started to suss out an unrelated stats output thing (we increment the x/y parts as soon as we *start* working on x instead as we finish) - I think I fixed the device count thing as a drive by - i'm sure it'll all come back to me I dust that off | 02:18 |
| *** dmorita has quit IRC | 02:18 | |
| *** sgundur_ has joined #openstack-swift | 02:19 | |
| kota_ | ah, exactly, it looks like it's missing for output assertions while reviewing process. | 02:21 |
| clayg | probalby nbd | 02:22 |
| *** asettle has quit IRC | 02:23 | |
| kota_ | yeah, patch set 1 has draft output check :/ https://review.openstack.org/#/c/283946/1/test/unit/obj/test_reconstructor.py | 02:23 |
| patchbot | patch 283946 - swift - Fix stats calculation in object-reconstructor (MERGED) | 02:23 |
| mattoliverau | kota_: yeah that's it. Hurricanerix is taking it over and will revise and continue it on. | 02:25 |
| kota_ | and iirc, it's a bit strong-arm and i changed something... | 02:25 |
| kota_ | and missing output assertion, maybe? | 02:25 |
| *** JimCheung has quit IRC | 02:26 | |
| kota_ | mattoliverau: nice | 02:26 |
| *** dmorita has joined #openstack-swift | 02:27 | |
| kota_ | notmyname: could you add me (kota3) to swift-team in trello? I didn't work on symlink staff yet but I'd get subscribed the change and progress from trello. | 02:30 |
| kota_ | to track upstream. | 02:30 |
| *** chlong has quit IRC | 02:32 | |
| *** sgundur_ has quit IRC | 02:34 | |
| *** sgundur_ has joined #openstack-swift | 02:40 | |
| *** sgundur_ has quit IRC | 02:46 | |
| *** sgundur_ has joined #openstack-swift | 02:47 | |
| *** winggundamth has quit IRC | 02:47 | |
| *** sgundur_ has quit IRC | 03:04 | |
| *** dmorita has quit IRC | 03:06 | |
| *** winggundamth has joined #openstack-swift | 03:07 | |
| kota_ | tdasilva: thanks for adding me to the symlink board | 03:19 |
| *** tqtran has joined #openstack-swift | 03:21 | |
| *** tqtran has quit IRC | 03:23 | |
| *** sams-gleb has joined #openstack-swift | 03:25 | |
| *** dschultz has joined #openstack-swift | 03:25 | |
| *** sams-gleb has quit IRC | 03:29 | |
| *** dmorita has joined #openstack-swift | 03:32 | |
| *** chosafine has joined #openstack-swift | 03:36 | |
| *** chosafine has joined #openstack-swift | 03:36 | |
| *** dmorita has quit IRC | 03:36 | |
| *** chosafin_ has joined #openstack-swift | 03:39 | |
| *** chosafine has quit IRC | 03:40 | |
| *** chosafin_ is now known as chosafine | 03:41 | |
| *** chosafine has joined #openstack-swift | 03:41 | |
| * MooingLemur is now seeing if bindfs will run at scale | 03:44 | |
| MooingLemur | changing from gentoo's swift uid/gid of 404 to CentOS's standard of 160 | 03:45 |
| *** links has joined #openstack-swift | 03:47 | |
| *** sgundur_ has joined #openstack-swift | 03:48 | |
| MooingLemur | and chowning all the files underneath before the conversion to centos :) | 03:49 |
| *** dmorita has joined #openstack-swift | 03:50 | |
| *** dmorita has quit IRC | 03:54 | |
| *** sgundur_ has quit IRC | 04:01 | |
| *** sgundur_ has joined #openstack-swift | 04:01 | |
| *** sgundur_ has quit IRC | 04:12 | |
| *** padkrish has joined #openstack-swift | 04:17 | |
| *** padkrish has left #openstack-swift | 04:18 | |
| *** varunsomani has joined #openstack-swift | 04:19 | |
| *** manij has joined #openstack-swift | 04:20 | |
| *** varunsomani has quit IRC | 04:20 | |
| * kota_ is starting to pick up the critical bug reported by acoles for looking. | 04:22 | |
| openstackgerrit | Mahati Chamarthy proposed openstack/swift: Cleanup tests from empty suffix quarantined db fix https://review.openstack.org/405134 | 04:23 |
| *** chsc has joined #openstack-swift | 04:25 | |
| *** chsc has joined #openstack-swift | 04:25 | |
| *** manij has quit IRC | 04:32 | |
| *** ppai has joined #openstack-swift | 04:33 | |
| *** sheel has joined #openstack-swift | 04:34 | |
| *** ppai has quit IRC | 04:35 | |
| *** chsc has quit IRC | 04:38 | |
| *** SkyRocknRoll has joined #openstack-swift | 04:40 | |
| *** psachin has joined #openstack-swift | 04:40 | |
| *** tqtran has joined #openstack-swift | 04:40 | |
| *** tqtran has quit IRC | 04:42 | |
| *** manij has joined #openstack-swift | 04:50 | |
| *** manij has quit IRC | 04:55 | |
| timburke | 404 seems like an ominous uid for a web service... particularly one often used to archive backups... | 05:12 |
| mattoliverau | lol | 05:25 |
| *** sams-gleb has joined #openstack-swift | 05:27 | |
| clayg | kota_: the suffix hashing one? | 05:30 |
| kota_ | clayg: yup | 05:30 |
| *** ppai has joined #openstack-swift | 05:30 | |
| kota_ | clayg: did you already start that one? | 05:30 |
| clayg | i *just* finished - took me three days - i have... three branches and 17 draft comments and 4 change-sets (3 mostly indpedent branches) | 05:31 |
| kota_ | clayg: awesome | 05:31 |
| *** sams-gleb has quit IRC | 05:32 | |
| kota_ | clayg: no worries to push your +2 if it's great. I didn't spent much my time yet. | 05:32 |
| kota_ | i'm just made sense with the bug description. | 05:33 |
| clayg | well... umm... no i'm not sure it should merge - I had to pull it apart to wrap my head around it - i'm hoping we maybe merge the smaller indepdent fixes | 05:36 |
| kota_ | yeah, i'm feeling the commit log compolicated a bit because that closes a bug but the title is "Optimize". | 05:39 |
| kota_ | complicated | 05:39 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Extract test pattern to helper https://review.openstack.org/418689 | 05:39 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Fix race in new partitions detecting new/invalid suffixes. https://review.openstack.org/418690 | 05:39 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Fix performance regression with hash invalidations https://review.openstack.org/418691 | 05:39 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Optimize noop case for suffix rehash https://review.openstack.org/418692 | 05:39 |
| kota_ | clayg: are they for replacing the patch 402043? | 05:40 |
| patchbot | https://review.openstack.org/#/c/402043/ - swift - Optimize hash calculation when suffix hash invalid... | 05:40 |
| clayg | kota_: it depends what acoles_ and pavel want to do? | 05:41 |
| kota_ | clayg: alright but probably i like that idea because each change (e.g. closing critical bug, https://review.openstack.org/#/c/418690/) smaller than the original. | 05:43 |
| patchbot | patch 418690 - swift - Fix race in new partitions detecting new/invalid s... | 05:43 |
| clayg | kota_: right, and that critical fix is acctually better (IMHO) than the change in patch 402043 | 05:43 |
| patchbot | https://review.openstack.org/#/c/402043/ - swift - Optimize hash calculation when suffix hash invalid... | 05:43 |
| clayg | which FWIW, again IMHO, address the performance issue better than patch 418691 | 05:44 |
| patchbot | https://review.openstack.org/#/c/418691/ - swift - Fix performance regression with hash invalidations | 05:44 |
| clayg | for some measure of "better" - I think part of the reason patch 418691 exists like it does is because it makes the problem/fix obvious - even thought it still needs some tweeks - I think even after addressing the race thing it'll still be "less stuff" that the entirty of patch 402043 | 05:45 |
| patchbot | https://review.openstack.org/#/c/418691/ - swift - Fix performance regression with hash invalidations | 05:45 |
| patchbot | https://review.openstack.org/#/c/402043/ - swift - Optimize hash calculation when suffix hash invalid... | 05:45 |
| clayg | i'm sure breaking it is the right thing to do - i'm not sure if not merging fixes purely on astetics is a wise choice | 05:46 |
| kota_ | clayg: sure, i'll try to look at your 4 patches, thanks for letting me to head up. | 05:48 |
| *** manij has joined #openstack-swift | 05:51 | |
| *** bkopilov has joined #openstack-swift | 06:06 | |
| *** dims has quit IRC | 06:13 | |
| *** dfflanders has quit IRC | 06:19 | |
| openstackgerrit | Janie Richling proposed openstack/swift: Symlink API documentation https://review.openstack.org/415943 | 06:25 |
| *** pcaruana has joined #openstack-swift | 06:25 | |
| jrichli | I will squash into the impl tomorrow ^^ | 06:25 |
| *** sheel has quit IRC | 06:37 | |
| *** geaaru has joined #openstack-swift | 06:46 | |
| *** chsc has joined #openstack-swift | 06:47 | |
| *** chsc has joined #openstack-swift | 06:47 | |
| *** chosafine has quit IRC | 06:51 | |
| *** chsc has quit IRC | 06:52 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Fix performance regression with hash invalidations https://review.openstack.org/418691 | 07:04 |
| *** cbartz has joined #openstack-swift | 07:08 | |
| *** tesseract has joined #openstack-swift | 07:08 | |
| clayg | shit - I think I messed up on patch 402043 - I +1 +A'd it :\ | 07:10 |
| patchbot | https://review.openstack.org/#/c/402043/ - swift - Optimize hash calculation when suffix hash invalid... | 07:10 |
| *** ChubYann has quit IRC | 07:15 | |
| *** sams-gleb has joined #openstack-swift | 07:29 | |
| *** sams-gleb has quit IRC | 07:34 | |
| *** sams-gleb has joined #openstack-swift | 07:46 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Fix race in new partitions detecting new/invalid suffixes. https://review.openstack.org/418690 | 07:50 |
| *** bkopilov has quit IRC | 07:55 | |
| *** hogepodge has quit IRC | 07:56 | |
| *** winggundamth has quit IRC | 07:56 | |
| *** hseipp has joined #openstack-swift | 07:59 | |
| *** tmoreira has quit IRC | 08:06 | |
| *** sams-gleb has quit IRC | 08:07 | |
| *** sams-gleb has joined #openstack-swift | 08:07 | |
| *** tmoreira has joined #openstack-swift | 08:08 | |
| *** rledisez has joined #openstack-swift | 08:11 | |
| *** sams-gleb has quit IRC | 08:11 | |
| zaitcev | tsk tsk :-) | 08:26 |
| *** kei_yama has quit IRC | 08:45 | |
| *** bkopilov has joined #openstack-swift | 08:51 | |
| kota_ | clayg: woow, the patch 402043 stops to get merged at Jenkis +2. Should I add -2 or workflow -1 for safety to prevent the merge? | 08:52 |
| patchbot | https://review.openstack.org/#/c/402043/ - swift - Optimize hash calculation when suffix hash invalid... | 08:52 |
| *** oshritf has joined #openstack-swift | 08:56 | |
| mahatic | kota_: if it can prevent the merge, maybe? But IDK..clayg didn't seem to suggest that? | 08:59 |
| kota_ | mahatic: i'm not sure too if it's a way. | 09:00 |
| *** cbartz has quit IRC | 09:00 | |
| mahatic | kota_: oh. clayg's latest comment seem to suggest that though - wait to see how the splitting up of patches work for others | 09:01 |
| *** jordanP has joined #openstack-swift | 09:02 | |
| kota_ | mahatic: splitting patches has existed already | 09:04 |
| kota_ | mahatic: 4 patches since https://review.openstack.org/418689 | 09:04 |
| patchbot | patch 418689 - swift - Extract test pattern to helper | 09:04 |
| mahatic | kota_: yeah I saw that, that's why he doesn't want this to be merged yet. Maybe you can give it a shot at workflow -1? | 09:05 |
| kota_ | mahatic: yeah, i can do that but idk he wants me to do that for now because he also is able to do... | 09:06 |
| mahatic | kota_: ah yeah, okay | 09:06 |
| *** asettle has joined #openstack-swift | 09:09 | |
| *** cbartz has joined #openstack-swift | 09:10 | |
| *** mvk has quit IRC | 09:32 | |
| *** tqtran has joined #openstack-swift | 09:44 | |
| *** tqtran has quit IRC | 09:45 | |
| *** sams-gleb has joined #openstack-swift | 09:59 | |
| *** geaaru has quit IRC | 10:07 | |
| *** geaaru has joined #openstack-swift | 10:09 | |
| *** hseipp has quit IRC | 10:09 | |
| *** mvk has joined #openstack-swift | 10:10 | |
| *** acoles_ is now known as acoles | 10:14 | |
| *** cbartz has quit IRC | 10:46 | |
| *** hseipp has joined #openstack-swift | 10:48 | |
| *** vint_bra has joined #openstack-swift | 10:49 | |
| *** cbartz has joined #openstack-swift | 10:52 | |
| *** ukaynar has joined #openstack-swift | 10:56 | |
| *** ppai has quit IRC | 11:08 | |
| kota_ | clayg: sorry, I couldn't finish all your patches today but only one patch which seems the most significant reviewed | 11:09 |
| kota_ | clayg: https://review.openstack.org/#/c/418690 <- this one | 11:10 |
| patchbot | patch 418690 - swift - Fix race in new partitions detecting new/invalid s... | 11:10 |
| kota_ | clayg: hope my comment would be worth | 11:10 |
| * kota_ is getting back home | 11:10 | |
| *** vint_bra has quit IRC | 11:14 | |
| *** ppai has joined #openstack-swift | 11:26 | |
| *** ukaynar has quit IRC | 11:27 | |
| *** ukaynar has joined #openstack-swift | 11:39 | |
| *** dims has joined #openstack-swift | 11:39 | |
| *** onboard has joined #openstack-swift | 11:58 | |
| *** hseipp has quit IRC | 12:00 | |
| *** mvk has quit IRC | 12:03 | |
| *** ukaynar has quit IRC | 12:03 | |
| *** mvk has joined #openstack-swift | 12:15 | |
| *** vint_bra has joined #openstack-swift | 12:24 | |
| *** SkyRocknRoll has quit IRC | 12:48 | |
| *** ganders has joined #openstack-swift | 12:48 | |
| *** hseipp has joined #openstack-swift | 12:51 | |
| *** chlong has joined #openstack-swift | 13:07 | |
| *** mvk has quit IRC | 13:21 | |
| *** mvk has joined #openstack-swift | 13:33 | |
| ganders | is there anyone who had change the python interpreter from cpython to pypy on an already running swift cluster with juno version? | 13:38 |
| *** Jeffrey4l_ has quit IRC | 13:45 | |
| *** AnxiousGarlic has joined #openstack-swift | 13:51 | |
| *** AnxiousGarlic_ has joined #openstack-swift | 13:51 | |
| *** david-lyle has quit IRC | 13:56 | |
| *** tongli has joined #openstack-swift | 13:59 | |
| *** david-lyle has joined #openstack-swift | 13:59 | |
| *** links has quit IRC | 14:02 | |
| *** AnxiousGarlic_ has quit IRC | 14:05 | |
| *** dmorita has joined #openstack-swift | 14:08 | |
| tdasilva | good morning | 14:09 |
| *** dmorita has quit IRC | 14:13 | |
| *** klamath has joined #openstack-swift | 14:14 | |
| *** jamielennox is now known as jamielennox|away | 14:14 | |
| *** klamath has quit IRC | 14:14 | |
| *** klamath has joined #openstack-swift | 14:15 | |
| *** david-lyle has quit IRC | 14:16 | |
| *** chlong has quit IRC | 14:37 | |
| *** chsc has joined #openstack-swift | 14:38 | |
| *** chsc has joined #openstack-swift | 14:38 | |
| *** chsc has quit IRC | 14:44 | |
| *** chosafine has joined #openstack-swift | 14:47 | |
| *** tongli has quit IRC | 14:49 | |
| *** ganders has quit IRC | 14:51 | |
| *** chosafine has quit IRC | 14:51 | |
| *** cloudm2 has joined #openstack-swift | 14:52 | |
| *** ppai has quit IRC | 14:52 | |
| *** ppai has joined #openstack-swift | 15:05 | |
| openstackgerrit | Janie Richling proposed openstack/swift: Symlink API documentation https://review.openstack.org/415943 | 15:05 |
| *** ppai has quit IRC | 15:06 | |
| *** sams-gleb has quit IRC | 15:09 | |
| *** sams-gleb has joined #openstack-swift | 15:09 | |
| *** ganders has joined #openstack-swift | 15:10 | |
| *** sams-gleb has quit IRC | 15:14 | |
| *** psachin has quit IRC | 15:14 | |
| *** _JZ_ has joined #openstack-swift | 15:15 | |
| *** chosafine has joined #openstack-swift | 15:24 | |
| *** adu has joined #openstack-swift | 15:24 | |
| *** sams-gleb has joined #openstack-swift | 15:31 | |
| *** Jeffrey4l has joined #openstack-swift | 15:40 | |
| timburke | ganders: i know notmyname has done some work with pypy... not sure it was as far back as juno, though | 15:42 |
| timburke | i seem to remember him saying things about how sockets are handled differently? | 15:43 |
| ganders | thanks a lot timburke, i'll try it on a lab cluster and see how things go with that.. | 15:44 |
| *** sgundur_ has joined #openstack-swift | 15:47 | |
| *** mvk has quit IRC | 15:48 | |
| notmyname | good morning | 15:49 |
| notmyname | kota_: done. I added you | 15:49 |
| *** sgundur_ has quit IRC | 15:54 | |
| *** AnxiousGarlic has quit IRC | 15:54 | |
| *** sgundur_ has joined #openstack-swift | 15:55 | |
| *** Shashikant86 has joined #openstack-swift | 16:01 | |
| notmyname | ganders: cool. what are you doing with swift and pypy | 16:02 |
| *** sgundur_ has quit IRC | 16:03 | |
| ganders | we're hosting millions and millions of images of our site, and we've a clus that consumes that, and the problem is that our disks are at 100% almost all the time.. we've 6 data nodes and 3 proxy servers, and for example, the three proxies had more or less 5.5M GET ops per day each | 16:06 |
| ganders | 800k put req, and the thing is that we don't have budget for expanding the cluster.. so.. it's a little bit hard to improve that condition | 16:07 |
| *** chosafine has quit IRC | 16:08 | |
| *** chosafine has joined #openstack-swift | 16:11 | |
| *** chosafine has joined #openstack-swift | 16:11 | |
| *** chlong has joined #openstack-swift | 16:11 | |
| rledisez | ganders: try to stop all the account/container/object auditor and replicator (just temporarily!). if it helps, tune their interval and speed to slow them down a bit | 16:11 |
| *** bkopilov has quit IRC | 16:16 | |
| *** hseipp has quit IRC | 16:23 | |
| ganders | thanks rledisez, i'll try that out | 16:25 |
| *** sgundur_ has joined #openstack-swift | 16:25 | |
| *** ganders has quit IRC | 16:27 | |
| *** bkopilov has joined #openstack-swift | 16:28 | |
| *** sgundur_ has quit IRC | 16:32 | |
| *** chsc has joined #openstack-swift | 16:35 | |
| *** chsc has joined #openstack-swift | 16:35 | |
| openstackgerrit | Janie Richling proposed openstack/swift: Symlink API documentation https://review.openstack.org/415943 | 16:39 |
| acoles | jrichli: re you failing api ref test - the yaml definitions need to be alphabetical | 16:40 |
| jrichli | acoles: yes, i think ill get it right this time. | 16:40 |
| acoles | jrichli: oh, did you just fix that? :) it's a pain, burnt me too a while back. | 16:41 |
| jrichli | we shall see if is really *fixed* ;-) | 16:41 |
| jrichli | once i know it builds successfully, i will squash | 16:42 |
| *** winggundamth has joined #openstack-swift | 16:47 | |
| notmyname | tdasilva: you're going to change your time zone? http://turnto10.com/ri-mass-lawmakers-propose-changing-time-zones | 16:47 |
| *** silor has joined #openstack-swift | 16:48 | |
| tdasilva | notmyname: i do hate the fact that (especially in december) it is pitch black dark at 4m | 16:49 |
| tdasilva | 4pm | 16:49 |
| *** sgundur_ has joined #openstack-swift | 16:50 | |
| notmyname | yeah, that doesn't sound good. | 16:50 |
| notmyname | but what about summer? light until mignith? | 16:51 |
| notmyname | what was that word? *midnight | 16:51 |
| tdasilva | heh | 16:51 |
| tdasilva | summer gets dark around 9pm | 16:51 |
| notmyname | acoles: where does a "yaml definition must be alphabetical" get enforced? | 16:53 |
| *** caiobrentano_ has joined #openstack-swift | 16:54 | |
| notmyname | is that a linter a la pep8 that's in an openstack project somewhere? is that enforced by sphinx/whatever? | 16:55 |
| acoles | notmyname: not sure, sorry. well in tox api-ref env but not sure what the root of it is. | 16:56 |
| *** acoles is now known as acoles_ | 16:57 | |
| notmyname | ok. no worries | 16:57 |
| notmyname | just seems like a wierd constraint | 16:57 |
| *** cloudm2 has quit IRC | 17:03 | |
| *** caiobrentano_ has left #openstack-swift | 17:08 | |
| *** caiobrentano_ has joined #openstack-swift | 17:08 | |
| *** ganders has joined #openstack-swift | 17:09 | |
| *** briancurtin has quit IRC | 17:11 | |
| *** Shashikant86 has quit IRC | 17:12 | |
| *** pcaruana has quit IRC | 17:12 | |
| *** tqtran has joined #openstack-swift | 17:16 | |
| dims | acoles_ : looks like it's from os-api-ref https://github.com/openstack/os-api-ref/blame/master/os_api_ref/__init__.py#L281 | 17:18 |
| *** tesseract has quit IRC | 17:20 | |
| *** klrmn has quit IRC | 17:26 | |
| *** asettle has quit IRC | 17:26 | |
| timburke | so the only way (currently) to validate it is to try building docs :-/ | 17:31 |
| notmyname | timburke: but that's possible locally, right? I mean, "try building docs" is pretty much the way any docs patch is validated, right? | 17:32 |
| *** rledisez has quit IRC | 17:32 | |
| timburke | yeah, i just appreciate how much faster `flake8 <file i actually editted>` is than `tox -e pep8` | 17:33 |
| *** dmorita has joined #openstack-swift | 17:42 | |
| *** cbartz has quit IRC | 17:42 | |
| clayg | morning! | 17:43 |
| clayg | kota_: I can't remember what test failed that prompted me to add that mkdirs calls - I agree it looks stupid - probably worth further investigation. | 17:44 |
| clayg | kota_: did you try to pull it out and see what failed? I sort of remember it prompted me to "un-tpool" the _get_hashes call so I could get a better traceback | 17:44 |
| *** dmorita has quit IRC | 17:47 | |
| *** dmorita has joined #openstack-swift | 17:49 | |
| *** jordanP has quit IRC | 17:50 | |
| clayg | ganders: how many parts per device? one thing that *loves* to eat up the iops is lots and lots of small files in lots and lots of sprase dirs. | 17:55 |
| clayg | well i guess that's two things | 17:55 |
| *** hseipp has joined #openstack-swift | 17:55 | |
| ganders | clayg: one part per device | 17:56 |
| clayg | there's some games you can play with memory and filesystem inode caching tunings | 17:56 |
| clayg | ganders: i'm sorry - what what? | 17:56 |
| tdasilva | fs partition != swift parts | 17:56 |
| clayg | maybe one unit of weight? like if you ls /srv/node/d1/objects there's only a single integer dir in there? | 17:56 |
| clayg | tdasilva: lol - omg thank you - i'm such a swift snob :'( | 17:56 |
| ganders | sorry my bad, i mean os part per dev | 17:56 |
| clayg | ganders: I like to just put the filesystem right on the block device! mbr's are a tool for people that don't eat harddrives for breakfast ;) | 17:57 |
| ganders | 547 parts | 17:58 |
| clayg | that's not bad! | 17:58 |
| clayg | sorta good acctually :\ | 17:58 |
| clayg | there's some /proc stats I think you can start to monitor that have to do with xfs inode reads and cahce hit rates | 17:58 |
| *** mvk has joined #openstack-swift | 17:59 | |
| clayg | mostly its the little file system updates/reads that chew on the iops - i mean disks obviously *can* get throughput saturated - but it's not the common case when you're bumping into utilization | 17:59 |
| clayg | how well distributed is the access patterns is 5.5M GET ops in a day for 5.5M different objects? | 18:00 |
| clayg | or is the working set like <1M or something? | 18:00 |
| ganders | <2M | 18:00 |
| clayg | still a lot - I think there's some caching options you can play with | 18:01 |
| ganders | do you have some detail info on those cache opts | 18:01 |
| clayg | notmyname: what is the one stupidly named option on the object server's that's like "dont_drop_unauthenticated_cahes_unless_they_are_big_or_not_maybe_what_does_this_do" option? | 18:01 |
| clayg | keep_cache_private defaults to false - changing it to true does "something" | 18:02 |
| *** JimCheung has joined #openstack-swift | 18:02 | |
| notmyname | clayg: is it the https://github.com/openstack/swift/blob/master/etc/object-server.conf-sample#L104-L109 stuff? | 18:02 |
| *** tqtran has quit IRC | 18:03 | |
| ganders | thnks!, i will take a look on that | 18:03 |
| clayg | default keep_cache_size is 5M - if your objects are 5.1M on average that might be annoying | 18:04 |
| patchbot | Error: 'supybot.keep_cache_size' is not a valid configuration variable. | 18:04 |
| ganders | big question..those opts.. keep_cache_size and keep_cache_private work on juno? | 18:04 |
| clayg | OTOH if you have a bunch of requests for objects 10K and a few for objects 4.9M - that might be annoying too? | 18:04 |
| openstackgerrit | Merged openstack/swift: Update the tag for Swift tox test https://review.openstack.org/410560 | 18:04 |
| clayg | what is juno? did they have indoor plumbing back then? | 18:05 |
| notmyname | FWIW juno = swift 2.2 | 18:05 |
| notmyname | 2 years ago | 18:05 |
| clayg | oh yeah, swift 2.2 is great | 18:05 |
| clayg | not as good as swift 2.12 - but still - pretty good | 18:05 |
| clayg | better than swift 1.13!? you remember that!? lol | 18:05 |
| ganders | yeah sorry guys.. update procs here are not very common lol... | 18:06 |
| clayg | ganders: doesn't matter swift 2.13 will be the best one yet - you can just wait and upgrade to that | 18:07 |
| *** chosafine has quit IRC | 18:07 | |
| clayg | does anyone here use vsaio? of that set of people does anyone like docs? of that set of people does anyone know api-ref docs? of that set of people (is there anyone left?) does anyone have time to consider -> https://github.com/swiftstack/vagrant-swift-all-in-one/issues/54 | 18:09 |
| clayg | notmyname: or did you say you were going to merge all that under the normal sphix tree anyway? | 18:10 |
| notmyname | clayg: https://review.openstack.org/#/c/386834/ <-- I need to update it | 18:11 |
| patchbot | patch 386834 - swift - Unify docs trees | 18:11 |
| clayg | notmyname: that looks *amazing* - can we *do* that? | 18:12 |
| clayg | like we don't *have* to have install-guide and api-ref TLD? | 18:13 |
| notmyname | well...seems like the CI isnt' set up to do it that way, so I don't know? | 18:13 |
| clayg | bummer | 18:13 |
| *** david-lyle has joined #openstack-swift | 18:16 | |
| *** hseipp has quit IRC | 18:16 | |
| tdasilva | clayg: hello, what do you mean by enforce that the etag of a symlink is always a zero-byte object? we currently validate that the sym is a zero-byte object, but by validating the content-length, not the etag | 18:18 |
| notmyname | but I think we should do it, so I need to look at what's actually blocking. IIRC mattoliverau had found the thing that was breaking so I'll look in to it | 18:18 |
| *** klrmn has joined #openstack-swift | 18:23 | |
| clayg | tdasilva: if i do a zero-byte PUT the etag is always ... d41d8cd98f00b204e9800998ecf8427e | 18:26 |
| clayg | it's probably NBD but like we could set an ETag header on all symlink put's even if the client didn't send it | 18:26 |
| clayg | but i suppose that's always true when content-length is 0 | 18:27 |
| clayg | i'm not really sure why I wrote it down | 18:27 |
| clayg | we were thinking about it when talking about container listings | 18:27 |
| tdasilva | clayg: right, and if the client tries to send somethign else an error is returned | 18:27 |
| clayg | < HTTP/1.1 422 Unprocessable Entity | 18:28 |
| tdasilva | yep | 18:28 |
| timburke | ganders: since i didn't notice a reply to your question: yeah, those options were available in 2.2.0 https://github.com/openstack/swift/blob/2.2.0/etc/object-server.conf-sample#L78-L83 | 18:28 |
| clayg | idk, it's w/e - just delete that story - or ask timburke he's pretty smart - maybe he can up with a reason to do ... something? | 18:28 |
| *** sgundur_ has quit IRC | 18:29 | |
| timburke | tdasilva: i saw the bit about checking for a content-length... if the client sends a transfer-encoding:chunked request, do we do anything to bail out when there's real content? forcing the etag would take care of it... though the error wouldn't be super helpful | 18:31 |
| tdasilva | timburke: good point, but couldn't we just not allow transfer-encoding:chunked request for symlink PUT request | 18:32 |
| timburke | tdasilva: and *if* we still wanted to support post-as-copy, i think we actually *could* start bypassing slo validation -- take a look at https://review.openstack.org/#/c/348495/ | 18:33 |
| patchbot | patch 348495 - swift - Make container sync copy SLO manifests (MERGED) | 18:33 |
| timburke | maybe? depends on how much control the user really has over their client. maybe it *always* does chunked transfers, even for zero-byte request bodies. i know i've run into it with swift3 and some s3 client before | 18:34 |
| *** chosafine has joined #openstack-swift | 18:35 | |
| *** chosafine has joined #openstack-swift | 18:35 | |
| *** chosafine has quit IRC | 18:37 | |
| tdasilva | timburke: was just checking dlo and copy, unfortunately it seems we are pretty inconsistent with how we validate our zero-byte objects | 18:43 |
| timburke | i don't believe we *do* validate that dlos are zero bytes. i seem to recall that you can turn *any* object into a dlo with a post | 18:44 |
| timburke | (maybe those are two separate issues?) | 18:44 |
| *** chosafine has joined #openstack-swift | 18:45 | |
| *** chosafine has joined #openstack-swift | 18:45 | |
| timburke | i'm all for making it a *hard requirement* that symlinks be zero bytes, though. if somebody can think of a reason to allow data in them, we can walk it back later | 18:45 |
| *** chlong has quit IRC | 18:46 | |
| tdasilva | timburke: yes on the dlo, i was just pointing out that I think the expecation is that a dlo manifest *should* be a zero byte object, it was just never enforced. But I'm not disagreeing with making it a hard requirement for syms, just trying to think of the best way to do it... | 18:50 |
| *** sgundur_ has joined #openstack-swift | 18:51 | |
| timburke | try reading a byte? either it comes back '' or we fail the request | 18:51 |
| timburke | idk | 18:51 |
| openstackgerrit | Richard Hawkins proposed openstack/swift: Refactoring the expiring objects feature https://review.openstack.org/252085 | 18:55 |
| *** tqtran has joined #openstack-swift | 18:57 | |
| openstackgerrit | Merged openstack/swift: Add test for object updater user-agent header https://review.openstack.org/398523 | 18:58 |
| *** chlong has joined #openstack-swift | 18:58 | |
| *** dmorita has quit IRC | 18:59 | |
| *** manij has quit IRC | 19:02 | |
| openstackgerrit | Richard Hawkins proposed openstack/swift: Refactoring the expiring objects feature https://review.openstack.org/252085 | 19:03 |
| *** briancurtin has joined #openstack-swift | 19:09 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 19:10 |
| *** dmorita has joined #openstack-swift | 19:14 | |
| *** ChubYann has joined #openstack-swift | 19:16 | |
| *** dmorita has quit IRC | 19:19 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Move documented reclaim_age option to correct location https://review.openstack.org/374419 | 19:25 |
| *** dmorita has joined #openstack-swift | 19:26 | |
| *** chosafine has quit IRC | 19:28 | |
| openstackgerrit | Tim Burke proposed openstack/swift: Tighten the move-one-replica test https://review.openstack.org/419107 | 19:29 |
| openstackgerrit | Gábor Antal proposed openstack/swift: Reduced complexity of _response_iter() method https://review.openstack.org/247560 | 19:30 |
| openstackgerrit | Tim Burke proposed openstack/swift: Move documented reclaim_age option to correct location https://review.openstack.org/374419 | 19:33 |
| timburke | clayg: fwiw, i cleaned up that break in https://review.openstack.org/#/c/419107/, where i also realized we only need one more rebalance | 19:38 |
| patchbot | patch 419107 - swift - Tighten the move-one-replica test | 19:38 |
| clayg | timburke: that parts_with_moved_count test fails when you add/fix the breaks? | 19:42 |
| timburke | which? is it over in test_ring.py? :-/ | 19:43 |
| clayg | patch 419107 | 19:43 |
| patchbot | https://review.openstack.org/#/c/419107/ - swift - Tighten the move-one-replica test | 19:43 |
| clayg | no, the test change the break cleanup are unrelated? | 19:44 |
| timburke | that. it's a general follow-up patch; i probably should have made that more clear | 19:44 |
| ganders | i change the opts on object-server.conf for keep_cache_size and keep_cache_private to true... is there any log or counter to see if this really helps out? I've already restart the object-server daemons but disks are still at 100% | 19:52 |
| openstackgerrit | Janie Richling proposed openstack/swift: Symlink API documentation https://review.openstack.org/415943 | 19:53 |
| *** joeljwright has joined #openstack-swift | 19:53 | |
| *** ChanServ sets mode: +v joeljwright | 19:53 | |
| jrichli | ok, sphinx builds with this now locally, so it should do the trick now. ^^ | 19:53 |
| *** ganders has quit IRC | 19:56 | |
| *** sgundur_ has quit IRC | 20:09 | |
| clayg | i guess ganders had to bolt :\ | 20:12 |
| clayg | who do I gripe at these days about EC bugs with? just acoles & kota? | 20:13 |
| jrichli | I am interested in learning more about EC, if that counts :-). Seems like I remember sgundur saying the same | 20:14 |
| clayg | yeah seems like folks love them some ec! | 20:15 |
| clayg | i just sort of remember when we were fighting the ssync protocol early disconnect fragment metadata corruption bug someone pointed out how EC dark data has this weird property of generating un-reconstructable forever "durable" fragments that we never clean up | 20:17 |
| clayg | ... then i see lp bug #1655608 | 20:17 |
| openstack | Launchpad bug 1655608 in OpenStack Object Storage (swift) "reconstructor tries to reconstruct a deleted object from an orphan fragment" [Undecided,New] https://launchpad.net/bugs/1655608 | 20:17 |
| clayg | so i guess we said it and never wrote it down :\ | 20:18 |
| clayg | make it hard to keep track of what we're supposed to be working on! | 20:18 |
| clayg | tdasilva: lol @ "just send the zero-byte etag" -> lp bug #1516579 | 20:20 |
| openstack | Launchpad bug 1516579 in OpenStack Object Storage (swift) "PUTing a 0-byte object and supplying an Etag results in a 503 response" [High,Fix committed] https://launchpad.net/bugs/1516579 - Assigned to Samuel Merritt (torgomatic) | 20:20 |
| clayg | at least it's fixed now ;) | 20:20 |
| *** sgundur_ has joined #openstack-swift | 20:26 | |
| *** asettle has joined #openstack-swift | 20:29 | |
| *** asettle has quit IRC | 20:33 | |
| *** asettle has joined #openstack-swift | 20:33 | |
| openstackgerrit | Merged openstack/swift: For any part, only one replica can move in a rebalance https://review.openstack.org/346475 | 20:34 |
| *** asettle has quit IRC | 20:38 | |
| mattoliverau | Morning | 20:45 |
| joeljwright | morning | 20:46 |
| *** bkeller` has joined #openstack-swift | 20:50 | |
| *** m_kazuhiro has joined #openstack-swift | 20:52 | |
| *** silor has quit IRC | 20:52 | |
| *** arch-nemesis has joined #openstack-swift | 20:54 | |
| clayg | man I am super pumped about that min_part_hours = 0 fix!? | 20:54 |
| clayg | cc swifterdarrell torgomatic ^ look no more fighting with min_part_hours after full replication dispersion cycle! | 20:55 |
| openstackgerrit | Tim Burke proposed openstack/swift: Tighten the move-one-replica test https://review.openstack.org/419107 | 20:56 |
| * clayg has a few clusters that are sorta small and a small gradual weight change can finish quickly :D | 20:58 | |
| cschwede | whoa, that's great that this patch merged! | 20:59 |
| notmyname | swift team meeting time in #openstack-meeting | 20:59 |
| *** acoles_ is now known as acoles | 21:00 | |
| kota_ | morning | 21:01 |
| *** jamielennox|away is now known as jamielennox | 21:01 | |
| openstackgerrit | Janie Richling proposed openstack/swift: Symlink implementation. https://review.openstack.org/232162 | 21:03 |
| acoles | clayg: I assumed trhat was going to be a duplicate bug report. did we not write one after/in Barcelona? | 21:06 |
| cschwede | notmyname: thx for starting at the bottom in the swift meeting b/c of timezones - appreciated! | 21:06 |
| acoles | clayg: i guess not | 21:06 |
| clayg | is it wednesday? | 21:07 |
| clayg | did I already miss the meeting? | 21:07 |
| notmyname | indeed it is | 21:07 |
| mattoliverau | its going now | 21:08 |
| notmyname | just talking about you in there | 21:08 |
| acoles | onovy: pls give our congrats to Pavel :) | 21:14 |
| onovy | acoles: yep :) | 21:15 |
| *** winggundamth has quit IRC | 21:19 | |
| jrichli | I need the last patchset to patch 232162 to be removed. Anyone know how to do that? | 21:19 |
| patchbot | https://review.openstack.org/#/c/232162/ - swift - Symlink implementation. | 21:19 |
| *** asettle has joined #openstack-swift | 21:21 | |
| timburke | jrichli: just drop it back to 44? i think something like `git review -d 232162,44 && git review` should work | 21:22 |
| timburke | it'll still be another patchset, but w/e | 21:23 |
| jrichli | ok. i just wondered if we could just blow it away altogether for a cleaner history, but thats ok | 21:23 |
| acoles | clayg: isn't the race...X sees no hashes.pkl so doesn't bother writing the invalidation, while Y was about to write hashes.pkl? | 21:26 |
| clayg | acoles: +100 simple obvious LBYL race - needs to be restructored - never abandon the log - the log is rule! | 21:28 |
| clayg | similarlly to patch 219165 - I want to do some reviews in the next few days - as always I'm particuarlly interested in patches that have been open for a long time and are ready to merge | 21:34 |
| patchbot | https://review.openstack.org/#/c/219165/ - swift - EC Fragment Duplication - Foundational Global EC C... | 21:34 |
| clayg | tdasilva: timburke: acoles: cschwede: tdasilva: jrichli ^ anything else come to mind? | 21:34 |
| *** Jeffrey4l_ has joined #openstack-swift | 21:34 | |
| *** asettle has quit IRC | 21:34 | |
| *** Jeffrey4l has quit IRC | 21:35 | |
| tdasilva | clayg: the part power increase patch from cschwede would be great | 21:35 |
| notmyname | m_kazuhiro: ok, just give me your trello id and I'll add you | 21:36 |
| tdasilva | clayg: i left some questions for you on some of the trello cards, if you have a chance, can you take a look? | 21:37 |
| m_kazuhiro | notmyname: thank you. I'm just making my trello id. | 21:37 |
| timburke | clayg: https://review.openstack.org/#/c/299225/ | 21:37 |
| patchbot | patch 299225 - swift - Treat invalid limit parameters as errors | 21:37 |
| tdasilva | clayg, timburke: on the override-etag, i started looking and it seems like it would require a change to support adding params to that header? I was wondering if it could be done separately from the symlink patch? and if someone could help me? | 21:38 |
| timburke | or https://review.openstack.org/#/c/390901/ | 21:38 |
| patchbot | patch 390901 - swift - Confirm receipt of SLO PUT with etag | 21:39 |
| timburke | tdasilva: it would? i thought it just sent whatever the value was. i'll look into it a bit | 21:40 |
| tdasilva | timburke: maybe i'm mistaken, but i was comparing to the content-type header and how it needs to parse the swift_bytes param | 21:40 |
| tdasilva | ? | 21:41 |
| m_kazuhiro | notmyname: I finished to making my id. My trello id is 'm_kazuhiro'. | 21:41 |
| tdasilva | timburke: i might not be understanding well what needs to be done. | 21:41 |
| notmyname | m_kazuhiro: great. I added you | 21:41 |
| m_kazuhiro | notmyname: thank you. I confirmed that I'm in openstack swift trello team. | 21:44 |
| timburke | tdasilva: as i recall, it's only a limitation on content-type, as enforced at https://github.com/openstack/swift/blob/@{2017-01-11}/swift/proxy/controllers/obj.py#L78-L85 | 21:46 |
| timburke | bah, maybe should've %-encoded that | 21:47 |
| timburke | https://github.com/openstack/swift/blob/@%7B2017-01-11%7D/swift/proxy/controllers/obj.py#L78-L85 | 21:47 |
| *** vint_bra has quit IRC | 21:52 | |
| clayg | onovy: there's not like some debuild magic I don't know about with the "package locking" - it's just like "python-pyeclib Requries liberasurecode == x.y.z" in it's control | 21:54 |
| onovy | it's just == x.y.z | 21:54 |
| onovy | nothing more | 21:54 |
| onovy | so i will build two packages | 21:54 |
| onovy | liberasurecode 1.0.0 | 21:55 |
| clayg | and then when you want to package a new liberasure code you check in updates to liberasurecode's deb source, and package that; then check in changes to pyeclib's deb source and package that ... | 21:55 |
| onovy | and python-pyeclib with depends on liberasurecode=1.0.0 | 21:55 |
| onovy | i will build this two packages at same time | 21:55 |
| onovy | from one source | 21:55 |
| onovy | because you will have only one repo | 21:55 |
| onovy | it's simple :) i want to | 21:55 |
| onovy | 1. offer liberasurecode lib for C | 21:56 |
| onovy | 2. offer python wrapper for Python world | 21:56 |
| onovy | you want: | 21:56 |
| onovy | 1. have one repo | 21:56 |
| onovy | 2. release both at same time | 21:56 |
| onovy | so i will have | 21:56 |
| onovy | 1. one source package | 21:56 |
| onovy | 2. two binary package | 21:56 |
| timburke | joeljwright: that's why i was thinking etag, since the symlink target must be set on PUT -- shouldn't have the same issues that come up with tweaking content-type | 21:56 |
| onovy | but i don't want to have duplicity in liberasure and python wrapper binary package | 21:57 |
| onovy | SO for me it's ideal to dynamic link python wrapper to liberasurecode C lib | 21:57 |
| onovy | and i will add deps which say: liberasurecode must be same version and python wrapper | 21:57 |
| onovy | because someone will add 3. offer go wrapper for GoLang world | 21:58 |
| clayg | onovy: why do you want liberasurecode c library - who wants that? who uses that? how can they!? it's so volitle still... | 21:58 |
| onovy | and that go wrapper will use liberasurecode too | 21:58 |
| mattoliverau | timburke: we could use the deleted field, it's an int, and we are only using 0 and 1. So a link could be link a 2.. although it would then be good if the field was called something like status rather then deleted. But that would mean no extra storage and miminal changes to code | 21:58 |
| timburke | onovy: ...and that will become even more important for you when we have go bindings, yeah? | 21:58 |
| onovy | timburke: yep | 21:58 |
| onovy | and another lang binding and another... | 21:58 |
| onovy | clayg: i don't think anybody is using it NOW | 21:58 |
| notmyname | clayg: IIRC others have been interested in using liberasurecode. ie any project that "just" uses C. IIRC, even ceph (c++) was looking at using it | 21:58 |
| clayg | good - let's never let them - it just sounds like more work for us - let the fork | 21:59 |
| timburke | mattoliverau: that wasn't all i was thinking of storing :-) may as well put the whole target in there | 21:59 |
| notmyname | clayg: or at least that was talked about | 21:59 |
| onovy | liberasurecode1 | 21:59 |
| onovy | Reverse Depends: liberasurecode-dev (= 1.4.0-1) | 21:59 |
| onovy | Reverse Depends: python-pyeclib (>= 1.4.0-2) | 21:59 |
| onovy | Reverse Depends: python3-pyeclib (>= 1.4.0-2) | 21:59 |
| clayg | Reverse Depends - interesting | 21:59 |
| mattoliverau | timburke: oh right, like an ls -l shows where it's pointing to | 21:59 |
| onovy | so in your solution there will be duplicity between py2 and py3 wrapper | 21:59 |
| onovy | in my there will be almost no duplicty | 21:59 |
| onovy | clayg: no, it's not work for you | 22:00 |
| onovy | ABI is volatile, Torvalds says that. | 22:00 |
| onovy | :) | 22:00 |
| clayg | the only that appears to get simpler if we just compile the binary bits into the python binding .so (instead of a dynamic handle to some shared library installed seperately) | 22:00 |
| onovy | so just break liberasurecode ABI, don't worry about it | 22:00 |
| mattoliverau | timburke: because you don't know the size of the link string, should you use some sysmeta or are you thinking of using etag field (seeing as it should always be a zero-byte file) | 22:00 |
| clayg | is that we can bump our requirements in the gate anytime we want - or in dev - and developers don't have to think about `sudo make install` | 22:01 |
| timburke | onovy: won't there still be the duplicity of *all the python files*? it's unavoidable for py vs. go, but our py2 & py3 code should be *the same code* | 22:01 |
| onovy | clayg: so if i install py2+py3+go wrapper i will get 3 copy of same code | 22:01 |
| clayg | onovy: wfm! | 22:01 |
| onovy | timburke: yep, that's "how it works now for py2+py3" :] | 22:01 |
| patchbot | Error: Spurious "]". You may want to quote your arguments with double quotes in order to prevent extra brackets from being evaluated as nested commands. | 22:01 |
| mattoliverau | i.e if delete == 2, etag == link string. | 22:01 |
| onovy | clayg: i think ideal is to have statically linked liberasurecode for pip and dynamical for distros | 22:02 |
| openstackgerrit | Janie Richling proposed openstack/swift: Symlink implementation. https://review.openstack.org/232162 | 22:02 |
| timburke | mattoliverau: ugh. i think i still prefer the idea of etag == <actual etag>;swift_symlink_target=a/c/o | 22:02 |
| clayg | onovy: you just blew my mind | 22:02 |
| onovy | :) | 22:03 |
| joeljwright | timburke: I see why... that makes some sort of sense | 22:03 |
| timburke | onovy: i can get behind that. not sure how to make it happen though :-( guess i need to learn more about python c extensions | 22:03 |
| onovy | timburke: i don't know too | 22:03 |
| onovy | but it should be possible | 22:03 |
| clayg | onovy: also the asnwer might be more obvious if some interested party tried to write golang bindings for libec - I know golang has previoulsy had some opinons on static linking .... | 22:03 |
| timburke | onovy: that's why i ended hopeful :-) | 22:04 |
| mattoliverau | timburke: etag of the symlink I guess? I see, that will be confusing for json/xml listing and existing clients, but I guess anyway you swing it, it'll be hard for em. | 22:04 |
| timburke | joeljwright: mattoliverau: and if need be, we can have the container server pull it out (like we already do for swift_bytes) and stuff it in a separate key | 22:04 |
| clayg | onovy: thanks again for your input on this | 22:04 |
| onovy | clayg: don't know how it's working inside golang sry | 22:04 |
| mattoliverau | ahh | 22:04 |
| *** sgundur_ has quit IRC | 22:04 | |
| onovy | clayg: ok, let's summarize it? | 22:05 |
| *** sgundur_ has joined #openstack-swift | 22:05 | |
| *** sams-gleb has quit IRC | 22:05 | |
| onovy | you want package for pip, without binary deps. install it anywhere, ... | 22:05 |
| tdasilva | timburke: i need to run, but will try to catch up with you tomorrow | 22:05 |
| onovy | so liberasurecode should be embedded for this king of distributions | 22:05 |
| *** sams-gleb has joined #openstack-swift | 22:06 | |
| onovy | *kind | 22:06 |
| onovy | embedded=statically linked inside python_c binding | 22:06 |
| onovy | this is how it was | 22:07 |
| onovy | liberasurecode was embbeded at pypi | 22:07 |
| clayg | how it ... was? | 22:07 |
| onovy | in old version, if i remember correctly | 22:07 |
| acoles | timburke: (i'm playing catch up...) if we embed the symlink target in container listing, are we sure it will never, ever change with a POST? | 22:08 |
| clayg | onovy: in old version it was *insane* - it was still dynamic AIUI - but like setup.py would randomly just start connecting to the internet and downloading things and running commands if the headers weren't around? | 22:08 |
| onovy | right | 22:09 |
| kota_ | onovy: maybe not statically linking but exactly keep the liberasurecode tar ball inside in pyeclib repo. | 22:09 |
| onovy | that's black magic we don't want | 22:09 |
| onovy | kota_: yep. but you don't want to break locally installed liberasurecode (from distros), so static link is better imho | 22:09 |
| kota_ | onovy: yup, *in the future* | 22:10 |
| *** sams-gleb has quit IRC | 22:10 | |
| onovy | so let's build static linked python_c with liberasurecode when installed from pypi | 22:10 |
| onovy | that's fine | 22:10 |
| onovy | but for distros, use dynamic linking and offer liberasurecode for C devs | 22:11 |
| onovy | so let's do it same way as now for distros :) | 22:12 |
| onovy | just imagine liberasurecode will be ~10MB library | 22:12 |
| onovy | i don't want to have it twice, one for py2 and one for py3 | 22:12 |
| onovy | and again for go, and again for PHP, and ... | 22:12 |
| *** sgundur- has joined #openstack-swift | 22:13 | |
| kota_ | sounds reasonable to me | 22:14 |
| timburke | acoles: we'll already be enforcing that it be a zero-byte object, so requiring a PUT to change targets shouldn't be onerous on clients -- there's no data to move either way. we already have the target recorded in *sysmeta*; if we start letting *that* change on POST, it seems like we'll have a whole host of other things breaking (or potentially breaking). | 22:14 |
| timburke | seems like the only hangup would be our own resolve to not subsequently allow it to be stuffed into transient-sysmeta or something, which we should never do because we'll already have data on-disk in non-transient-sysmeta | 22:14 |
| onovy | if it's possible (simple) | 22:14 |
| kota_ | yes, if it's possible | 22:14 |
| onovy | i can patch it in packing, that's possible too, but i don't prefer this solution | 22:15 |
| onovy | because there is more distros, not only debian packages :] | 22:15 |
| patchbot | Error: Spurious "]". You may want to quote your arguments with double quotes in order to prevent extra brackets from being evaluated as nested commands. | 22:15 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 22:15 |
| onovy | patchbot: blah, ignore me pls! | 22:15 |
| onovy | btw: and stable/unstable ABI. Every upstream breaks ABI (really) often. Distros knows it, and have are using ABI bump for it | 22:16 |
| onovy | and we are... | 22:16 |
| timburke | onovy: ugh. is it really so big? i really hope not... | 22:17 |
| onovy | ideally upstream indicate this breaking ABI change, for example with major version bump. But every packager is checking this | 22:17 |
| onovy | timburke: "so big"? you mean often ABI bumps? | 22:17 |
| timburke | no no :-) the 10MB. i understand that ABI bumps are a big deal for dynamically linked libraries | 22:18 |
| onovy | timburke: just __imagine__ :] | 22:18 |
| patchbot | Error: Spurious "]". You may want to quote your arguments with double quotes in order to prevent extra brackets from being evaluated as nested commands. | 22:18 |
| onovy | -rw-r--r-- 1 onovy onovy 56K Dec 29 14:31 liberasurecode.so.1.4.0 | 22:19 |
| onovy | ~ 112K all of them | 22:19 |
| timburke | yeah; still not exactly trivial. but if we ever let this thing balloon to the MB range i'll be very mad :-) | 22:20 |
| onovy | it's not about size now, but about principle how distros is doing this | 22:20 |
| onovy | for example we are removing embedded jquery and replacing it with deps | 22:21 |
| * kota_ is away sorry, will take a look the log about packagin discussion for pyeclib/liberasurecode | 22:22 | |
| onovy | to be honest, i would like to go to bed | 22:23 |
| *** m_kazuhiro has quit IRC | 22:23 | |
| timburke | do it! i think we all have a better understanding of what should be done and why because of your helpful explanations :-) | 22:24 |
| onovy | thanks :) let's continue later. gn | 22:25 |
| notmyname | onovy: thanks, as always, for helping out :-) | 22:25 |
| *** sgundur_ has left #openstack-swift | 22:28 | |
| *** bkeller` has quit IRC | 22:32 | |
| *** geaaru has quit IRC | 22:34 | |
| acoles | timburke: ah, target link is in sysmeta, of course. thank you , I will now sleep easier :) | 22:48 |
| *** acoles is now known as acoles_ | 22:49 | |
| *** catintheroof has quit IRC | 22:50 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Fix race in new partitions detecting new/invalid suffixes. https://review.openstack.org/418690 | 22:51 |
| *** catintheroof has joined #openstack-swift | 22:51 | |
| *** chlong has quit IRC | 22:55 | |
| *** catintheroof has quit IRC | 22:55 | |
| *** joeljwright has quit IRC | 22:59 | |
| *** tqtran has quit IRC | 23:17 | |
| *** arch-nemesis has quit IRC | 23:22 | |
| *** tqtran has joined #openstack-swift | 23:29 | |
| *** klamath has quit IRC | 23:35 | |
| *** chsc has quit IRC | 23:35 | |
| *** vint_bra has joined #openstack-swift | 23:50 | |
| *** dpak has joined #openstack-swift | 23:56 | |
| *** kei_yama has joined #openstack-swift | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!