| torgomatic | notmyname: fragment size is how big a fragment is (yes, I know)... e.g. if 10+4 coding, and 1MB segments, then fragment_size is roughly 100000 bytes plus ~32 header bytes so 100032 | 00:01 |
|---|---|---|
| torgomatic | seg_size / ec_ndata + fragment headers | 00:01 |
| notmyname | ah good. thanks | 00:01 |
| notmyname | so, importantly, segment_size is bigger than fragment_size | 00:02 |
| notmyname | but they don't have any particular integer multiple relationship | 00:03 |
| *** ho has quit IRC | 00:04 | |
| *** ho has joined #openstack-swift | 00:06 | |
| notmyname | torgomatic: this doesn't look right to me. https://gist.github.com/notmyname/ed7c81e6290fcc337d79 | 00:08 |
| notmyname | fragment_size = 200, segment_size = 1000 | 00:08 |
| torgomatic | notmyname: so, what looks wrong about it? | 00:08 |
| notmyname | I think it's my understanding :-) | 00:09 |
| notmyname | to the fragment start/end is what's then passed down as the translated range request | 00:09 |
| torgomatic | I mean, the fact that segment size is a multiple of fragment size is sort of surprising; are you making those numbers up, or do they come from pyeclib? | 00:09 |
| notmyname | so how does 0, 199 have anything to do with 790,810 | 00:09 |
| notmyname | totally making them up | 00:09 |
| torgomatic | oh, ok | 00:09 |
| notmyname | I'm trying to understand the math in a REPL with numbers smaller than MB | 00:10 |
| torgomatic | do you see how they turn into a segment start/end? | 00:10 |
| torgomatic | that is, does client -> segment make sense? | 00:10 |
| notmyname | yes I think so. so that's the start and end ranges for the segment that the range is in | 00:11 |
| notmyname | right? | 00:11 |
| torgomatic | yup | 00:11 |
| torgomatic | and then a segment is bigger than a fragment, so we scale those ends down by the ratio segsize/fragsize, and there you go | 00:11 |
| notmyname | but how do the 0,199 have to do with the 20 bytes requested? | 00:12 |
| notmyname | (21) | 00:12 |
| torgomatic | we have to decode entire fragments and get entire segments | 00:13 |
| torgomatic | er, "to get" | 00:13 |
| torgomatic | and then trim the resulting segments to fit what the client asked for | 00:13 |
| torgomatic | if we try to go grabbing partial fragments, pyeclib complains mightily | 00:14 |
| notmyname | ok | 00:14 |
| notmyname | I get that | 00:14 |
| notmyname | and since the fragment size is 200, that makes sense that it's 200 bytes | 00:14 |
| notmyname | but that's not translated to an offset of which fragment | 00:15 |
| torgomatic | I don't understand that last statement | 00:15 |
| notmyname | like, I'd expect it to be the range of bytes in the segment to fetch. | 00:15 |
| notmyname | basically, I have zero clue how setting req.range to the fragment start/end has anything to do with the bytes the client requested | 00:16 |
| notmyname | I mean, yes I see the math. I just don't know why | 00:16 |
| notmyname | why that math | 00:16 |
| torgomatic | so we start with the client start/end | 00:17 |
| torgomatic | and then we subtract some from client-start, and add some to client-end, so that they encompass only whole segments | 00:17 |
| torgomatic | those things are now called segment-start and segment-end | 00:17 |
| notmyname | ok, I've got that. the range is adjusted to be the whole segments | 00:18 |
| torgomatic | and then we scale those by fragsize/segsize to turn them into ranges encompassing whole fragments | 00:18 |
| notmyname | that's the part I don't get. or I don't understand the scaling | 00:18 |
| * notmyname wishes for a whiteboard | 00:18 | |
| notmyname | so here's what I'm expecting | 00:19 |
| notmyname | client range -> whole segments. eg if a boundary is crossed, the range will be for the whole segments that are spanned | 00:19 |
| torgomatic | yup | 00:19 |
| notmyname | then fragment range-> offset in the segment range of the fragments that have the client-requested ranges | 00:20 |
| notmyname | which is not at all what I'm seeing | 00:20 |
| torgomatic | so let's say you've got 5+0 coding, so just a 5-way stripe | 00:20 |
| notmyname | oh, and fragment range anchored (ie 0) at the segment start | 00:20 |
| notmyname | ok | 00:20 |
| torgomatic | you take your 3000-byte object and turn it into 1000, 1000, 1000 segments | 00:20 |
| notmyname | ok | 00:20 |
| torgomatic | then you take your 1000-byte segment and turn it into 200, 200, 200, 200, 200 fragments | 00:21 |
| torgomatic | then do it twice more | 00:21 |
| notmyname | ok | 00:21 |
| torgomatic | so 5 object servers have 600 bytes apiece | 00:21 |
| torgomatic | for a total of 3000 | 00:21 |
| torgomatic | so if someone wants bytes 1500-1600, you have to go get the middle segment | 00:21 |
| torgomatic | so that's bytes 1000-1999 | 00:21 |
| notmyname | right | 00:22 |
| torgomatic | and that's the middle third of each segment, so bytes 200-399 | 00:22 |
| torgomatic | and that's what goes to each object server | 00:22 |
| notmyname | ok. I think I get it now | 00:23 |
| notmyname | to quote myself, "so there's this thing called erasure coding. it's complicated" | 00:23 |
| torgomatic | yep, it is | 00:25 |
| torgomatic | anyhow, did you get a repeatable failure for a ranged GET? If you did, tell me and I'll see about making a unit test out of it | 00:26 |
| notmyname | so much in EC, including this, only makes sense when I'm visualizing it as a 2-dimensional array across the drives | 00:26 |
| notmyname | nah, I got the hangs on the functional tests and then went to the range code to figure that out. and here we are :-) | 00:26 |
| torgomatic | it's basically https://www.youtube.com/watch?v=qHQZcRPqUkY | 00:27 |
| notmyname | I was thinking more of https://www.youtube.com/watch?v=lVZ8Ko-nss4 | 00:28 |
| notmyname | gotta run. kid duty time | 00:29 |
| notmyname | torgomatic: thanks for the help | 00:30 |
| torgomatic | np | 00:30 |
| notmyname | I might tackle it again tonight. or more likely, tomorrow am in the office | 00:30 |
| clayg | whoa | 00:47 |
| clayg | shri: what version of swift? are they all from that same account hash? are the other primary nodes for that account having lock timeouts? | 00:49 |
| *** shri has quit IRC | 00:50 | |
| *** kota_ has joined #openstack-swift | 01:01 | |
| *** zhill has quit IRC | 01:26 | |
| *** dmorita has quit IRC | 01:26 | |
| *** jrichli has joined #openstack-swift | 01:32 | |
| *** zhill has joined #openstack-swift | 01:42 | |
| *** tsg has quit IRC | 01:50 | |
| *** kota_ has quit IRC | 02:21 | |
| *** zhill has quit IRC | 02:23 | |
| *** jamielennox is now known as jamielennox|lunc | 02:32 | |
| *** jamielennox|lunc is now known as jamielennox|food | 02:32 | |
| *** haomaiwang has joined #openstack-swift | 02:35 | |
| *** tsg has joined #openstack-swift | 02:55 | |
| *** greghaynes has quit IRC | 02:58 | |
| *** jamielennox|food is now known as jamielennox | 03:01 | |
| *** greghaynes has joined #openstack-swift | 03:06 | |
| *** jbonjean has quit IRC | 03:20 | |
| *** gyee has quit IRC | 03:41 | |
| *** jrichli has quit IRC | 03:43 | |
| *** jrichli has joined #openstack-swift | 03:45 | |
| clayg | we have some stupid crazy slow proxy.test_server.TestObjectController test methods | 04:03 |
| *** kbee has joined #openstack-swift | 04:04 | |
| *** ho has quit IRC | 04:08 | |
| clayg | hell yeah --with-timer! | 04:10 |
| clayg | wtf are you doing test.unit.proxy.test_mem_server.TestObjectController.test_version_manifest_utf8: 1.2433s !? | 04:10 |
| *** ppai has joined #openstack-swift | 04:25 | |
| mattoliverau | obviously utf8's fault :P | 04:37 |
| openstackgerrit | Janie Richling proposed openstack/swift: WIP - non-in-process functest mods to support different policies https://review.openstack.org/166097 | 04:38 |
| clayg | i think i've tried to make this test used pipelined requests before - i'm getting strange de-ja-vu | 04:40 |
| mattoliverau | clayg: or your just psycic :P | 04:41 |
| mattoliverau | *psychic | 04:43 |
| *** jrichli has quit IRC | 04:44 | |
| *** kota_ has joined #openstack-swift | 04:44 | |
| clayg | bah, stupid ... like who thought it was a good idea to write a bunch of HTTP by hand? | 04:49 |
| clayg | is it any surprise we got it wrong? | 04:49 |
| *** nonix4 has quit IRC | 04:50 | |
| *** chlong has quit IRC | 05:02 | |
| clayg | some of a -test.unit.proxy.test_mem_server.TestObjectController.test_version_manifest_utf8: 1.1549s | 05:02 |
| clayg | so all the tcp_connect's are not the problem - sign | 05:03 |
| clayg | sigh | 05:03 |
| clayg | fuck these tests!? test.unit.proxy.test_mem_server.TestObjectController.test_version_manifest_utf8: 1.1443s | 05:18 |
| clayg | i would have swore the read(1) in readuntil2crlfs was the problem! | 05:18 |
| *** chlong has joined #openstack-swift | 05:19 | |
| clayg | well, if it's not makefile i don't know what it is - test.unit.proxy.test_mem_server.TestObjectController.test_version_manifest_utf8: 1.1466s | 05:28 |
| *** tsg has quit IRC | 05:35 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Try to make real socket version tests run faster https://review.openstack.org/166104 | 05:38 |
| clayg | it helps - but not enough - i respectively bow out | 05:39 |
| mattoliverau | clayg: baby steps are better then backwards ones | 05:46 |
| *** reed has quit IRC | 05:55 | |
| *** echevemaster has joined #openstack-swift | 05:57 | |
| *** echevemaster has joined #openstack-swift | 05:57 | |
| *** chlong has quit IRC | 06:01 | |
| *** chlong has joined #openstack-swift | 06:13 | |
| *** fifieldt has joined #openstack-swift | 06:32 | |
| *** jamielennox is now known as jamielennox|away | 06:35 | |
| *** Bsony has quit IRC | 07:01 | |
| *** echevemaster has quit IRC | 07:03 | |
| *** fandi has joined #openstack-swift | 07:05 | |
| *** fandi has quit IRC | 07:06 | |
| *** fandi has joined #openstack-swift | 07:07 | |
| *** fandi has quit IRC | 07:10 | |
| *** fandi has joined #openstack-swift | 07:10 | |
| *** fandi has quit IRC | 07:13 | |
| *** fandi has joined #openstack-swift | 07:14 | |
| *** fandi has quit IRC | 07:17 | |
| *** fandi has joined #openstack-swift | 07:17 | |
| *** fandi has quit IRC | 07:21 | |
| *** fandi has joined #openstack-swift | 07:22 | |
| *** ppai has quit IRC | 07:24 | |
| *** fandi has quit IRC | 07:25 | |
| *** fandi has joined #openstack-swift | 07:26 | |
| *** fandi has quit IRC | 07:29 | |
| *** fandi has joined #openstack-swift | 07:30 | |
| *** silor has joined #openstack-swift | 07:30 | |
| *** echevemaster has joined #openstack-swift | 07:32 | |
| *** fandi has quit IRC | 07:33 | |
| *** fandi has joined #openstack-swift | 07:33 | |
| *** ppai has joined #openstack-swift | 07:35 | |
| *** fandi has quit IRC | 07:37 | |
| *** fandi has joined #openstack-swift | 07:37 | |
| *** fandi has quit IRC | 07:39 | |
| *** Bsony has joined #openstack-swift | 07:43 | |
| *** chlong has quit IRC | 07:45 | |
| *** jistr has joined #openstack-swift | 07:58 | |
| *** Trixboxer has joined #openstack-swift | 07:59 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Fix a lack of method arguments at tempauth https://review.openstack.org/166129 | 08:07 |
| *** geaaru has joined #openstack-swift | 08:18 | |
| *** openstackgerrit has quit IRC | 08:22 | |
| *** acoles_away is now known as acoles | 08:22 | |
| *** openstackgerrit has joined #openstack-swift | 08:22 | |
| *** mmcardle has joined #openstack-swift | 08:38 | |
| *** Bsony_ has joined #openstack-swift | 08:39 | |
| *** Bsony has quit IRC | 08:40 | |
| *** echevemaster has quit IRC | 08:51 | |
| *** mmcardle1 has joined #openstack-swift | 08:51 | |
| *** mmcardle has quit IRC | 08:52 | |
| *** joeljwright has joined #openstack-swift | 08:53 | |
| *** jordanP has joined #openstack-swift | 09:02 | |
| *** classics1 is now known as classicsnail | 09:10 | |
| *** jamielennox|away is now known as jamielennox | 09:14 | |
| *** ppai has quit IRC | 09:20 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Fix a lack of method arguments at tempauth https://review.openstack.org/166129 | 09:21 |
| *** ppai has joined #openstack-swift | 09:30 | |
| acoles | kota_: ^^ thanks! | 09:30 |
| *** mtreinish has quit IRC | 09:35 | |
| *** mtreinish has joined #openstack-swift | 09:36 | |
| kota_ | acoles: yeah, you are welcome :) | 09:52 |
| *** sileht has quit IRC | 10:07 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: wip: ec reconstructor probe test https://review.openstack.org/164291 | 10:13 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Erasure Code Reconstructor https://review.openstack.org/131872 | 10:13 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Multiple Fragment Archive support for suffix hashes https://review.openstack.org/159637 | 10:13 |
| clayg | alright we'll see what gerrit thinks of this mess | 10:14 |
| clayg | peluse: I think I've either broken some of the probetests - are they weren't quite working right to begin with :\ | 10:14 |
| *** joeljwright has left #openstack-swift | 10:14 | |
| clayg | peluse: we should skype about suffix hashes later | 10:14 |
| *** joeljwright has joined #openstack-swift | 10:19 | |
| *** sileht has joined #openstack-swift | 10:22 | |
| *** sluo_laptop has quit IRC | 10:26 | |
| *** kbee has quit IRC | 10:26 | |
| acoles | clayg! what you doing here at this time? | 10:34 |
| *** erlon has joined #openstack-swift | 10:42 | |
| *** silor has quit IRC | 10:44 | |
| *** ppai has quit IRC | 10:47 | |
| clayg | acoles: you're an ssync expert - have you given any thought to how we could ship just a .durable marker to a node that just failed right before the commit message? | 10:51 |
| acoles | clayG; you mean it has the .data but just missing .durable? | 10:51 |
| clayg | acoles: yes | 10:52 |
| acoles | clayg ^^ | 10:52 |
| acoles | clayg: not yet but i will give it some thought | 10:52 |
| clayg | thanks | 10:52 |
| *** ppai has joined #openstack-swift | 10:53 | |
| acoles | clayg: ugh. so ssync_receiver tries to open its diskfile to compare timestamp, that open will fail if there is no durable. sounds like another case where we need to be able to say 'no, please construct this diskfile, even if you don't have a durable' | 10:54 |
| clayg | ah... curious | 10:55 |
| acoles | clayg: ok i'm going to have to ponder that some more, cos we'll either need a way for the receiver to say 'i want something, but it ain't the whole data file', or for the receiver to 'infer' that it can unilaterally add a durable file | 10:57 |
| clayg | acoles: it should look a lot like what you did to teach ssync how to POST right? | 10:57 |
| acoles | clayg: yeah, i was just tapping that into keyboard ;) | 10:58 |
| acoles | clayg: well similar | 10:58 |
| clayg | bbiab | 10:58 |
| acoles | clayg: for fast-post work i modified the receiver return to be a tuple of hash and encoding of all the timestamps - .data, .meta etc - so adding explicit .durable timestamp in there somehow might work | 10:59 |
| acoles | clayg: i'd need to be able to encode a 'None' when durable is missing | 11:00 |
| acoles | clayg: then sender sends a new verb ('RUGGEDIZE' makes a comeback?) or a POST with .durable timestamp and some special header X-Make-Me-Durable?? | 11:05 |
| *** ppai has quit IRC | 11:19 | |
| *** kota_ has quit IRC | 11:28 | |
| *** ppai has joined #openstack-swift | 11:32 | |
| *** dencaval has joined #openstack-swift | 11:43 | |
| *** haomaiwang has quit IRC | 11:45 | |
| openstackgerrit | Merged openstack/swift: Fix a lack of method arguments at tempauth https://review.openstack.org/166129 | 11:48 |
| *** fifieldt has quit IRC | 11:54 | |
| *** fifieldt_ has joined #openstack-swift | 11:54 | |
| *** fifieldt__ has joined #openstack-swift | 11:58 | |
| *** fifieldt_ has quit IRC | 12:00 | |
| *** stockpirate has quit IRC | 12:05 | |
| *** ppai has quit IRC | 12:15 | |
| *** ppai has joined #openstack-swift | 12:25 | |
| *** ppai has quit IRC | 12:38 | |
| *** ppai has joined #openstack-swift | 12:52 | |
| *** panbalag has joined #openstack-swift | 13:01 | |
| *** jrichli has joined #openstack-swift | 13:31 | |
| openstackgerrit | Prashanth Pai proposed openstack/swift: Refactor server side copy as middleware https://review.openstack.org/156923 | 13:36 |
| *** jamielennox is now known as jamielennox|away | 13:36 | |
| openstackgerrit | Prashanth Pai proposed openstack/swift: Refactor server side copy as middleware https://review.openstack.org/156923 | 13:39 |
| *** acoles has left #openstack-swift | 13:51 | |
| *** acoles has joined #openstack-swift | 13:51 | |
| *** ChanServ sets mode: +v acoles | 13:51 | |
| *** mariusv has quit IRC | 13:51 | |
| *** mariusv has joined #openstack-swift | 13:53 | |
| *** mariusv has quit IRC | 13:53 | |
| *** mariusv has joined #openstack-swift | 13:53 | |
| *** raginbajin has joined #openstack-swift | 13:55 | |
| openstackgerrit | Janie Richling proposed openstack/swift: WIP - non-in-process functest mods to support different policies https://review.openstack.org/166097 | 14:01 |
| *** thumpba has joined #openstack-swift | 14:06 | |
| *** thumpba_ has joined #openstack-swift | 14:07 | |
| *** Gu_______ has joined #openstack-swift | 14:07 | |
| *** thumpba has quit IRC | 14:11 | |
| *** Gu_______ has quit IRC | 14:15 | |
| *** ppai has quit IRC | 14:15 | |
| *** vinsh has joined #openstack-swift | 14:22 | |
| *** vinsh has quit IRC | 14:23 | |
| *** ppai has joined #openstack-swift | 14:28 | |
| *** tsg_ has joined #openstack-swift | 14:28 | |
| *** ppai has quit IRC | 14:34 | |
| *** dencaval has quit IRC | 14:39 | |
| *** vinsh has joined #openstack-swift | 14:44 | |
| *** mahatic has joined #openstack-swift | 14:47 | |
| *** os1 has joined #openstack-swift | 14:48 | |
| *** vinsh has quit IRC | 14:48 | |
| acoles | peluse: hi, i'm adding cards to trello for future 'todo's', is that the right thing? is there a way to mark them as low priority? | 14:51 |
| *** a1|away is now known as AbyssOne | 14:53 | |
| notmyname | acoles: sounds good, thanks. tag them with some color. I don't think there's any color used yet except red for the gotta-be-in-beta items | 14:56 |
| *** dencaval has joined #openstack-swift | 14:59 | |
| peluse | correct, I will add a yellow color today for those things that are 'stretch goals' but not requirements so feel free to use that color if something is obviously in that category (or bring it up here if its not clear) | 15:01 |
| peluse | and no color, of course, means "future sometime" :) | 15:01 |
| acoles | peluse: notmyname ok | 15:02 |
| acoles | peluse: i prefer stuff on trello, comments in gerrit get easily lost | 15:03 |
| peluse | ditto | 15:03 |
| acoles | in other news, we had a partial solar eclipse this morning | 15:03 |
| acoles | but it was cloudy :( | 15:04 |
| *** reed has joined #openstack-swift | 15:06 | |
| *** openstackgerrit has quit IRC | 15:21 | |
| *** openstackgerrit has joined #openstack-swift | 15:22 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Add fragment index support to obj server and ssync https://review.openstack.org/165188 | 15:33 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Per-policy DiskFile classes https://review.openstack.org/165125 | 15:33 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Multiple Fragment Archive support for suffix hashes https://review.openstack.org/159637 | 15:33 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Diskfile decides if durable is written based on policy https://review.openstack.org/165208 | 15:33 |
| *** nellysmitt has joined #openstack-swift | 15:35 | |
| acoles | mattoliverau: ^^ hopefully addressed your comments on 165125, thanks for reviewing! | 15:40 |
| *** Nadeem has joined #openstack-swift | 15:48 | |
| *** welldannit has joined #openstack-swift | 15:55 | |
| *** GlennS has joined #openstack-swift | 15:59 | |
| GlennS | Hi, I'm wondering what would be the quickest way in Swift to delete a folder and all its sub-folders? I haven't found any documentation for this. (I appreciate folders are somehow not real in swift.) | 16:04 |
| *** mahatic has quit IRC | 16:05 | |
| *** silor has joined #openstack-swift | 16:11 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Enable in-process functional test policy to be configured https://review.openstack.org/159205 | 16:14 |
| clayg | hyeoh! | 16:17 |
| acoles | clayg: morning! | 16:18 |
| acoles | clayg: if you can put your +2 back on patch 165125 maybe we can get it landed (if you're happy with it still) | 16:19 |
| clayg | GlennS: scrool down the "Bulk Delete" -> http://docs.openstack.org/developer/swift/middleware.html?highlight=bulk#module-swift.common.middleware.bulk | 16:19 |
| acoles | come on patchbot ... patch 165125 | 16:20 |
| patchbot | acoles: https://review.openstack.org/#/c/165125/ | 16:20 |
| clayg | acoles: NO! you've *ruined* it! | 16:20 |
| acoles | clayg: i aim to please ;) | 16:20 |
| *** mahatic has joined #openstack-swift | 16:20 | |
| clayg | acoles: hey you didn't by chance fix my suptid missing @patch_policies rebase screw up on patch 159637 did you? | 16:20 |
| patchbot | clayg: https://review.openstack.org/#/c/159637/ | 16:20 |
| acoles | i did | 16:20 |
| acoles | more ruination | 16:21 |
| acoles | clayg: i'm going to put some thoughts about ssync & missing durable on a trello card in a while | 16:22 |
| acoles | peluse: ping | 16:23 |
| clayg | acoles: sweet! | 16:23 |
| clayg | acoles: i don't think i quite followed your explination of fname_to_ts on the replicated diskfile | 16:24 |
| clayg | acoles: like you said reconstructor - but I guess you meant ssync? | 16:24 |
| acoles | clayg: see my follow on comment | 16:25 |
| clayg | i thougth i did :\ | 16:25 |
| acoles | but yeah it would be ssync | 16:25 |
| acoles | ok hang on, lets reload that context | 16:25 |
| peluse | pong | 16:26 |
| clayg | so it's *yield_hashes* acctually it's used as part of the trailing filter impl | 16:26 |
| peluse | yes | 16:26 |
| clayg | but that's a manager method - soo...... | 16:26 |
| GlennS | thanks clayg, I'll tkae a look | 16:26 |
| peluse | tox | 16:28 |
| acoles | clayg: if i ripped it out of 165125 the the dependent patch would have a problem | 16:28 |
| clayg | ERROR: py27: commands failed | 16:28 |
| clayg | oh damn, too slow ^ peluse that was for you | 16:28 |
| clayg | screw that patch! we'll fix it then | 16:28 |
| acoles | peluse: hi, got a question for you, can i just finish with clay on fname_to_ts? | 16:29 |
| peluse | sure | 16:30 |
| acoles | clayg: yeah, so it needs fixing to be diskfile manager impl specific yield hashes (and i'm buzzing about that taking advantage of gather_ondisk_files) | 16:30 |
| peluse | heh, nice clayg. you responded faster than my VM :) | 16:30 |
| GlennS | clayg: I take it middleware is something that my CDN needs to have installed on their end for me to be able to use? | 16:30 |
| acoles | clayg: but i'm all frazzled by the patch dependencies ?!? | 16:30 |
| peluse | I'm bedazzled | 16:30 |
| acoles | peluse: ok, i'm looking at trello https://trello.com/c/W4TkwVDS/144-cleanup-also-what-s-the-difference-between-x-backend-node-index-and-x-object-sysmeta-ec-archive-index | 16:32 |
| acoles | peluse: and i can't see where these headers are set, like in the proxy i presume but i couldn't find it - am i beng stupid or is it in a patch that hasn't yet landed? | 16:32 |
| peluse | OK, give me a min | 16:33 |
| acoles | sure | 16:34 |
| clayg | GlennS: which provider? try to call /info on it. | 16:35 |
| clayg | GlennS: http://docs.openstack.org/developer/swift/api/discoverability.html?highlight=discover#discoverability | 16:36 |
| clayg | GlennS: mine says it has Additional middleware: bulk_delete | 16:36 |
| clayg | acoles: I've given them more useful names like 'git checkout -b rebase-here' | 16:37 |
| openstackgerrit | paul luse proposed openstack/swift: Make get_dev_path() treat moun_check literally... https://review.openstack.org/166307 | 16:37 |
| clayg | the literal *moun*_check sounds tricky :\ | 16:38 |
| peluse | clayg, so this was a bit more of a PITA than I thought it would be ^^^ but I do need it for ECrecon. Stupid commit msg! | 16:38 |
| peluse | acoles, OK wil go look at your question now | 16:38 |
| acoles | did he mean moon_check? | 16:38 |
| clayg | ecRec ? | 16:38 |
| peluse | ha! | 16:38 |
| acoles | did he mean ecWreck :) | 16:39 |
| peluse | holy cow it must be Friday! | 16:39 |
| clayg | yeah thats how to say it - the eck-rec | 16:39 |
| clayg | - the frag-fixer? | 16:40 |
| peluse | acoles, so the second one is in the footer on an EC put and defined in the policy. Will check on the first right onw | 16:40 |
| *** EmilienM is now known as EmilienM|afk | 16:41 | |
| peluse | acoles, and the 2nd is on the reconstructor patch | 16:42 |
| clayg | acoles: idk if self._diskfile.manager is so much better than self._diskfile._mgr? | 16:43 |
| acoles | peluse: ok i see the second now, thx | 16:43 |
| peluse | I think I can probably just use the first in the reconstructor. It didn't exist when I started | 16:44 |
| *** tsg_ has quit IRC | 16:44 | |
| peluse | so if you want to ignore that trello card, that's fine, I'll throw it in my name as part of the reconstructor | 16:44 |
| clayg | what was getting me is that I'm never sure if I should call ._mgr or ._diskfile._mgr - adding the alias just means we'll also have instances where the code will say ._diskfile.manager and .manager - but instead of only one way working we'll have two | 16:45 |
| clayg | if the @properties somehow made it so you could always say self.manager and get the thing you wanted - then I'd be down, because people would probably just do that than try to remeber which under attr has the damn thing | 16:46 |
| GlennS | clayg: it's Memset | 16:46 |
| clayg | *and* i want to get rid of some of those other changes to the replicated diskfile | 16:46 |
| clayg | hey that's cool -> https://github.com/Memset/sftpcloudfs | 16:48 |
| clayg | didn't chmouel and redbo both write one of those two? maybe it's like the thing to do. | 16:48 |
| clayg | chmouel: what the heck is this -> https://github.com/cloudfs/ftp-cloudfs | 16:50 |
| acoles | clayg: hmm. so DiskFile has a manger property, it constructs a DiskFileWriter and DiskFileWriter either creates its own manager property so it can use self.manager or it just holds a ref to its DiskFile parent and calls self.diskfile.manager? Are you saying you prefer the first? Either works, I tend to prefer the second because if you take the first approach for every property that DiskFileWriter might want to inherit fro | 16:55 |
| acoles | m DiskFile you end up with more lines of code in DiskFileWriter where one line (self.diskfile = diskfile) would do. | 16:55 |
| clayg | man, i can never seem to find storage urls from public providers - like the auth endpoint is always published - but without the catalog you never know what's the real swift endpoint | 16:56 |
| clayg | i wonder if you could crowd source it | 16:56 |
| acoles | clayg: Arguably for manager it makes sense to do first because its like a 'hub' for all those helper methods | 16:56 |
| clayg | acoles: i don't much care for def some_useful_thing(self): return self._mgr.some_useful_thing() | 16:57 |
| clayg | i'd much rather see the place calling self.some_useful_thing() just flipping tell me it's defined on the manager with self.manager.some_useful_thing() | 16:58 |
| clayg | in the weird case of the writer where it has to proxy through the diskfile to get to a manager I think i could stand a @property so the writer can still call self.manager.some_useful_thing() (which the property translates to self._diskfile.manager or something | 16:59 |
| acoles | clayg: ok which specific example are we looking at, i thought i'd reverted the place i did that with get_ondisk_files? | 16:59 |
| clayg | or ._mgr or ._manager - i don't really care what it's called | 16:59 |
| clayg | invalidate_hash I think? | 16:59 |
| acoles | right, but there's a reason for that, iirc, let me check | 16:59 |
| *** zhill has joined #openstack-swift | 17:00 | |
| clayg | acoles: nah, i had originally splew'd the module functions all over the classes that were using them - but that turned out to be wrong - i eventually realized we should just do it all on the manager | 17:04 |
| acoles | clayg: oh boy you know i actually reverted invalidate_hash back to a self_strip or whatever today, and then convinced myself that i would break a dependent patch :( | 17:05 |
| clayg | because while the writer is the *primary* call site for invalidate_hash and it would make sense to override implementation specific behavior there - the manager's got a bunch of quarantine functionality that eventually call into invalidate_hash | 17:06 |
| acoles | i'm going to work on crypto... | 17:06 |
| clayg | i'm *pretty* sure I implemented it on the manager | 17:06 |
| clayg | acoles: wfm - I'm going to go on a rebase cleanup tear | 17:06 |
| clayg | acoles: anyway - are you buying into diskfile.manager = manager and DiskFileWriter.manager = @property self._diskfile.manager ? | 17:08 |
| acoles | clayg: let me go look at inavlidate_hash, cos unless i ahd a reason then you are right | 17:08 |
| clayg | acoles: if you had a reason I'll find it when I get there | 17:08 |
| acoles | k that works too | 17:08 |
| clayg | acoles: but I'll also add it then - so we can both see why it has to work that way | 17:09 |
| acoles | sorry, i thought i'd moved things the way you were asking but misunderstood you | 17:09 |
| clayg | acoles: i wasn't complaing - i thogutht they were find - two obvious ways to do it but only one way works | 17:09 |
| clayg | mattoliverau was complaining about accessing a under atter - but the alias ended up making it worse for me | 17:10 |
| clayg | man i misspell fine as find *a lot* | 17:10 |
| clayg | there was something I couldn't type yesterday... | 17:11 |
| clayg | stupid muscles | 17:11 |
| acoles | i ahd no porbelm undrestadning :) | 17:11 |
| clayg | lol | 17:11 |
| clayg | acoles: have you looked at https://review.openstack.org/#/c/156825/8 | 17:12 |
| clayg | it's the proxy method extraction refactor on master | 17:12 |
| acoles | no not yet | 17:13 |
| clayg | acoles: i was to head over there I think i'd be looking at re-organizing and auditing some of the test coverage - i mean the code is sorta obviously better chunked up - so the only real question is if we missed something trying to copy and paste that stuff into new methods | 17:14 |
| acoles | k | 17:15 |
| acoles | tbh i don't think i'll get there today | 17:16 |
| *** jordanP has quit IRC | 17:16 | |
| *** zaitcev has joined #openstack-swift | 17:17 | |
| *** ChanServ sets mode: +v zaitcev | 17:17 | |
| *** jistr has quit IRC | 17:19 | |
| *** openstackgerrit has quit IRC | 17:21 | |
| *** openstackgerrit has joined #openstack-swift | 17:22 | |
| *** dmorita has joined #openstack-swift | 17:22 | |
| *** Bsony_ has quit IRC | 17:25 | |
| acoles | peluse: just ome more thought on those node index headers - if the node index does not need to be persisted in object metadata then X-Backend-Node-Index is a lot more succinct than X-Object-Sysmeta-Ec-Archive-Index :) | 17:27 |
| *** gyee has joined #openstack-swift | 17:27 | |
| *** jkugel has joined #openstack-swift | 17:28 | |
| acoles | clayg: so is this what you mean re invalidate_hash http://paste.openstack.org/show/193983/ ? | 17:31 |
| acoles | clayg: give or take the "_diskfile." | 17:31 |
| acoles | clayg: i think i got it in my head that we'd be passing frag_index to invalidate_hash, but yeah, that was wrong | 17:32 |
| *** tsg has joined #openstack-swift | 17:33 | |
| clayg | acoles: yeah i've got that | 17:34 |
| clayg | and I'm trying to decide what to do with quarantine_renamer | 17:34 |
| clayg | because *honestly* - while a diskfile *should* be able to override that stuff we don't need it | 17:34 |
| clayg | it calls invalidate_hash - but at the end of the day the ecdsikfilemanager's invalidate_hash is *exactly* the module method | 17:35 |
| clayg | and not plubming quarantine_renamer (so it can be "fixed" to call the the right invalidate_hash) means the reader doesn't need a diskfile reference | 17:35 |
| clayg | so I think i'm going to rip it out - and invalidate_hash too | 17:36 |
| clayg | bah, crap i have a bunch of tests that expect invalidate_hash to be on a amanger | 17:37 |
| *** nellysmitt has quit IRC | 17:37 | |
| acoles | clayg: its always the tests that make small change into a big one ! | 17:37 |
| clayg | lol | 17:38 |
| acoles | clayg: ok i am leaving invalidate_hash fixing and/or ripping out to you man ;) i'm going to write my ssync thoughts down before i disappear | 17:39 |
| *** jbonjean has joined #openstack-swift | 17:42 | |
| *** mmcardle1 has quit IRC | 17:47 | |
| *** dmorita has quit IRC | 17:47 | |
| *** fandi has joined #openstack-swift | 17:47 | |
| *** fandi has quit IRC | 17:47 | |
| *** dmorita has joined #openstack-swift | 17:48 | |
| clayg | does anyone else thing we can't have X-Backend-Node-Index, X-Backend-Ssync-Frag-Index, and X-Object-Sysmeta-Ec-Archive-Index that all mean the same thing? | 17:51 |
| acoles | +1 | 17:52 |
| *** dmorita has quit IRC | 17:54 | |
| clayg | acoles: ok - you get to pick then! | 17:55 |
| acoles | clayg: i think peluse is on it. I like the shortest one. | 17:55 |
| acoles | clayg: of course, another answer is to have them all mean *different* things ;) | 17:56 |
| *** dmorita has joined #openstack-swift | 17:57 | |
| *** Bsony has joined #openstack-swift | 17:58 | |
| clayg | i love it when I find some wft code - then go look at the review where I approved it - with a comment like "i'm not sure about this bit buth looks ok otherwise" | 18:12 |
| acoles | clayg: https://trello.com/c/UjVTwXj7/154-optimize-ssync-to-not-replicate-data-file-when-only-durable-is-missing | 18:13 |
| *** Nadeem has quit IRC | 18:15 | |
| *** rdaly2 has joined #openstack-swift | 18:16 | |
| *** acoles is now known as acoles_away | 18:17 | |
| *** geaaru has quit IRC | 18:18 | |
| peluse | clayg, acoles_away: yeah I'll cleanup the dup headers and pick the shortest name :) | 18:20 |
| clayg | torgomatic: do you think it'd be really hard on PUT to know what node-index gaps you're filling in for as you're connecting? | 18:33 |
| clayg | torgomatic: like what if Putter was more like GETorHEADHandler - and took a node_iter | 18:34 |
| clayg | oh well, i'm not going to do it until after patch 164950 anyways... | 18:37 |
| patchbot | clayg: https://review.openstack.org/#/c/164950/ | 18:37 |
| *** jdaggett has quit IRC | 18:39 | |
| *** dmorita has quit IRC | 18:46 | |
| *** tdasilva has quit IRC | 18:47 | |
| *** Fin1te has joined #openstack-swift | 18:53 | |
| *** tdasilva has joined #openstack-swift | 18:54 | |
| *** tdasilva has quit IRC | 19:04 | |
| *** EmilienM|afk is now known as EmilienM | 19:10 | |
| *** silor has quit IRC | 19:13 | |
| *** h_m has quit IRC | 19:14 | |
| *** tdasilva has joined #openstack-swift | 19:17 | |
| *** tdasilva is now known as tdasilva_ | 19:18 | |
| *** tdasilva has joined #openstack-swift | 19:20 | |
| tdasilva_ | acoles_away: panicbnc is a little shaky today | 19:24 |
| *** h_m has joined #openstack-swift | 19:26 | |
| openstackgerrit | Janie Richling proposed openstack/swift: WIP - non-in-process functest mods to support different policies https://review.openstack.org/166097 | 19:32 |
| *** mahatic has quit IRC | 19:47 | |
| peluse | nice Janie! | 19:57 |
| *** tdasilva_ has quit IRC | 20:03 | |
| jrichli | peluse: thanks! | 20:06 |
| *** jdaggett has joined #openstack-swift | 20:08 | |
| *** Bsony has quit IRC | 20:10 | |
| *** Bsony has joined #openstack-swift | 20:11 | |
| *** erlon is now known as erlon_away | 20:21 | |
| *** tellesnobrega has quit IRC | 20:34 | |
| clayg | peluse: so i've managed to tease out why get's from handoffs isn't working | 20:35 |
| clayg | i think we need to merge some of the out standing per-policy-diskfile -> ec-probe-test chain so we can start fixing stuff | 20:36 |
| *** tsg_ has joined #openstack-swift | 20:40 | |
| *** tsg has quit IRC | 20:41 | |
| *** tellesnobrega has joined #openstack-swift | 20:44 | |
| notmyname | looks like the OpenStack summit presentation accept/reject emails are getting delivered now | 20:46 |
| jrichli | notmyname: good results so far? | 20:53 |
| notmyname | https://www.openstack.org/summit/vancouver-2015/schedule/ | 20:54 |
| notmyname | http://sched.co/2qcL is going to be the best evar! | 20:55 |
| notmyname | and http://sched.co/2qbz will be good too :-) | 20:57 |
| *** Guest__ has joined #openstack-swift | 21:00 | |
| *** tab____ has joined #openstack-swift | 21:01 | |
| *** silor has joined #openstack-swift | 21:03 | |
| *** dmorita has joined #openstack-swift | 21:05 | |
| *** Fin1te has quit IRC | 21:05 | |
| *** Guest__ has quit IRC | 21:07 | |
| *** Guest__ has joined #openstack-swift | 21:09 | |
| *** GearboxL_ has joined #openstack-swift | 21:13 | |
| GearboxL_ | Question about the object ring... Is it possible to change the zone of a given entry ? Say I've accidentally ended up with ~20 zones, of which 19 have ~12 entries, and 1 has ~100 entries, thusly giving me very unbalanced zones. | 21:14 |
| GearboxL_ | (And leading to those 19 smaller zones being really / completely full, and that 1 zone substantially less full) | 21:15 |
| *** silor has quit IRC | 21:15 | |
| clayg | GearboxL_: you must be pre swift 2.0 maybe? | 21:29 |
| GearboxL_ | Yeah | 21:29 |
| GearboxL_ | Havana | 21:29 |
| GearboxL_ | 1.10 | 21:29 |
| *** jrichli has quit IRC | 21:31 | |
| clayg | anyway - no you can't really change the zone | 21:32 |
| GearboxL_ | Ok, thanks | 21:33 |
| clayg | but if you remove a device and add a device (with different zone) in the same rebalance you have some control over how many parts move - but some shuffling will occur | 21:33 |
| GearboxL_ | *nod* | 21:33 |
| GearboxL_ | Is upgrading to 2.2 or later a viable approach here? | 21:33 |
| clayg | basically equal to the amount that you'd get if you just "changed" a zone on a device | 21:34 |
| clayg | maybe - if you wanna make your builders available I could look at the balance/dispersion with the new ring-builder code and see what I would recommend | 21:34 |
| clayg | you could also just go clone it somewhere (or use a vagrant-swift-all-in-one) and then get a copy of your builders and play around with it | 21:35 |
| *** Guest__ has quit IRC | 21:35 | |
| GearboxL_ | basically it's this: We have 21 zones. 20 of them have 12 device nodes in them. 1 has 95. There are 376 total nodes. Each node is 4 TB. :/ | 21:35 |
| clayg | ultimately the on-disk ring.gz file has been stable even with all the balance changes - i wouldn't *recommend* it - but you could balance rings with swift master and then distribute them to your older cluster if you like the partition placement better until you can figure out how to do an upgrade | 21:36 |
| clayg | GearboxL_: sounds like a great deployment! with all those zones I'm not really sure why even back in swift 1.10 you're not getting a decent balance | 21:37 |
| notmyname | balance is good. just that the smaller ones are filling up. makes sense for swift 1.10 | 21:37 |
| *** zhill has quit IRC | 21:37 | |
| clayg | notmyname: but all the *disks* are the same size - they should have equal parts - if balance is good I think that means by definition the devices are assigned a number of parts == parts wanted | 21:39 |
| GearboxL_ | And this is all in a single cluster. | 21:39 |
| clayg | GearboxL_: can you paste your swift-ring-builder blah.builder list output so we can make sure we're on the same page? | 21:40 |
| GearboxL_ | yes | 21:40 |
| clayg | GearboxL_: are you having problems issues with some of the 4TB drives being more *full* (like acctual df GB on disk) that others? | 21:40 |
| GearboxL_ | https://gist.github.com/gabriel-seomoz/1009f08f750a3babe8d3 | 21:41 |
| notmyname | clayg: but with as-unique-as-possible, it put partitions into a different zone and will not place replicas in the same one. which means that small zones fill up. especially since num_zones >> num_replicas | 21:41 |
| GearboxL_ | yup! | 21:41 |
| GearboxL_ | lots of drives in the small zones are full | 21:41 |
| *** Guest__ has joined #openstack-swift | 21:41 | |
| *** tdasilva has quit IRC | 21:42 | |
| GearboxL_ | this is absolutely our fault for badly managing the zone placement, and I recognize going forward what to change... but recovering from the issue is where I'm at now ;) | 21:42 |
| clayg | oic - right - *that* problem - the total weight of the 1 zone with 95 devices is more than one-replicanth - bah | 21:42 |
| GearboxL_ | yup | 21:43 |
| clayg | so for balance you'd need to assign multiple copies of some parts to the huge zone, and unique-as-possible won't do it - unique-as-balanceable ftw! | 21:43 |
| *** EmilienM is now known as EmilienM|PTO | 21:44 | |
| GearboxL_ | so upgrade required to make that happen? | 21:44 |
| GearboxL_ | we've been meaning to do the upgrade for ... too long ... but today might be when we have to | 21:45 |
| GearboxL_ | Fridays, right? :P | 21:45 |
| clayg | heck yeah! | 21:45 |
| *** rdaly2 has quit IRC | 21:45 | |
| clayg | GearboxL_: I mean you don't *have* to upgrade - you can do the remove/add devices trick - and just split up the larger zone into smaller ones | 21:46 |
| GearboxL_ | Ok | 21:46 |
| GearboxL_ | We've got a couple extra boxes that we've been meaning to add as well | 21:46 |
| GearboxL_ | so I'm hearing that we should add new capacity in different zones and that it'll pull out of *all* the other zones, inclusively our giant one, reducing the overall problem space && we should also pull nodes out of our giant zone && reallocate it | 21:46 |
| clayg | it's always better to have roughly evenly sized zones - but yeah newer swift was will let you trade dispersion (multiple replicas in a failure domain) for balance (making disks not have more parts assigned than wanted) | 21:47 |
| clayg | swift *loves* when you add capacity - adding more capacity is *always* an option ;) | 21:47 |
| GearboxL_ | Yeah, we goofed by ending up with this disparity of zone size | 21:47 |
| GearboxL_ | ;) | 21:47 |
| clayg | GearboxL_: no worries - we've seen it happen before | 21:48 |
| GearboxL_ | thanks for your help | 21:48 |
| GearboxL_ | I think I have a sense of how to proceed. | 21:48 |
| *** jamielennox|away is now known as jamielennox | 21:49 | |
| *** rdaly2_ has joined #openstack-swift | 21:50 | |
| clayg | GearboxL_: remeber you can do all of this swith swift-ring-builder offline - just makes copies of your rings - play around and see what you like | 21:50 |
| *** rdaly2_ has quit IRC | 21:50 | |
| clayg | GearboxL_: I'd try removing devices (maybe a lot) from the too big zone - and adding them back in under a different zone name - rebalance and see what your balance looks like | 21:51 |
| GearboxL_ | Are there risks to the actual data in doing that? | 21:51 |
| GearboxL_ | Like, could I bisect the too-big-zone safely? | 21:52 |
| clayg | heh - it *should* work - you've got three copies ;) | 21:52 |
| os1 | Is anyone aware of why /usr/bin/swift-recon-cron places the following lock file/directory: /var/lock/swift-recon-object-cron | 21:52 |
| os1 | Also, doesn't Swift need to be deployed such that the swift user is included in the 'lock' group, then? | 21:53 |
| os1 | Because otherwise, it will fail whenever trying to write the file. | 21:53 |
| GearboxL_ | Since we've filled up a number of the disks on the small zones (to 100%, ugh) is it the case that the data really is in 3 different zones? | 21:54 |
| clayg | yeah being full sucks - if only z95 has room that's where they're at | 21:54 |
| clayg | rebalancing can't really account for partitions out of place because of handoffs | 21:55 |
| GearboxL_ | *nod* | 21:55 |
| GearboxL_ | so we gotta get space utilization down by deleting old stuff before we can realistically safely rebalance, huh? | 21:56 |
| clayg | you may be in for some availability woes - but you might already be returning 507's and 404's depending on how many disks are full and what your request node count is | 21:56 |
| clayg | GearboxL_: a delete is just another write | 21:56 |
| GearboxL_ | Yeah, my users detected this problem. :-( | 21:57 |
| GearboxL_ | so I'm clearly suffering an availability problem right now :D | 21:57 |
| clayg | and if the nodes with the data you want to delete is on full devices the tombstones just go elsewhere and can't replicte back to cleanup the data files | 21:57 |
| *** Guest__ has quit IRC | 21:57 | |
| clayg | well then screw it - just split the big zone | 21:57 |
| *** thumpba_ has quit IRC | 21:58 | |
| *** jkugel has left #openstack-swift | 22:00 | |
| clayg | GearboxL_: this is what I was looking for - > https://bugs.launchpad.net/swift/+bug/1359160 | 22:03 |
| openstack | Launchpad bug 1359160 in OpenStack Object Storage (swift) "Problem with disk overflowing" [Undecided,Expired] | 22:03 |
| GearboxL_ | easy for you to say... ;) I have a few hundred TB of user data online that I can't lose. Not being available is ok, but losing the data wouldn't be. | 22:03 |
| GearboxL_ | *looks* | 22:03 |
| clayg | oh no - it's ok - you're not going to *loose* anything - it's not a *durablity* issue - just *availability* | 22:04 |
| clayg | if you're already getting complaints you're better off just getting the cluster on the path to rightness as quickly as possible | 22:04 |
| *** jamielennox is now known as jamielennox|away | 22:05 | |
| clayg | if you had noticed last week that things were starting to look hairy you could have come up with a plan to move more slowly and maybe avoid anyone noticing | 22:05 |
| GearboxL_ | Much joy in hindsight. ;) Strong is the bias there. :D | 22:05 |
| GearboxL_ | But I do appreciate what you're saying | 22:05 |
| GearboxL_ | thanks for the link. | 22:07 |
| clayg | if you don't have some new disks you can add asap getting that one big zone cut down into less than one-replicanth of your weight is the next best thing | 22:09 |
| clayg | aside from the rebalance asap - setting handoffs first on the fullest nodes would be the next biggest help | 22:10 |
| clayg | if your metadata servers (containers & accounts) are also full - there's some risk of inconsistencies (deleted objects stuck in the listings) the longer that goes on | 22:11 |
| *** tdasilva has joined #openstack-swift | 22:12 | |
| GearboxL_ | fortunately, we've done a less terrible job with metadata | 22:15 |
| GearboxL_ | we actually do have a bunch of disk that we can bring online | 22:15 |
| *** erlon_away has quit IRC | 22:41 | |
| peluse | clayg, you still hanging around? | 22:50 |
| openstackgerrit | paul luse proposed openstack/swift: wip: ec reconstructor probe test https://review.openstack.org/164291 | 22:59 |
| *** tsg_ has quit IRC | 23:00 | |
| *** welldannit has quit IRC | 23:03 | |
| *** welldannit has joined #openstack-swift | 23:10 | |
| *** GearboxL_ has quit IRC | 23:11 | |
| *** welldannit has quit IRC | 23:14 | |
| *** os1 has left #openstack-swift | 23:14 | |
| *** Bsony has quit IRC | 23:18 | |
| *** joeljwright has quit IRC | 23:23 | |
| *** tsg has joined #openstack-swift | 23:34 | |
| openstackgerrit | SHIGEMATSU Mitsuhiro proposed openstack/swift: Fix typo in swift/test/unit/account/test_backend.py https://review.openstack.org/166421 | 23:43 |
| notmyname | torgomatic: if you have a WIP for the range stuff by the time you finish up today, put it on gerrit and I'll take a look | 23:49 |
| torgomatic | notmyname: sure | 23:49 |
| *** gyee has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!