| *** klrmn has joined #openstack-swift | 00:11 | |
| *** jerrygb has quit IRC | 00:17 | |
| *** vint_bra has joined #openstack-swift | 00:19 | |
| *** PsionTheory has joined #openstack-swift | 00:33 | |
| *** vint_bra has quit IRC | 01:18 | |
| *** vinsh has joined #openstack-swift | 01:20 | |
| *** vinsh has quit IRC | 01:24 | |
| *** klrmn has quit IRC | 02:13 | |
| *** PsionTheory has quit IRC | 02:30 | |
| openstackgerrit | Karen Chan proposed openstack/swift: Mirror X-Trans-Id to X-Openstack-Request-Id https://review.openstack.org/387354 | 03:02 |
|---|---|---|
| *** vinsh has joined #openstack-swift | 03:13 | |
| *** jerrygb has joined #openstack-swift | 03:33 | |
| *** jerrygb has quit IRC | 03:37 | |
| openstackgerrit | Pete Zaitcev proposed openstack/swift: Fake GetRing in order to test on systems without Swift https://review.openstack.org/391700 | 03:38 |
| *** Jeffrey4l has joined #openstack-swift | 03:44 | |
| *** SkyRocknRoll has joined #openstack-swift | 03:47 | |
| kota_ | hello | 04:05 |
| kota_ | back from barcelona | 04:06 |
| kota_ | clayg: nice vsaio installs liberasurecode from source. if you finde any problems in the master source, please notify me to fix them. | 04:07 |
| *** vinsh has quit IRC | 04:12 | |
| *** vinsh has joined #openstack-swift | 04:16 | |
| *** chsc has joined #openstack-swift | 04:18 | |
| *** chsc has joined #openstack-swift | 04:18 | |
| *** jerrygb has joined #openstack-swift | 04:37 | |
| *** chsc has quit IRC | 04:39 | |
| *** links has joined #openstack-swift | 04:48 | |
| *** jerrygb has quit IRC | 05:04 | |
| *** vinsh has quit IRC | 05:14 | |
| *** SkyRocknRoll has quit IRC | 05:15 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:16 | |
| *** SkyRocknRoll has quit IRC | 05:22 | |
| *** nakagawamsa has joined #openstack-swift | 05:32 | |
| *** bkopilov_ has joined #openstack-swift | 05:45 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Fix stats calculation in object-reconstructor https://review.openstack.org/283946 | 05:50 |
| *** chirag has joined #openstack-swift | 05:58 | |
| chirag | I have Icehouse version of swift and I want to use it with Keystone Mitaka(v3) and I am facing auth failure issues. Can someone help ?? | 05:59 |
| *** silor has joined #openstack-swift | 06:11 | |
| *** silor1 has joined #openstack-swift | 06:17 | |
| *** silor has quit IRC | 06:19 | |
| *** silor1 is now known as silor | 06:19 | |
| *** vinsh has joined #openstack-swift | 06:45 | |
| *** nakagawamsa has quit IRC | 06:47 | |
| *** vinsh has quit IRC | 06:50 | |
| chirag | [Repost]I have Icehouse version of swift and I want to use it with Keystone Mitaka(v3) and I am facing auth failure issues. Can someone help ?? | 06:56 |
| *** jerrygb has joined #openstack-swift | 07:02 | |
| *** jerrygb has quit IRC | 07:08 | |
| *** tesseract has joined #openstack-swift | 07:11 | |
| *** tesseract is now known as Guest58324 | 07:11 | |
| openstackgerrit | Christian Schwede proposed openstack/swift: Respect server type for --md5 check in swift-recon https://review.openstack.org/240726 | 07:26 |
| kota_ | chirag: i'm not familiar with the history for keystone auth devel, but it looks keystone v3 with swift has been supported since Juno (past icehouse), https://github.com/openstack/swift/blob/master/CHANGELOG#L748 | 07:28 |
| *** Jeffrey4l has quit IRC | 07:34 | |
| kota_ | chirag: do you have a chance to try either v2 with Keystone Mitaka or Swift newer than Juno? | 07:36 |
| chirag | Hello, unfortunately no I have to do mitaka(v3) with swift icehouse. Is there any way ? | 07:41 |
| *** LZ has joined #openstack-swift | 07:46 | |
| *** vinsh has joined #openstack-swift | 07:46 | |
| *** LZ has quit IRC | 07:49 | |
| *** sams-gleb has joined #openstack-swift | 07:56 | |
| *** vinsh has quit IRC | 08:01 | |
| *** hseipp has joined #openstack-swift | 08:37 | |
| *** vinsh has joined #openstack-swift | 08:58 | |
| *** vinsh has quit IRC | 09:02 | |
| *** bkopilov has quit IRC | 09:24 | |
| *** bkopilov_ has quit IRC | 09:24 | |
| *** bkopilov_ has joined #openstack-swift | 09:25 | |
| *** bkopilov has joined #openstack-swift | 09:25 | |
| *** chirag has quit IRC | 09:51 | |
| *** vinsh has joined #openstack-swift | 09:59 | |
| *** vinsh has quit IRC | 10:03 | |
| *** eranrom has joined #openstack-swift | 10:31 | |
| *** jerrygb has joined #openstack-swift | 10:57 | |
| *** vinsh has joined #openstack-swift | 10:59 | |
| *** vinsh has quit IRC | 11:04 | |
| *** chlong has joined #openstack-swift | 11:40 | |
| *** cdelatte has joined #openstack-swift | 11:53 | |
| *** vinsh has joined #openstack-swift | 12:00 | |
| *** jerrygb has quit IRC | 12:01 | |
| *** vinsh has quit IRC | 12:05 | |
| *** MVenesio has joined #openstack-swift | 12:06 | |
| *** kei_yama has quit IRC | 12:08 | |
| *** links has quit IRC | 12:50 | |
| *** MVenesio has quit IRC | 12:51 | |
| *** caiobrentano_ has joined #openstack-swift | 12:56 | |
| *** vinsh has joined #openstack-swift | 13:02 | |
| *** jerrygb has joined #openstack-swift | 13:03 | |
| *** jerrygb has quit IRC | 13:07 | |
| *** vinsh has quit IRC | 13:09 | |
| *** jerrygb has joined #openstack-swift | 13:11 | |
| *** jerrygb has quit IRC | 13:11 | |
| *** klamath has joined #openstack-swift | 13:13 | |
| *** klamath has quit IRC | 13:13 | |
| *** klamath has joined #openstack-swift | 13:14 | |
| *** bkopilov_ has quit IRC | 13:14 | |
| *** bkopilov_ has joined #openstack-swift | 13:15 | |
| *** amoralej is now known as amoralej|lunch | 13:22 | |
| *** vint_bra has joined #openstack-swift | 13:32 | |
| *** vinsh has joined #openstack-swift | 14:04 | |
| *** jerrygb has joined #openstack-swift | 14:08 | |
| *** amoralej|lunch is now known as amoralej | 14:11 | |
| *** briancurtin has quit IRC | 14:19 | |
| *** tuan_luong has joined #openstack-swift | 14:22 | |
| *** briancurtin has joined #openstack-swift | 14:22 | |
| *** d0ugal has joined #openstack-swift | 14:22 | |
| *** _JZ_ has joined #openstack-swift | 14:26 | |
| *** jamielennox is now known as jamielennox|away | 14:34 | |
| *** sgundur has joined #openstack-swift | 14:48 | |
| *** sams-gleb has quit IRC | 14:53 | |
| *** sams-gleb has joined #openstack-swift | 14:54 | |
| *** sams-gleb has quit IRC | 14:58 | |
| *** Tuan_ has joined #openstack-swift | 14:59 | |
| *** sgundur has quit IRC | 15:00 | |
| *** tuan_luong has quit IRC | 15:01 | |
| *** jerrygb has quit IRC | 15:07 | |
| *** jerrygb has joined #openstack-swift | 15:08 | |
| *** Tuan_ has quit IRC | 15:08 | |
| *** klrmn has joined #openstack-swift | 15:09 | |
| *** ukaynar has joined #openstack-swift | 15:12 | |
| *** sams-gleb has joined #openstack-swift | 15:14 | |
| *** sgundur has joined #openstack-swift | 15:22 | |
| *** sgundur has quit IRC | 15:29 | |
| *** sgundur has joined #openstack-swift | 15:30 | |
| openstackgerrit | Shashirekha Gundur proposed openstack/swift: Invalidate cached tokens api https://review.openstack.org/370319 | 15:36 |
| *** StraubTW has joined #openstack-swift | 15:37 | |
| *** chsc has joined #openstack-swift | 15:37 | |
| *** Jeffrey4l has joined #openstack-swift | 15:38 | |
| notmyname | good morning | 15:39 |
| *** Jeffrey4l has quit IRC | 15:39 | |
| *** Jeffrey4l has joined #openstack-swift | 15:40 | |
| *** Jeffrey4l has quit IRC | 15:41 | |
| *** lcurtis has joined #openstack-swift | 15:42 | |
| *** Jeffrey4l has joined #openstack-swift | 15:42 | |
| *** urth has left #openstack-swift | 15:42 | |
| *** chsc has quit IRC | 15:48 | |
| *** Guest58324 has quit IRC | 15:56 | |
| *** LZ has joined #openstack-swift | 16:11 | |
| *** sgundur has quit IRC | 16:17 | |
| *** sgundur has joined #openstack-swift | 16:18 | |
| *** sgundur has quit IRC | 16:20 | |
| *** sgundur has joined #openstack-swift | 16:21 | |
| *** klrmn has quit IRC | 16:21 | |
| LZ | Hi everyone | 16:22 |
| LZ |  Can you recommend me some good online training of swift storage for beginner ? | 16:24 |
| LZ | Is there some online course? thanks very much | 16:24 |
| notmyname | LZ: what sort of training? operating it? writing apps for it? understanding the architecture? wanting to contribute? | 16:24 |
| LZ | understanding the architecture and operating it | 16:25 |
| *** nikivi has joined #openstack-swift | 16:26 | |
| notmyname | LZ: I don't know of anyone who provides training for it like that, but you might try https://www.openstack.org/marketplace/training/f/a/a/swift | 16:27 |
| notmyname | those are probably pretty strongly coupled to other openstack projects, though | 16:27 |
| LZ | Ok, I will try it, thanks | 16:27 |
| notmyname | LZ: https://www.swiftstack.com/docs/introduction/openstack_swift.html is a great starting place | 16:27 |
| notmyname | LZ: as is swift.openstack.org | 16:27 |
| clayg | morning! | 16:28 |
| clayg | or... Hola! | 16:28 |
| notmyname | clayg: yo! | 16:28 |
| clayg | happy Halloween! | 16:28 |
| notmyname | clayg: do probetests work for you? they seem to pass in the community cluster, so it's likely something on my machine | 16:28 |
| notmyname | clayg: I woke up at 4:07 today. how about you? | 16:29 |
| clayg | last time i was running them was on hummingbird | 16:29 |
| patchbot | hummingbird | 16:29 |
| notmyname | lol | 16:29 |
| clayg | i'm doing pretty good - i woke up at 7:30 with the kiddos | 16:29 |
| clayg | patchbot: is hummingbird the new docker? did someone say hummingbird? | 16:29 |
| LZ | in prague is 17:30 :-) | 16:30 |
| clayg | everywhere it is 16:27 UTC | 16:30 |
| * clayg goes to check ntp is running on this machine | 16:30 | |
| LZ | yes, it is | 16:30 |
| *** links has joined #openstack-swift | 16:31 | |
| *** chlong has quit IRC | 16:32 | |
| clayg | ntpd[17822]: adjusting local clock by 174.196515s | 16:34 |
| clayg | better | 16:34 |
| *** hseipp has quit IRC | 16:40 | |
| timburke | good morning | 16:41 |
| *** links has quit IRC | 16:42 | |
| tdasilva | notmyname: FWIW, i ran probetests on master a few minutes back and it passed ok for me | 16:44 |
| notmyname | tdasilva: thanks | 16:44 |
| *** sgundur has quit IRC | 16:48 | |
| *** sgundur has joined #openstack-swift | 16:48 | |
| *** tuan_luong has joined #openstack-swift | 16:51 | |
| *** david-lyle has joined #openstack-swift | 16:51 | |
| *** Jeffrey4l has quit IRC | 16:52 | |
| *** sgundur has quit IRC | 16:53 | |
| *** tuan_luong has quit IRC | 16:54 | |
| *** d0ugal has quit IRC | 16:54 | |
| *** d0ugal has joined #openstack-swift | 16:58 | |
| *** d0ugal has quit IRC | 16:58 | |
| *** d0ugal has joined #openstack-swift | 16:58 | |
| *** sgundur has joined #openstack-swift | 16:58 | |
| *** hseipp has joined #openstack-swift | 16:59 | |
| *** chsc has joined #openstack-swift | 17:01 | |
| *** chsc has joined #openstack-swift | 17:01 | |
| *** mvk has quit IRC | 17:01 | |
| openstackgerrit | John Dickinson proposed openstack/swift: Add checksum to object extended attributes https://review.openstack.org/336323 | 17:05 |
| openstackgerrit | John Dickinson proposed openstack/swift: Add checksum to object extended attributes https://review.openstack.org/336323 | 17:07 |
| notmyname | @torgomatic_ ^ | 17:10 |
| *** klrmn has joined #openstack-swift | 17:16 | |
| *** jordanP has joined #openstack-swift | 17:19 | |
| *** jordanP has joined #openstack-swift | 17:20 | |
| *** hseipp has quit IRC | 17:21 | |
| zaitcev | Was there ever an Etherpad for Hummingbird in Barcelona? | 17:46 |
| zaitcev | It's not in https://etherpad.openstack.org/p/ocata_swift_summit_topics | 17:46 |
| clayg | zaitcev: we only really talked about how someone needs to start pushing up some patches to delete some code soon and the layout of the object-server/replicator/repconn | 17:47 |
| clayg | general consensus I *think* was that we mostly like the repconn connection on not an "object-server" but it's maybe not great to have it on the replicator either because it's harder to manage/reason about (e.g. "once" mode doesn't even start a repconn listener) | 17:48 |
| zaitcev | okay | 17:48 |
| clayg | zaitcev: yeah you're all caught up! | 17:49 |
| zaitcev | BTW, I'm going to add you to review https://review.openstack.org/391700 | 17:49 |
| clayg | it wasn't a big topic | 17:49 |
| patchbot | patch 391700 - swift (feature/hummingbird) - Fake GetRing in order to test on systems without S... | 17:49 |
| clayg | ^ missed you nadeem hurricanerix | 17:49 |
| clayg | zaitcev: awesome sounds good | 17:49 |
| clayg | I think i pushed something up like "better error message handling" in some probe test when on-disk rings were missing because of the order that test patches policies and not rings | 17:50 |
| zaitcev | clayg: I see you added multiply policy into replicator, so I started adding them to proxy (put a map[int]Ring inside ProxyDirectClient basically). | 17:50 |
| clayg | zaitcev: no that was redbo ! | 17:50 |
| clayg | zaitcev: don't work on the proxy - delete the proxy! | 17:51 |
| zaitcev | clayg: you did and I put -1 on it so you can circle to it and remove 1 unneded function call | 17:51 |
| clayg | oh cool - thanks! | 17:51 |
| zaitcev | https://review.openstack.org/#/c/390903/ | 17:51 |
| patchbot | patch 390903 - swift (feature/hummingbird) - Better error messages in go/probe | 17:51 |
| zaitcev | Anyway, I don't even care about policies, I just need something, anything, a place to attach my liberasurecode stub | 17:52 |
| clayg | zaitcev: awsome! learning go via gerrit review while trying to refactor the replicator is going to be so productive!? | 17:52 |
| zaitcev | And my existing cluster has 2 policies, so I ended doing policies.... | 17:52 |
| *** sgundur has quit IRC | 17:52 | |
| openstackgerrit | Tim Burke proposed openstack/swift: Treat invalid limit parameters as errors https://review.openstack.org/299225 | 17:53 |
| openstackgerrit | Tim Burke proposed openstack/swift: Return 400 on bad requests https://review.openstack.org/391932 | 17:53 |
| notmyname | timburke: oh, https://review.openstack.org/#/c/391932/ won't be controversial at all. ;-) | 18:00 |
| patchbot | patch 391932 - swift - Return 400 on bad requests | 18:00 |
| *** asettle has joined #openstack-swift | 18:00 | |
| notmyname | I'm interested to see what temptest tests do with it | 18:00 |
| timburke | notmyname: no chance at all :P | 18:00 |
| timburke | i'm telling you, though: a 412 is just wildly inappropriate there | 18:01 |
| *** LZ has quit IRC | 18:01 | |
| *** sgundur has joined #openstack-swift | 18:01 | |
| clayg | timburke: and? | 18:02 |
| timburke | clayg: i figure not speaking http correctly is probably a bad idea | 18:03 |
| clayg | notmyname: do you have a sense for just how many environments where we currently run unittests are going to skip all of those modules where your doing the xattr check? | 18:03 |
| *** Harin has joined #openstack-swift | 18:09 | |
| clayg | timburke: I feel like I have this discussion with you a lot - I'm not really sure if we disagree or if you keep forgetting | 18:11 |
| *** jordanP has quit IRC | 18:11 | |
| clayg | notmyname: half of me thinks we should let timburke merge some of his api cleans up so he can learn from his mistakes like we have? | 18:12 |
| clayg | timburke: also fwiw - i made the same observation years ago about percondition failed - it *is* very sad that we didn't know http better five years ago | 18:12 |
| clayg | i try to watch out for imapproprriate use of http status codes *as much as possible* when reviewing changes to the API going forward - ¡Vive y aprende! | 18:14 |
| *** Harin has quit IRC | 18:17 | |
| notmyname | clayg: with torgomatic_'s patch to stop mocking xattrs, I expect every existing SAIO to break unless they run tests on an xfs partition. my updates mock those out so they skip instead of break. but yeah, I expect every dev box to need to be updated in order to run all the tests (and I'm ok with that) | 18:20 |
| notmyname | clayg: so i'll be making another CI test that has xfs in it. and that will be the "real" one. the tactic I took was to do the simplest thing possible without causing a national crisis | 18:21 |
| timburke | clayg: "using a 412 to indicate a error condition other than some If-X header did not match is a bit of a mis-use; unless tempest already validates that as the documented error code for a PUT to a version container when the version target does not exist we should consider changing it to a 400 with a better description." -- https://review.openstack.org/#/c/265015/ | 18:24 |
| patchbot | patch 265015 - swift - Autovivify X-Versions-Location container | 18:24 |
| clayg | notmyname: it's a pretty good strategy - you should write that on the change that fixes and rebase torgomatic_'s patch to depend on that | 18:24 |
| timburke | you! you were the one that gave me hope! | 18:24 |
| clayg | timburke: and doesn't tempest validate it? | 18:24 |
| notmyname | clayg: yeah, the reason I bundled it in was twofold: the base patch from torgomatic_ is small and the existing stuff without torgomatic_'s patch "works" without my changes (by virtue of not actually testing xattr stuff) | 18:25 |
| clayg | timburke: also if I told you to change an existing error code i was drunk | 18:25 |
| timburke | no idea. i guess we'll see | 18:25 |
| notmyname | clayg: so it comes down to that I can see going either way, but I thought one patch is simpler than two and combined it's not too big to fit in your head | 18:25 |
| clayg | notmyname: ok, that makes sense | 18:26 |
| notmyname | timburke: there is no hope. we're stuck with our warts | 18:26 |
| clayg | notmyname: i mean - it's reasonable as well - i'm not sure splitting them or keeping them together is acctually *better* | 18:26 |
| timburke | V2! V2! V2! :P | 18:26 |
| clayg | notmyname: I think your change to the test infrastructure is cirtainly "a bigger deal" than torgomatic_'s patch | 18:26 |
| clayg | but hopefully not "a national crisis" ;) | 18:27 |
| timburke | we've got enough of those already, thank you | 18:28 |
| notmyname | clayg: I'd prefer to simply edit the image so we have xfs, but I got a *lot* of pushback on that from infra people. along the lines of "dont' do something different than every other project". so, yeah, national crisis (in openstack terms). so my plan is to have one job that skips stuff and another with xfs that tests everything | 18:28 |
| clayg | notmyname: yes, i agree with the strategy - I think it deserves to be on the patch | 18:31 |
| notmyname | clayg: meaning more words in the commit message? | 18:31 |
| notmyname | or a "depends-on" for the CI gate job patch (that I haven't written yet)? | 18:32 |
| clayg | notmyname: yeah something in the commit messsage ... and yeah probably need the CI job ... | 18:33 |
| notmyname | ok | 18:33 |
| clayg | notmyname: like I think it's a significant enough change to how swift is tested that it's worth a commit titled "everything you know about unitests is different now" | 18:33 |
| clayg | notmyname: maybe see if anyone else agrees? torgomatic_ ? | 18:33 |
| notmyname | looks like the functests failed in the gate (but not in my saio), so there's definitely at least one more patch set | 18:34 |
| *** ukaynar has quit IRC | 18:34 | |
| clayg | notmyname: rock n' roll | 18:34 |
| torgomatic_ | I wonder if doing stuff to increase the number of 'S' printed out would help... if I see a test run with hundreds of '.' and one or two 'S', I'm going to assume it's pretty much run the whole thing | 18:35 |
| clayg | timburke: I'm not sure what point I was trying to make on patch 265015 - or what bugs we have that need to be fixed | 18:35 |
| patchbot | https://review.openstack.org/#/c/265015/ - swift - Autovivify X-Versions-Location container | 18:35 |
| torgomatic_ | however, if it's a third 'S', then I can see something's up | 18:35 |
| torgomatic_ | (not "third" as in "first, second, third", but as in "one in three of these things is an S") | 18:36 |
| notmyname | torgomatic_: "why can't you people write more code with shared responsibilities?!?" | 18:36 |
| torgomatic_ | notmyname: ? | 18:36 |
| notmyname | torgomatic_: sounded like that's what you're advocating ;-) | 18:36 |
| torgomatic_ | notmyname: ...no? | 18:37 |
| notmyname | torgomatic_: if we'd only call setting xattrs on more of our tests, we'd get more skips :-) | 18:37 |
| torgomatic_ | notmyname: more that skipping the module seems to print one "S" regardless of the number of unit tests run | 18:37 |
| torgomatic_ | s/run/in that module/ | 18:37 |
| * torgomatic_ can't brain today | 18:37 | |
| notmyname | torgomatic_: ah, right. so if every single test in that module had an S, then it would be better | 18:37 |
| notmyname | I see what you're getting at | 18:37 |
| torgomatic_ | notmyname: yep, or at least louder | 18:38 |
| notmyname | no, it was me not following you | 18:38 |
| *** ukaynar has joined #openstack-swift | 18:38 | |
| notmyname | louder? it's already a capital letter. that's like shouting, man. (...maybe we could do some escape codes for bold?) | 18:38 |
| notmyname | ;-) | 18:38 |
| notmyname | yeah, so in a class setup it might be better. I'll look in to that | 18:39 |
| clayg | notmyname: I guess "it's hard to know that we're skipping everything we need to w/o the xattr checksumming change" is a good reason to keep them together | 18:39 |
| timburke | notmyname: clayg: 14 func test failures on https://review.openstack.org/#/c/391932/ (i suppose i should have run those before submitting), but ZERO tempest failures | 18:39 |
| patchbot | patch 391932 - swift - Return 400 on bad requests | 18:39 |
| clayg | timburke: does that mean tempest doesn't use versioned containers? | 18:40 |
| notmyname | I'm pretty sure they have some versioning tests in tempest | 18:40 |
| *** sgundur has quit IRC | 18:43 | |
| timburke | https://github.com/openstack/tempest/blob/422ded8/tempest/api/object_storage/test_object_version.py#L51-L95 | 18:44 |
| timburke | they only ever test with both containers set up | 18:45 |
| *** sgundur has joined #openstack-swift | 18:46 | |
| notmyname | jrichli has someone at ibm working on swift tests in tempest. might be someone who could make those better, then | 18:46 |
| * notmyname lunch | 18:46 | |
| *** asettle has quit IRC | 18:47 | |
| *** asettle has joined #openstack-swift | 18:48 | |
| *** asettle has quit IRC | 18:48 | |
| *** asettle has joined #openstack-swift | 18:48 | |
| jrichli | notmyname: that is clu_. I can see what she has going on | 18:49 |
| openstackgerrit | Joel Wright proposed openstack/swift: WIP: Add Preamble and Postamble to SLO and SegmentedIterable https://review.openstack.org/365371 | 18:49 |
| *** sgundur has quit IRC | 18:50 | |
| clayg | timburke: well patch 391932 touches lots of stuff besdies versioned containers - I was looking through the negative tests in temptest and they're kinda sparse - but i'm still surprised they didn't break | 18:50 |
| patchbot | https://review.openstack.org/#/c/391932/ - swift - Return 400 on bad requests | 18:50 |
| clayg | I *suppose* there's probably not that many clients out there doing a bunch of requests they expect to return an error - so maybe this is a special exception | 18:51 |
| *** asettle has quit IRC | 18:52 | |
| *** asettle has joined #openstack-swift | 18:52 | |
| *** vint_bra has quit IRC | 19:02 | |
| notmyname | clayg: it's talk like that which gives timburke hope! | 19:09 |
| clayg | notmyname: you're right - i'm sorry timburke - there is no hope | 19:10 |
| *** silor has quit IRC | 19:12 | |
| *** vint_bra has joined #openstack-swift | 19:13 | |
| timburke | So we beat on, boats against the current, borne back ceaselessly into the past. | 19:15 |
| clayg | notmyname: acoles_: so what is alexis handle? | 19:15 |
| clayg | because I don't know if it's expected, observed or observed, expected? | 19:15 |
| notmyname | clayg: that is a great question. IDK | 19:18 |
| *** ukaynar has quit IRC | 19:20 | |
| *** sgundur has joined #openstack-swift | 19:23 | |
| *** vint_bra1 has joined #openstack-swift | 19:25 | |
| openstackgerrit | Tim Burke proposed openstack/swift: SLO: Concurrently HEAD segments https://review.openstack.org/391682 | 19:26 |
| *** vint_bra has quit IRC | 19:29 | |
| *** sgundur has quit IRC | 19:37 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Suppress unexpected file warnings for rsync temp files https://review.openstack.org/391658 | 19:43 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Include debug message for rsync tempfiles https://review.openstack.org/391960 | 19:43 |
| *** amoralej is now known as amoralej|off | 19:46 | |
| openstackgerrit | Merged openstack/python-swiftclient: Correct the example keystone session code imports https://review.openstack.org/391591 | 19:47 |
| *** sgundur has joined #openstack-swift | 20:14 | |
| *** cdelatte has quit IRC | 20:17 | |
| *** clsacramento has joined #openstack-swift | 20:24 | |
| *** mvk has joined #openstack-swift | 20:31 | |
| *** sgundur has quit IRC | 20:35 | |
| notmyname | torgomatic_: "Ran 4866 tests in 70.084s OK (SKIP=1213)" | 20:37 |
| notmyname | that should work for you ;-) | 20:37 |
| *** sgundur has joined #openstack-swift | 20:37 | |
| *** d0ugal has quit IRC | 20:40 | |
| openstackgerrit | John Dickinson proposed openstack/swift: Add checksum to object extended attributes https://review.openstack.org/336323 | 20:40 |
| *** d0ugal has joined #openstack-swift | 20:42 | |
| clayg | notmyname: wait, if the "problem" is just xattr > 4k? why not put write_metadata's 'xattr_size' kwarg just use a swift.common.constraint and have if fixed to 4k in the environments running tests? | 20:43 |
| clayg | or even better - have your code force it dynamically *unless* the environment overrides it? | 20:44 |
| notmyname | clayg: if you're running with default constraints and with /tmp as ext4, then the only thing that breaks are the crypto metadata tests (since those expand the metadata beyond the blocksize limit in ext4) | 20:46 |
| notmyname | that's where I started, but AFAIK the ext4 limit is blocksize and not 4k (I should check on this) and tempfs is just right out. doesn't work at all | 20:46 |
| notmyname | I've heard a rumor that fedora has /tmp as tempfs | 20:47 |
| notmyname | zaitcev: ^ is that true? | 20:47 |
| *** jerrygb__ has joined #openstack-swift | 20:48 | |
| notmyname | In the current ext2, ext3, and ext4 filesystem implementations, the | 20:48 |
| notmyname | total bytes used by the names and values of all of a files extended | 20:48 |
| notmyname | attributes must fit in a single filesystem block (1024, 2048 or 4096 | 20:48 |
| notmyname | bytes, depending on the block size specified when the filesystem was | 20:48 |
| notmyname | created). | 20:48 |
| zaitcev | notmyname: http://marc.info/?l=linux-mm&m=146734777402582&w=2 | 20:48 |
| notmyname | zaitcev: ah yes. I remember that. kernet patch to add xattrs to tempfs | 20:49 |
| zaitcev | notmyname: oh, you mean ext4. I dunno, sorry. There is some limit, but you can run on ext4. Our biggest and baddest to date Swift at a customer site is on ext4. | 20:49 |
| *** jerrygb has quit IRC | 20:49 | |
| zaitcev | although, this was probably still with chaining | 20:49 |
| notmyname | zaitcev: no, I was asking about what /tmp is mounted as in fedora, cent, rhel | 20:49 |
| notmyname | in ubuntu (xenial) it defaults to ext4 | 20:50 |
| zaitcev | notmyname: yes, it is mounted as tmpfs | 20:50 |
| notmyname | got it | 20:50 |
| zaitcev | also /var/run points to /run, which is also tmpfs | 20:50 |
| notmyname | same on ubuntu | 20:50 |
| notmyname | ok, so I can't simply skiptest on just the crypto ones. gotta skiptest on anything that uses xattrs on tempfiles | 20:50 |
| notmyname | clayg: ^ that's why it can't simply be a constraint check. rhel variants use tempfs where there's zero metadata support | 20:51 |
| zaitcev | or like https://review.openstack.org/#/c/328508/ | 20:51 |
| patchbot | patch 328508 - swift (feature/hummingbird) - Force /var/tmp as the test directory because of xattr | 20:51 |
| zaitcev | see redbo's comment | 20:51 |
| notmyname | yeah | 20:52 |
| *** jerrygb__ has quit IRC | 20:52 | |
| notmyname | but I'm guessing you'll still run in to an ext* limit there, right? | 20:52 |
| zaitcev | Yes... But we fully support XFS in RHEL 7, so it usually happens when people create a gigantic root on ext4, then run with mount_check=false | 20:53 |
| zaitcev | just use XFS | 20:54 |
| notmyname | yeah, exactly | 20:54 |
| zaitcev | I moved my $HOME from ext4 to XFS just so I don't have to deal with this anymore with I do ./.unittests | 20:54 |
| notmyname | I'm totally fine with breaking swift dev and test environments where they aren't using xfs. it's what we expect every production deployment to use. there's no (technical) reason I know of to keep trying to make up things to support non-xfs in tests | 20:56 |
| notmyname | but I may be wrong. I've been wrong many times before, and I'll be wrong again | 20:57 |
| *** _JZ_ has quit IRC | 21:02 | |
| clayg | i'm still not sure I fully understand all the justifications here | 21:03 |
| *** sams-gleb has quit IRC | 21:03 | |
| clayg | we have some test environments that are testing on files systems that don't *have* xattrs? great make a way for the test a) direct all tests to use a filesystem location that *does* have xattr support or b) skip tests if your absolutely unable to provide such a file system (gate my not be able to do that, that's ok by me) | 21:04 |
| *** sgundur has quit IRC | 21:04 | |
| clayg | or we have some test environments that are testing on filesystems that can only hold 4K worth of xattrs? great make it so that tests use less xattrs - operators should have a way limit this already (granting some single digit slop for crypto/etc) - so let's just those tools to enforce the same limits on unittests (which should really not care if the limit is 1K instead of 65K) | 21:06 |
| *** jamielennox|away is now known as jamielennox | 21:06 | |
| notmyname | environments where we don't even have xattrs are rhel-based. they have /tmp mounted as tempfs which doesn't support any xattrs at all | 21:07 |
| notmyname | envs where we have xattrs, but it's limited to 4k, are anything where /tmp is mounted with ext*. this includes the gate images | 21:08 |
| notmyname | when I talked to the infra team and qa people about setting up a different job that has xfs in it, they were all for that. but not with us dropping the existing job that doesn't have xfs. so we need a skiptest for that | 21:10 |
| notmyname | one could argue that our unittests shouldn't care what the limit is at all and just work. but then we end us mocking out the xattrs (and just testing the mock) and the same logic gets us to mocking out a whole filesystem | 21:11 |
| notmyname | current state, right now on master, is that we halfway mock out a filesystem for tests | 21:11 |
| notmyname | this patch drops the half we mock (xattrs) and therefore requires that xattrs are supported on the underlying tempdir | 21:11 |
| notmyname | one option I considered was to set up a new gate image that sets up an xfs partition somewhere and passes it in via an envvar. then our tests use that as the tmpdir and work. this is similar to what zaitcev's golang patch he linked does | 21:12 |
| notmyname | so, sure, that would work | 21:12 |
| notmyname | but we'd still need the "if the xfs-enabled tmpdir envvar isn't set, raise skiptest", and the plumbing is 95% similar to what i proposed | 21:13 |
| notmyname | so my current plan is to make a gate job template to have /tmp as xfs and then the current stuff works | 21:14 |
| notmyname | I think that's the simplest thing that will work and avoids "swift is being different" and actually tests the new functionality we want in | 21:14 |
| *** caiobrentano_ has quit IRC | 21:15 | |
| notmyname | and, for the record, yes, I think much of this is silly. I'd much prefer to land torgomatic_'s original patch and simply say "yeah, of course it broke. use xfs". but since we can't change the existing job template in CI, we can't do that | 21:15 |
| notmyname | http://www.quickmeme.com/img/52/52cfa562021c4c144f132c85c40a1ec19d1e1fedb4dc0764ed06e137d1ae826c.jpg | 21:16 |
| *** openstackgerrit has quit IRC | 21:18 | |
| *** openstackgerrit has joined #openstack-swift | 21:19 | |
| openstackgerrit | Tim Burke proposed openstack/pyeclib: Avoid segfault when raising exceptions https://review.openstack.org/385186 | 21:21 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Remove executable flag from some test modules https://review.openstack.org/391606 | 21:22 |
| zaitcev | Maybe I really should just spend a couple of weeks on tmpfs with xattr. It does not seem that hard, just needs some focus and concentration. | 21:26 |
| mattoliverau | Almost home, in Sydney, one last flight \o/ | 21:29 |
| clayg | notmyname: correct - our tests should not need to write >4K of xattrs on any one inode to know the code they're testing is working correctly | 21:36 |
| clayg | notmyname: correct - our tests should have a way to make all "scatch space" on an external mount point that's been configured for xfs | 21:36 |
| clayg | notmyname: maybe we don't *need* to do either of those things to land torgomatic_'s patch, but if we're going to disable ~1/3 of our tests there we probably *do* need to get this other job that *does* do those tests working in the gate before we land torgomatic_'s patch | 21:37 |
| torgomatic_ | clayg: yeah, that's definitely a good idea | 21:38 |
| clayg | ... so do you wanna stick a -2 on there or should I? | 21:38 |
| clayg | also... thanks for working on this! :D | 21:39 |
| clayg | wow, so no inprocess fuctests have ever stored xattrs on their files they create - awesome | 21:41 |
| clayg | i'm all for getting rid of these mocks | 21:41 |
| *** nikivi has quit IRC | 21:50 | |
| *** nikivi has joined #openstack-swift | 21:51 | |
| torgomatic_ | yeah, the mocks are pretty gross | 21:54 |
| torgomatic_ | although I'm lazy enough that if there were a good way to fix the mocks, I'd have been tempted to do that | 21:55 |
| *** jerrygb has joined #openstack-swift | 21:59 | |
| *** jerrygb has quit IRC | 22:04 | |
| *** chlong has joined #openstack-swift | 22:07 | |
| *** nikivi has quit IRC | 22:10 | |
| *** StraubTW has quit IRC | 22:11 | |
| clayg | do we mostly pass in the open fd int or do we give it a file object? i figure if we had the name there'd be some chance... the reuse of fd int's seems liekly to be what bone's the mock? | 22:18 |
| notmyname | clayg: from what torgomatic_ told me, the problem is that we get the inode and the fs number, but the kernel resuses both of those. so we don't have enough info to actually give different xattrs to each call to the mocked out xattr | 22:32 |
| notmyname | we pass in the fs | 22:32 |
| notmyname | *we pass in the fd | 22:32 |
| torgomatic_ | clayg: it's not that, it's that the mock isn't plumbed into unlinking at all. Imagine Test1 makes a file, sets some xattrs, then cleans up the file. Test2 then makes a new file with a different name but happens to get the same inode number (ruh roh), so the new file still mock-has the old xattrs | 22:32 |
| * notmyname needs to commute. will check in later | 22:33 | |
| *** lcurtis has quit IRC | 22:40 | |
| *** jerrygb has joined #openstack-swift | 22:42 | |
| *** klrmn has quit IRC | 22:45 | |
| *** jerrygb_ has joined #openstack-swift | 22:45 | |
| *** klrmn has joined #openstack-swift | 22:46 | |
| *** jerrygb has quit IRC | 22:48 | |
| *** rvasilets___ has joined #openstack-swift | 22:52 | |
| *** klamath has quit IRC | 22:52 | |
| *** geaaru has quit IRC | 22:58 | |
| *** geaaru has joined #openstack-swift | 23:00 | |
| clayg | I think it's *likely* to see the reuse of a fd int - and not likely *at all* to see the reuse of a filename - but the reuse of a fd int is a problem yes, specifically because of unlink | 23:08 |
| clayg | anyway - when I first saw the mock break down I had assumed a global patch/side-effect on unlink would be reasonable - but again we're back to mapping fname -> fd int and all hell breaks loose | 23:10 |
| clayg | oh, wait i see - the fd int gets translated to file system inode through fstat | 23:11 |
| clayg | so maybe a global side-effect on unlink to clear the inodes' cache is *insane* - but could also work? | 23:11 |
| clayg | did anyone try it? | 23:11 |
| *** asettle has quit IRC | 23:11 | |
| *** clsacramento has quit IRC | 23:25 | |
| *** clsacramento has joined #openstack-swift | 23:26 | |
| notmyname | clayg: I didn't. not sure if torgomatic_ looked in to it | 23:28 |
| torgomatic_ | clayg: no, I didn't try it | 23:29 |
| torgomatic_ | clayg: also note that the mock was keyed off inode number, not fd number, but the problem is that sometimes we'd get passed an integer fd, and there's no way to turn that back into a filename | 23:30 |
| torgomatic_ | if we had filenames, then we'd just use those and be done with it :| | 23:30 |
| *** chsc has quit IRC | 23:30 | |
| *** nikivi has joined #openstack-swift | 23:39 | |
| *** geaaru has quit IRC | 23:40 | |
| timburke | clayg: on https://review.openstack.org/#/c/391960/ -- if there's "garbage" in the datadir that we care enough about to get annoyed that our warning is getting skipped... aren't we going to be *even more annoyed* that we're going around *purging* it every 30mins or w/e? | 23:42 |
| patchbot | patch 391960 - swift - Include debug message for rsync tempfiles | 23:42 |
| timburke | that line of argument just makes me think we need to tighten our regex. and the second one i offered is still loosey-goosey enough that i can't imagine us changing the filename format enough to accidentally start warning about rsync tempfiles again | 23:44 |
| timburke | at least, not for another 200+ years | 23:44 |
| timburke | honestly, i'd worry more about rsync changing their tempfile naming convention more than that | 23:45 |
| openstackgerrit | Merged openstack/swift: Remove executable flag from some test modules https://review.openstack.org/391606 | 23:52 |
| openstackgerrit | Merged openstack/swift: Suppress unexpected file warnings for rsync temp files https://review.openstack.org/391658 | 23:53 |
| clayg | oh that's exciting a bunch of patches are in merge conflict now! | 23:56 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Make ECDiskFileReader check fragment metadata https://review.openstack.org/387655 | 23:57 |
| timburke | clayg: just a sign that things are merging! it's great! | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
