| *** siva_krish has joined #openstack-swift | 00:03 | |
| *** siva_krish has quit IRC | 00:07 | |
| *** Suyash has quit IRC | 00:08 | |
| *** zaitcev has joined #openstack-swift | 00:15 | |
| *** ChanServ sets mode: +v zaitcev | 00:15 | |
| *** diogogmt has quit IRC | 00:17 | |
| *** gyee has quit IRC | 00:29 | |
| *** ManojK has quit IRC | 00:30 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Preserve X-Static-Large-Object from .data file after POST https://review.openstack.org/334719 | 00:34 |
|---|---|---|
| clayg | timburke: acoles_: kota_: ^ I think it's is a better way to get solve lp bug #1453807 | 00:35 |
| openstack | Launchpad bug 1453807 in OpenStack Object Storage (swift) "Post (not as copy) to SLO manifest destroys its state as a manifest" [Undecided,Fix released] https://launchpad.net/bugs/1453807 - Assigned to Kota Tsuyuzaki (tsuyuzaki-kota) | 00:35 |
| *** lyrrad has quit IRC | 00:43 | |
| *** siva_krish has joined #openstack-swift | 00:51 | |
| *** ManojK has joined #openstack-swift | 00:53 | |
| kota_ | morning | 00:55 |
| kota_ | oh, what? I've been poked? | 00:55 |
| clayg | kota_: it's not super important - timburke pointed out to me something about x-static-large-object metadata handling that I had missed; you and acoles_ had basically already squared it - but he thought there was a consistency issue... i'm not sure if the comments on patch 334719 really make it any clearer | 00:57 |
| patchbot | clayg: https://review.openstack.org/#/c/334719/ - swift - Preserve X-Static-Large-Object from .data file aft... | 00:57 |
| *** siva_krish has quit IRC | 00:57 | |
| kota_ | clayg: alright, gotcha. | 00:58 |
| kota_ | clayg: I added the link to my review stack :) | 00:58 |
| *** tqtran has quit IRC | 01:01 | |
| *** klamath has quit IRC | 01:01 | |
| *** klamath has joined #openstack-swift | 01:02 | |
| *** ManojK has quit IRC | 01:03 | |
| *** ManojK has joined #openstack-swift | 01:06 | |
| *** ManojK has quit IRC | 01:11 | |
| *** rcernin has quit IRC | 01:11 | |
| *** klrmn has quit IRC | 01:12 | |
| *** klamath has quit IRC | 01:24 | |
| kota_ | clayg: with my quick look, your patch changed the x-static-large-object header holder from .meta into the root .data file right? | 01:24 |
| kota_ | it seems better (and easy) to tracking an issue of .meta racing. | 01:25 |
| clayg | aww you put it at the top of the stack :) | 01:26 |
| clayg | yeah that's pretty much the idea | 01:26 |
| kota_ | clayg: k, proably https://gist.github.com/bloodeagle40234/1badfff746df1db00fbac95e7e351b68 is a new test for the change :) | 01:27 |
| kota_ | When I opened the link, I noticed it's small change so easy to start on today's work beginning ;) | 01:28 |
| kota_ | ah, swiftstack/vagrant-swift-all-in-one] support vagrant AWS provider (#39) merged! | 01:33 |
| kota_ | great tools for me :) | 01:33 |
| clayg | ah, yeah that's nice | 01:34 |
| kota_ | oh, jenkins failed :/ | 01:36 |
| kota_ | ugh, test_mem_server stands up in front of me again :.( | 01:39 |
| kota_ | clayg: that change affects diskfile and mem_server file system doesn't support PRESERVED_DATAFILE/DATAFILE_SYSTEM_META change | 01:44 |
| kota_ | clayg: https://gist.github.com/bloodeagle40234/1badfff746df1db00fbac95e7e351b68 is a new version for passing test, I ran localy test/unit/proxy/test_server and that succeded. Now I'm running whole py27 tests yet. | 01:45 |
| *** vinsh_ has quit IRC | 01:47 | |
| *** vinsh has joined #openstack-swift | 01:49 | |
| clayg | aw, that makes sense - oops - good catch | 01:53 |
| *** baojg has joined #openstack-swift | 01:54 | |
| clayg | kota_: I'll circle back around to it tmrw and I can pull in any additional fixups you have - or just push over and we can each put a half a +2 on it ;) | 01:54 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Patch for Clay's diskfile change https://review.openstack.org/334728 | 01:55 |
| clayg | if you get it looking sharp I'm sure we could trick timburke or acoles_ into the +A | 01:55 |
| kota_ | clayg: pushed into the fix to gerrit :) | 01:55 |
| clayg | kota_: ah, np, that works too! | 01:55 |
| clayg | kota_: the changes in mem_diskfile in your patch probably should get merged in with the other fix tho - I might just squash everything and co-author you if that's ok? | 01:56 |
| clayg | AND THANKS! | 01:56 |
| kota_ | clayg: enough for me :) tentatively, I drop my +2, and will add +2 again after you merged it :) | 01:57 |
| *** baojg has quit IRC | 01:58 | |
| *** tqtran has joined #openstack-swift | 01:59 | |
| kota_ | clayg: anyway, have a good evening/night :D | 02:01 |
| *** tqtran has quit IRC | 02:03 | |
| *** mwheckmann has joined #openstack-swift | 02:06 | |
| *** Jeffrey4l__ has quit IRC | 02:07 | |
| *** Suyash has joined #openstack-swift | 02:09 | |
| *** baojg has joined #openstack-swift | 02:11 | |
| *** klrmn has joined #openstack-swift | 02:12 | |
| *** ManojK has joined #openstack-swift | 02:16 | |
| *** remimin has joined #openstack-swift | 02:22 | |
| *** remimin has quit IRC | 02:27 | |
| *** baojg has quit IRC | 02:33 | |
| *** mtreinish has quit IRC | 02:36 | |
| *** mtreinish has joined #openstack-swift | 02:36 | |
| *** chlong has quit IRC | 02:37 | |
| *** ManojK has quit IRC | 02:44 | |
| *** arch-nemesis has quit IRC | 02:47 | |
| *** chlong has joined #openstack-swift | 02:49 | |
| *** ManojK has joined #openstack-swift | 02:58 | |
| *** sheel has joined #openstack-swift | 03:07 | |
| *** SkyRocknRoll has joined #openstack-swift | 03:21 | |
| openstackgerrit | Pete Zaitcev proposed openstack/swift: Add a global fix for anything that involves GetHashPrefixAndSuffix https://review.openstack.org/333108 | 03:22 |
| *** ppai has joined #openstack-swift | 03:23 | |
| *** ManojK has quit IRC | 03:35 | |
| openstackgerrit | YaoZheng proposed openstack/swift: make print python3 compatible https://review.openstack.org/333801 | 03:41 |
| *** links has joined #openstack-swift | 03:50 | |
| *** tqtran has joined #openstack-swift | 04:01 | |
| *** tqtran has quit IRC | 04:06 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Patch for Clay's diskfile change https://review.openstack.org/334728 | 04:06 |
| *** Jeffrey4l has joined #openstack-swift | 04:31 | |
| *** psachin has joined #openstack-swift | 04:33 | |
| *** manous has joined #openstack-swift | 04:39 | |
| *** manous has quit IRC | 04:44 | |
| *** ppai has quit IRC | 04:45 | |
| *** rcernin has joined #openstack-swift | 04:46 | |
| *** manous has joined #openstack-swift | 04:57 | |
| *** ppai has joined #openstack-swift | 04:58 | |
| *** manous has quit IRC | 05:02 | |
| *** mariusv has joined #openstack-swift | 05:03 | |
| *** mariusv has quit IRC | 05:03 | |
| *** mariusv has joined #openstack-swift | 05:03 | |
| *** mariusv has quit IRC | 05:03 | |
| *** zaitcev has quit IRC | 05:08 | |
| *** mariusv has joined #openstack-swift | 05:11 | |
| *** ppai has quit IRC | 05:13 | |
| *** manous has joined #openstack-swift | 05:15 | |
| *** manous has quit IRC | 05:19 | |
| *** baojg has joined #openstack-swift | 05:22 | |
| timburke | clayg: yay! i should complain about things more often! | 05:22 |
| timburke | i'm not sure i understand the link between that bug and the cve on it, though... | 05:25 |
| *** ChubYann has quit IRC | 05:27 | |
| *** manous has joined #openstack-swift | 05:32 | |
| *** kei_yama has quit IRC | 05:35 | |
| *** rcernin has quit IRC | 05:37 | |
| *** mariusv has quit IRC | 05:37 | |
| *** manous has quit IRC | 05:38 | |
| *** kei_yama has joined #openstack-swift | 05:42 | |
| *** baojg has quit IRC | 05:45 | |
| *** manous has joined #openstack-swift | 05:49 | |
| *** rcernin has joined #openstack-swift | 06:09 | |
| *** ppai has joined #openstack-swift | 06:09 | |
| *** Suyash has quit IRC | 06:20 | |
| *** kei_yama_ has joined #openstack-swift | 06:29 | |
| *** kei_yama has quit IRC | 06:29 | |
| openstackgerrit | YaoZheng proposed openstack/swift: make print python3 compatible https://review.openstack.org/333801 | 06:29 |
| *** klrmn has quit IRC | 06:41 | |
| *** jamielennox is now known as jamielennox|away | 06:44 | |
| *** pcaruana has joined #openstack-swift | 06:46 | |
| *** manous has quit IRC | 06:58 | |
| *** tesseract- has joined #openstack-swift | 06:59 | |
| *** asettle has joined #openstack-swift | 07:03 | |
| *** daemontool has joined #openstack-swift | 07:04 | |
| *** mwheckmann has quit IRC | 07:05 | |
| *** manous has joined #openstack-swift | 07:09 | |
| *** manous has quit IRC | 07:14 | |
| *** manous has joined #openstack-swift | 07:17 | |
| *** jordanP has joined #openstack-swift | 07:18 | |
| *** jordanP has quit IRC | 07:18 | |
| *** koko has joined #openstack-swift | 07:22 | |
| *** koko is now known as Guest74976 | 07:22 | |
| *** rledisez has joined #openstack-swift | 07:23 | |
| *** manous has quit IRC | 07:25 | |
| *** Guest74976 has quit IRC | 07:27 | |
| *** manous has joined #openstack-swift | 07:38 | |
| *** manous has quit IRC | 07:42 | |
| *** diogogmt has joined #openstack-swift | 07:53 | |
| *** joeljwright has joined #openstack-swift | 07:56 | |
| *** ChanServ sets mode: +v joeljwright | 07:56 | |
| *** asettle has quit IRC | 07:58 | |
| *** acoles_ is now known as acoles | 07:58 | |
| *** geaaru has joined #openstack-swift | 08:00 | |
| *** hseipp has joined #openstack-swift | 08:02 | |
| *** dmk0202 has joined #openstack-swift | 08:06 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/swift: [WIP] Testing latest u-c https://review.openstack.org/318441 | 08:10 |
| *** mvk_ has quit IRC | 08:19 | |
| acoles | tdasilva: like I said, never assume anything :( | 08:24 |
| *** daemontool has quit IRC | 08:40 | |
| *** asettle has joined #openstack-swift | 08:45 | |
| *** sanchitmalhotra has joined #openstack-swift | 08:45 | |
| *** mvk_ has joined #openstack-swift | 08:51 | |
| *** jistr is now known as jistr|mtg | 09:04 | |
| openstackgerrit | YaoZheng proposed openstack/swift: make print python3 compatible https://review.openstack.org/333801 | 09:09 |
| *** hseipp has quit IRC | 09:20 | |
| *** asettle has quit IRC | 09:21 | |
| *** jistr|mtg is now known as jistr | 09:31 | |
| *** baojg has joined #openstack-swift | 09:37 | |
| *** baojg has quit IRC | 09:37 | |
| *** asettle has joined #openstack-swift | 09:40 | |
| *** baojg has joined #openstack-swift | 09:46 | |
| *** mvk_ has quit IRC | 09:48 | |
| *** sanchitmalhotra has quit IRC | 09:49 | |
| *** baojg has quit IRC | 09:50 | |
| *** sanchitmalhotra has joined #openstack-swift | 09:50 | |
| *** mvk_ has joined #openstack-swift | 10:01 | |
| *** kei_yama_ has quit IRC | 10:35 | |
| *** kei_yama has joined #openstack-swift | 10:36 | |
| *** kei_yama has quit IRC | 10:41 | |
| *** hosanai has quit IRC | 10:43 | |
| *** silor has joined #openstack-swift | 10:45 | |
| *** silor has quit IRC | 10:47 | |
| *** daemontool has joined #openstack-swift | 10:48 | |
| *** silor has joined #openstack-swift | 10:49 | |
| *** gvernik has joined #openstack-swift | 11:13 | |
| *** gvernik has quit IRC | 11:23 | |
| *** wanghua has quit IRC | 11:25 | |
| *** joeljwright has quit IRC | 11:28 | |
| *** manous has joined #openstack-swift | 11:43 | |
| *** arcimboldo has joined #openstack-swift | 11:46 | |
| arcimboldo | hi all, I have a swift/liberty and a container is not getting deleted. There is a ".pending" file and when I try to delete I get "409 conflict" reply | 11:47 |
| *** manous has quit IRC | 11:47 | |
| arcimboldo | what's happening? | 11:47 |
| arcimboldo | Besides: it is supposed to be a replica-2 container but for some reason (maybe a past test) I have 3 copies of the container data | 11:48 |
| arcimboldo | conatiner is empty | 11:48 |
| arcimboldo | (ah ok, that's my fault: containers are stored in 3 replicas ring, so that's fine) | 11:48 |
| *** raildo-afk is now known as raildo | 11:53 | |
| *** manous has joined #openstack-swift | 12:00 | |
| *** ppai has quit IRC | 12:04 | |
| *** manous has quit IRC | 12:05 | |
| *** manous has joined #openstack-swift | 12:05 | |
| *** manous has quit IRC | 12:10 | |
| *** ppai has joined #openstack-swift | 12:18 | |
| *** jamielennox|away is now known as jamielennox | 12:21 | |
| *** diogogmt has quit IRC | 12:21 | |
| *** diogogmt has joined #openstack-swift | 12:22 | |
| *** _JZ_ has joined #openstack-swift | 12:24 | |
| openstackgerrit | Sachin Patil proposed openstack/swift: Fix swift-get-nodes arg parsing for missing ring https://review.openstack.org/334238 | 12:25 |
| *** daemontool has quit IRC | 12:28 | |
| arcimboldo | I'll ask again: what's the meaning of ".pending" files? I have containers with ".pending" file and ".lock", and I cannot delete the container because I get a 409 - HTTPConflict error | 12:31 |
| * arcimboldo suspects everyone here is living on the west coast | 12:32 | |
| *** diogogmt has quit IRC | 12:34 | |
| *** daemontool has joined #openstack-swift | 12:36 | |
| *** joeljwright has joined #openstack-swift | 12:37 | |
| *** ChanServ sets mode: +v joeljwright | 12:37 | |
| *** links has quit IRC | 12:40 | |
| *** ManojK has joined #openstack-swift | 12:46 | |
| *** links has joined #openstack-swift | 12:52 | |
| *** klamath has joined #openstack-swift | 12:54 | |
| *** daemontool has quit IRC | 12:57 | |
| *** klamath_ has joined #openstack-swift | 13:02 | |
| *** klamath has quit IRC | 13:02 | |
| *** klamath_ has quit IRC | 13:02 | |
| *** klamath has joined #openstack-swift | 13:03 | |
| *** pauloewerton has joined #openstack-swift | 13:03 | |
| *** daemontool has joined #openstack-swift | 13:13 | |
| *** diogogmt has joined #openstack-swift | 13:15 | |
| *** jamielennox has quit IRC | 13:25 | |
| *** daemontool has quit IRC | 13:25 | |
| *** daemontool has joined #openstack-swift | 13:41 | |
| *** fifieldt has joined #openstack-swift | 13:43 | |
| *** ManojK has quit IRC | 13:44 | |
| *** mmcardle has quit IRC | 13:44 | |
| *** mmcardle has joined #openstack-swift | 13:48 | |
| acoles | arcimboldo: containers must be empty before they can be deleted, the 409 may be indicating that the container is not empty | 13:51 |
| openstackgerrit | Christian Schwede proposed openstack/swift: Fix inline tempurl/formpost signature examples https://review.openstack.org/335044 | 13:53 |
| acoles | arcimboldo: the .pending file contains a journal of recent updates waiting to be committed to the container objects table, it is normal to see that. The .pending file is processed periodically, and always when an attempt is made to delete the container. | 13:54 |
| tdasilva | good morning | 13:54 |
| *** mwheckmann has joined #openstack-swift | 13:55 | |
| *** ametts has joined #openstack-swift | 13:57 | |
| tdasilva | acoles: tough game, but this video is awesome to watch: http://espn.go.com/video/clip?id=espn:16565508 | 14:01 |
| acoles | tdasilva: what game? what are you talking about? ;) it has been erased from history | 14:03 |
| acoles | tdasilva: heh, I saw some footage of the commentator | 14:03 |
| tdasilva | acoles: hehe, believe me, I know the feeling all too well now | 14:03 |
| *** delattec has joined #openstack-swift | 14:14 | |
| *** cdelatte has quit IRC | 14:16 | |
| *** delattec has quit IRC | 14:20 | |
| *** ManojK has joined #openstack-swift | 14:22 | |
| *** ppai has quit IRC | 14:30 | |
| *** diogogmt has quit IRC | 14:32 | |
| *** diogogmt has joined #openstack-swift | 14:32 | |
| *** siva_krish has joined #openstack-swift | 14:33 | |
| *** jamielennox has joined #openstack-swift | 14:36 | |
| *** links has quit IRC | 14:38 | |
| *** arch-nemesis has joined #openstack-swift | 14:39 | |
| *** cdelatte has joined #openstack-swift | 14:40 | |
| *** cdelatte has quit IRC | 14:42 | |
| *** macsun has joined #openstack-swift | 14:43 | |
| *** ManojK has quit IRC | 14:44 | |
| *** ManojK has joined #openstack-swift | 14:44 | |
| *** thumpba has joined #openstack-swift | 14:49 | |
| *** cdelatte has joined #openstack-swift | 14:55 | |
| *** cdelatte has quit IRC | 15:05 | |
| *** jistr is now known as jistr|mtg | 15:13 | |
| notmyname | good morning | 15:17 |
| *** klrmn has joined #openstack-swift | 15:17 | |
| *** cdelatte has joined #openstack-swift | 15:31 | |
| *** jistr|mtg is now known as jistr | 15:32 | |
| *** Suyash has joined #openstack-swift | 15:33 | |
| *** cdelatte has quit IRC | 15:34 | |
| *** psachin has quit IRC | 15:37 | |
| *** dmk0202 has quit IRC | 15:42 | |
| *** jmccarthy has quit IRC | 15:44 | |
| *** jmccarthy has joined #openstack-swift | 15:44 | |
| notmyname | acoles: WIP on patch 328207? | 15:47 |
| patchbot | notmyname: https://review.openstack.org/#/c/328207/ - swift (feature/crypto-review) - Allow middleware to override metadata header checking | 15:47 |
| acoles | notmyname: I saw enough comments pushing towards a different approach (shifting encrypted vals to transient sysmeta) | 15:48 |
| acoles | notmyname: don't want people wasting time reviewing that patch if it is not needed | 15:48 |
| notmyname | well *something* is needed | 15:49 |
| acoles | notmyname: I have the alternative working ;) | 15:49 |
| notmyname | oh, ok. carry on then ;-) | 15:49 |
| acoles | notmyname: yes, just leave me to type ... ;-) | 15:49 |
| acoles | JK | 15:49 |
| acoles | notmyname: I am just summarising the change on an etherpad so some of us can discuss | 15:50 |
| notmyname | ok | 15:50 |
| notmyname | feature that is working and landed and updated later is much preferred to perfectly designed feature that will land some time later | 15:51 |
| *** Jeffrey4l has quit IRC | 15:55 | |
| *** tesseract- has quit IRC | 15:56 | |
| *** daemontool_ has joined #openstack-swift | 16:00 | |
| *** daemontool has quit IRC | 16:03 | |
| acoles | notmyname: ^^ that is very much in my mind as I deal with review comments | 16:08 |
| *** cdelatte has joined #openstack-swift | 16:12 | |
| *** SkyRocknRoll has quit IRC | 16:15 | |
| *** geaaru has quit IRC | 16:17 | |
| *** gyee has joined #openstack-swift | 16:17 | |
| *** lyrrad has joined #openstack-swift | 16:17 | |
| acoles | notmyname: timburke jrichli kota_ torgomatic mattoliverau - we have an option to change user metadata handling and avoid the contentious patch 328207 altogether, change is summarised here https://etherpad.openstack.org/p/swift_encryption_meta_headers, I'd like feedback before I push to gerrit (or not) | 16:22 |
| patchbot | acoles: https://review.openstack.org/#/c/328207/ - swift (feature/crypto-review) - Allow middleware to override metadata header checking | 16:22 |
| *** cdelatte has quit IRC | 16:22 | |
| jrichli | acoles: ill take a look | 16:23 |
| *** asettle has quit IRC | 16:27 | |
| jrichli | acoles: sounds like a good plan - still thinking it all through. | 16:31 |
| *** daemontool_ has quit IRC | 16:35 | |
| *** sambhare has joined #openstack-swift | 16:37 | |
| timburke | good morning | 16:39 |
| *** Suyash has quit IRC | 16:39 | |
| jrichli | acoles: I think this solution could also be applied to account and container metadata, but I know that the "transient-sysmeta" was originally developed only for object metadata. | 16:40 |
| *** joeljwright has quit IRC | 16:40 | |
| jrichli | would you see this being the same scheme we'd maybe use for acct and cont in the future? | 16:40 |
| *** Suyash has joined #openstack-swift | 16:40 | |
| *** rledisez has quit IRC | 16:42 | |
| *** sambhare has quit IRC | 16:43 | |
| *** ManojK has quit IRC | 16:43 | |
| *** permalac has joined #openstack-swift | 16:43 | |
| *** asettle has joined #openstack-swift | 16:44 | |
| *** ManojK has joined #openstack-swift | 16:45 | |
| *** permalac_ has quit IRC | 16:45 | |
| acoles | jrichli: No. they would use sysmeta. but otherwise, could be similar pattern. | 16:45 |
| timburke | jrichli: i think it might be a bit messier for account/container data since we'd need to reconcile between the encrypted sysmeta and any lurking user meta. it may be sufficient to have the encrypter *always* set both (clearing the encrypted sysmeta when encryption is disabled), but we'd need to ensure it stays in the pipeline | 16:46 |
| *** Suyash has quit IRC | 16:48 | |
| *** zul_ is now known as zul | 16:48 | |
| *** cdelatte has joined #openstack-swift | 16:49 | |
| *** pcaruana has quit IRC | 16:49 | |
| *** rcernin has quit IRC | 16:51 | |
| *** cdelatte has quit IRC | 16:53 | |
| *** macsun has quit IRC | 16:54 | |
| *** tqtran has joined #openstack-swift | 16:59 | |
| timburke | acoles: your new (unsubmitted) patch still calls check_metadata in the encrypter, right? | 17:01 |
| acoles | timburke: it should ;) | 17:01 |
| acoles | timburke: yes, just double checked | 17:02 |
| timburke | acoles: thanks :) | 17:02 |
| *** gyee has quit IRC | 17:03 | |
| timburke | i'm starting to wonder whether the version in the key_id needs to have some sort of namespacing for the keymaster that generated it... | 17:04 |
| *** vinsh has quit IRC | 17:04 | |
| *** ChubYann has joined #openstack-swift | 17:04 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Enable middleware to set metadata on object POST https://review.openstack.org/328206 | 17:04 |
| *** asettle has quit IRC | 17:04 | |
| *** vinsh has joined #openstack-swift | 17:04 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Allow middleware to override metadata header checking https://review.openstack.org/328207 | 17:04 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Enable object body and metadata encryption https://review.openstack.org/328208 | 17:04 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Add encryption overview doc https://review.openstack.org/328209 | 17:05 |
| *** Suyash has joined #openstack-swift | 17:05 | |
| acoles | ^^ today's updates, thanks for everyone's reviews | 17:05 |
| *** arcimboldo has quit IRC | 17:07 | |
| jrichli | timburke: I am in favor of the key_id namespacing. good idea | 17:08 |
| *** cdelatte has joined #openstack-swift | 17:11 | |
| *** cdelatte has quit IRC | 17:11 | |
| clayg | i think it's perfectly reasonable to expect there to be only one keymaster | 17:14 |
| acoles | jrichli: timburke are you planning to support multiple keymasters contributing keys for the *same* object request? I only imagined a single keymaster providing keys for any given request, so the key_id meta does not need to accommodate contributions from multiple sources | 17:15 |
| clayg | let's wait to have two keymasters because we invent a use-case for two keymasters - and then also lets just write a single keymaster that can talk to multiple backends specific for that use-case | 17:16 |
| clayg | *before we invent | 17:16 |
| * acoles back later | 17:17 | |
| *** flwang1 has joined #openstack-swift | 17:17 | |
| flwang1 | greetings, i have some questions about swift monitoring | 17:17 |
| flwang1 | if we would like to focus on the cluster health, except dispersion and recon, anything else i can leverage? | 17:19 |
| flwang1 | thanks | 17:19 |
| *** hoonetorg has quit IRC | 17:19 | |
| jrichli | clayg acoles: I see that only one key could apply at one time. I guess I was thinking of the breadcrumbs to indicate which keymaster may be associated. Somebody using a custom keymaster may be able to add to the crypto-meta - ill think about that | 17:20 |
| timburke | acoles: i'd imagine that for conditional requests we'd need to get all possible keys for the target, for PUTs we'd need some designated "primary" key, and for GETs we'd need to be able to request a particular key | 17:20 |
| clayg | flwang1: log aggregation and statsd are useful tools for various operational scenarios - althought perhaps not quite as useful as dispersion reporting and recon for like... leveled health/alert triggers | 17:24 |
| flwang1 | clayg: thanks, based on my understanding, 'health' of swift means all the data can have the eventual consistency, swift has already provide a lot of process/services to ensure that | 17:26 |
| flwang1 | so does that mean generally, we just need to make sure those services/process are in a good status? | 17:26 |
| clayg | flwang1: yeah sure *generally* | 17:27 |
| flwang1 | clayg: for example, when we see '77.56% of object copies found (6094 of 7857)' from dispersion report, does that mean swift is not healthy? | 17:28 |
| clayg | flwang1: it probably means it's replicating - or you have some capacity that's not available | 17:28 |
| flwang1 | clayg: yep, that's what i'm trying to understand | 17:29 |
| flwang1 | so what's the metric can help us define if the cluster is healthy or not? | 17:29 |
| clayg | there's no one number - it takes lots of input | 17:30 |
| jrichli | clay acoles timburke : ah, I am just catching up with reviews from the last couple days. acoles had already replied to these concerns with key['id'] can be a dict (in fact, it is in the keymaster), so that should allow other key manager impls to persist arbitrary private metadata with the object crypto-meta. | 17:30 |
| clayg | dispersion is a good canary - it's worth investigating - do you have devices that are not available - can you check? are the replicators logging lots of errors? | 17:31 |
| flwang1 | clayg: so dispersion is like a overview alarm/alert, and if we see it's not 'good', then we need to check the others, right? | 17:34 |
| *** d0ugal has quit IRC | 17:37 | |
| *** d0ugal has joined #openstack-swift | 17:37 | |
| *** d0ugal has quit IRC | 17:37 | |
| *** d0ugal has joined #openstack-swift | 17:37 | |
| *** vinsh_ has joined #openstack-swift | 17:40 | |
| *** d0ugal has quit IRC | 17:41 | |
| *** d0ugal has joined #openstack-swift | 17:42 | |
| *** d0ugal has quit IRC | 17:42 | |
| *** d0ugal has joined #openstack-swift | 17:42 | |
| *** vinsh has quit IRC | 17:44 | |
| *** rcernin has joined #openstack-swift | 17:44 | |
| *** mvk_ has quit IRC | 17:45 | |
| clayg | flwang1: yeah, but even as an alarm/alert you may *know* that you have a node down waiting for a new mb, or you may *know* that you just added six nodes and a bunch of replication is still pushing data to the new capacity | 17:49 |
| flwang1 | clayg: yep, i see. another question is, if one of the replicator or auditor is down, can we see any difference from the recon api response? | 17:50 |
| clayg | so it doesn't mean like nesscessarily "ZOMG YOU'RE GOINGTO LOOSE DATAZ!?/11" as much as "hey bro, just FYI stuff ain't where I'd expect, pls look, kthxbai" | 17:50 |
| clayg | flwang1: well they wouldn't be updating their recon drops - so you'd seem timestamps getting old - but it'd probably be a good idea to add some process monitoring/alerting - stuff shouldn't just "die" | 17:52 |
| *** cschwede has quit IRC | 17:52 | |
| flwang1 | clayg: yep, we do monitor the daemon as well. but some times process is alive but it may have died. hence i asked above question | 17:53 |
| flwang1 | clayg: btw, i think the recon api's data come from database, right? where can i find the database? | 17:54 |
| clayg | ah, yeah liked locked up on a tarpit disk or something - yeah if it's hung sometimes your best indicator is the recon drop not getting updates | 17:54 |
| clayg | it's just a json file | 17:54 |
| clayg | /var/[lib|cache]/swift/something.[recon|json] or something? | 17:54 |
| clayg | i'm going with /var/cache/swift/service.recon | 17:55 |
| acoles | timburke: OIC, https://review.openstack.org/#/c/328206/9/swift/common/request_helpers.py line 47, yes makes sense | 17:55 |
| patchbot | acoles: patch 328206 - swift (feature/crypto-review) - Enable middleware to set metadata on object POST | 17:55 |
| timburke | acoles: but like i said, could be done as a follow-up | 17:56 |
| *** diogogmt has quit IRC | 17:58 | |
| clayg | acoles: everyone picking on you about going with the obviously more correct and clear get_*_meta(key) instead of the gross and obviously leaky get_*_meta_prefix([type]) + 'key' (or is it + '-key' !?) | 17:58 |
| *** cschwede has joined #openstack-swift | 17:59 | |
| *** diogogmt has joined #openstack-swift | 18:00 | |
| flwang1 | clayg: cool, does swift use any database to save something? | 18:00 |
| pdardeau | clayg: tarpit disk. i like it. that one's gonna stick :-^ | 18:01 |
| clayg | flwang1: not for... monitoring - accounts and containers use sqlite databases... | 18:03 |
| timburke | clayg: acoles: i *like* passing the key well enough; i'm just somewhat hesitant to create a second style without enabling authors to also use that style for the existing functions | 18:04 |
| flwang1 | clayg: cool, i'm good now, thank you so much | 18:05 |
| flwang1 | clayg: i may bug you again :) | 18:05 |
| *** gyee has joined #openstack-swift | 18:08 | |
| *** klamath has quit IRC | 18:13 | |
| *** diogogmt has quit IRC | 18:16 | |
| *** mvk_ has joined #openstack-swift | 18:20 | |
| *** chsc has joined #openstack-swift | 18:22 | |
| *** chsc has joined #openstack-swift | 18:22 | |
| *** cdelatte has joined #openstack-swift | 18:24 | |
| *** pcaruana has joined #openstack-swift | 18:25 | |
| *** cdelatte has quit IRC | 18:25 | |
| *** mvk_ has quit IRC | 18:26 | |
| *** diogogmt has joined #openstack-swift | 18:28 | |
| *** PsionTheory has joined #openstack-swift | 18:29 | |
| *** zaitcev has joined #openstack-swift | 18:36 | |
| *** ChanServ sets mode: +v zaitcev | 18:36 | |
| *** pauloewerton has quit IRC | 18:37 | |
| *** PsionTheory has quit IRC | 18:39 | |
| *** MooingLemur has quit IRC | 18:39 | |
| *** ManojK has quit IRC | 18:44 | |
| *** ManojK has joined #openstack-swift | 18:45 | |
| *** mvk has joined #openstack-swift | 18:49 | |
| openstackgerrit | Michael Barton proposed openstack/swift: go: move hummingbird.go https://review.openstack.org/335171 | 18:51 |
| *** mvk has quit IRC | 18:52 | |
| *** sheel has quit IRC | 18:55 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 18:56 |
| acoles | notmyname: quick crypto-review update...the first 4 of 7 patches seem to be stablilising and collecting +2. The next patch 328207 is the subject of this etherpad discussion https://etherpad.openstack.org/p/swift_encryption_meta_headers. That leaves patch 328208 and the docs. The docs have had plenty of review so I hope are stable-ish. | 18:57 |
| patchbot | acoles: https://review.openstack.org/#/c/328207/ - swift (feature/crypto-review) - Allow middleware to override metadata header checking | 18:57 |
| patchbot | acoles: https://review.openstack.org/#/c/328208/ - swift (feature/crypto-review) - Enable object body and metadata encryption | 18:57 |
| acoles | notmyname: I am focussed now more on changes that might affect what we are writing to disk or config i.e. things that could make life hard if we need to change. Changes where we can to follow up on master I may or may not apply. | 19:00 |
| notmyname | acoles: sounds great. looking over the etherpad now | 19:01 |
| *** fifieldt has quit IRC | 19:01 | |
| notmyname | acoles: what's your gut feeling on if we can finish this up by the end of the week? | 19:01 |
| acoles | notmyname: unless there are unknown issues still to arise, I think we can, obviously, the reviewers decide. The two significant *known* issues that I think should be finalised before master first the subject of the etherpad https://etherpad.openstack.org/p/swift_encryption_meta_headers, and second a discussion that has started in review of 328208 over whether we need/want 3 discrete middlewares, or could simplify. IMHO w | 19:04 |
| acoles | e can simplify if we want to by at least merging the encrypter and decrypter (mostly a case of moving code between modules), and of course it is best to do that before we land a config pattern on master. | 19:04 |
| acoles | notmyname: but hopefully 4/7 are done, 5 is either done or not needed, 6 is the meat, and 7 is docs | 19:05 |
| acoles | I say 5 (patch 328207) is done or not needed because reviews have said any changes to address concerns could be done as a follow up. | 19:06 |
| patchbot | acoles: https://review.openstack.org/#/c/328207/ - swift (feature/crypto-review) - Allow middleware to override metadata header checking | 19:06 |
| *** asettle has joined #openstack-swift | 19:11 | |
| timburke | on combining middlewares -- note that we *could* keep the current structure (having discrete encrypter/decrypter filters that take an app and return a filtered app), and just roll them up ourselves instead of having paste do it for us | 19:13 |
| *** MooingLemur has joined #openstack-swift | 19:14 | |
| notmyname | timburke: ie one python module with 2 classes? or, 2 filter_factory() methods, I'd guess | 19:14 |
| acoles | timburke: ok. so the goal is to reduce the number of ways to break your pipeline by moving to 'keymaster encryption' - is that what you are thinking too? (I'm inclined to leave keymaster independent, but the other two can become one with no change to the callback interface) | 19:16 |
| timburke | notmyname: or even keep the two python modules. drop the encrypter/decrypter filter_factory entrypoints we have now and add a new crypto entrypoint that points to (idk) some filter_factory in crypto_utils | 19:16 |
| timburke | acoles: yeah, more or less | 19:16 |
| *** fifieldt has joined #openstack-swift | 19:16 | |
| *** vinsh_ has quit IRC | 19:17 | |
| timburke | i think operators are fairly aware of the fact that order of things matter. if we can get this down to two middlewares and we can say "look, this one needs to be to the left so the keys are available to the one on the right", i think it'll be a good improvement | 19:18 |
| acoles | timburke: the critical thing is what we say goes in the pipeline - how the code lives in modules, where the factory method is can all change later, so yeah whatever minimises code churn works for me. which is probably just creating a single factory method somewhere that chains the two filters. | 19:18 |
| timburke | yup | 19:19 |
| *** mmcardle has quit IRC | 19:19 | |
| acoles | timburke: yeah, its reasonable to expect ops to know left from right, but putting things in the middle might be a step too far :P | 19:19 |
| acoles | notmyname: timburke ok, how about this plan - I'll push etherpad plan B as separate review (cos i have limited time right now to run all my "QA" on the chain) - then if anyone has time they can review that and I can squash it in tomorrow - plus others can still disagree on the etherpad. | 19:22 |
| notmyname | +1 | 19:22 |
| * acoles thinks it will conflict with timburke's fantastic patch, so will be a little time | 19:22 | |
| timburke | which patch am i supposed to be doing now? | 19:22 |
| acoles | then tomorrow we can beat on filter factory changes | 19:23 |
| acoles | timburke: you already did it - the one you left for me to append etag meta to the end of the header | 19:23 |
| acoles | ;) | 19:23 |
| acoles | I already merged and publsihed that in latest version | 19:23 |
| timburke | yay! i was afraid i'd already promised something else :-) | 19:24 |
| acoles | timburke: sorry for the scare | 19:24 |
| *** gyee has quit IRC | 19:25 | |
| clayg | ok, so no more encrypted x-object-meta-<key> then? it's all about the composite x-object-transient-sysmeta-crypto-<key> now? | 19:29 |
| clayg | lol | 19:32 |
| *** siva_krish has quit IRC | 19:32 | |
| *** ametts has quit IRC | 19:40 | |
| *** flwang1 has quit IRC | 19:49 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Switch to transient sysmeta for stashing encrypted user metadata https://review.openstack.org/335209 | 19:51 |
| acoles | notmyname: timburke jrichli ^^ like i said, if we like it I will squash it in to the chain tomorrow | 19:52 |
| acoles | clayg: transient is where its at man | 19:52 |
| *** siva_krish has joined #openstack-swift | 19:53 | |
| notmyname | acoles: how is that in the dependency chain? | 19:53 |
| *** siva_krish has left #openstack-swift | 19:53 | |
| acoles | it depends on patch 328208 | 19:54 |
| patchbot | acoles: https://review.openstack.org/#/c/328208/ - swift (feature/crypto-review) - Enable object body and metadata encryption | 19:54 |
| *** hoonetorg has joined #openstack-swift | 19:54 | |
| notmyname | got it. thanks | 19:54 |
| acoles | it doesn't need patch 328207 but I don't want branch further down the chain since this is just testing the water | 19:55 |
| patchbot | acoles: https://review.openstack.org/#/c/328207/ - swift (feature/crypto-review) - Allow middleware to override metadata header checking | 19:55 |
| *** ametts has joined #openstack-swift | 19:56 | |
| notmyname | started running local tests on it. now to go pick up some lunch | 19:56 |
| acoles | I'm done for today, have to fix the washing machine now :/ | 19:57 |
| *** acoles is now known as acoles_ | 20:03 | |
| *** vinsh has joined #openstack-swift | 20:08 | |
| *** joeljwright has joined #openstack-swift | 20:16 | |
| *** ChanServ sets mode: +v joeljwright | 20:16 | |
| *** joeljwright has quit IRC | 20:16 | |
| *** silor has quit IRC | 20:18 | |
| *** asettle has quit IRC | 20:20 | |
| *** zul has quit IRC | 20:20 | |
| *** Suyash has quit IRC | 20:32 | |
| *** gyee has joined #openstack-swift | 20:39 | |
| mwheckmann | hello. Can anyone tell me what swift process would be responsible for *removing* partitions if I lower the replica count of a ring? | 20:41 |
| *** asettle has joined #openstack-swift | 20:42 | |
| notmyname | mwheckmann: the replication process (for replicated storage) or the reconciler for EC storage | 20:42 |
| *** Suyash has joined #openstack-swift | 20:42 | |
| mwheckmann | ok. That's what I was thinking. I'm guess the removal of partitions would be treated the same way as shuffling partitions of new devices are added? | 20:43 |
| notmyname | yup | 20:43 |
| mwheckmann | thanks | 20:43 |
| *** ManojK has quit IRC | 20:43 | |
| *** thumpba has quit IRC | 20:45 | |
| *** ManojK has joined #openstack-swift | 20:45 | |
| *** ametts has quit IRC | 20:49 | |
| *** ukaynar has joined #openstack-swift | 21:03 | |
| notmyname | timburke: I feel like this was asked before, but I don't remember the answer: why is encrypt_header_val() in encrypter.py instead of crypto_utils.py? | 21:08 |
| timburke | notmyname: just compartmentalizing encryption operations to encrypter.py, IIRC | 21:10 |
| *** cdelatte has joined #openstack-swift | 21:11 | |
| *** dmk0202 has joined #openstack-swift | 21:11 | |
| *** asettle has quit IRC | 21:40 | |
| *** zul has joined #openstack-swift | 21:41 | |
| *** dmk0202 has quit IRC | 21:42 | |
| *** ManojK has quit IRC | 21:46 | |
| *** flwang1 has joined #openstack-swift | 21:59 | |
| *** ManojK has joined #openstack-swift | 22:01 | |
| *** cdelatte has quit IRC | 22:05 | |
| *** ManojK has quit IRC | 22:16 | |
| *** _JZ_ has quit IRC | 22:29 | |
| *** ukaynar has quit IRC | 22:33 | |
| mattoliverau | morning | 22:41 |
| *** mwheckmann has quit IRC | 22:49 | |
| *** ManojK has joined #openstack-swift | 23:01 | |
| *** rcernin has quit IRC | 23:03 | |
| *** kei_yama has joined #openstack-swift | 23:11 | |
| *** zul has quit IRC | 23:24 | |
| notmyname | hi mattoliverau | 23:30 |
| *** chsc has quit IRC | 23:31 | |
| notmyname | clayg: without a keymaster goes boom | 23:34 |
| notmyname | gerrit down for everyone or just me? | 23:37 |
| notmyname | (that was temporary) | 23:43 |
| mattoliverau | notmyname: hey hey | 23:45 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!