Friday, 2016-07-01

*** arcimboldo has quit IRC00:04
*** lyrrad has quit IRC00:05
*** thumpba has joined #openstack-swift00:07
zaitcevThat should be fine... I just found this:00:07
zaitcev[zaitcev@lembas ~]$ grep TMPFS_XATTR /boot/config-4.*00:07
zaitcev/boot/config-4.3.5-300.fc23.x86_64:CONFIG_TMPFS_XATTR=y00:07
zaitcevSo it should work, just doesn't in reality for some weird reason.00:07
*** Suyash has quit IRC00:10
*** catintheroof has joined #openstack-swift00:13
*** NM has quit IRC00:15
*** ManojK has joined #openstack-swift00:15
jrichlitorgomatic notmyname mattoliverau: I love the cross-cultural education we all get from IRC chat00:20
*** NM has joined #openstack-swift00:25
*** thumpba has quit IRC00:31
*** cdelatte has quit IRC00:34
*** cdelatte has joined #openstack-swift00:37
*** NM has quit IRC00:38
*** kevinc has quit IRC00:38
openstackgerritMerged openstack/swift: Refactor tests and add tests  https://review.openstack.org/32820300:38
openstackgerritMerged openstack/swift: Support for http footers - Replication and EC  https://review.openstack.org/32820400:39
*** Jeffrey4l has joined #openstack-swift00:42
openstackgerritSamuel Merritt proposed openstack/swift: Add checksum to object extended attributes  https://review.openstack.org/33632300:44
openstackgerritMerged openstack/swift: Make container update override headers persistent  https://review.openstack.org/32820500:46
openstackgerritMerged openstack/swift: Enable middleware to set metadata on object POST  https://review.openstack.org/32820600:46
claygyou know, despite being obviously marketing doc - this is pretty fair to swift - it's just out-dated - http://docs.basho.com/riak/cs/2.1.1/references/appendices/comparisons/swift/00:51
*** rcernin has quit IRC00:55
*** cdelatte has quit IRC00:55
*** chlong has quit IRC01:06
*** tqtran_ has quit IRC01:07
notmynamenice to see those have landed on crypto-review. I'm going to eat, and I'll check in later tonight about the last two and getting a merge commit ready01:13
*** klamath has quit IRC01:27
*** klamath has joined #openstack-swift01:27
*** hoonetorg has joined #openstack-swift01:32
*** klamath has quit IRC01:33
*** klamath has joined #openstack-swift01:33
*** MooingLemur has joined #openstack-swift01:36
*** NM has joined #openstack-swift01:40
*** vint_bra has joined #openstack-swift01:46
*** dmorita has quit IRC01:55
*** dmorita has joined #openstack-swift01:56
*** ManojK has quit IRC01:58
jrichliyay01:59
*** NM has quit IRC02:04
*** vinsh has quit IRC02:25
*** chlong has joined #openstack-swift02:33
*** klamath has quit IRC02:35
*** dmorita has quit IRC02:39
*** _JZ_ has quit IRC02:45
*** tanee is now known as tanee_away02:52
*** sheel has joined #openstack-swift02:54
*** ChubYann has quit IRC02:55
*** catintheroof has quit IRC03:00
*** gyee has quit IRC03:00
*** jamielennox is now known as jamielennox|away03:03
zaitcevOkay, I'm going to test tempfs with attributes now. On my laptop. With EFI bootloader.03:05
*** zaitcev has quit IRC03:05
*** asettle has joined #openstack-swift03:08
*** ChubYann has joined #openstack-swift03:09
*** asettle has quit IRC03:15
*** vint_bra has quit IRC03:15
*** vint_bra has joined #openstack-swift03:19
notmynamekota_: mattoliverau: just checking in...03:19
kota_notmyname: still looking03:20
kota_sorry03:20
notmynameok :-)03:20
notmynameno need to apologize :-)03:20
kota_notmyname: quick question.03:21
notmynamesure03:21
kota_notmyname: docs said the conditional GET header (i.e. If-[None]-Match) translates to If-Match: match_etag,HMAC(object_key, match_etag)03:21
kota_I know we are using HMAC but03:22
kota_it still holds the raw match_etag?03:22
kota_I'm now checking the implementation03:22
*** vinsh has joined #openstack-swift03:25
kota_ah, yeah if-match can take more than one value.03:26
*** links has joined #openstack-swift03:26
notmynamesorry, was just doing a little testing locally to answer your question03:28
notmynamewhat do you mean by if-match taking more than one value?03:28
notmynamebut yeah, the client sends int he real etag and the encrypted object will end up doing an hmac on the passed in value and comparing that to the stored hmac of the real etag03:29
*** vinsh has quit IRC03:30
kota_https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.2403:31
kota_notmyname: ^^03:31
*** jamielennox|away is now known as jamielennox03:31
kota_that's just my lack of knowlege of HTTP rule.03:31
kota_ok, it probably works.03:31
notmynameok03:31
*** vint_bra has quit IRC03:32
kota_so that means not replaced but append the HMAC value to if-match header.03:32
kota_except "*" case :)03:32
kota_make sense.03:32
*** chlong has quit IRC03:34
*** ManojK has joined #openstack-swift03:41
*** SkyRocknRoll has joined #openstack-swift03:46
*** sgundur1 has left #openstack-swift03:48
*** klrmn1 has quit IRC04:02
*** tqtran has joined #openstack-swift04:05
*** tqtran has quit IRC04:10
*** tanee_away is now known as tanee04:15
*** tanee is now known as tanee_away04:15
*** kevinc has joined #openstack-swift04:22
*** kevinc has quit IRC04:23
*** kevinc has joined #openstack-swift04:23
*** vinsh has joined #openstack-swift04:26
*** vinsh has quit IRC04:31
*** zaitcev has joined #openstack-swift04:39
*** ChanServ sets mode: +v zaitcev04:39
*** dmorita has joined #openstack-swift04:39
*** psachin has joined #openstack-swift04:39
*** ManojK has quit IRC04:41
*** kevinc has quit IRC04:43
*** dmorita has quit IRC04:43
mattoliveraunotmyname: found a small bug during playing with it on my SAIO. nothing super major. But noted it in patch 32820804:50
patchbotmattoliverau: https://review.openstack.org/#/c/328208/ - swift (feature/crypto-review) - Enable object body and metadata encryption04:50
mattoliverauhappy for it to be a follow up, but it is a bug.04:51
mahatic_wow, glad to see crypto merges! two more to go I think04:53
mattoliveraumahatic_: yeah almost there04:54
mahatic_mattoliverau: valid point, it'd be a quick fix04:54
* mattoliverau goes for a beach walk cause its too nice and sunny out :)04:55
*** _JZ_ has joined #openstack-swift04:55
kota_notmyname: done on docs, I didn't get whole things on adding middleware yet but it's ok to start to get merged because probably I partially understood about the changes while I was considering some additional stuff (merging Encyrpter/Decrypter, check_metadata skipping, reading docs)04:58
kota_notmyname: and looks like some cores approve... oh, mattoliverau set it as -1 :P04:59
notmynamelooking at that now05:00
mattoliverauI know, happy to +2, but it is a bug.. Follow up is fine05:01
*** tanee_away is now known as tanee05:02
*** tanee is now known as tanee_away05:02
kota_mattoliverau: I think you got good catch from my eyes :)05:02
*** tanee_away is now known as tanee05:02
*** tanee is now known as tanee_away05:02
notmynamemattoliverau: so you're concerned about x-object-meta-Foo instead of X-Object-Meta-Foo?05:03
kota_notmyname: i think so05:08
mattoliverauYup05:10
mattoliverauUser metadata looks different, and different to what it used too05:10
*** chlong has joined #openstack-swift05:14
notmynamemattoliverau: but headers are case-insensitive05:15
timburkei was about to say. from https://tools.ietf.org/html/rfc7230#section-3.2 -- "Each header field consists of a case-insensitive field name..."05:16
notmynameI'm ok with changing to a HeaderKeyDict. that would probably be nicer. but I don't think it's a bug the way it is05:16
timburkeit'd probably be good to fix, but i don't know that it's a high priority05:16
mattoliverauSure, but when the response has mostly cased and the one of a few lower it looks untidy05:16
timburkethough notmyname, haven't we run into trouble before by changing the casing of header names? something about Etag vs. ETag?05:17
notmynameheh, yeah05:17
notmynamestupid java clients05:17
mattoliverauMaybe the gatekeeper could headerdict things seeing as it probably parses all the headers anyway,05:17
mattoliverauThen we never have to worry about it again :p05:17
timburkei like never having to worry about things again :-)05:18
mahatic_timburke: yeah, I remember that being an issue. casing of header names05:18
mattoliverauI say probably cause I'm still on the beach ;)05:18
mahatic_timburke: you probably must be talking in a diff context from mine :)05:19
mahatic_but I gotta recall where05:19
*** ppai has joined #openstack-swift05:20
timburkehow much do we care about the order of the headers?05:22
notmynameorder? I'd guess about zero05:22
timburkeor...screw it. don't really *need* to use HeaderKeyDict. can just use the str.title()05:23
claygmattoliverau: I think -1 is the right vote there fwiw05:23
notmynameclayg: about the casing?05:24
claygyup05:24
notmynameclayg: because of previous experience with ETag vs Etag?05:24
claygit's a tell too05:24
claygyeah but mostly clients05:25
timburkeoh...except...i think i want a way to have a middleware say "i know what i'm doing, don't touch my headers". hmm...05:26
*** vinsh has joined #openstack-swift05:28
*** hseipp has joined #openstack-swift05:28
timburkeand tests are annoying. don't get to use swob.Response05:29
*** vinsh has quit IRC05:33
*** ChubYann has quit IRC05:37
*** _JZ__ has joined #openstack-swift05:39
*** _JZ_ has quit IRC05:40
claygtimburke: mattoliverau: notmyname: I don't man, I'm not sure it's worth trying to do anything more fancy than this -> https://gist.github.com/clayg/cb7b670801cbcbc1a03abe6f140fe48305:41
notmynameyeah, that works too05:43
openstackgerritTim Burke proposed openstack/swift: Always title-case headers on the way out of gatekeeper  https://review.openstack.org/33637105:43
timburkein case we want it^^05:43
notmynameFWIW, I dont' see anywhere we do Etag->ETag in the code. and I'm getting Etag in my responses. but maybe that was just a long time ago and I'm misremembering05:43
claygit's not like you can control caseing on any of the other parts of the header - we're just mixing wire case with method case and that stupid option in common.wsgi https://github.com/openstack/swift/blob/029c2782ddfd933af20b697c00a46b1375e0f23e/swift/common/wsgi.py#L43605:44
*** hseipp has quit IRC05:45
timburkei was thinking the trouble was trying to go the other way? iirc, all the RFCs talke about ETag...so someone may have tried to move us away from simple title-casing, then we realized we'd break clients if we did05:45
claygnotmyname: try turning off that wsgi setting05:45
claygnotmyname: on disk it seems to be ETag05:46
claygwell...05:46
notmynameclayg: oh. TBH I don't remember that setting or what it does at all05:46
timburkeclayg: good link! yeah, i was thinking of swift3 when i said i might want an option to control it...05:46
claygyeah..05:46
claygtimburke: yeah but like that only effected mainly like swift3 being able to cook stuff up and get out of wsgi - not that swift will store the case you give it on disk05:47
kota_back, was part a bit.05:48
timburkeoh, kota_ *would* be good for this discussion!05:48
kota_ah, if we do at the gate keeper, swift3 absolutely affected for that.05:49
kota_I don't think, it's wsgi setting.05:51
kota_Swift is now using Etag (in title calse), S3 returens ETag instead.05:52
kota_and s3 java sdk hung up due to the difference.05:52
kota_Swift3 is now converting from Etag to ETag in itself.05:52
kota_in the custom HeaderKeyDict.05:53
kota_around here, https://github.com/openstack/swift3/blob/master/swift3/response.py#L31-L3705:54
kota_ah, it wrapps title() method :/05:54
claygheh05:54
notmynamethanks. that's what I was remembering. I forgot that it was related to S3 clients05:54
kota_and I don't think ETag is not suitable, I like Etag instead05:55
notmynamekota_: everybody agrees that the client is wrong in this case. but we can't change it ;-)05:55
kota_notmyname: yeah, if possible, I'd like to fix at S3 itself ;-)05:56
psachinping clayg, https://review.openstack.org/#/c/334238/. Please have a look. I implemented your suggestions.05:56
patchbotpsachin: patch 334238 - swift - Fix swift-get-nodes arg parsing for missing ring05:56
*** _JZ_ has joined #openstack-swift05:57
*** _JZ__ has quit IRC05:57
claygpsachin: hey!  thanks for jumpin' in channel05:57
notmynameso what do we want to do here with the headers? use clayg's simple .title()? use mattoliverau's slightly more verbose HeaderKeyDict?05:58
claygi saw that the other day and glanced at it - i thought it looked good!  but we're in the middle of crypto and I didn't leave a score because I didn't really test out the changes05:58
claygi was sort of thinking there was something that could might still break... or maybe I didn't see a unittest for the other defect I saw about -P <name> /etc/swift/account.ring.gz05:58
kota_oh,05:58
kota_oh,oh05:58
*** ChanServ sets mode: +v cschwede05:58
*** tqtran has joined #openstack-swift05:58
psachinclayg, Ohk, No problem05:59
mattoliverauor gatekeeper-ify it all.. or is that bad for swift3, was that the problem?05:59
kota_clayg: yeah, I forgot it, exctly we need the eventlet setting05:59
notmynamemattoliverau: yeah, seems like a gatekeeper change would break swift305:59
kota_clayg: exactly, if we don't set it, ETag will be changed to Etag anyway05:59
claygI really think the problem is that we're using lower case for that prefix there - not that generally we have an issue with bad handling of case headers?06:00
kota_need to look at again, probably I forgot something on it.06:01
notmynameclayg: yeah, looking at it again, I agree with you06:01
claygdo we use a headerkeydict when we send requests to storage nodes?06:01
*** _JZ_ has quit IRC06:02
timburkemattoliverau: notmyname: kota_: i could make the gatekeeper change still work for swift3 (assuming that we update swift3 as well)06:02
mattoliveraunotmyname, clayg +1. Clay's is nice and simple. I was just hacking it make sure it was happening while testing, not mine wasn't very effiecent :)06:02
claygi guess we should add a unittest before we push over, sigh06:03
*** tqtran has quit IRC06:03
mattoliverautimburke: I like the idea of it in gatekeeper, cause then people should use headerkeydict in middleware, but if not we clean it up.. but I don't want to break things like swift3 and force versions or what not.. so to get this merged clayg's fix works06:03
timburkeyeah, i'm not opposed to a quick, precise fix for this. i might still work on a more permanent fix (and risk making more work for third-party middlewares that want non-title-cased headers)06:05
notmynameclayg: seems that after I unpickle the xattr on disk, I get mostly title case headers. except for ETag06:13
openstackgerritTim Burke proposed openstack/swift: Always title-case headers on the way out of gatekeeper  https://review.openstack.org/33637106:15
notmynameand some are unicode, and some are strings. fun. that probably won't cause any problems at all at some point06:15
timburkefeel free to argue over the header name used in that06:15
timburkenotmyname: nah, sounds perfectly safe ;-)06:16
notmynametimburke: can we argue about your iterator that selects every header that's exactly that string?06:17
notmynamewould we have multiple ones (ie with different case)?06:17
mattoliveraunotmyname: lol, i was going to say timburke I prefer to see  x-backend-gatekeeper-preserve-casing titled :P06:18
timburkewell, i did in the commit message :P06:18
timburkenotmyname: we may; if any of them are true, preserve. the nice thing about any() is that it'll bail on the first positive hit06:20
kota_notmyname: with different case is unsafe actually06:21
mattoliverauThere has got to be a way to unroll one of the loops, cause in that method we are now looping through respose headers 3 times.06:21
notmynameclayg: are you writing a test for your .title() patch?06:21
kota_notmyname: I bet "Don't do that"06:21
mattoliverautimburke: oh yeah, any would do that, that's a little better06:21
kota_some virous scan program pick up them as malformed HTTP06:22
kota_notmyname: ah, techinically speaking, we could push the same header names (but different case) to the client as response but it should not be done.06:23
notmynameIIRC,that's allowed in HTTP06:24
timburkemattoliverau: four times :-( i'll think about it some more when it's not 11:30 or so06:24
kota_notmyname: ah, might be yes. so we need to fix security scan checks? :P06:25
mattoliverautimburke: lol, fair enough :P06:25
timburkeas i recall, we can push several of the same header with the same casing to the client. but client's may behave badly as a result06:26
kota_traking conversations, oh, we employ the new header namespace to save swift3?06:26
timburkekota_: that was my thinking. just have swift3 always send that flag back to the gatekeeper06:27
notmynamea swift developer, when confronted with a problem, says, "I know! We'll put it in its own namespace". Now the developer only has his own problems06:27
kota_timburke: to call "swift3 is safe on everything" :)06:27
timburkewe know *exactly* what we're doing. honest ;-)06:28
clayganyone still lookin at this?  https://gist.github.com/clayg/de2282be944d5ceee5c511e32732024206:28
timburkenotmyname: could be worse. i could've found a way to add a callback :P06:29
*** rcernin has joined #openstack-swift06:29
claygnotmyname: that namespace joke is *really* funny06:29
*** vinsh has joined #openstack-swift06:29
claygnotmyname: you could put that on a t-shirt or make a meme or something06:29
timburkeclayg: seems reasonable06:31
timburke(the gist. as well as the t-shirt idea, i suppose)06:32
claygwho's going to tell me not to type yes!?  https://gist.github.com/clayg/823f5b213b8b865250ccee3b6ac840a306:32
notmynameclayg: thanks for the test. looks good06:32
claygSTAND BACK - I'VE DONE THIS BEFORE06:32
notmynameclayg: oh good. quite the opposite, it means that I don't have to do it :-)06:32
* clayg breifly considers i'll ruin all of acoles hard work06:32
openstackgerritClay Gerrard proposed openstack/swift: Enable object body and metadata encryption  https://review.openstack.org/32820806:32
openstackgerritClay Gerrard proposed openstack/swift: Add encryption overview doc  https://review.openstack.org/32820906:32
notmynamenope. all the others are landed ;-)06:32
*** vinsh has quit IRC06:33
kota_clayg: nice, you merged the tests into them?06:34
notmynamelooks good so far06:34
notmynameclayg: sure you can still +2 it ;-)06:35
claygyeah i think if you diff against patch set 15 it looks pretty good06:35
claygnotmyname: i forgot the ;)06:35
claygi can +2 any change I want06:36
notmynameI feel like http://xkcd.com/1700/ is relevant06:37
notmyname(to last-minute bugs that are noticed)06:38
claygtimburke: doesn't 336371 like sort of ruin whatever that eventlet setting was trying to do for whatever reason kota explained06:38
timburkeclayg: well, it makes it a runtime decision06:39
claygnotmyname: salt them with emoji!06:39
*** silor has joined #openstack-swift06:39
claygtimburke: it makes what a runtime decision?  oh... sorry - i don't think i was following that test correctly06:39
*** ppai has left #openstack-swift06:40
*** ppai has joined #openstack-swift06:40
timburkei could probably make that more clear...06:40
*** hseipp has joined #openstack-swift06:42
timburkebut not tonight06:46
*** silor1 has joined #openstack-swift06:47
notmynamekota_: mattoliverau: could you leave your thoughts on patch 328208 with clayg's patch applied, before shutting down for the weekend?06:47
patchbotnotmyname: https://review.openstack.org/#/c/328208/ - swift (feature/crypto-review) - Enable object body and metadata encryption06:47
mattoliveraunotmyname: sure06:47
notmynamemattoliverau: thanks06:48
*** arcimboldo has joined #openstack-swift06:48
kota_notmyname: yup. And that looks good so far.06:48
notmynamekota_: thanks06:48
*** silor has quit IRC06:49
*** silor1 is now known as silor06:49
clayggetting late for me, I'll have to continue being passive aggressive with timburke in the am ;)06:52
* clayg hugs06:52
timburkeclayg: i don't know that anyone could ever call you passive...06:52
claygrofl06:53
clayg'night06:53
*** geaaru has joined #openstack-swift06:54
*** pcaruana has joined #openstack-swift06:57
kota_clayg (and probably west coast guys): have a good night o/06:57
*** tesseract- has joined #openstack-swift07:02
timburkeclayg: i still don't get it, but i can bug you about it in the morning. it seems like we're either not testing what we think we're testing, or we're just testing FakeSwift's behavior, and doing so in kind of a weird and non-obvious place07:09
openstackgerritClay Gerrard proposed openstack/swift: Stop digging for publicly_accessible ourselves  https://review.openstack.org/33630807:11
timburkeclayg: isn't that what the Related-Change: tag is for? that i already added? :P07:12
claygoh!07:13
claygno i didn't see that!  i think i don't know how to read those - thanks!07:13
*** asettle has joined #openstack-swift07:13
claygshit did i erase it07:13
timburkenope, still there. no worries :-)07:13
timburkevery end07:14
claygwtf, when I click it my back button doesn't work anymore07:14
timburkestupid redirects. hit back twice, quickly07:15
clayglol07:15
claygi'll watch out for those related-change tags more carefully, i'm sure it helps with context - but if the goal was consistency I think it's worth calling that out - but i'll try to do better snooping around for context07:15
claygtimburke: so if our Fake didn't use the HeaderKeyDict it'd be a shitty fake, if the real thing ignores case the fake should too07:16
timburkei'll buy that argument07:17
claygI didn't put them in all jumbled indicate that I expected the code under test to get jumbled headers - I did it to demonstrate (if you get into a debugger) that our app doesn't recieve cased up headers07:17
*** asettle has quit IRC07:17
claygand you can reason that really quickly if you a) look on disk at an object server or b) add some prints in your proxy07:17
clayg... if we wanted to test what that code does if the obect server were to send case-sensitive keys (which I think it would do if you store them that way) - I think we'd be dissaapointed with how it behaves07:18
clayganyway - obvs I should have said # the fake and the real ignore case, so here I'm going to type in whatever I want to make that obvious07:19
claygidk, maybe I was thinking of the fake setup more like a PUT (which is where the keys are normalized in the real)07:20
timburkeeh, doesn't really matter. just struck me as something weird. important part is the bug's fixed07:21
claygohhhhhhh kay07:21
psachinclayg, swift-get-nodes -P <name> /etc/swift/account.ring.gz, should not break any thing, but thanx, I added a test-case07:22
*** chlong has quit IRC07:23
claygpsachin: ah sweet -i'll look out ofr it?  on patch 334238 yeah?07:24
patchbotclayg: https://review.openstack.org/#/c/334238/ - swift - Fix swift-get-nodes arg parsing for missing ring07:24
*** openstackgerrit has quit IRC07:33
*** openstackgerrit has joined #openstack-swift07:33
openstackgerritJohn Dickinson proposed openstack/swift: Merge at-rest encryption feature into master  https://review.openstack.org/33640707:36
notmynamethere's the merge commit to master ^07:37
notmynameit's of course dependent on the last 2 open patches on crypto-review07:37
notmynameacoles_ is set as the patch author, with everyone else from the patch chain listed as co-authors07:38
notmynameand I think I'll go to bed now07:43
notmynamekota_: mattoliverau: I'll look for any more feedback tomorrow morning for the crypto branch07:44
notmynameand if things look good, we'll land these last 2 and the merge commit07:44
*** dmorita has joined #openstack-swift07:45
kota_notmyname: thanks for working on late time, I'll leave some comments but right now they are blockers07:45
kota_notmyname: good night.07:46
kota_are not blockers07:46
kota_my mis typing :/07:46
kota_sorry07:46
*** dmorita has quit IRC07:49
*** rcernin has quit IRC07:57
mahatic_notmyname: thanks for staying up late and working07:59
*** mmcardle has joined #openstack-swift08:00
*** dmorita has joined #openstack-swift08:05
*** asettle has joined #openstack-swift08:09
openstackgerritDavanum Srinivas (dims) proposed openstack/swift: [WIP] Testing latest u-c  https://review.openstack.org/31844108:10
*** dmorita has quit IRC08:10
*** rcernin has joined #openstack-swift08:11
*** acoles_ is now known as acoles08:16
*** openstackgerrit has quit IRC08:18
kota_notmyname: I left some comments on patch 328208 but it looks ok to merge it as it is.08:18
patchbotkota_: https://review.openstack.org/#/c/328208/ - swift (feature/crypto-review) - Enable object body and metadata encryption08:18
acolesgood morning08:18
kota_acoles: good morning08:18
acolesis it done?08:18
*** openstackgerrit has joined #openstack-swift08:18
acolesoh, maybe not08:18
kota_for my side, I did what I can do.08:19
kota_acoles: I think it is ready to merge. Probably final check with mattoliverau08:19
kota_is needed.08:19
acoleskota_: great, thanks for all your reviewing!08:20
kota_acoles: ;-)08:20
mattoliverauacoles: it's looking good to me, just checking that clayg's test works. and looks like it does. I remove the title() and it fails with the wrong case.08:21
kota_acoles: sorry, I couldn't finish to run whole tests in my local but I think it's ok for other cores did that.08:21
acolesmattoliverau: kota_ I've not caught up yet, sounds like clayg found something08:21
* acoles goes to fetch coffee and postpone relaxing08:22
kota_acoles: and I'm getting cough since yesterday so getting back home and going to hospital :/08:22
acoleskota_: ok take it easy and have a good weekend08:22
kota_acoles: mattolvieau found !08:22
mattoliverauacoles: I found a bug where the user metadata coming out of the decrypter didn't have the same case as all other headers in a response.08:22
kota_acoles: matt found a bug for the response header on case insentive and clayg fixed.08:23
mattoliverauso we just needed to call .title() like headerdict does.08:23
mattoliverauacoles: it's all in scroll back for when you get back with your coffee :)08:23
kota_bye guys, I'm looking forward to encryption landing while I'm enjoying weekend ;-)08:26
mattoliveraukota_: o/ have a great weekend and I hope you feel better so you can enjoy it :)08:28
acolesmattoliverau: yeah long scroll back. so aren't headers case insensitive? when does it matter that they are title-cased?08:28
mattoliverauacoles: yeah it's my OCD :P also remember the old java clients out there that had problems with etags (swift3 me thinks).08:29
mattoliverauits more, it used to be one way, and now it isn't so clients may break cause they have in the past.08:30
mattoliveraubesides, it's now fixed. and resposes are like what they were before. The unit tests just passed locally.08:31
mattoliveraualso they should be ok being case insensitve.. that's what the RFC says.. but who reads that :P08:31
acolesmattoliverau: service development would be so much easier if we had no clients :P08:32
mattoliveraulol, exactly08:32
*** Jeffrey4l has quit IRC08:33
acolesmattoliverau: so what's the plan? are we merging patch 328208 or waiting for more reviews?08:34
patchbotacoles: https://review.openstack.org/#/c/328208/ - swift (feature/crypto-review) - Enable object body and metadata encryption08:34
mattoliverauacoles: clayg pushed a new patchset, I'm reviewing it. I'll give it a +2, if kota does too I guess we could merge it ;) it did have a billion +2's before hand :)08:38
acolesmattoliverau: ok. TBH I think I had assumed that somehere in wsgi the header casing just got taken care of, but now I think more that would be in swob. You see, assumptions precede failure!08:39
mattoliverauLol, Tim and I were talking about doing it in gatekeeper, he's gone and created a patch, but again swift3 would break.08:40
*** janonymous has quit IRC08:41
*** Jeffrey4l has joined #openstack-swift08:55
acolesmattoliverau: hmm, confusing, wsgi spec says header names are case sensitive https://www.python.org/dev/peps/pep-3333/#the-start-response-callable, but then wsgi by default capitalizes them, but then we disable that in eventlet wsgi !?!08:57
*** dmorita has joined #openstack-swift08:59
acolesmattoliverau: anyway, glad you guys spotted it and fixed it, it would have triggered my OCD too if I had noticed. Presumably out func tests are insensitive to resp header case or we'd have noticed earlier.08:59
kota_(Waiting at hospital) AFAIK, the chang clayg suggested in wsgi was proposed assuming Swift response headers are safe while using HeaderKeyDict09:01
*** dmorita has quit IRC09:03
*** arcimboldo has quit IRC09:03
kota_So Swift3 assuming the response header should be title case and think of Swift3 is only guy wants to make non title case09:04
kota_But decrypter did09:04
*** janonymous has joined #openstack-swift09:06
kota_That caused the bug09:06
claygacoles: nowhey - mattoliverau found it09:10
claygacoles: i'm *so* disappointed too - because it was exactly the kind of bug i thought might be lurking - i missed it09:10
claygacoles: I didn't even find anything good in the Putters' patch!09:10
kota_clayg!? Night man!?09:12
acolesclayg: howdy, yeah, but now I'm concerned about the Etag header that we swap in, or should it be ETag ? 't' or 'T'09:13
claygyeah lets merge it09:13
acolesclayg: cos this says AWS SDK wants a 'T' in etag https://review.openstack.org/#/c/93780/09:13
patchbotacoles: patch 93780 - swift - Disable case-changing behavior in Eventlet (MERGED)09:13
claygacoles: oh, interesting...09:13
acolesbut then on master we return Etag with a 't'09:14
kota_Yes09:14
kota_Ah, my comment seems lost duo to my client09:15
kota_Tim suggested to solve it09:15
claygso strange... the object server *stores* ETag in the metadata!?09:15
claygunencrypted pipeline sure does return Etag tho...09:16
acolesyeah but swob Response title everything, so on master we won't get ETag09:16
kota_No Swift3 transrate the header name at Get response09:17
kota_In the pipeline09:17
acoleskota_: so swift3 translates Etag -> ETag ??09:17
kota_Yes09:17
acolesso why can't it title case all my decrypter x-object-meta- headers too ?!? <- that's a joke ;)09:18
kota_Hehe09:18
* acoles had a headache *before* starting work today :(09:19
kota_Wait, its time to open my laptop09:19
kota_Mobile is too slow to type09:20
kota_back with laptop09:21
acoleskota_: :) typing etag ETag and Etag consistently on a mobile can't be fun09:21
kota_https://github.com/openstack/swift3/blob/master/swift3/response.py#L31-L37 is the place to translate Etag -> ETag09:21
acolesthanks09:22
kota_plus x-amz-<usermeta> swift3 need with smaller case.09:22
kota_and timburke is thinking of making a new backend header namespace to pass to gatekeeper "this headers shouldn't be changed to titlecase"09:24
kota_and then, set title() at the gatekeeper.09:24
kota_After that, swift3 can pass ETag and all of x-amz-* headers as non-title case and then, gatekeeper will title() other headers.09:25
*** sheel has quit IRC09:25
*** ma9 has joined #openstack-swift09:27
ma9I would like to create a new user who should be able to list the contents of a container and read its files, but not create any new container nor file…. should he be member of a role which belongs to operator_roles?09:27
ma9I gets this warning message if he's not an operator: Access was denied to this resourc09:28
claygsounds like a use-case for hisashi's policy.json stuff - account ACLs have that concept - but I don't think Keystone supports them09:29
mattoliverauthis is timburke's patch 33637109:30
patchbotmattoliverau: https://review.openstack.org/#/c/336371/ - swift - Always title-case headers on the way out of gateke...09:30
* kota_ is waiting at hospital so long....09:30
mattoliverauAlso, are we going to merge thsi thing, or did you want to functest something clayg?09:31
mattoliveraukota_: :(09:31
*** vinsh has joined #openstack-swift09:31
kota_mattoliverau: but it allows to me to talk at irc ;)09:31
mattoliverau\o/09:31
kota_mattoliveau: thanks, I didn't notice that for patch 33637109:32
patchbotkota_: https://review.openstack.org/#/c/336371/ - swift - Always title-case headers on the way out of gateke...09:32
claygyeah i guess the obj/server uses a swob.Response (who puts all headers in the HeaderKeyDict09:32
mattoliverauk, I need to go bath the bub then run a quick errand to the shops. bbs09:33
kota_ah, looks like a bit different with my idea for passing the hing to gatekeeper.09:33
* mattoliverau notes that he's been a linux geek for too long, bath almost came out as bash.. and that changed the sentence quite a bit!09:34
kota_I don't it should not be just a boole (true/false) but csv_list which says "this header name keeps current case!"09:34
mattoliveraukota_: so long as you pass the backend header in the response in swift3, it shouldn't change things.09:35
acolesma9: operator role on a project will grant read and write access to the project. you may be able to achieve your goal using ACLs http://docs.openstack.org/developer/swift/overview_auth.html#access-control-using-keystoneauth09:35
*** vinsh has quit IRC09:35
kota_mattoliverau: take fun with baby ;-)09:35
*** arcimboldo has joined #openstack-swift09:36
claygwebob seems to like to to ETag09:37
claygand also with "" around it09:38
kota_clayg: lol09:38
kota_swift cannot follow on that :/09:38
claygI think we've always returned Etag since swob?09:41
claygwe should have functional tests for some of these warts tho09:41
acoles+109:42
kota_clayg: I think so, swob.Response.headers is a HeaderKeyDict09:42
kota_and it returns response.headers.items() for start_response09:42
kota_right?09:42
claygmattoliverau: merge it!09:42
claygkota_: yes definately09:43
acoleskota_: right09:43
kota_so any middleware can change the header name case after start_response of proxy_app if the middleware is not using Response.headers or something to ensure the title case.09:44
kota_that's why swift3 can do that.09:45
mahatic_I'm taking off early today, by the time I get back to check on things, maybe crypto will be merged and I will have a long scroll back with great insights09:45
mahatic_have a great weekend everyone09:45
acoleskota_: btw, yes the middleware.rst needs some more work, I just put something there as a placeholder09:46
kota_mahatic_: have a good time, you too.09:46
acolesmahatic_: you too! it's done...nearly09:46
kota_acoles: ;-)09:47
claygwhere *is* hisashi anyway?09:47
claygworking on monasca i guess?  http://stackalytics.com/?user_id=osanai-hisashi09:48
claygwtf is monasca?09:48
kota_looks like hosanai is not online09:48
kota_interesting09:49
acolesclayg: monitoring service09:50
claygHyperconverged Meets Cloud09:51
* kota_ is thinking what's happen at this hospital. I'm waiting about an hour even though no waiting patient when I arrived here !?!?09:52
* clayg hopes everyone is ok :\09:53
*** tqtran has joined #openstack-swift10:01
*** tqtran has quit IRC10:05
claygpsachin: you still about?  I don't think -P syntax is working at all, and when you hit the InfoExit block from the cli it's not displaying the error to the user :'(10:06
*** kei_yama has quit IRC10:09
*** rcernin has quit IRC10:10
acolesclayg: mattoliverau kota_ so who is hitting +A on patch 328208? feel like it shouldn't really be me10:21
patchbotacoles: https://review.openstack.org/#/c/328208/ - swift (feature/crypto-review) - Enable object body and metadata encryption10:21
*** rcernin has joined #openstack-swift10:23
*** dmorita has joined #openstack-swift10:24
*** haypo has joined #openstack-swift10:26
haypoyarkot: hello. for my dict.iteritems() vs dict.items() change https://review.openstack.org/#/c/333298/ , i don't have a strong opinion. i tried to follow the same rule for all openstack projects10:27
patchbothaypo: patch 333298 - swift - Python 3: dict.iteritems() and dict.itervalues()10:27
haypoyarkot: i expect that the creation of a temporary python list is a negligible impact on performance10:27
haypoyarkot: in the past, i got negative feedback in swift when i started to use six "everywhere", the readability argument matter10:28
mattoliverauacoles: I'm happy too :)10:28
*** dmorita has quit IRC10:28
acolesmattoliverau: ok it looks good to me, thanks to you and clayg for the fix10:29
haypodoes anyone have an opinion on https://review.openstack.org/#/c/333298/ : six.iteritems(dict) (less readable) versus dict.items() (create a temporary list on python 2)10:29
patchbothaypo: patch 333298 - swift - Python 3: dict.iteritems() and dict.itervalues()10:29
haypothe rationale for dict.items() can be found in this email thread, http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html10:30
mattoliverauacoles: lol, clayg already did, but I'll do the doco ne10:30
mattoliverau*doc one10:30
mattoliverauacoles: congrats, tis all +Aed!10:32
mattoliverauand on that note, I'm going to call it a night!10:32
acoleshehe clayg left us his key10:32
*** vinsh has joined #openstack-swift10:32
acolesmattoliverau: thanks, have a good weekend10:32
mattoliverauacoles: have a great night and weekend, relax you deserve it!10:32
*** vinsh has quit IRC10:37
haypoacoles, clayg : hello. would you mind to review my python3 patch serie? https://review.openstack.org/#/c/333297/ Pete Zaitcev already voted +2 ;)10:38
patchbothaypo: patch 333297 - swift - Python 3: Fix basestring, long and StringIO10:38
claygohsnap - it's all +A'd now - i really gotta knock off10:40
*** mmcardle has quit IRC10:57
*** asettle has quit IRC11:01
*** SkyRocknRoll has quit IRC11:06
*** SkyRocknRoll has joined #openstack-swift11:19
*** cbartz has joined #openstack-swift11:27
openstackgerritMerged openstack/swift: Enable object body and metadata encryption  https://review.openstack.org/32820811:29
openstackgerritMerged openstack/swift: Add encryption overview doc  https://review.openstack.org/32820911:30
*** thumpba has joined #openstack-swift11:37
*** ManojK has joined #openstack-swift11:58
*** raildo-afk is now known as raildo12:01
*** asettle has joined #openstack-swift12:02
*** asettle has quit IRC12:07
*** derekjhyang has quit IRC12:11
*** SkyRocknRoll has quit IRC12:14
*** mmcardle has joined #openstack-swift12:16
*** SkyRocknRoll has joined #openstack-swift12:19
*** mmcardle has quit IRC12:21
*** SkyRocknRoll has quit IRC12:25
tdasilvaacoles: good morning and congrats! looks like everything merged12:27
openstackgerritGábor Antal proposed openstack/swift: Replaced some assertTrue with more specific asserts  https://review.openstack.org/28764312:33
*** vinsh has joined #openstack-swift12:33
*** vinsh has quit IRC12:38
*** asettle has joined #openstack-swift12:41
*** asettle has quit IRC12:47
*** NM has joined #openstack-swift12:52
*** d0ugal has quit IRC12:54
*** d0ugal has joined #openstack-swift12:54
*** d0ugal has quit IRC12:54
*** d0ugal has joined #openstack-swift12:54
*** d0ugal has quit IRC12:55
*** d0ugal has joined #openstack-swift12:55
*** d0ugal has quit IRC12:55
*** d0ugal has joined #openstack-swift12:55
acolestdasilva: good morning. yes all merged onto feature-crypto and notmyname has merge commit patch ready for master :) clayg and mattoliverau found and fixed something last night, but all done now.12:59
*** d0ugal has quit IRC13:00
*** d0ugal has joined #openstack-swift13:00
*** _JZ_ has joined #openstack-swift13:01
*** d0ugal has quit IRC13:02
*** d0ugal has joined #openstack-swift13:03
*** d0ugal has quit IRC13:03
*** d0ugal has joined #openstack-swift13:03
*** d0ugal has quit IRC13:03
*** ManojK has quit IRC13:03
*** siva_krish has joined #openstack-swift13:03
*** Guest12790 has joined #openstack-swift13:03
*** Guest12790 has quit IRC13:04
*** d0ugal has joined #openstack-swift13:04
*** dmorita has joined #openstack-swift13:16
*** psachin has quit IRC13:16
notmynamegood morning13:19
notmynamereading scrollback13:19
*** dmorita has quit IRC13:21
*** ManojK has joined #openstack-swift13:22
*** ppai has quit IRC13:23
notmynameall right!13:26
acolesnotmyname: all merged!13:33
notmynameI just got through IRC playback and the review emails.13:33
notmynameyup, looks good :-)13:33
notmynameacoles: does https://review.openstack.org/#/c/336407/1 look good to you (author, commit message)?13:34
patchbotnotmyname: patch 336407 - swift - Merge at-rest encryption feature into master13:34
acolesnotmyname: just checking the co-authors (sorry, should have done that before)13:35
notmynamelooks like a lot of failures from the community cluster. checking those now13:36
*** SkyRocknRoll has joined #openstack-swift13:37
*** Jeffrey4l has quit IRC13:40
notmynamehmm..tests just passed locally for me13:40
acolesnotmyname: I want to say that I've seen the community CI randomly fail before, but I'd also love to see it pass for this patch!13:41
acolesnotmyname: I am just running tests with the merge commit branch13:42
notmynameyeah. well, it passed with EC and all the gate tests passed13:42
notmynameand it passed for the crypto doc patch13:43
*** arcimboldo has quit IRC13:43
*** daemontool has joined #openstack-swift13:43
notmynameacoles: let me know how your test run goes. I'm good with it on this side13:47
acolesnotmyname: we appear to have some patches land on master, just going through to look for anything that could matter13:47
*** asettle has joined #openstack-swift13:49
acolesnotmyname: is it ok that the patch to change .gitreview on feature/crypto-review is included in the merge to master? if I git checkout HEAD~1 then I skip back to before any of the crypto patches, but if I git checkout 96a0e077 then i will have the default branch set to crypto-review.13:56
acolesI guess it has to be that way13:56
acolesmy func tests pass, EC w/wo fast-post13:56
notmynamesince we're landing the feature as a merge commit, I don't think there's a reasonable expectation that everything is just like normal if you check out some commit on the feature branch. it's good to be able to do that for future debugging (and was good for our reviewing), but not reasonable to start from one of those and do dev or prod stuff13:58
notmynamethe way around that next time would be to have the -review branch maintainer make sure to specify the -review branch in every call to `git review`13:59
notmynamethat way we wouldn't land the .gitreview update on the feature-review branch14:00
acolesnotmyname: ok. I'm happy.14:01
notmynameok, let's make you even happier now14:02
notmynamehttps://review.openstack.org/#/c/336407/  <-- +2/+A14:02
patchbotnotmyname: patch 336407 - swift - Merge at-rest encryption feature into master14:02
acolesnotmyname: I think the way we did it with .gitreview undoubtedly saved some mistakes :)14:02
acoleswait....14:03
notmynamenot funny14:03
acolesJK14:03
notmyname;-)14:03
notmyname...that joke happens every time...14:03
*** ManojK has quit IRC14:03
acolesyeah I know, sorry14:04
notmynameno, it's still funny :-)14:04
*** arcimboldo has joined #openstack-swift14:05
notmynameok, I added a small update to http://not.mn/swift/swift_community_dashboard.html14:06
acolesnotmyname: lol (the dots)14:06
notmynameit's what I found to copy/paste in last night at 1am ;-)14:07
acolesjrichli: mahatic_ are you here ? the fireworks are going off14:08
notmyname#success the 1.5 year long effort to get at-rest encryption in openstack swift has been finished. at-rest crypto has landed in master14:09
openstackstatusnotmyname: Added success to Success page14:09
acoleswhere is the success page?14:09
acoles/dev/null ?14:09
notmynameI think it's a wiki page somewhere14:09
NMopenstackstatus: Go home, you're drunk.14:10
acoleswiki? well that's going away then!14:10
notmynamehttps://wiki.openstack.org/wiki/Successes14:10
notmynameIIRC it goes in a weekly mailing list post too14:11
acolesnotmyname: hmm, we're not honest as a community -> https://wiki.openstack.org/wiki/Failures14:11
notmynamelol14:11
notmynamefor those of you who do the twitters, https://twitter.com/notmyname/status/74888185851596390414:12
*** daemontool has quit IRC14:12
notmynametime for me to go get ready for the day14:12
notmynameacoles: enjoy your weekend14:13
notmynameI'll be back online later this morning14:13
acolesnotmyname: thanks, and enjoy your holiday14:13
*** daemontool has joined #openstack-swift14:14
*** haypo has left #openstack-swift14:14
*** catintheroof has joined #openstack-swift14:14
*** vint_bra has joined #openstack-swift14:19
*** arcimboldo has quit IRC14:20
*** ManojK has joined #openstack-swift14:21
*** links has quit IRC14:25
*** links has joined #openstack-swift14:27
*** cbartz has left #openstack-swift14:28
jrichliOMG!  It has really happened.  Thanks to everyone!  Just read scrollback - lots of activity since i went to sleep!14:32
*** vint_bra has quit IRC14:33
*** arcimboldo has joined #openstack-swift14:34
jrichlinotmyname: love the dashboard!14:35
*** vinsh has joined #openstack-swift14:35
*** kevinc has joined #openstack-swift14:36
*** kevinc has quit IRC14:36
*** nadeem has joined #openstack-swift14:36
*** kevinc has joined #openstack-swift14:36
*** nadeem has quit IRC14:37
*** dmorita has joined #openstack-swift14:38
*** nadeem has joined #openstack-swift14:38
*** vint_bra has joined #openstack-swift14:38
*** vinsh has quit IRC14:40
*** SkyRocknRoll has quit IRC14:51
tdasilvajrichli, acoles: congrats to you both and everyone else involved!14:51
*** SkyRocknRoll has joined #openstack-swift14:52
jrichlitdasilva: thank you for all your help!14:56
openstackgerritMerged openstack/swift: Merge at-rest encryption feature into master  https://review.openstack.org/33640715:02
*** links has quit IRC15:02
*** ManojK has quit IRC15:04
*** vinsh has joined #openstack-swift15:06
*** jistr is now known as jistr|mtg15:07
*** pcaruana has quit IRC15:08
*** SkyRocknRoll has quit IRC15:08
*** vint_bra has quit IRC15:09
*** sgundur1 has joined #openstack-swift15:09
acoles^^ wheee15:10
acolestdasilva: you're part of it!15:10
*** ma9 has left #openstack-swift15:10
*** rcernin has quit IRC15:14
mathiasbgreat job everyone!15:16
*** admin6 has left #openstack-swift15:20
*** SkyRocknRoll has joined #openstack-swift15:21
yarkothaypo - re: six.iter* ... my comment isn't a particularly strong one;  it's more of pick a way for a changeset, and stick with it, otherwise info gets lost;  I'm not sure about the subtle unexpected openining of a door for an unexpected behavioral change (certainly in the one "six" exception you made, that was manually anticipated), but the old engineer in me reacted w/ discomfort to that too.   If you're really not intent on making it consistent,15:21
yarkot I can change my comment to a zero rating - but I'd be happy if you'd consider the safer, consistent route (rather than being swayed by the social "anti six.*" pressure, which I think makes more sense in a general decision point:  "don't use unless really warranted";  I just added "then stay consistent";  ymmv.)15:21
yarkot^haypo15:22
openstackgerritAlistair Coles proposed openstack/swift: Add encrypter and decrypter links to middleware.rst  https://review.openstack.org/33661815:23
*** Jeffrey4l has joined #openstack-swift15:24
acolestimburke: ^^ includes fix for the dodgy test, you are co-authored. kota_: follow up on your last review comments15:24
*** Jeffrey4l has quit IRC15:24
*** klrmn has joined #openstack-swift15:27
*** ukaynar has joined #openstack-swift15:27
*** daemontool has quit IRC15:28
*** chsc has joined #openstack-swift15:28
*** chsc has joined #openstack-swift15:28
*** tesseract- has quit IRC15:28
acolestimburke: mahatic_ jrichli notmyname fyi i have tidied the trello board and I'm still tracking follow up tasks there https://trello.com/b/63l5zQhq/swift-encryption15:38
acolesplease add any left over from review15:38
*** manous has joined #openstack-swift15:40
notmynameok, thanks15:42
notmynametoday I'll be looking at other patches that should land shortly that were on hold until crypto landed15:42
acolesnotmyname: I intend to start reviewing the container sync patche snext week15:42
tdasilvainteresting problem triple folks hit with swift: https://bugs.launchpad.net/tripleo/+bug/159591615:42
openstackLaunchpad bug 1595916 in tripleo "Swift memory usage grows until it is killed" [High,New] - Assigned to Emilien Macchi (emilienm)15:42
tdasilvatl:dr it was a pipeline configuration issue15:43
notmynametdasilva: indeed. nice to see some further work being done on it15:43
*** lyrrad has joined #openstack-swift15:43
*** david-lyle has joined #openstack-swift15:44
notmynameI see only one instance of proxy-logging in pipelines in nearly every config that people looking for help show me15:44
notmynameso even if it's misconfigured, it would be good to find and fix a memory bug there15:44
*** NM has quit IRC15:45
*** dmorita has quit IRC15:45
*** dmorita has joined #openstack-swift15:46
*** jmccarthy has quit IRC15:46
cschwedejrichli: acoles: (and everyone else who worked on it): congrats! very well done \o/15:47
*** jmccarthy has joined #openstack-swift15:47
*** NM has joined #openstack-swift15:48
*** jistr|mtg is now known as jistr15:50
*** ukaynar has quit IRC15:56
*** manous has quit IRC15:58
*** david-lyle_ has joined #openstack-swift16:00
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873616:01
*** nadeem has quit IRC16:02
*** tqtran has joined #openstack-swift16:03
*** nadeem has joined #openstack-swift16:04
*** nadeem has quit IRC16:04
*** david-lyle has quit IRC16:04
*** david-lyle_ is now known as david-lyle16:04
*** tqtran has quit IRC16:07
*** cdelatte has joined #openstack-swift16:10
*** manous has joined #openstack-swift16:11
*** vinsh has quit IRC16:15
*** derekh has joined #openstack-swift16:15
*** ChubYann has joined #openstack-swift16:16
derekhclayg: you were commenting on a tripleo bug a few days ago we had with swift memory usage https://bugs.launchpad.net/tripleo/+bug/159591616:17
openstackLaunchpad bug 1595916 in tripleo "Swift memory usage grows until it is killed" [High,New] - Assigned to Emilien Macchi (emilienm)16:17
derekhclayg: see my last comment would the absence of a second entry for proxy-logging in the proxy pipeline explain things?16:18
claygderekh: i don't think so - did adding it make a difference (!?)16:21
claygderekh: I really can't imagine any good reason for that sort of memory usage :'(16:21
derekhclayg: yes, I can reliable reproduce the problem by removing it, then adding it back in fixes the problem16:21
claygPHASPFOIUHWEF9H??!!16:22
* clayg exploded16:22
claygsomething like... eventlet posthook loggers set in the middleware piling up or something ?16:24
*** diogogmt has joined #openstack-swift16:26
derekhclayg: maybe, I don't see any logs of use but will poke at it some more to see if I can get info16:27
*** klrmn has quit IRC16:27
*** lyrrad has quit IRC16:28
claygderekh: yeah thanks for the heads up I'll try and repro some in dev - that's a pretty big landmind16:28
derekhclayg: it only seems to happen with a certain image I have, want me to upload it somewhere ?16:29
*** geaaru has quit IRC16:30
claygmerge commits happen fast?16:31
*** daemontool has joined #openstack-swift16:31
derekhwas that a question for me?16:32
*** gyee has joined #openstack-swift16:33
*** vint_bra has joined #openstack-swift16:35
acolestdasilva: I updated patch 324607 to keep just the tests that didn't make it onto feature/crypto-review16:37
patchbotacoles: https://review.openstack.org/#/c/324607/ - swift - Add test for copy using a Range header16:37
acolestdasilva: you are co-author16:37
*** diogogmt has quit IRC16:37
*** asettle has quit IRC16:38
tdasilvaacoles: ack, thanks!16:38
*** derekh has quit IRC16:39
*** thumpba_ has joined #openstack-swift16:42
openstackgerritThiago da Silva proposed openstack/swift: Add "history" mode to versioned_writes middleware  https://review.openstack.org/21492216:42
tdasilvatimburke: ^^^ just a rebase16:42
*** thumpba has quit IRC16:44
*** hseipp has quit IRC16:45
notmynameheh, the raspberry pi showing the community dashboard here in the office is really struggling with that javascript confetti16:46
zaitcevGosh I have i7 with 16 GB struggling with JavaScript in Gerrit16:46
acolesgood night. enjoy the holiday weekend those in US.16:51
*** acoles is now known as acoles_16:52
notmynamezaitcev: http://d.not.mn/rpi_cpu.png16:52
*** asettle has joined #openstack-swift16:54
notmynameoh, interesting. looks like there's an -infra maintenance window in a few hours. I'm very happy we got crypto landed before that :-)16:56
*** vint_bra has quit IRC16:56
zaitcevlucky16:56
*** SkyRocknRoll has quit IRC16:57
zaitcevEarly bird gets the worm.16:57
openstackgerritTim Burke proposed openstack/swift: Add sanity checks to strip_*meta_prefix functions  https://review.openstack.org/33666917:06
openstackgerritTim Burke proposed openstack/swift: Only use Timeout if we don't have a final_resp  https://review.openstack.org/33667117:07
openstackgerritTim Burke proposed openstack/swift: Use functools.partial to provide read/readline methods  https://review.openstack.org/33667217:08
*** manous has quit IRC17:09
*** nadeem has joined #openstack-swift17:13
*** tqtran has joined #openstack-swift17:15
*** klamath has joined #openstack-swift17:19
*** klamath has quit IRC17:20
notmynamevirtual box has a 20-step process from alerting you to a new version to launch a VM with the upgraded version17:20
*** klamath has joined #openstack-swift17:20
*** manous has joined #openstack-swift17:21
timburkemattoliverau: acoles_: the header casing isn't just about OCD-ness. i don't think we want the knowledge that encryption is enabled bleeding out to clients (that's why we have the admin=True flag in register_swift_info, right?)17:21
notmynametimburke: right, that's the big deal. exposing the knowledge17:22
*** vint_bra has joined #openstack-swift17:24
*** manous has quit IRC17:26
*** SkyRocknRoll has joined #openstack-swift17:28
*** manous has joined #openstack-swift17:38
*** SkyRocknRoll has quit IRC17:45
*** arch-nemesis has joined #openstack-swift17:50
*** silor has quit IRC17:51
*** asettle has quit IRC18:00
*** daemontool has quit IRC18:00
*** klrmn has joined #openstack-swift18:05
claygif a client set their mind to it they should be able to tell by measuring the speed at which proxies serve files/metadata of various bytes - if they can figureout passwords measuring strcmp they can probably make a pretty solid guess about crypto18:06
claygtimburke: in patch 336672 can you just say `read = functools.partial(...` and drop the the @property?18:07
patchbotclayg: https://review.openstack.org/#/c/336672/ - swift - Use functools.partial to provide read/readline met...18:07
*** ManojK has joined #openstack-swift18:15
*** nadeem has quit IRC18:18
*** ManojK has quit IRC18:21
*** ManojK has joined #openstack-swift18:28
*** klamath has quit IRC18:30
openstackgerritSamuel Merritt proposed openstack/swift: DO NOT MERGE: probe the gate to figure out what kind of filesystems are in there  https://review.openstack.org/33670318:30
claygtorgomatic: oh yeah that was it - the inprocess func seemed to only be passing because of the mock - rofl18:31
torgomaticclayg: yep. once I figure out what kind of environment the gate is running in, I can see WTF is going on18:32
mahatic_woohoo, crypto merged!18:34
mahatic_twitter directed me here18:35
mahatic_notmyname: acoles_ excellent fireworks!18:35
timburketdasilva: thanks for the rebase18:36
tdasilvatimburke: reviewing now :)18:36
timburkemahatic_: what are you doing awake? go sleep!18:37
timburketdasilva: extra thanks!18:37
mahatic_timburke: I saw notmyname's update on twitter, I wanted to dropby to see the fireworks ;)18:38
mahatic_now that i'm here, i'm gonna do a rebase of an old patch I guess18:38
*** arcimboldo has quit IRC18:39
notmynameclayg: interestingly, after having tested a bunch of public swift clusters, I wouldn't be able to guess about which ones have crytpo based on timings. the fastest clusters were 10x the slowest clusters18:47
claygnotmyname: no I mean there's like zero-byte stuff that's *not* encrypted, and the *amount* of metadata means more to encrypt - i bet you could do it all relative to a cluster - if there's some varience because of disk load in that cluster you just make more requests and average it out18:48
*** siva_krish has quit IRC19:00
*** siva_krish has joined #openstack-swift19:02
*** manous has quit IRC19:10
*** manous has joined #openstack-swift19:23
*** siva_krish has quit IRC19:32
*** siva_krish has joined #openstack-swift19:35
*** okdas has quit IRC19:40
*** pdardeau has quit IRC19:40
*** pdardeau has joined #openstack-swift19:41
*** tristanC has quit IRC19:41
*** andymccr has quit IRC19:41
*** gmmaha has quit IRC19:41
*** furlongm_ has joined #openstack-swift19:42
*** arch-nemesis has quit IRC19:42
*** notmyname has quit IRC19:42
*** j_king has quit IRC19:42
*** furlongm has quit IRC19:42
*** bhakta has quit IRC19:42
*** jidar has quit IRC19:42
*** glange has quit IRC19:42
*** manous has quit IRC19:43
*** tristanC has joined #openstack-swift19:43
*** j_king has joined #openstack-swift19:44
*** glange has joined #openstack-swift19:44
*** ChanServ sets mode: +v glange19:44
*** okdas has joined #openstack-swift19:47
*** okdas has joined #openstack-swift19:47
*** notmyname has joined #openstack-swift19:47
*** ChanServ sets mode: +v notmyname19:47
*** andymccr has joined #openstack-swift19:47
*** bhakta has joined #openstack-swift19:47
*** gmmaha has joined #openstack-swift19:48
-openstackstatus- NOTICE: The infrastructure team is taking Gerrit offline for maintenance beginning shortly after 20:00 UTC to upgrade the Zuul and static.openstack.org servers. We aim to have it back online around 22:00 UTC.19:54
*** arch-nemesis has joined #openstack-swift19:55
*** jidar has joined #openstack-swift19:56
*** asettle has joined #openstack-swift19:58
*** asettle has quit IRC19:58
*** nadeem has joined #openstack-swift19:58
*** ManojK has quit IRC20:06
*** chsc has quit IRC20:06
-openstackstatus- NOTICE: Gerrit is offline for maintenance until approximately 22:00 UTC20:07
*** ChanServ changes topic to "Gerrit is offline for maintenance until approximately 22:00 UTC"20:07
notmynameso gerrit's offline for another 2 hours. hmm..ok20:21
*** siva_krish has quit IRC20:22
torgomaticdid logs.openstack.org get taken down too?20:22
torgomaticit's up, but all my requests for logs are 404ing20:22
notmynamethey said something oabout static.openstack.org20:22
notmynameso maybe it links there? or is related?20:23
notmynamebah! I'm running out of excuses to deal with the email requests that have piled up as crypto got done20:23
notmynameI was going to look at some review, but I guess not now ;-)20:23
torgomaticweren't you going to shampoo your cat this afternoon?20:23
timburkeclayg: oh! i remember why i did the @property now! i needed the reference to self.wsgi_input. it also has the nice property of properly AttributeError'ing if wsgi_input doesn't have read/readline20:27
*** dmorita has quit IRC20:32
*** dmorita has joined #openstack-swift20:33
*** ManojK has joined #openstack-swift20:41
*** siva_krish has joined #openstack-swift20:44
*** ManojK has quit IRC20:48
*** dmorita has quit IRC20:55
*** dmorita has joined #openstack-swift20:57
*** asettle has joined #openstack-swift20:59
*** raildo is now known as raildo-afk21:01
*** vint_bra has quit IRC21:01
*** dmorita has quit IRC21:09
*** siva_krish has quit IRC21:15
*** dmorita has joined #openstack-swift21:21
*** arcimboldo has joined #openstack-swift21:23
*** asettle has quit IRC21:24
*** NM has quit IRC21:33
*** catintheroof has quit IRC21:36
*** siva_krish has joined #openstack-swift21:38
*** arcimboldo has quit IRC21:41
claygtimburke: oh of couse, very well then21:43
openstackgerritTim Burke proposed openstack/swift: Add encryption_root_secret_path option to keymaster  https://review.openstack.org/33672021:44
openstackgerritTim Burke proposed openstack/swift: Make load_crypto_meta more py3-friendly  https://review.openstack.org/33672121:44
*** patchbot has quit IRC21:47
*** patchbot` has joined #openstack-swift21:47
*** siva_krish has quit IRC21:48
*** furlongm has joined #openstack-swift21:48
*** patchbot` is now known as patchbot21:48
*** david-lyle has quit IRC21:51
*** tristanC_ has joined #openstack-swift21:51
*** furlongm_ has quit IRC21:53
*** tristanC has quit IRC21:53
*** jaakkos has joined #openstack-swift21:56
*** jaakkos_ has quit IRC21:57
*** wer has quit IRC22:13
*** thumpba_ has quit IRC22:14
*** dmorita has quit IRC22:27
*** chsc has joined #openstack-swift22:27
*** dmorita has joined #openstack-swift22:32
*** dmorita has quit IRC22:33
*** chsc has quit IRC22:35
*** asettle has joined #openstack-swift22:37
*** dmorita has joined #openstack-swift22:41
*** NM has joined #openstack-swift22:42
*** asettle has quit IRC22:42
brianclinekinda late to the party, but \o/ congrats to all the crypto contributors!22:45
*** mvk has quit IRC22:53
*** nadeem has quit IRC22:53
*** tristanC_ is now known as tristanC22:58
*** dmorita has quit IRC23:01
notmynameI'm taking off for the weekend. I'll be back online tuesday night or wednesday morning23:02
notmynamehave a good weekend, everyone23:02
brianclineyou too23:03
*** ManojK has joined #openstack-swift23:09
openstackgerritClay Gerrard proposed openstack/swift: Minor cleanups in Putter  https://review.openstack.org/33674023:11
openstackgerritClay Gerrard proposed openstack/swift: Add basic functests for user object metadata  https://review.openstack.org/33674123:12
*** kevinc has quit IRC23:12
openstackgerritTim Burke proposed openstack/swift: Fix gettext_ calls  https://review.openstack.org/33674323:13
*** dmorita has joined #openstack-swift23:13
claygbase64 decode will do *what* now!?23:14
*** dmorita has quit IRC23:18
openstackgerritSamuel Merritt proposed openstack/swift: Add checksum to object extended attributes  https://review.openstack.org/33632323:30
openstackgerritSamuel Merritt proposed openstack/swift: DO NOT MERGE: probe the gate to figure out what kind of filesystems are in there  https://review.openstack.org/33670323:30
*** gyee has quit IRC23:33
*** arch-nemesis has quit IRC23:38
*** NM has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!