21:00:21 #startmeeting swift 21:00:24 Meeting started Wed Jun 1 21:00:21 2016 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:25 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:27 The meeting name has been set to 'swift' 21:00:30 who's here for the swift team meeting? 21:00:32 o/ 21:00:34 o/ 21:00:35 hello 21:00:36 hey 21:00:36 hello 21:00:36 hi 21:00:38 o/ 21:00:39 o/ !buenos dias! 21:00:41 hi 21:00:42 o/ 21:00:42 hi 21:00:43 eu 21:00:43 hi 21:00:59 hi 21:01:06 o/ 21:01:08 o/ 21:01:13 o/ 21:01:26 hi 21:01:38 welcome everyone. good group today :-) 21:01:39 here 21:01:49 woo 21:01:56 agenda is at 21:01:57 #link https://wiki.openstack.org/wiki/Meetings/Swift 21:02:18 I want to skip around just a bit to put the cschwede things up front (it's latest for him, I think) 21:02:33 hello 21:02:34 #topic rolling upgrade tests update 21:02:36 notmyname: oh, thx a lot! :) 21:02:58 cschwede: first up is status on the rolling upgrades tests. seems there was some conversation yesterday 21:03:01 \o/ 21:03:09 cschwede: anything blocking on our side or that you need help with here? 21:03:10 not much news here, rebased and answered a few comments this morning, test passes, failures are not related to my patch (IMO) 21:03:26 no, thx, nothing to do right (except getting reviews) 21:03:39 ok, thanks for patiently workign on that 21:03:46 #topic translation bugs 21:04:06 cschwede: last week you volunteered to look in to tests that would find translation issues 21:04:12 eg https://bugs.launchpad.net/swift/+bug/1580678 21:04:12 Launchpad bug 1580678 in OpenStack Object Storage (swift) "UnicodeDecodeError when rebalancing a ring" [Undecided,In progress] - Assigned to Christian Schwede (cschwede) 21:04:15 that was fun… i have a patch with some improved tests, as well as a fix for the issue itself: https://review.openstack.org/#/c/323950/ 21:04:59 great 21:05:22 side note, why do we have a nonvoting neutron test in our gate again? 21:05:31 something to look in to later 21:05:50 cschwede: thanks 21:05:52 cschwede: ok, so with patch 323950 we need reviews? 21:05:52 notmyname: https://review.openstack.org/#/c/323950/ - swift - Refactor locale tests and unicode issue 21:05:58 yes, please 21:06:18 thanks for working on this 21:06:33 cschwede: anything else to bring up on this for this week? 21:06:34 you’re welcome! 21:06:49 notmyname: not from my side, thx 21:07:05 great. any questions from anyone else for cschwede? 21:08:16 #topic crypto update 21:08:27 looks like good progress this week 21:08:34 acoles: jrichli: where are we today? 21:08:40 #link https://trello.com/b/63l5zQhq/swift-encryption 21:08:45 the todo list is very short now 21:09:07 similar to last week...still need some reviews to land final patches on feature/crypto, the priority list is up to date https://wiki.openstack.org/wiki/Swift/PriorityReviews 21:09:43 there is nothing on the todo list that stops us moving to crypto-review imo, just the outstanding reviews 21:09:52 ok 21:10:39 since -infra will need an actual SHA from which to base a crypto-review branch (and because it's historically been a very simple, fast process) I'll get that set up as soon as we have these patches landed 21:10:52 with one exception perhaps, the more I hae worked on the code the more I have become uncomfortable with the interface to the keymaster (the callback) 21:11:06 ok 21:11:18 but I'll put together a patch for us to discuss any change there 21:11:35 notmyname: is that a SHA from master? 21:11:41 yes 21:12:04 acoles: it doesn't matter too much as long as we avoid obvious merge conflicts 21:12:30 eg if we're pretty certain there aren't any patches that will land on master that will cause a conflict, we could do it for master HEAD now. 21:13:29 k. what about conflicts that happen after we start the release branch - we merge from master to crypto-review and rebase? 21:13:45 on these 4 listed patches, what are you looking for? if they come in with the crypto_review branch, they'll be reviewed there. do you need in-depth review now, or can they go ahead and land these 4 as long as they're "reasonably good" 21:14:02 acoles: we do our best to avoid landing stuff that would cause a conflict :-) 21:14:41 notmyname: I have been wondering exactly that, whether I land the remaining patches and we take it up on crypto-review 21:15:27 notmyname: there is an issue that tdasilva spotted on https://review.openstack.org/320579 that we should resolve 21:15:53 acoles: yeah, i was wondering if that would need a fix on master? 21:17:21 tdasilva: both I think, on master the req.etag thing should be fixed (or removed), on crypto we need to do the right thing with the override headers. starting point is a test for a ranged copy if we don't have one already 21:17:46 acoles: agree 21:17:46 ranged copy is crazy 21:18:07 tdasilva: are you looking at a ranged copy test? 21:18:08 notmyname: let's give it a day or so, try to sort that ^^ and hopefully get some more reviews 21:18:14 ok 21:18:40 I think webdav defined COPY verb - i wonder if they have any words on how to reject COPY requests with a Range header? 21:18:45 notmyname: haven't started yet, but I can start looking after this... 21:19:17 acoles: and think on the "land if reasonable" for the rest of them. it would be wonderful to have a start on the for-review patch chain by the end of the week 21:19:43 tdasilva: I made a simple change to functional.tests.TestFile.testCopy locally 21:20:07 notmyname: sure, there's likely some I can just land, cleanup stuff, tests etc 21:20:23 anything else about crypto from anyone? 21:20:57 clayg: ack. maybe we should start to reject them. 21:22:14 acoles, clayg: if we would start to reject it, does slo range work? 21:22:30 kota_: seems orthogonal 21:22:49 it'd still work; a ranged GET will work, just a ranged COPY wouldn't 21:22:50 acoles: if there is a reasonable and useful defined behavior that's implementable I'm not against having them - but as a client it's not obvious to me what I'd be requesting? 21:23:18 clayg: ok, probably, I should look at the problem more deeply. 21:23:33 kota_: you and me both brother! 21:23:52 I was just throwing stuff out there - not doing a thing has to be easier than doing a thing - well maybe not... dunno 21:23:59 agreed; it started out as a thing that nobody explicitly coded up but happened to do something seemingly sane, but then you get into the edge cases... 21:24:02 clayg: ranged copies make sense enough: apply range to source, copy to destination 21:24:07 torgomatic: some part of the http spec says you can just ignore range headers right? 21:24:10 ... as a server 21:24:19 timburke: and for multi-range? 21:24:26 timburke: you want a mime document? 21:24:29 clayg: yeah, you can... the spec also says they're only for GET requests :) 21:24:38 torgomatic: BOOM! 21:24:42 timburke: does swift3 use ranged copy? 21:24:54 of course, the spec doesn't contain the COPY verb anywhere, so we're sort of on our own there 21:25:05 clayg: well, yeah, that part's stupid. which is probably part of why S3 limits copies to a single range 21:25:08 I was about to say "let's think on it for next week", but torgomatic just dropped the rfc bomb on us 21:25:18 kota_: yep, but only for multipart upload parts 21:25:28 timburke: I don't have clear memory but I brought COPY range discussion seeing your patch. 21:25:38 IIRC we had this discussion before and kota added some doc or comment somewhere to flag up that we used the Range header on non-GETs 21:25:39 * torgomatic is helping, maybe? 21:25:57 timburke: S3 supports it is probably not the worse as signal as anything that clients have a use-case for it :D 21:26:17 timburke: didn't realize S3 had ranged copy requests - i'll try to dig up the docs 21:26:22 torgomatic: if we just look at the spec, we should probably just reject all COPY requests :P 21:26:43 technically correct is the best kind of correct! ;) 21:26:48 it seems like there's more reading and thinking to be done about this 21:27:09 so I'll add it as a topic for next week, and we'll come back with new thinking 21:27:10 clayg: see http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPartCopy.html#mpUploadUploadPartCopy-requests-request-headers 21:27:13 timburke: server-side copy is obviously useful - you're being snarky - but having a ill defined api is not helpful to clients 21:27:38 timburke: see being helpful is nice! 21:28:23 let's move on 21:28:36 #topic pyeclib/liberasurecode status 21:28:42 tdasilva: I think status is "done" right? 21:28:46 everythign migrated 21:28:51 new releases tagged 21:28:56 we at least pop the Range header for a post-as-copy :) 21:29:05 everything migrated and we have a new release, but I"m still working on gate jobs 21:29:07 need to put some words in bitbucket, but that's it 21:29:14 ah, ok. the gate jobs 21:29:25 yeah, we also need to figure out what to do with PR and issues on bitbutcket 21:29:25 tdasilva: thanks for working on this and getting these moved 21:29:56 anything from tsg or kevin about that? 21:29:58 also zaitcev had a question on the tag so I wanted to clarify 21:30:13 tdasilva: hopefully I'd like to add doc for how to contribute likely Swift does 21:30:31 previous releases had a v, like v1.1.0, openstack-infra asked to drop the v for pyeclib 21:30:40 so i dropped for liberasurecode too 21:31:02 notmyname: i have not heard from kevin or tsg, will ping them again 21:31:10 ok, thanks 21:31:11 kota_: yeah, sounds good :) 21:31:46 clayg: did you get everything you needed this (last?) week from those libraries? new tags/releases? 21:32:15 notmyname: i'm trying to build with all the new depends now - no new information - hopefully everythign is wonderful! 21:32:26 I'm sure it is :-) 21:32:38 #topic symlinks 21:32:42 on the gate jobs, i'm trying to write some custom jobs, that would build liberasurecode and then test pyeclib. I thought we could have two jobs, one that builds libEC from master and another that builds libEC from a stable tag 21:32:58 #undo 21:32:58 Removing item from minutes: 21:33:03 cool 21:33:10 does that sounds like a sane plan? 21:33:26 tdasilva: nice. I'm interested in that too, since I'm starting to look at what it takes to do basically the same thing with golang 21:33:48 someone should put a __unicode__ on that Topic object 21:33:54 notmyname: ok, i might ping you then cause you might already know some stuff i don'tk now 21:33:55 torgomatic: https://review.openstack.org/#/c/272727/ 21:33:55 timburke: patch 272727 - openstack-infra/meetbot - Add __str__ methods for items 21:34:05 tdasilva: thanks 21:34:14 torgomatic: of course timburke already has a patch for it 21:34:28 lol 21:34:38 #topic symlinks 21:34:40 tdasilva: did I mistakenly leave this on the agenda for this week? 21:34:45 or is there new stuff to discuss? 21:34:46 i think so 21:34:48 no updates 21:35:02 ok :-) 21:35:12 unless somebody wants to ask something 21:35:17 #topic patch 317475 21:35:17 notmyname: https://review.openstack.org/#/c/317475/ - swift - Send correct size in POST async update for EC object 21:35:26 looks like kota_ just approved this, so that's great 21:35:36 \o/ 21:35:41 thanks kota_ and timburke 21:35:55 #topic other 21:36:01 a couple of general updates 21:36:15 golang discussion has mostly stopped on the ML, from what I can see 21:36:26 next week's tuesday TC meetign will take it up again 21:36:46 today I'm feeling ok with the current state of the conversation 21:37:11 in other news, yesterday the TC approved a resolution that says that defcore tests must live in tempest (not a tempest plugin 21:37:27 boo! 21:37:39 this means that our in-tree tests cannot be used to validate something calling itself "swift" is actually swift or not 21:37:40 notmyname are they going to maintain them? 21:38:03 hurricanerix: yes, by definition, since the QA team is the only one with +2 access 21:38:26 so any tests that can validate a defcore defined capability must be reimplemented in tempest 21:38:38 there was an email just a few minutes ago about this, from the defcore team 21:38:41 hurricanerix: that shouldn’t stop us from submitting improved tests (if required) 21:38:43 So we can submit a big patch to tempest with all our tests ;P 21:39:04 #link http://lists.openstack.org/pipermail/openstack-dev/2016-June/096417.html 21:39:18 cschwede: right 21:39:25 mattoliverau: probably not ;-) 21:39:25 :( 21:39:31 cschwede just now we have to watch two repos to make sure the tests they create are correct. 21:40:02 what happens if we fix a bug, and it causes their tests to fail =) 21:40:02 but a test in tempest is not necessarily a "defcore test" is that correct? 21:40:11 tdasilva: correct 21:40:22 hurricanerix: or vice versa 21:40:31 hurricanerix: well, tempest already has swift tests for a long time - someone had to fix them anyways 21:40:44 test has to be in tempest, then it can be submitted for inclusion in the defcore definition 21:40:57 overall, this isn't really much of a change from the way things have been all along 21:41:01 who drives that process? 21:41:13 tdasilva: the defcore subcommittee 21:41:15 and it’s mostly API tests, there shouldn’t be too many changes/fixes 21:41:16 notmyname, who takes the initiative to make sure tests in tempest are up to date with swift code base? 21:41:44 hurricanerix: we shouldn't break tempest - they're a client like anyone else 21:42:00 if our api used to work one way and then stops its probably an indication we're doing something wrong 21:42:08 yep 21:42:08 notmyname: hopefully someone from swift community is in the defcore object storage subcommittee ??? 21:42:15 ...as long as they aren't testing some under-specified API, like including Range headers with COPY :P 21:42:25 timburke: haha 21:42:28 tdasilva: no, it's a board thing. I get pinged occasionally 21:42:29 timburke: YOU'RE ON FIRE! 21:42:59 actually, ntata identified the real issue. who takes initiative and keeps it up to date 21:43:31 I think it's like it is now: basically some intersection of us or the tempest team. so mostly us, but we don't look at those too often 21:43:41 and that's the way it will keep working 21:44:10 we're not planning on breaking anything, but there could be a lot more coverage there than there is now 21:44:16 will we have to submit tests to tempest when new features are implemented? or will that qa/qe team do that? 21:44:31 hurricanerix: likely us. I wouldn't count on the tempest team to do it 21:45:04 so when we review things, we also have to search tempest reviews to see if there is a matching new tempest test? =) 21:45:20 hurricanerix: you and timburke are like peas in a pod today 21:45:21 hurricanerix: no. no more than you do today 21:45:28 yeah, and then there's also interesting stuff like this: https://review.openstack.org/#/c/272062/ 21:45:29 tdasilva: patch 272062 - tempest - Fix checks for content length in object storage te... 21:45:49 tdasilva: is that sill going!? 21:46:00 * hurricanerix highfives timburke 21:46:22 so if you want to read all about it, you can see it at https://review.openstack.org/#/c/312718/ (including my opposition). 21:46:23 notmyname: patch 312718 - governance - add resolution explaining which tests we think def... (MERGED) 21:46:58 clayg: not sure, patch was last updated 13 days ago 21:48:21 last thing I wanted to bring up this week was a crazy idea I had last night. right now we normally want 2 +2s on a patch and sometimes are ok with 1 +2. it's a social norm. what if we swapped that? normally be fine with 1 +2 and sometimes get 2 +2s. would that help people feel better about the speed of progress and the ability to get stuff to happen? 21:48:52 it's something I wanted to bounce off of people and maybe get people to think about 21:49:24 but not something I wanted to make a decision on today 21:49:51 hmm, for many patches i really like to see 2 +2s - that’s a great level of confidence for things that could break clusters at night… 21:49:58 notmyname: second set of eyes is good - almost always 21:50:14 yeah, will need to think about that. I like we can 1 +2 on obvious things, but 2 x +2 means 2 sets of eyes are looking and finding things the other missed... ok I mean I missed :) 21:50:33 notmyname: to my knowledge no one has ever been "repremanded" for a +A you're the first +2 - so folks judgement seems to be working there 21:50:50 notmyname: I would suggest we continue to encourage folks to +A when appropriate 21:51:15 notmyname: in your data crunching, have you found that what slows down reviews is the second +2 or the first? 21:51:25 reprimanded - that's a weird word 21:51:27 tdasilva: that's a good question, and I haven't 21:51:32 +2/-1 from two cores is not infrequent, and reassures me of the value of review 21:51:37 only small things (typos, docs maybe, addons to tests without other modifications and similar things) should require one +2 - IMO 21:51:53 notmyname: i like your out of the box thinking tho! 21:52:04 but basically - we're all scardy cats :D 21:52:11 lol 21:52:13 heh 21:52:18 I don't need that kind of pressure!! 21:52:22 clayg: yeah, there's nowhere to hide if you solo +A ;) 21:52:22 :) 21:52:24 rofl 21:52:30 acoles: share the blame baby! 21:52:34 yeah, that's how I expected everyone to react :-) 21:52:50 but seriosuly - +A when appropriate 21:52:52 notmyname was just checking we we're all listening :P 21:52:52 how about 3 +2's ?? :P 21:52:55 notmyname: I could swear we had this convo some time back 21:53:30 yeah, I agree that the second +2 is very nice. but also I think that we don't +A simple things as often as we could and most of the time there's a 2nd +2 with no disagreement 21:53:37 tdasilva: yeah, maybe so 21:53:40 tdasilva: it's notmyname's way of reminding us to do our jobs ;) 21:53:46 lol 21:53:50 acoles: everyon must +2 everything daly - the beatins will continue until merge rate improves 21:54:17 heh, ok. like I said, just a conversation starter :-) 21:54:38 clayg: :D 21:54:55 tdasilva shared this with me earlier today. it's really interesting in light of our global community and how we normally think and interact. good stuff to read https://hbr.org/2015/12/getting-to-si-ja-oui-hai-and-da 21:55:03 anything else from anyone this week? 21:55:09 notmyname: let's track "time from +2 to +A" independently as something we'd like to improve - and +2 to -1 as a reminder why the second +2 matters? 21:55:31 clayg: I can do that :-) 21:55:52 quick fyi there is a proposal to add a non-voting functional test job that uses keystone with only v3 API , patch 313659 21:55:52 acoles: https://review.openstack.org/#/c/313659/ - openstack-infra/project-config - Run Swift functional tests in Identity v3-only 21:56:33 I anticipate it will pass, I think swift tests already only use the v3 API 21:57:03 it's aprt of a cross-project effort to finally deprecate the v2 API 21:57:15 that's good to know 21:57:57 so if nothing else... 21:58:14 thanks for coming everyone. keep up the great work on swift 21:58:18 #endmeeting