21:00:21 <notmyname> #startmeeting swift
21:00:24 <openstack> Meeting started Wed Jun  1 21:00:21 2016 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:25 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:27 <openstack> The meeting name has been set to 'swift'
21:00:30 <notmyname> who's here for the swift team meeting?
21:00:32 <mattoliverau> o/
21:00:34 <bkeller`> o/
21:00:35 <ntata> hello
21:00:36 <joeljwright> hey
21:00:36 <jrichli> hello
21:00:36 <dmorita> hi
21:00:38 <hosanai> o/
21:00:39 <cutforth> o/    !buenos dias!
21:00:41 <timburke> hi
21:00:42 <pdardeau> o/
21:00:42 <kmARC> hi
21:00:43 <tdasilva> eu
21:00:43 <sgundur1> hi
21:00:59 <mmotiani> hi
21:01:06 <mmotiani> o/
21:01:08 <cschwede> o/
21:01:13 <gmmaha> o/
21:01:26 <mathiasb> hi
21:01:38 <notmyname> welcome everyone. good group today :-)
21:01:39 <acoles> here
21:01:49 <torgomatic> woo
21:01:56 <notmyname> agenda is at
21:01:57 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift
21:02:18 <notmyname> I want to skip around just a bit to put the cschwede things up front (it's latest for him, I think)
21:02:33 <kota_> hello
21:02:34 <notmyname> #topic rolling upgrade tests update
21:02:36 <cschwede> notmyname: oh, thx a lot! :)
21:02:58 <notmyname> cschwede: first up is status on the rolling upgrades tests. seems there was some conversation yesterday
21:03:01 <hurricanerix> \o/
21:03:09 <notmyname> cschwede: anything blocking on our side or that you need help with here?
21:03:10 <cschwede> not much news here, rebased and answered a few comments this morning, test passes, failures are not related to my patch (IMO)
21:03:26 <cschwede> no, thx, nothing to do right (except getting reviews)
21:03:39 <notmyname> ok, thanks for patiently workign on that
21:03:46 <notmyname> #topic translation bugs
21:04:06 <notmyname> cschwede: last week you volunteered to look in to tests that would find translation issues
21:04:12 <notmyname> eg https://bugs.launchpad.net/swift/+bug/1580678
21:04:12 <openstack> Launchpad bug 1580678 in OpenStack Object Storage (swift) "UnicodeDecodeError when rebalancing a ring" [Undecided,In progress] - Assigned to Christian Schwede (cschwede)
21:04:15 <cschwede> that was fun… i have a patch with some improved tests, as well as a fix for the issue itself: https://review.openstack.org/#/c/323950/
21:04:59 <notmyname> great
21:05:22 <notmyname> side note, why do we have a nonvoting neutron test in our gate again?
21:05:31 <notmyname> something to look in to later
21:05:50 <acoles> cschwede: thanks
21:05:52 <notmyname> cschwede: ok, so with patch 323950 we need reviews?
21:05:52 <patchbot> notmyname: https://review.openstack.org/#/c/323950/ - swift - Refactor locale tests and unicode issue
21:05:58 <cschwede> yes, please
21:06:18 <notmyname> thanks for working on this
21:06:33 <notmyname> cschwede: anything else to bring up on this for this week?
21:06:34 <cschwede> you’re welcome!
21:06:49 <cschwede> notmyname: not from my side, thx
21:07:05 <notmyname> great. any questions from anyone else for cschwede?
21:08:16 <notmyname> #topic crypto update
21:08:27 <notmyname> looks like good progress this week
21:08:34 <notmyname> acoles: jrichli: where are we today?
21:08:40 <notmyname> #link https://trello.com/b/63l5zQhq/swift-encryption
21:08:45 <notmyname> the todo list is very short now
21:09:07 <acoles> similar to last week...still need some reviews to land final patches on feature/crypto, the priority list is up to date https://wiki.openstack.org/wiki/Swift/PriorityReviews
21:09:43 <acoles> there is nothing on the todo list that stops us moving to crypto-review imo, just the outstanding reviews
21:09:52 <notmyname> ok
21:10:39 <notmyname> since -infra will need an actual SHA from which to base a crypto-review branch (and because it's historically been a very simple, fast process) I'll get that set up as soon as we have these patches landed
21:10:52 <acoles> with one exception perhaps, the more I hae worked on the code the more I have become uncomfortable with the interface to the keymaster (the callback)
21:11:06 <notmyname> ok
21:11:18 <acoles> but I'll put together a patch for us to discuss any change there
21:11:35 <acoles> notmyname: is that a SHA from master?
21:11:41 <notmyname> yes
21:12:04 <notmyname> acoles: it doesn't matter too much as long as we avoid obvious merge conflicts
21:12:30 <notmyname> eg if we're pretty certain there aren't any patches that will land on master that will cause a conflict, we could do it for master HEAD now.
21:13:29 <acoles> k. what about conflicts that happen after we start the release branch - we merge from master to crypto-review and rebase?
21:13:45 <notmyname> on these 4 listed patches, what are you looking for? if they come in with the crypto_review branch, they'll be reviewed there. do you need in-depth review now, or can they go ahead and land these 4 as long as they're "reasonably good"
21:14:02 <notmyname> acoles: we do our best to avoid landing stuff that would cause a conflict :-)
21:14:41 <acoles> notmyname: I have been wondering exactly that, whether I land the remaining patches and we take it up on crypto-review
21:15:27 <acoles> notmyname: there is an issue that tdasilva spotted on https://review.openstack.org/320579 that we should resolve
21:15:53 <tdasilva> acoles: yeah, i was wondering if that would need a fix on master?
21:17:21 <acoles> tdasilva: both I think, on master the req.etag thing should be fixed (or removed), on crypto we need to do the right thing with the override headers. starting point is a test for a ranged copy if we don't have one already
21:17:46 <tdasilva> acoles: agree
21:17:46 <clayg> ranged copy is crazy
21:18:07 <notmyname> tdasilva: are you looking at a ranged copy test?
21:18:08 <acoles> notmyname: let's give it a day or so, try to sort that ^^ and hopefully get some more reviews
21:18:14 <notmyname> ok
21:18:40 <clayg> I think webdav defined COPY verb - i wonder if they have any words on how to reject COPY requests with a Range header?
21:18:45 <tdasilva> notmyname: haven't started yet, but I can start looking after this...
21:19:17 <notmyname> acoles: and think on the "land if reasonable" for the rest of them. it would be wonderful to have a start on the for-review patch chain by the end of the week
21:19:43 <acoles> tdasilva: I made a simple change to functional.tests.TestFile.testCopy locally
21:20:07 <acoles> notmyname: sure, there's likely some I can just land, cleanup stuff, tests etc
21:20:23 <notmyname> anything else about crypto from anyone?
21:20:57 <acoles> clayg: ack. maybe we should start to reject them.
21:22:14 <kota_> acoles, clayg: if we would start to reject it, does slo range work?
21:22:30 <clayg> kota_: seems orthogonal
21:22:49 <torgomatic> it'd still work; a ranged GET will work, just a ranged COPY wouldn't
21:22:50 <clayg> acoles: if there is a reasonable and useful defined behavior that's implementable I'm not against having them - but as a client it's not obvious to me what I'd be requesting?
21:23:18 <kota_> clayg: ok, probably, I should look at the problem more deeply.
21:23:33 <clayg> kota_: you and me both brother!
21:23:52 <clayg> I was just throwing stuff out there - not doing a thing has to be easier than doing a thing - well maybe not... dunno
21:23:59 <torgomatic> agreed; it started out as a thing that nobody explicitly coded up but happened to do something seemingly sane, but then you get into the edge cases...
21:24:02 <timburke> clayg: ranged copies make sense enough: apply range to source, copy to destination
21:24:07 <clayg> torgomatic: some part of the http spec says you can just ignore range headers right?
21:24:10 <clayg> ... as a server
21:24:19 <clayg> timburke: and for multi-range?
21:24:26 <clayg> timburke: you want a mime document?
21:24:29 <torgomatic> clayg: yeah, you can... the spec also says they're only for GET requests :)
21:24:38 <clayg> torgomatic: BOOM!
21:24:42 <kota_> timburke: does swift3 use ranged copy?
21:24:54 <torgomatic> of course, the spec doesn't contain the COPY verb anywhere, so we're sort of on our own there
21:25:05 <timburke> clayg: well, yeah, that part's stupid. which is probably part of why S3 limits copies to a single range
21:25:08 <notmyname> I was about to say "let's think on it for next week", but torgomatic just dropped the rfc bomb on us
21:25:18 <timburke> kota_: yep, but only for multipart upload parts
21:25:28 <kota_> timburke: I don't have clear memory but I brought COPY range discussion seeing your patch.
21:25:38 <acoles> IIRC we had this discussion before and kota added some doc or comment somewhere to flag up that we used the Range header on non-GETs
21:25:39 * torgomatic is helping, maybe?
21:25:57 <clayg> timburke: S3 supports it is probably not the worse as signal as anything that clients have a use-case for it :D
21:26:17 <clayg> timburke: didn't realize S3 had ranged copy requests - i'll try to dig up the docs
21:26:22 <timburke> torgomatic: if we just look at the spec, we should probably just reject all COPY requests :P
21:26:43 <torgomatic> technically correct is the best kind of correct! ;)
21:26:48 <notmyname> it seems like there's more reading and thinking to be done about this
21:27:09 <notmyname> so I'll add it as a topic for next week, and we'll come back with new thinking
21:27:10 <timburke> clayg: see http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPartCopy.html#mpUploadUploadPartCopy-requests-request-headers
21:27:13 <clayg> timburke: server-side copy is obviously useful - you're being snarky - but having a ill defined api is not helpful to clients
21:27:38 <clayg> timburke: see being helpful is nice!
21:28:23 <notmyname> let's move on
21:28:36 <notmyname> #topic pyeclib/liberasurecode status
21:28:42 <notmyname> tdasilva: I think status is "done" right?
21:28:46 <notmyname> everythign migrated
21:28:51 <notmyname> new releases tagged
21:28:56 <acoles> we at least pop the Range header for a post-as-copy :)
21:29:05 <tdasilva> everything migrated and we have a new release, but I"m still working on gate jobs
21:29:07 <notmyname> need to put some words in bitbucket, but that's it
21:29:14 <notmyname> ah, ok. the gate jobs
21:29:25 <tdasilva> yeah, we also need to figure out what to do with PR and issues on bitbutcket
21:29:25 <notmyname> tdasilva: thanks for working on this and getting these moved
21:29:56 <notmyname> anything from tsg or kevin about that?
21:29:58 <tdasilva> also zaitcev had a question on the tag so I wanted to clarify
21:30:13 <kota_> tdasilva: hopefully I'd like to add doc for how to contribute likely Swift does
21:30:31 <tdasilva> previous releases had a v, like v1.1.0, openstack-infra asked to drop the v for pyeclib
21:30:40 <tdasilva> so i dropped for liberasurecode too
21:31:02 <tdasilva> notmyname: i have not heard from kevin or tsg, will ping them again
21:31:10 <notmyname> ok, thanks
21:31:11 <tdasilva> kota_: yeah, sounds good :)
21:31:46 <notmyname> clayg: did you get everything you needed this (last?) week from those libraries? new tags/releases?
21:32:15 <clayg> notmyname: i'm trying to build with all the new depends now - no new information - hopefully everythign is wonderful!
21:32:26 <notmyname> I'm sure it is :-)
21:32:38 <notmyname> #topic symlinks
21:32:42 <tdasilva> on the gate jobs, i'm trying to write some custom jobs, that would build liberasurecode and then test pyeclib. I thought we could have two jobs, one that builds libEC from master and another that builds libEC from a stable tag
21:32:58 <notmyname> #undo
21:32:58 <openstack> Removing item from minutes: <ircmeeting.items.Topic object at 0x7f2dd3b0ea10>
21:33:03 <notmyname> cool
21:33:10 <tdasilva> does that sounds like a sane plan?
21:33:26 <notmyname> tdasilva: nice. I'm interested in that too, since I'm starting to look at what it takes to do basically the same thing with golang
21:33:48 <torgomatic> someone should put a __unicode__ on that Topic object
21:33:54 <tdasilva> notmyname: ok, i might ping you then cause you might already know some stuff i don'tk now
21:33:55 <timburke> torgomatic: https://review.openstack.org/#/c/272727/
21:33:55 <patchbot> timburke: patch 272727 - openstack-infra/meetbot - Add __str__ methods for items
21:34:05 <notmyname> tdasilva: thanks
21:34:14 <notmyname> torgomatic: of course timburke already has a patch for it
21:34:28 <tdasilva> lol
21:34:38 <notmyname> #topic symlinks
21:34:40 <notmyname> tdasilva: did I mistakenly leave this on the agenda for this week?
21:34:45 <notmyname> or is there new stuff to discuss?
21:34:46 <tdasilva> i think so
21:34:48 <tdasilva> no updates
21:35:02 <notmyname> ok :-)
21:35:12 <tdasilva> unless somebody wants to ask something
21:35:17 <notmyname> #topic patch 317475
21:35:17 <patchbot> notmyname: https://review.openstack.org/#/c/317475/ - swift - Send correct size in POST async update for EC object
21:35:26 <notmyname> looks like kota_ just approved this, so that's great
21:35:36 <kota_> \o/
21:35:41 <acoles> thanks kota_ and timburke
21:35:55 <notmyname> #topic other
21:36:01 <notmyname> a couple of general updates
21:36:15 <notmyname> golang discussion has mostly stopped on the ML, from what I can see
21:36:26 <notmyname> next week's tuesday TC meetign will take it up again
21:36:46 <notmyname> today I'm feeling ok with the current state of the conversation
21:37:11 <notmyname> in other news, yesterday the TC approved a resolution that says that defcore tests must live in tempest (not a tempest plugin
21:37:27 <pdardeau> boo!
21:37:39 <notmyname> this means that our in-tree tests cannot be used to validate something calling itself "swift" is actually swift or not
21:37:40 <hurricanerix> notmyname are they going to maintain them?
21:38:03 <notmyname> hurricanerix: yes, by definition, since the QA team is the only one with +2 access
21:38:26 <notmyname> so any tests that can validate a defcore defined capability must be reimplemented in tempest
21:38:38 <notmyname> there was an email just a few minutes ago about this, from the defcore team
21:38:41 <cschwede> hurricanerix: that shouldn’t stop us from submitting improved tests (if required)
21:38:43 <mattoliverau> So we can submit a big patch to tempest with all our tests ;P
21:39:04 <notmyname> #link http://lists.openstack.org/pipermail/openstack-dev/2016-June/096417.html
21:39:18 <notmyname> cschwede: right
21:39:25 <notmyname> mattoliverau: probably not ;-)
21:39:25 <ntata> :(
21:39:31 <hurricanerix> cschwede just now we have to watch two repos to make sure the tests they create are correct.
21:40:02 <hurricanerix> what happens if we fix a bug, and it causes their tests to fail =)
21:40:02 <tdasilva> but a test in tempest is not necessarily a "defcore test" is that correct?
21:40:11 <notmyname> tdasilva: correct
21:40:22 <tdasilva> hurricanerix: or vice versa
21:40:31 <cschwede> hurricanerix: well, tempest already has swift tests for a long time - someone had to fix them anyways
21:40:44 <notmyname> test has to be in tempest, then it can be submitted for inclusion in the defcore definition
21:40:57 <notmyname> overall, this isn't really much of a change from the way things have been all along
21:41:01 <tdasilva> who drives that process?
21:41:13 <notmyname> tdasilva: the defcore subcommittee
21:41:15 <cschwede> and it’s mostly API tests, there shouldn’t be too many changes/fixes
21:41:16 <ntata> notmyname, who takes the initiative to make sure tests in tempest are up to date with swift code base?
21:41:44 <clayg> hurricanerix: we shouldn't break tempest - they're a client like anyone else
21:42:00 <clayg> if our api used to work one way and then stops its probably an indication we're doing something wrong
21:42:08 <notmyname> yep
21:42:08 <tdasilva> notmyname: hopefully someone from swift community is in the defcore object storage subcommittee ???
21:42:15 <timburke> ...as long as they aren't testing some under-specified API, like including Range headers with COPY :P
21:42:25 <tdasilva> timburke: haha
21:42:28 <notmyname> tdasilva: no, it's a board thing. I get pinged occasionally
21:42:29 <clayg> timburke: YOU'RE ON FIRE!
21:42:59 <notmyname> actually, ntata identified the real issue. who takes initiative and keeps it up to date
21:43:31 <notmyname> I think it's like it is now: basically some intersection of us or the tempest team. so mostly us, but we don't look at those too often
21:43:41 <notmyname> and that's the way it will keep working
21:44:10 <notmyname> we're not planning on breaking anything, but there could be a lot more coverage there than there is now
21:44:16 <hurricanerix> will we have to submit tests to tempest when new features are implemented?  or will that qa/qe team do that?
21:44:31 <notmyname> hurricanerix: likely us. I wouldn't count on the tempest team to do it
21:45:04 <hurricanerix> so when we review things, we also have to search tempest reviews to see if there is a matching new tempest test?  =)
21:45:20 <clayg> hurricanerix: you and timburke are like peas in a pod today
21:45:21 <notmyname> hurricanerix: no. no more than you do today
21:45:28 <tdasilva> yeah, and then there's also interesting stuff like this: https://review.openstack.org/#/c/272062/
21:45:29 <patchbot> tdasilva: patch 272062 - tempest - Fix checks for content length in object storage te...
21:45:49 <clayg> tdasilva: is that sill going!?
21:46:00 * hurricanerix highfives timburke
21:46:22 <notmyname> so if you want to read all about it, you can see it at https://review.openstack.org/#/c/312718/ (including my opposition).
21:46:23 <patchbot> notmyname: patch 312718 - governance - add resolution explaining which tests we think def... (MERGED)
21:46:58 <tdasilva> clayg: not sure, patch was last updated 13 days ago
21:48:21 <notmyname> last thing I wanted to bring up this week was a crazy idea I had last night. right now we normally want 2 +2s on a patch and sometimes are ok with 1 +2. it's a social norm. what if we swapped that? normally be fine with 1 +2 and sometimes get 2 +2s. would that help people feel better about the speed of progress and the ability to get stuff to happen?
21:48:52 <notmyname> it's something I wanted to bounce off of people and maybe get people to think about
21:49:24 <notmyname> but not something I wanted to make a decision on today
21:49:51 <cschwede> hmm, for many patches i really like to see 2 +2s - that’s a great level of confidence for things that could break clusters at night…
21:49:58 <clayg> notmyname: second set of eyes is good - almost always
21:50:14 <mattoliverau> yeah, will need to think about that. I like we can 1 +2 on obvious things, but 2 x +2 means 2 sets of eyes are looking and finding things the other missed... ok I mean I missed :)
21:50:33 <clayg> notmyname: to my knowledge no one has ever been "repremanded" for a +A you're the first +2 - so folks judgement seems to be working there
21:50:50 <clayg> notmyname: I would suggest we continue to encourage folks to +A when appropriate
21:51:15 <tdasilva> notmyname: in your data crunching, have you found that what slows down reviews is the second +2 or the first?
21:51:25 <clayg> reprimanded - that's a weird word
21:51:27 <notmyname> tdasilva: that's a good question, and I haven't
21:51:32 <acoles> +2/-1 from two cores is not infrequent, and reassures me of the value of review
21:51:37 <cschwede> only small things (typos, docs maybe, addons to tests without other modifications and similar things) should require one +2 - IMO
21:51:53 <clayg> notmyname: i like your out of the box thinking tho!
21:52:04 <clayg> but basically - we're all scardy cats :D
21:52:11 <mattoliverau> lol
21:52:13 <notmyname> heh
21:52:18 <joeljwright> I don't need that kind of pressure!!
21:52:22 <acoles> clayg: yeah, there's nowhere to hide if you solo +A ;)
21:52:22 <joeljwright> :)
21:52:24 <clayg> rofl
21:52:30 <clayg> acoles: share the blame baby!
21:52:34 <notmyname> yeah, that's how I expected everyone to react :-)
21:52:50 <clayg> but seriosuly - +A when appropriate
21:52:52 <mattoliverau> notmyname was just checking we we're all listening :P
21:52:52 <acoles> how about 3 +2's ?? :P
21:52:55 <tdasilva> notmyname: I could swear we had this convo some time back
21:53:30 <notmyname> yeah, I agree that the second +2 is very nice. but also I think that we don't +A simple things as often as we could and most of the time there's a 2nd +2 with no disagreement
21:53:37 <notmyname> tdasilva: yeah, maybe so
21:53:40 <acoles> tdasilva: it's notmyname's way of reminding us to do our jobs ;)
21:53:46 <tdasilva> lol
21:53:50 <clayg> acoles: everyon must +2 everything daly - the beatins will continue until merge rate improves
21:54:17 <notmyname> heh, ok. like I said, just a conversation starter :-)
21:54:38 <joeljwright> clayg: :D
21:54:55 <notmyname> tdasilva shared this with me earlier today. it's really interesting in light of our global community and how we normally think and interact. good stuff to read https://hbr.org/2015/12/getting-to-si-ja-oui-hai-and-da
21:55:03 <notmyname> anything else from anyone this week?
21:55:09 <clayg> notmyname: let's track "time from +2 to +A" independently as something we'd like to improve - and +2 to -1 as a reminder why the second +2 matters?
21:55:31 <notmyname> clayg: I can do that :-)
21:55:52 <acoles> quick fyi there is a proposal to add a non-voting functional test job that uses keystone with only v3 API , patch 313659
21:55:52 <patchbot> acoles: https://review.openstack.org/#/c/313659/ - openstack-infra/project-config - Run Swift functional tests in Identity v3-only
21:56:33 <acoles> I anticipate it will pass, I think swift tests already only use the v3 API
21:57:03 <acoles> it's aprt of a cross-project effort to finally deprecate the v2 API
21:57:15 <notmyname> that's good to know
21:57:57 <notmyname> so if nothing else...
21:58:14 <notmyname> thanks for coming everyone. keep up the great work on swift
21:58:18 <notmyname> #endmeeting