19:01:21 <notmyname> #startmeeting swift
19:01:21 <openstack> Meeting started Wed Sep 18 19:01:21 2013 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:22 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:01:24 <torgomatic> OH JOY
19:01:25 <openstack> The meeting name has been set to 'swift'
19:01:32 <notmyname> who's here?
19:01:35 <portante> o/
19:01:38 <zaitcev> o7
19:01:48 <clayg> o8
19:01:57 <creiht> ☃
19:02:19 * portante can barely read that
19:02:21 <notmyname> looks like dfg and gholt jsut recently joined in here
19:02:27 <notmyname> agenda for this week is at https://wiki.openstack.org/wiki/Meetings/Swift
19:02:56 <notmyname> first thing up is swift client dependency
19:03:02 <notmyname> #topic swift client dependency
19:03:18 <notmyname> torgomatic: you added this to the agenda. ant to give an overview?
19:03:31 <notmyname> s/and/want/
19:03:39 <torgomatic> basically, there's a bunch of stuff we could be doing with the client, like py3 compatibility, or whatever
19:03:53 <torgomatic> but any time there's a new dependency, it transitively winds up in swift, and that's really annoying
19:03:59 <notmyname> indeed
19:04:14 <dfg> i'm cool with it
19:04:24 <torgomatic> so basically, i want to move swift-bench out to its own repo, then there's like two calls in container sync that need to be rewritten to be plain old HTTP
19:04:29 <clayg> dfg: what is "it" in that sentence referring too?
19:04:30 <notmyname> k
19:04:30 <torgomatic> and that's about it
19:04:57 * portante is in favor of this work
19:05:04 <torgomatic> then we can do things like turn on SSL cert checking by default in client version 2.0.0.0.0.0 without hosing swift
19:05:15 <creiht> that sounds reasonable to me
19:05:17 <dfg> clayg: i'm cool with what torgomatic is saying
19:05:40 <notmyname> I've talked to the -infra guys about splitting out swift-bench into another repo. it's fairly simple to do and doesn't really require any extra work from anyone to get set up
19:06:13 <torgomatic> alright, sounds like a plan then
19:06:14 <notmyname> is anyone opposed to moving in that direction? splitting out swift-bench and updating container-sync?
19:06:27 <zaitcev> and direct_client.py
19:07:05 * portante aquaducts?
19:07:16 <clayg> zaitcev: you want direct_client to go out with swift-bench because he's the only guy using it?
19:07:50 <zaitcev> clayg: I meant maybe keep json_load locally perhaps.
19:08:28 <zaitcev> it's trivial by comparison with container-sync
19:08:42 <torgomatic> could move direct_client into python-swiftclient too, or its own repo, or swift-bench, or whatever
19:08:58 <dfg> why direct_client? cause nobody likes it?
19:09:06 <torgomatic> because it depends on swiftclient
19:09:09 <notmyname> in direct_client, there is "from swiftclient import ClientException, json_loads"
19:09:20 <notmyname> so that doesn't look like a heavy dependency
19:09:22 <creiht> we could fix that pretty easily
19:09:25 <torgomatic> also "import swiftclient as client"
19:09:26 <notmyname> ay
19:09:28 <torgomatic> which is heavy
19:09:29 <creiht> and leave direct_client in
19:09:39 <dfg> ya- missed that part- i'd rather just refactor it
19:10:03 <notmyname> torgomatic: where? not in direct_client.py
19:10:09 <zaitcev> swift/common/bench.py:import swiftclient as client
19:10:12 <torgomatic> notmyname: oh, bench.py
19:10:15 <torgomatic> sorry, wrong file
19:10:25 <notmyname> which will move out, so no worries there :-)
19:10:48 <creiht> swift-bench will still have swift as a dependency
19:10:55 <creiht> will that be an issue?
19:11:17 <dfg> what about probetests?
19:11:21 <portante> what is that level of dependency?
19:11:28 <notmyname> ya, I was wondering about tests
19:11:37 <torgomatic> dfg: what about them? do they use direct_client?
19:11:48 <dfg> no swiftclient
19:11:51 <torgomatic> ah
19:11:55 <notmyname> swift_Test_client.py
19:12:27 <torgomatic> even just moving swiftclient from a runtime to a test-time dependency would be a decent win, IMO
19:12:39 <dfg> who are we kidding- nobody runs probetests anyway ;)
19:12:45 <creiht> hah
19:12:46 <notmyname> creiht: I'm ok with swift-bench initially depending on swift (ie it does today, so that's not new). but a longer-term plan would be nice to remove it and only make siwft-bench rely on swiftclient
19:12:53 <creiht> yeah
19:13:45 <torgomatic> fwiw, bench's dependencies on Swift code don't look too heavy (other than direct-client)
19:13:46 <notmyname> so the functional and probe tests are an issue
19:13:51 <torgomatic> a couple constants, config_true_value
19:14:04 <torgomatic> a json lib
19:14:15 <creiht> torgomatic: yeah, and then you could make swift not a hard pre-req, only if you want to do direct benching
19:14:22 <torgomatic> creiht: yep
19:14:42 <creiht> I don't want to remove that feature as it does come in handy from time to time
19:16:35 <torgomatic> so it sounds like we're pretty much agreed: step 1a is pull out swift-bench into its own repo, step 1b is to pull the client out of container-sync, step 2 is TBD
19:16:42 <notmyname> so initially, move swift-bench to another repo, patch up contianer_sync, and use swiftclient only for testing in swift
19:17:04 <creiht> we should probably wait for post release for that though right?
19:17:10 <notmyname> yes, absolutely
19:17:29 <torgomatic> sure
19:17:34 <dfg> and fix up direct_client too right?
19:17:40 <torgomatic> and fix up direct_client
19:17:41 <notmyname> yes
19:17:45 <notmyname> any other comments on that?
19:17:59 <creiht> wait, shouldn't we vote for clayg's sake? :)
19:18:04 <torgomatic> if we need another repo, I nominate our fearless leader to go fetch it from the infra guys :)
19:18:16 <notmyname> ya, I'll do that. that part's easy :-)
19:18:16 <creiht> repos as a service
19:18:30 <notmyname> #agreed move swift_bench to another repo and fix up the few places in swift that need it for runtime
19:18:44 <notmyname> ok, moving on
19:18:50 <notmyname> #topic havana release timeline
19:19:13 <notmyname> we've been asked to deliver an RC sometime between sept 19 (ie tomorrow) and oct 8
19:20:10 <notmyname> I don't think there's any way to do so by tomorrow, but I'd like to shoot for having patches landed by the end of next week
19:20:24 <notmyname> ie Sepy 27
19:20:52 <notmyname> #topic patches for havana
19:21:05 <notmyname> so, on the topic of patches for the release
19:21:36 <notmyname> I've put on the agenda a list of stuff that I'd like to see in this release and a list of things that are nice to haves for the release
19:21:36 <creiht> I wouldn't mind some more eyes on: https://review.openstack.org/#/c/45134/
19:21:39 <notmyname> https://wiki.openstack.org/wiki/Meetings/Swift
19:21:39 <creiht> :)
19:22:11 <notmyname> creiht: yup. definitely want that one in
19:22:18 <zaitcev> This is competitor to redbo's UDP or a compliment?
19:22:20 <creiht> cool, yeah just saw your list
19:22:28 <creiht> zaitcev: I would say complimentary
19:22:39 <portante> is there other work to consider besides what is on the wiki?
19:22:51 <portante> bugs?
19:23:11 <creiht> portante: that's for post RC ;)
19:23:11 <notmyname> portante is basically trying to win all the internet points by getting the review count up, but the disk file stuff is the last "new" feature I'd like to see.
19:23:37 * portante is found out
19:23:57 <notmyname> please take a look at the diskfile refactors
19:24:46 <notmyname> one of the main reasons for wanting the refactoring in what's called "havana" is because so many people (red hat included) will grab that and not update until next spring
19:25:11 <portante> red hat also wants pete's work on the pluggable backends
19:25:40 <notmyname> the compressing file reader patch from glange should get in (it's pretty small), and the large object patches should get in too
19:26:06 <notmyname> portante: zaitcev: that makes a long list of patches
19:26:14 <portante> ack
19:26:21 <zaitcev> I know
19:26:25 * portante does his best bill the cat impression
19:26:50 <zaitcev> I thought it would be too ambitious from the start, that's why the attempts to codify the existing API and move on to proxy. But you guys shot it down.
19:27:07 <notmyname> my hope for havana is that a 3rd party will be able to take the code and provide a diskfile implementation. docs and auditing/etc can be added after havana, but the basic support shoudl be in havana
19:27:52 <portante> it is a partial win for red hat, but does not solve the customer requests here
19:28:17 <notmyname> will rackspace devs be able to look at the diskfile refactors? y'all have been pretty silent on it
19:28:33 * portante would like that as well
19:28:39 <notmyname> portante: it enables a lot of new use cases in swift
19:29:10 <zaitcev> John, once again, DiskFile is in decent shape... Peter has reviews posted. But my half is in trouble as it lags on the account of things like ReplicatorRpc.
19:29:11 <portante> the combination of the pluggable backend work with that makes it much easier to support
19:29:30 <creiht> notmyname: I think most of us have been a bit ambivalent about it
19:30:01 <creiht> One thing that I think some of us have struggled with, is it would help tremendously if there was an actual other implementation that used the refactoring
19:30:20 <zaitcev> Yeah... If we could pull Ceph over from their RGW thingie to using the Pluggable Backend, that would be nice. But I did not even talk to them yet.
19:30:24 <portante> thre are two
19:30:29 <notmyname> portante has a sample in-memory config proposed
19:30:32 <portante> the in-memory diskfile module in the last post
19:30:37 <portante> and gluster (being worked on now)
19:30:40 <creiht> well besides something simple
19:30:42 <creiht> :)
19:30:57 <portante> will be posting the gluster refactoring later today (hopefully)
19:31:00 <notmyname> I'd love to see an xfs-specific one
19:31:01 <creiht> having something that actually works as a real world use case, helps validate the usefulness of it
19:31:17 <portante> the gluster one is basically XFS specific
19:31:30 <portante> it does not do any really glustery stuff yet
19:31:42 <notmyname> zaitcev: I talked to sage over lunch about it. maybe there is something there, but it's not incredibly likely
19:31:56 <portante> the gluster-swift team lead by lpabon will be using libgfapi instead of direct system calls later
19:32:06 <creiht> notmyname, zaitcev: I really doubt this type of refactor would work with ceph
19:32:13 <notmyname> creiht: it wouldn't
19:32:38 <notmyname> and, that's not actually a goal
19:32:57 <portante> ceph would plug in once we rework the proxy layer to allow pluggable controllers
19:33:56 <notmyname> this work (diskfile and db broker) combined with the erasure code stuff that's in progress, will allow for a deployer to choose 1) what subset of hardware is used 2) how the data is stored on that hardware and 3) how the communication with a particular storage volume actually happens
19:34:11 <portante> agreed
19:34:13 <notmyname> that's the cool stuff that the current work is enabling
19:35:03 <notmyname> a single swift cluster, with optimized communication to the volume, that has multiple replicated or non-replicated storage policies
19:35:40 <portante> creiht: would still like your input on the backend work
19:35:45 <notmyname> including migration storage policies like an s3 policy so that a swift container can "mirror" or proxy data in an s3 bucket
19:36:01 <notmyname> but that's at least 3 months out ;-)
19:36:32 <notmyname> so back to the havana patches (ie stuff to get merged by the end of next week)...
19:36:39 <notmyname> what is missing from the list?
19:36:51 <notmyname> zaitcev: do you have a link to your's?
19:37:27 <creiht> portante: and most of us are quite busy, so it may be difficult for us to find time to give it priority
19:37:35 <creiht> sorry :/
19:37:38 <notmyname> zaitcev: is it just https://review.openstack.org/#/c/40037/ ?
19:37:42 <zaitcev> There's this https://review.openstack.org/45786, but we figured it's probably no good, so I'm re-doing it. And there should be just 1 more
19:38:40 <notmyname> zaitcev: k, thanks. I'll update the list and ask you in -swift if I have any questions
19:38:48 <notmyname> moving on to the last topic...
19:39:03 <notmyname> #topic dealing with the large review queue
19:39:23 <notmyname> we've had a very large review queue recently, and we need a better way to tackle it
19:39:30 <zaitcev> 40037 should not that controversial. Funny though initially it had split configs, Ayal and Peter thought it was too confusing, so I moved that into swift.conf. Now Clay said it's too inflexible, split it back up. But that's details really.
19:39:34 <glange> make fewer changes :)
19:39:43 <notmyname> patches are staying around far to long, and it's frustrating for both reviewers and submitters
19:40:07 <notmyname> so we need some ideas of what to do
19:40:08 <creiht> yeah chain portante down somwhere so he can't make so many refacotrs :)
19:40:14 <notmyname> heh
19:40:28 <notmyname> we can see who's been active in gerrit at http://russellbryant.net/openstack-stats/swift-reviewers-90.txt
19:40:31 * portante smiles quietly to himself
19:40:41 <notmyname> so there's the wall of shame / wall of pride
19:41:05 <notmyname> besides "just do less programming", what can we do?
19:41:31 <notmyname> I can provide a list of prioritized reviews at each meeting if that would be helpful
19:41:39 <notmyname> do we need review days?
19:41:42 <zaitcev> Well, I'm doing to deliver the two I mentioned and just review the heck out of it
19:41:46 <notmyname> do we need a dedicated review sprint?
19:41:59 <portante> I would be up for a review sprint
19:42:08 <torgomatic> the swiftclient ones are backed up because I'm scared of introducing new transitive dependencies on Swift, hence the first 20 minutes of this meeting
19:42:16 <torgomatic> er, into swift, not on swift
19:42:19 <notmyname> do we need more (or different) core devs who can spend more time reviewing?
19:42:27 <torgomatic> so once that's fixed, those'll unwedge
19:42:31 <zaitcev> One in particular was David's ACL thingie, I thoght most of it was good except this writer ACL... He disagreed. Fine, nobody else chimed in to break us up, then it went inactive
19:43:03 <zaitcev> Currently I try not to look into reviews which have intricate impact. Only nibble on the sides.
19:43:09 <notmyname> zaitcev: that one has been long, but I'm not thinking of a particular patch. I mean, we've got 3 pages of stuff now
19:44:17 <notmyname> do we need more active nagging in IRC?
19:44:31 <creiht> notmyname: that will probably just make it worse
19:44:36 * portante is also up for a prioritized list to burn down
19:44:40 <zaitcev> I tune that out. It works while it's rare
19:44:50 <notmyname> I agree with that
19:45:00 <torgomatic> agreed w/zaitcev; if it becomes common, I'll just learn to ignore it
19:45:04 <zaitcev> Maybe official sprint
19:45:06 <notmyname> just trying to throw ideas out to get feedback :-)
19:45:07 <torgomatic> whether I want to learn that or not
19:46:07 <notmyname> hmm...the current top reviewers are agreeing that a review sprint is good ;-)
19:46:10 <torgomatic> we could try a prioritized list for a little while; if that's small, it'll help provide focus
19:46:11 <portante> so today, if an organization proposes a change, where they only have one core dev, they are hosed
19:46:13 <creiht> lol
19:46:15 <zaitcev> I have a ton of auto-nag e-mails "please rebuild glusterfs" from Fedora which I ignore
19:46:58 <notmyname> portante: in general, an org pushing changes in without external input is bad too
19:47:04 <portante> agreed
19:47:22 <notmyname> ok, let's start with the prioritized list and see how that helps (like torgomatic said)
19:47:29 <torgomatic> eh, my coworkers are quite efficient at ripping my diffs to little tiny pieces
19:47:31 <portante> but doubly worse because you can possible not ever get one reviewer to voich
19:47:33 <notmyname> and not try to fix too much all at once
19:47:33 <portante> vouch
19:47:59 <portante> notmyname: sounds good
19:48:28 <notmyname> kk, I'll start that, well I guess today with the havana list. ta da!
19:48:46 <creiht> yeah having a general priority list will help a lot
19:48:55 <torgomatic> notmyname: throw a link in the topic on #openstack-swift so it's easy to find once I close this browser tab
19:49:15 <notmyname> torgomatic: will do (although any voiced person can set topic)
19:49:41 <torgomatic> ah, nifty. I just always assume I have peon-level rights in any channel I didn't start
19:49:57 <notmyname> #topic general
19:50:06 <notmyname> ok, 2 small things FYI
19:50:15 <notmyname> one, openstack elections are coming up soon
19:50:25 <torgomatic> vote for pedro!
19:50:45 <notmyname> basically every PTL is up (as normal), and the TC election is general this time (ie PTLs don't get a seat automatically)
19:51:09 <creiht> oh yeah, forgot about the TC change
19:51:33 <notmyname> ttx sent an email to the mailing list recently with info
19:51:42 <notmyname> second, the swift sprint in austin is coming up in a few weeks. I'll be starting on a rough schedule next week.
19:51:57 <notmyname> for now, the only scheduled thing is BBQ on wednesday night
19:52:17 * portante can't wait to fly in and pig out
19:52:20 <notmyname> anything else? any questions or concerns?
19:52:31 <torgomatic> what are we having for dinner the other nights?
19:52:35 <torgomatic> :)
19:52:37 <zaitcev> BigMac
19:52:57 <zaitcev> Russian programmers eat instant noodles.
19:53:36 <creiht> torgomatic: leftover BBQ :)
19:53:40 <notmyname> lol
19:53:46 <portante> works for me
19:53:47 <notmyname> thanks for being here next week. thanks for making swift awesome
19:53:49 <notmyname> #endmeeting