19:01:21 #startmeeting swift 19:01:21 Meeting started Wed Sep 18 19:01:21 2013 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:22 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:01:24 OH JOY 19:01:25 The meeting name has been set to 'swift' 19:01:32 who's here? 19:01:35 o/ 19:01:38 o7 19:01:48 o8 19:01:57 ☃ 19:02:19 * portante can barely read that 19:02:21 looks like dfg and gholt jsut recently joined in here 19:02:27 agenda for this week is at https://wiki.openstack.org/wiki/Meetings/Swift 19:02:56 first thing up is swift client dependency 19:03:02 #topic swift client dependency 19:03:18 torgomatic: you added this to the agenda. ant to give an overview? 19:03:31 s/and/want/ 19:03:39 basically, there's a bunch of stuff we could be doing with the client, like py3 compatibility, or whatever 19:03:53 but any time there's a new dependency, it transitively winds up in swift, and that's really annoying 19:03:59 indeed 19:04:14 i'm cool with it 19:04:24 so basically, i want to move swift-bench out to its own repo, then there's like two calls in container sync that need to be rewritten to be plain old HTTP 19:04:29 dfg: what is "it" in that sentence referring too? 19:04:30 k 19:04:30 and that's about it 19:04:57 * portante is in favor of this work 19:05:04 then we can do things like turn on SSL cert checking by default in client version 2.0.0.0.0.0 without hosing swift 19:05:15 that sounds reasonable to me 19:05:17 clayg: i'm cool with what torgomatic is saying 19:05:40 I've talked to the -infra guys about splitting out swift-bench into another repo. it's fairly simple to do and doesn't really require any extra work from anyone to get set up 19:06:13 alright, sounds like a plan then 19:06:14 is anyone opposed to moving in that direction? splitting out swift-bench and updating container-sync? 19:06:27 and direct_client.py 19:07:05 * portante aquaducts? 19:07:16 zaitcev: you want direct_client to go out with swift-bench because he's the only guy using it? 19:07:50 clayg: I meant maybe keep json_load locally perhaps. 19:08:28 it's trivial by comparison with container-sync 19:08:42 could move direct_client into python-swiftclient too, or its own repo, or swift-bench, or whatever 19:08:58 why direct_client? cause nobody likes it? 19:09:06 because it depends on swiftclient 19:09:09 in direct_client, there is "from swiftclient import ClientException, json_loads" 19:09:20 so that doesn't look like a heavy dependency 19:09:22 we could fix that pretty easily 19:09:25 also "import swiftclient as client" 19:09:26 ay 19:09:28 which is heavy 19:09:29 and leave direct_client in 19:09:39 ya- missed that part- i'd rather just refactor it 19:10:03 torgomatic: where? not in direct_client.py 19:10:09 swift/common/bench.py:import swiftclient as client 19:10:12 notmyname: oh, bench.py 19:10:15 sorry, wrong file 19:10:25 which will move out, so no worries there :-) 19:10:48 swift-bench will still have swift as a dependency 19:10:55 will that be an issue? 19:11:17 what about probetests? 19:11:21 what is that level of dependency? 19:11:28 ya, I was wondering about tests 19:11:37 dfg: what about them? do they use direct_client? 19:11:48 no swiftclient 19:11:51 ah 19:11:55 swift_Test_client.py 19:12:27 even just moving swiftclient from a runtime to a test-time dependency would be a decent win, IMO 19:12:39 who are we kidding- nobody runs probetests anyway ;) 19:12:45 hah 19:12:46 creiht: I'm ok with swift-bench initially depending on swift (ie it does today, so that's not new). but a longer-term plan would be nice to remove it and only make siwft-bench rely on swiftclient 19:12:53 yeah 19:13:45 fwiw, bench's dependencies on Swift code don't look too heavy (other than direct-client) 19:13:46 so the functional and probe tests are an issue 19:13:51 a couple constants, config_true_value 19:14:04 a json lib 19:14:15 torgomatic: yeah, and then you could make swift not a hard pre-req, only if you want to do direct benching 19:14:22 creiht: yep 19:14:42 I don't want to remove that feature as it does come in handy from time to time 19:16:35 so it sounds like we're pretty much agreed: step 1a is pull out swift-bench into its own repo, step 1b is to pull the client out of container-sync, step 2 is TBD 19:16:42 so initially, move swift-bench to another repo, patch up contianer_sync, and use swiftclient only for testing in swift 19:17:04 we should probably wait for post release for that though right? 19:17:10 yes, absolutely 19:17:29 sure 19:17:34 and fix up direct_client too right? 19:17:40 and fix up direct_client 19:17:41 yes 19:17:45 any other comments on that? 19:17:59 wait, shouldn't we vote for clayg's sake? :) 19:18:04 if we need another repo, I nominate our fearless leader to go fetch it from the infra guys :) 19:18:16 ya, I'll do that. that part's easy :-) 19:18:16 repos as a service 19:18:30 #agreed move swift_bench to another repo and fix up the few places in swift that need it for runtime 19:18:44 ok, moving on 19:18:50 #topic havana release timeline 19:19:13 we've been asked to deliver an RC sometime between sept 19 (ie tomorrow) and oct 8 19:20:10 I don't think there's any way to do so by tomorrow, but I'd like to shoot for having patches landed by the end of next week 19:20:24 ie Sepy 27 19:20:52 #topic patches for havana 19:21:05 so, on the topic of patches for the release 19:21:36 I've put on the agenda a list of stuff that I'd like to see in this release and a list of things that are nice to haves for the release 19:21:36 I wouldn't mind some more eyes on: https://review.openstack.org/#/c/45134/ 19:21:39 https://wiki.openstack.org/wiki/Meetings/Swift 19:21:39 :) 19:22:11 creiht: yup. definitely want that one in 19:22:18 This is competitor to redbo's UDP or a compliment? 19:22:20 cool, yeah just saw your list 19:22:28 zaitcev: I would say complimentary 19:22:39 is there other work to consider besides what is on the wiki? 19:22:51 bugs? 19:23:11 portante: that's for post RC ;) 19:23:11 portante is basically trying to win all the internet points by getting the review count up, but the disk file stuff is the last "new" feature I'd like to see. 19:23:37 * portante is found out 19:23:57 please take a look at the diskfile refactors 19:24:46 one of the main reasons for wanting the refactoring in what's called "havana" is because so many people (red hat included) will grab that and not update until next spring 19:25:11 red hat also wants pete's work on the pluggable backends 19:25:40 the compressing file reader patch from glange should get in (it's pretty small), and the large object patches should get in too 19:26:06 portante: zaitcev: that makes a long list of patches 19:26:14 ack 19:26:21 I know 19:26:25 * portante does his best bill the cat impression 19:26:50 I thought it would be too ambitious from the start, that's why the attempts to codify the existing API and move on to proxy. But you guys shot it down. 19:27:07 my hope for havana is that a 3rd party will be able to take the code and provide a diskfile implementation. docs and auditing/etc can be added after havana, but the basic support shoudl be in havana 19:27:52 it is a partial win for red hat, but does not solve the customer requests here 19:28:17 will rackspace devs be able to look at the diskfile refactors? y'all have been pretty silent on it 19:28:33 * portante would like that as well 19:28:39 portante: it enables a lot of new use cases in swift 19:29:10 John, once again, DiskFile is in decent shape... Peter has reviews posted. But my half is in trouble as it lags on the account of things like ReplicatorRpc. 19:29:11 the combination of the pluggable backend work with that makes it much easier to support 19:29:30 notmyname: I think most of us have been a bit ambivalent about it 19:30:01 One thing that I think some of us have struggled with, is it would help tremendously if there was an actual other implementation that used the refactoring 19:30:20 Yeah... If we could pull Ceph over from their RGW thingie to using the Pluggable Backend, that would be nice. But I did not even talk to them yet. 19:30:24 thre are two 19:30:29 portante has a sample in-memory config proposed 19:30:32 the in-memory diskfile module in the last post 19:30:37 and gluster (being worked on now) 19:30:40 well besides something simple 19:30:42 :) 19:30:57 will be posting the gluster refactoring later today (hopefully) 19:31:00 I'd love to see an xfs-specific one 19:31:01 having something that actually works as a real world use case, helps validate the usefulness of it 19:31:17 the gluster one is basically XFS specific 19:31:30 it does not do any really glustery stuff yet 19:31:42 zaitcev: I talked to sage over lunch about it. maybe there is something there, but it's not incredibly likely 19:31:56 the gluster-swift team lead by lpabon will be using libgfapi instead of direct system calls later 19:32:06 notmyname, zaitcev: I really doubt this type of refactor would work with ceph 19:32:13 creiht: it wouldn't 19:32:38 and, that's not actually a goal 19:32:57 ceph would plug in once we rework the proxy layer to allow pluggable controllers 19:33:56 this work (diskfile and db broker) combined with the erasure code stuff that's in progress, will allow for a deployer to choose 1) what subset of hardware is used 2) how the data is stored on that hardware and 3) how the communication with a particular storage volume actually happens 19:34:11 agreed 19:34:13 that's the cool stuff that the current work is enabling 19:35:03 a single swift cluster, with optimized communication to the volume, that has multiple replicated or non-replicated storage policies 19:35:40 creiht: would still like your input on the backend work 19:35:45 including migration storage policies like an s3 policy so that a swift container can "mirror" or proxy data in an s3 bucket 19:36:01 but that's at least 3 months out ;-) 19:36:32 so back to the havana patches (ie stuff to get merged by the end of next week)... 19:36:39 what is missing from the list? 19:36:51 zaitcev: do you have a link to your's? 19:37:27 portante: and most of us are quite busy, so it may be difficult for us to find time to give it priority 19:37:35 sorry :/ 19:37:38 zaitcev: is it just https://review.openstack.org/#/c/40037/ ? 19:37:42 There's this https://review.openstack.org/45786, but we figured it's probably no good, so I'm re-doing it. And there should be just 1 more 19:38:40 zaitcev: k, thanks. I'll update the list and ask you in -swift if I have any questions 19:38:48 moving on to the last topic... 19:39:03 #topic dealing with the large review queue 19:39:23 we've had a very large review queue recently, and we need a better way to tackle it 19:39:30 40037 should not that controversial. Funny though initially it had split configs, Ayal and Peter thought it was too confusing, so I moved that into swift.conf. Now Clay said it's too inflexible, split it back up. But that's details really. 19:39:34 make fewer changes :) 19:39:43 patches are staying around far to long, and it's frustrating for both reviewers and submitters 19:40:07 so we need some ideas of what to do 19:40:08 yeah chain portante down somwhere so he can't make so many refacotrs :) 19:40:14 heh 19:40:28 we can see who's been active in gerrit at http://russellbryant.net/openstack-stats/swift-reviewers-90.txt 19:40:31 * portante smiles quietly to himself 19:40:41 so there's the wall of shame / wall of pride 19:41:05 besides "just do less programming", what can we do? 19:41:31 I can provide a list of prioritized reviews at each meeting if that would be helpful 19:41:39 do we need review days? 19:41:42 Well, I'm doing to deliver the two I mentioned and just review the heck out of it 19:41:46 do we need a dedicated review sprint? 19:41:59 I would be up for a review sprint 19:42:08 the swiftclient ones are backed up because I'm scared of introducing new transitive dependencies on Swift, hence the first 20 minutes of this meeting 19:42:16 er, into swift, not on swift 19:42:19 do we need more (or different) core devs who can spend more time reviewing? 19:42:27 so once that's fixed, those'll unwedge 19:42:31 One in particular was David's ACL thingie, I thoght most of it was good except this writer ACL... He disagreed. Fine, nobody else chimed in to break us up, then it went inactive 19:43:03 Currently I try not to look into reviews which have intricate impact. Only nibble on the sides. 19:43:09 zaitcev: that one has been long, but I'm not thinking of a particular patch. I mean, we've got 3 pages of stuff now 19:44:17 do we need more active nagging in IRC? 19:44:31 notmyname: that will probably just make it worse 19:44:36 * portante is also up for a prioritized list to burn down 19:44:40 I tune that out. It works while it's rare 19:44:50 I agree with that 19:45:00 agreed w/zaitcev; if it becomes common, I'll just learn to ignore it 19:45:04 Maybe official sprint 19:45:06 just trying to throw ideas out to get feedback :-) 19:45:07 whether I want to learn that or not 19:46:07 hmm...the current top reviewers are agreeing that a review sprint is good ;-) 19:46:10 we could try a prioritized list for a little while; if that's small, it'll help provide focus 19:46:11 so today, if an organization proposes a change, where they only have one core dev, they are hosed 19:46:13 lol 19:46:15 I have a ton of auto-nag e-mails "please rebuild glusterfs" from Fedora which I ignore 19:46:58 portante: in general, an org pushing changes in without external input is bad too 19:47:04 agreed 19:47:22 ok, let's start with the prioritized list and see how that helps (like torgomatic said) 19:47:29 eh, my coworkers are quite efficient at ripping my diffs to little tiny pieces 19:47:31 but doubly worse because you can possible not ever get one reviewer to voich 19:47:33 and not try to fix too much all at once 19:47:33 vouch 19:47:59 notmyname: sounds good 19:48:28 kk, I'll start that, well I guess today with the havana list. ta da! 19:48:46 yeah having a general priority list will help a lot 19:48:55 notmyname: throw a link in the topic on #openstack-swift so it's easy to find once I close this browser tab 19:49:15 torgomatic: will do (although any voiced person can set topic) 19:49:41 ah, nifty. I just always assume I have peon-level rights in any channel I didn't start 19:49:57 #topic general 19:50:06 ok, 2 small things FYI 19:50:15 one, openstack elections are coming up soon 19:50:25 vote for pedro! 19:50:45 basically every PTL is up (as normal), and the TC election is general this time (ie PTLs don't get a seat automatically) 19:51:09 oh yeah, forgot about the TC change 19:51:33 ttx sent an email to the mailing list recently with info 19:51:42 second, the swift sprint in austin is coming up in a few weeks. I'll be starting on a rough schedule next week. 19:51:57 for now, the only scheduled thing is BBQ on wednesday night 19:52:17 * portante can't wait to fly in and pig out 19:52:20 anything else? any questions or concerns? 19:52:31 what are we having for dinner the other nights? 19:52:35 :) 19:52:37 BigMac 19:52:57 Russian programmers eat instant noodles. 19:53:36 torgomatic: leftover BBQ :) 19:53:40 lol 19:53:46 works for me 19:53:47 thanks for being here next week. thanks for making swift awesome 19:53:49 #endmeeting