18:59:49 #startmeeting swift 18:59:50 Meeting started Wed Feb 25 18:59:49 2015 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:59:52 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:59:54 The meeting name has been set to 'swift' 18:59:56 who's here for the swift meeting? 19:00:01 howdy 19:00:05 here 19:00:14 hi 19:00:16 hello 19:00:18 👋 19:00:40 torgomatic: you always intro yourself with an emoji 19:00:48 o/ 19:00:58 mattoliverau: good morning 19:00:58 hi 19:01:18 kota_: glad you're here. we were just looking at one of your patches 19:01:24 hello 19:01:32 notmyname: 😃 19:01:34 cschwede: dinner was good? 19:01:37 #link https://wiki.openstack.org/wiki/Meetings/Swift 19:01:44 notmyname: yes, thx :) 19:02:06 ok, just a few things to cover this week. mostly a status check 19:02:27 first up, as promised last week, I'll be focusing on EC status 19:02:36 #topic EC status 19:02:46 GET patches have landed 19:02:49 yay 19:02:55 torgomatic: peluse: what's missing from them? 19:02:59 write *and* read, wow 19:03:12 all the partial PUT failure stuff on the GET side is missing 19:03:17 but there's a trello card for that 19:03:26 gotcha 19:03:31 tsg is, any minute, going to push the updated multiphase patch that has all of PUT complete 19:03:40 great 19:03:51 except the correct ETAG/content-length in the DB 19:03:55 and peluse has the reconstructor patch 19:04:12 that will be a new patch (item on trello) that thiago has agreed to knock out after alistair's override path clands 19:04:24 nice 19:04:27 yes, on the reconstructor 19:04:40 more progress on unit tests, etc. hwoever I think I'm going to take a day and break it up a bit 19:04:47 multiple patches? 19:04:50 at least try to spearate the non-reconstructor diskfile chagnes out 19:04:53 yes 19:04:59 ok 19:05:04 essentially support for multiple fragment archive indexes 19:05:09 it does seem kinda big 19:05:11 * acoles apologises for late arrival 19:05:18 acoles: no worries 19:05:24 because that will be needed by the remaining GET work which won't want to depend on my growing reconstructor patch 19:05:57 so that will be the rest of my week... trying to carve that out and push an update to reconstructor as well 19:05:59 peluse: and so you expect that to be split up tomorrow some time? 19:06:05 or Fri 19:06:05 ah. endof-week 19:06:10 ok 19:06:18 maybe tomorrow :) 19:06:20 so looking at trello 19:06:22 #link https://trello.com/b/LlvIFIQs/swift-erasure-codes 19:06:35 I will be setting up some time to online code review on Fri as well 19:06:52 we've got the GET handling you mentioned. the PUT to land. and the multi part reconstructor piece 19:06:59 and tests and docs 19:07:12 yup 19:07:21 but looking at the functional pieces required to have a beta in kilo, that's what we've got 19:07:36 so, for everyone, is there something missing? 19:07:54 I *think* the red items on trello make a good beta 19:08:28 we've talked a little about the docs parts. I think some docs are necessary (at least "how to set it up and use it"), but all those IMO aren't required 19:08:39 eg perf considerations or best prectices 19:08:40 what's our target for landing the beta requirements on feature/ec? 19:08:43 I don't think we have any idea 19:08:53 (about the docs) 19:08:56 :) 19:08:58 I figured 19:09:01 peluse: right before we propose to master 19:09:05 :-) 19:09:06 ha! 19:09:07 maybe i missed that, but what if the reconciler has to move from 3-replicas to EC? is that/will that be supported? 19:09:24 cschwede: good questions 19:09:35 clayg: any ideas? just works? needs code? 19:09:43 cschwede: we'll have to test it now that we have most of EC working - but internal client shouldn't really care about the source and destination policies 19:09:53 yup, agreed 19:10:01 GET -> PUT if it works for the client it should work for the reconciler 19:10:15 back to the data question.... 19:10:20 notmyname? 19:10:23 date, not data 19:10:30 ok, dates.. 19:11:16 by next week we need to make a call. do we have enough velocity and small enough scope to finish what we'd be ok calling a beta by kilo 19:11:31 peluse: i don't understand - we want everything on feature ec as soon as possible, but having docs on feature/ec two weeks before the two week merge period to master doesn't seem useful 19:11:32 we have about 4 weeks from now to start the merge to master 19:11:36 and then 2 weeks for the merge 19:11:39 then the RC 19:11:49 looking at calendar, one sec 19:12:18 so around Mar 25 start the merge to master you're thinking? 19:12:29 not very far away 19:12:48 nope, and I'll be drinking wine in Napa Mar 26/27 BTW 19:12:59 * notmyname pulls out notes 19:13:03 20 yr anniversary and all... 19:13:07 nice 19:13:23 mar 27 19:13:32 but if that's the case the timing for our Mar 18 mini-F2F is just about perfect for identifying/wrapping loose ends, planning merge stratgy 19:13:37 estimating the RC on april 10 19:13:46 nothing say anniversary like some EC work :P 19:14:17 yeah, that's go over well! Hold on honey, gottta check zuul.... 19:14:26 lol 19:15:10 so I think this last week (since last meeting), things have gone well. thank you to everyone for that 19:15:11 OK, so anyway. yeah I think we can make it. Big thing for me is breaking the recon up a bit and spending some quality review time especially in some of the test code 19:15:55 can we get another core set of eyes on acoles header patch? 19:16:00 for those looking to review something, https://review.openstack.org/#/c/158821/ by acoles is necessary for EC and crypto 19:16:03 peluse: yep me 19:16:11 #link https://review.openstack.org/#/c/158821/ 19:16:12 thanks 19:16:23 peluse: and tsg will be pushing new patch sets asap? 19:16:28 that's the blocker there, I think 19:16:42 he was aiming to have it done before this meeting. since he's not here I assume he is frantically working on it :) 19:16:47 :-) 19:16:48 ok 19:16:48 I looked at that patch some, but then I fell into the rabbit hole of failing probetests 19:16:54 hope to get back to it today 19:16:56 same here 19:17:04 ok, thanks for the status update 19:17:18 peluse: let me know if there is anything more that i can usefully do building on that patch 19:17:19 look for tsg's patch today, I'mm mention it in IRC when it comes out 19:17:29 kota_: FYI you might be interested in some of those probetest failures. see the playback in #openstack-swift for details 19:17:37 peluse: thanks 19:17:40 ok, moving on 19:17:44 #topic fsync on dirs 19:17:53 this is something we spent time on at the hackathon 19:17:56 acoles, I think its good. tdasilva already did some quick testing w/the DLO failure on EC GET and it worked (he can comment more) and will be finishing the EC specific stuff in that area after it lands 19:17:59 #link https://review.openstack.org/#/c/126923/ 19:18:09 tdasilva: I think you wanted a status update on it 19:18:17 looks like ppai has pushed a new patch set 19:18:31 acoles: I can provide a paste so you can see the patch I have for EC and how i'm building on top of your patch 19:18:43 and acoles has some updated benchmark numbers 19:18:44 tdasilva, me too please 19:18:58 notmyname: yeah...we left the hackathon trying to make a decision and waiting to hear from rackspace guys 19:18:59 tdasilva: ok cool 19:19:04 * peluse apoligizes for not switching gears fat enough :) 19:19:05 notmyname: ya, interesting (about probetests) 19:19:41 re fsync on dirs i repeated my benchmark just fsyncing the leaf dir and it made no difference 19:19:45 tdasilva: ah, yes. we didn't hear anything from rackers yet. I'll add them to the review. but otherwise, Ithink it's ready to go 19:20:07 tdasilva: the latest patch set has the comments we made in person? eg the always on, no config? 19:20:12 notmyname: ok, just wanted to make sure 19:20:44 tdasilva: IMO it should be reviewed, and I'll add dfg and glange and hurricanerix_ to it 19:20:49 notmyname: need to double check, but I believe so 19:20:53 ok 19:21:17 notmyname: yep, no option 19:21:18 I'll also add it to priority reviews 19:21:30 anythign else on that patch? 19:21:36 tdasilva: thanks for bringing it up? 19:21:41 s/?// 19:21:47 notmyname: nothing from me 19:21:53 ok 19:21:57 sorry, i'm late folks 19:22:03 cutforth: welcome 19:22:07 #topic python-swiftclient 19:22:22 there's been a lot of stuff that's landed on swiftclient since the last release was tagged 19:22:37 I think MD5 is the biggest, the rest is meh. 19:22:53 * cschwede agrees with zaitcev 19:22:53 for those of you who have been looking at it, what are your thoughts on tagging a new release? 19:23:24 zaitcev: you had the pseudo dirs crash fix. that's good 19:23:39 notmyname: there's one of jeol's patches might be good to land 19:23:46 acoles: ok. got a link? 19:23:52 min 19:24:09 oh, that was a while ago. I also fixed Darrell's crash. But those were safe from regressions, relatively speaking. 19:24:33 #link https://review.openstack.org/#/c/130339/ 19:24:34 sorry gerrit slow https://review.openstack.org/130339 19:24:43 clayg: acoles: thanks 19:24:44 Right, the shuffle 19:24:47 clayg: beat me :) 19:25:08 been a while since i looked at it so need to revisit 19:25:28 I think I have a question, if anyone understands. We used to shuffle everything. Now we shuffle inside a page (even if it's enabled). So... How big is the page? 19:25:33 ok. I'll track that one and look at a release after it lands 19:25:34 thanks 19:25:45 zaitcev: ~10K 19:26:19 That patch has surprisingly big benefit, if you run "time swift download blah" it shows hundreds of kilobytes less even for rather small downloads. 19:26:37 cool 19:26:54 wait a second, is the latest release from september 2014? ok, there are a lot of interesting commits then 19:27:27 :-) 19:27:55 Probably might as well cut it before the shuffle patch, to be safe. I'll package it into Fedora right away. 19:28:07 before? why? 19:28:26 well, to have a known-good release... 19:29:36 if it's reviewed and tested and landed, I'd call it known-good 19:30:05 unless there's a big gotcha in this one. anyone else have strong thoughts? acoles? clayg? 19:30:40 I see 19:30:41 ;-) 19:30:46 iirc from when i last looked the only issue i had was how to verify the download sequencing 19:30:54 in unit tests 19:31:29 this is the part of the meeting that you make fun of me about: "anything? anyone?" "ok, moving on...." 19:32:30 i'll look at it again tomorrow 19:32:32 zaitcev: if you've got some strong concerns, i'm fine with cutting one before. but if it's reviewed and tested, I'm fine with including it in a release 19:32:34 acoles: thanks 19:32:48 notmyname: sorry, memory has faded a bit on that one 19:32:54 no worries 19:33:02 notmyname: I don't have strong concerns. 19:33:06 ok 19:33:07 #topic open discussion 19:33:10 anything else? 19:33:20 that's all I've got for this week 19:33:47 19:33:57 tick - tock 19:34:02 :-) 19:34:02 ok 19:34:12 thanks everyone for attending and working on swift! 19:34:20 #endmeeting