21:00:08 #startmeeting swift 21:00:09 Meeting started Wed Sep 30 21:00:08 2015 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:10 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:12 The meeting name has been set to 'swift' 21:00:17 who's here for the swift meeting? 21:00:19 o/ 21:00:23 yello 21:00:24 o/ 21:00:25 hola 21:00:29 o/ 21:00:30 O/ 21:00:31 hi 21:00:41 here 21:00:52 o/ 21:00:52 hi 21:01:42 hi 21:01:56 tdasilva: hurricanerix: cschwede: ? 21:02:08 yes, i'm here 21:02:10 ah. tdasilva, I missed that :-) 21:02:49 ok, let's get started. thanks for coming 21:02:57 yo 21:03:23 it's a cold, rainy day in san francisco. so hopefully that doesn't dampen the mood ;-) 21:03:33 #link https://wiki.openstack.org/wiki/Swift/PriorityReviews 21:03:49 lots of great work done this week. thank you for all the code and reviews 21:04:28 looking at the list of patches we'd like to see land in liberty, we're down to 3 (unless someone mentions some more) 21:04:47 but from a higher level, here's the openstack liberty story and our place in it: 21:05:00 so far, all the other projects have released their RC for liberty 21:05:12 I've told people that we'll have one at the end of this week 21:05:30 RCs are working slightly differently for the liberty release 21:06:00 instead of a tag and release version like 2.5.0rc1, we'll have a normal tag, undecorated with any "rc" nomenclature 21:06:20 so this release will be 2.5.0. that's what we'll tag 21:06:43 and if we have something that comes up that in the past would result in an rc2, that will be tagged as 2.5.1 21:06:48 and so on for anythign we find 21:06:58 notmyname: that's nice 21:07:49 the only real change this means for us is that we have to "leave space" in the release numbers for any future backports on releases that are maintained longer (ie what was the "integrated" release) 21:08:09 so that means that whenever we do our first release after liberty, it will be 2.6.0. 21:08:31 it cannot be 2.5.x because that release namespace will be used for backports and later subsequent releases 21:08:36 make sense? any questions on that? 21:09:01 sounds reasonable 21:09:15 makes sense so micro release numbers used for RCs 21:09:22 also, FWIW, that's also the same thing that's being used for swiftclient too 21:09:48 the official liberty release date is october 15. that means we have pretty much until the end of _next_ week to do any further "release candidates" 21:10:13 acoles: ^ see it's fine! 21:10:23 * acoles breathes out 21:10:23 we have litterally *days* to fix the whole thing 21:10:34 more than a week! 21:10:35 so, this week: tag a release that we think will be in liberty. next week: our chance to land bug fixes for liberty before the release 21:11:11 ok, one final thing I want to say before we talk about these three specific patches 21:11:30 sounds ominous 21:11:33 (in order to be transparent and let us all participate) 21:11:36 no, not at all 21:12:22 if we were to take the commit SHA at the current HEAD on master and say that is the liberty release, nothing about what I've told people is in swift for liberty would be wrong 21:13:10 also, most of us who are really concerned about these last few patches (ie EC in swift) don't really care all that much about "liberty" or not--we all spin our own packages 21:13:31 however, that being said, I think there are 2 good reasons to still try to get these patches in this release 21:14:04 first, it's a goal for us, the swift devs, to get behind and accomplish and feel good about 21:14:48 second, outside of our own little group, the liberty release has significant impact from what people say about swift and when or if they use it (ie "marketing") 21:15:58 so when we get these things landed and we tell people we've made significant progress on making swift (even more) awesome, it's something we can be proud of and it something that end users start to hear from the whole openstack community. and that's great! 21:16:24 so that's why I'm pushing for getting these patches in by friday. thanks for helping with that :-) 21:17:03 so shall we talk about patches now? 21:17:20 lets 21:17:24 #link https://wiki.openstack.org/wiki/Swift/PriorityReviews 21:17:31 #topic patch 227133 21:17:32 notmyname: https://review.openstack.org/#/c/227133/ - Fix copy requests to service accounts in Keystone 21:17:41 first up, a non-ec one 21:17:48 mattoliverau: thanks for looking at this one 21:17:56 yes thanks mattoliverau 21:18:04 acoles: you've updated the patch with some updates 21:18:14 AFAICT this is ready for reviews. 21:18:23 mattoliverau: you should probably reverify it today 21:18:55 is cschwede here? i pushed over with mattoliverau comment fixed and some changes of mine 21:19:09 acoles: are your changes in the last patch set significant enough in your opinion that you shouldn't also be a reviewer of it? 21:19:12 no problem, thank cschwede, acoles and donagh really 21:19:15 mattoliverau: i made no change to the actual fix, just tests 21:19:16 notmyname: wil do 21:19:50 notmyname: let's see if someone else can but as last resort i would add +2 21:19:52 I was planning to look at this today as well, although I don't have a keystone handy (I think. I'll look), so I might only leave a +1 21:19:54 acoles: ok 21:20:08 I'd love to see this one land today 21:20:27 ho would be a good person to review it 21:20:33 yeah! 21:20:57 mattoliverau: I may need to ask you some questions about it related to the keystone roles. you'll be online in a few hours? 21:20:57 mattoliverau: kota_ could you as ho in your day time, he's not here? 21:21:02 ask 21:21:20 acoles: sure 21:21:20 acoles: yup 21:21:29 notmyname: sure 21:21:43 ok, great. that one is easy :-) 21:21:52 #topic patch 222799 21:21:52 notmyname: https://review.openstack.org/#/c/222799/ - Validate against duplicate device part replica ass... 21:21:59 clayg: how goes it? 21:22:36 we really had a good thing going with that "you need at least five zones to run swift" thing - then someone had to go screw it up and and make it possible to run on smaller topologies - been down hill ever sense 21:22:46 heh 21:23:02 that was the beginning of the end 21:23:07 I think overload was the end of the end, though 21:23:08 that is to say - i have also broken against the tide of the perfect part placement algorithm 21:23:35 I for one welcome our new ring overloads 21:23:37 meaning you're done with that for now? or at least for the short term? 21:23:38 and yes overload - a response for dealing with topology migrations that dramatically effective your dispersion - was a bad idea 21:24:07 fudging with weights and trick the fill algorithm isn't going to work in the long term 21:24:30 in the short term - we really should place a replica of all the parts on unique devices - but it's not so simple 21:25:02 neway - I think we have a part-replica placement algorith that works pleanty of times - but sometimes it sucks and we should fix it 21:25:07 but I'm not going to do that by friday 21:25:21 ok 21:25:32 so that gets us to the validator, right? 21:25:35 SO - I think we should leave it work about the same as it has for a least the last six months - but try to warning about a little better - let people know when we're struggling 21:26:09 I think the "don't do the dumb thing" fix in the validate change will help quite a bit for lots of folks (i'm running some datasets now) 21:26:34 I think i might be able to squeeze in a "get you out of hot water fix" too 21:26:40 but it won't be a general solution 21:26:41 it does sound like a much better solution that where we are now, even if it's not the perfect general-case one 21:26:54 +1 21:27:26 clayg: thanks for picking up that ring work 21:27:35 so... i pushed a patch - folks should look over the tests 21:27:36 +1 good work 21:27:42 for patch 222799, it's ready to review? 21:27:42 notmyname: https://review.openstack.org/#/c/222799/ - Validate against duplicate device part replica ass... 21:28:10 I tried to make it so that we don't have any tests that setup these land mines - because I think a lot of these "warnings" *should* be errors - and I'd like our tests to pass 21:28:15 nice subject shortening, patchbot 21:28:35 lol 21:28:49 I think we *can* make small rings that will rebalance and look good - *but* I don't think they'll do a good job of showing the behaviors the existing tests were trying to demonstrate 21:29:57 clayg: are you expecting a subsequent patch to follow the current one? eg for the "get out of hot water fix"? 21:30:09 so I'm moving existing tests to more devices and demonstrate reasonable behaviors - then we'll add more tests with smaller devices that demonstrate reasonable (although not nessecarily obviously optimal in the cases where more devices would allow better part-replcia placement) 21:30:21 notmyname: not 100% sure on that 21:30:36 ok 21:30:54 are the more tests in this current patch, or will that be "sometime later"? 21:30:54 I guess I'd probably like to just pull it into that one and make it the one and only change that we try to review this week wrt rings 21:31:50 I added an example strawman test in this latest change - I don't think it's worth adding them now because they would having nothing sane at the end to validate 21:32:04 ... it'd be more like an "expected failure" 21:32:19 ok. and thus your very first comment of "take a look at the tests" ;-) 21:32:52 I don't think their critical - with the warnings in place I'm finding it quite trivial to identify topologies that result in bad placement 21:33:02 ok 21:33:05 but that's just IMHO 21:33:21 clayg: beyond reviewing what's up there now (and I supposed if it's all good also landing it), is there anything else you need help with for that whole set of work? 21:33:45 I would love some sanity checks on the tests where I'm turning 4 devices in 12 and stuff - the intent was that they read about the same - so it's very much a game of "what makes the diff easier to grok" 21:33:49 so outside eyes would be helpful 21:33:57 ok 21:34:20 the "get you out of hot water fix" would just add something into gather that picks up these screwed up placements with doubled devices and tries harder to put them down right 21:34:47 ... might also require making get_effective_overload not return 0 just because it thinks the topology looks right when the *current* dispersion is non-zero 21:35:01 so for everyone, it would be very helpful to look at this patch today. start with tests, and be active in IRC with questions 21:35:04 clayg: thanks 21:35:09 anything else on this one? 21:35:15 but I'm going to try and validate those changes are helpful on a larger dataset than just tests and if I like the results push up one more time 21:35:20 sorry to gab so long 21:35:25 also I hate rings and math 21:35:35 no worries. it's super important 21:35:45 thanks clayg 21:35:57 #topic patch 215276 21:35:57 notmyname: https://review.openstack.org/#/c/215276/ - Enable object server to return non-durable data 21:35:57 yeah thanks clayg 21:36:03 acoles: where are we here? 21:36:13 you've got it WIP 21:36:39 -2 actually 21:36:49 I hit a snag this evening with this 21:37:01 ok 21:37:14 in general its good and ready for review, and boy will it need review! 21:37:47 yeah, it's big. but most of the line count are well-formatted tests, so don't be too fearful about just the line count 21:37:48 but there is an issue when used with object_post_as_copy=false 21:38:22 It is big, and I'd love people to tell me where I got carried away, but yes there's a lot of test and hopefully helpful commentary 21:39:05 There's a failing probe test when object_post_as_copy=false 21:39:07 the test change was awesome, good refactor for us, imho. 21:39:08 acoles: what's the best place to start to get a sense of it (beyond the commit book^Wmessage) 21:39:46 notmyname: ok, good question. read the commit message, then maybe look at test/unit/proxy/test_server.py:TestECGets 21:40:03 those tests are meant to cover the GET cases we're trying to fix 21:40:13 ok 21:40:34 are you intending to push another patch set tonight or will it be your tomorrow? 21:41:14 I will try for tonight. To fix the issue I found I need patch 138498 21:41:15 acoles: https://review.openstack.org/#/c/138498/ - Add POST capability to ssync for .meta files 21:41:24 yeah, that was my next question :-) 21:41:28 acoles: +2 21:41:29 which is also not small, but again, lots of test lines in there 21:41:55 so this means that patch 138498 *also* needs to land. and before patch 215276? 21:41:56 notmyname: https://review.openstack.org/#/c/138498/ - Add POST capability to ssync for .meta files 21:41:57 notmyname: https://review.openstack.org/#/c/215276/ - Enable object server to return non-durable data 21:42:15 * notmyname loves the patchbot subjects. thanks timburke 21:42:40 notmyname: yes. otherwise I try a different approach in obj.py and stop assuming that all frags for same object have same timestamp 21:42:47 ok 21:42:51 also, yikes 21:42:51 ;-) 21:43:03 but i kinda feel i ought to be able to assume that and that ssync should be fixed 21:43:20 yeah. that makes sense 21:43:28 like i said before, if someone can show me an easier way please do! 21:43:57 acoles: anything else on this set of work to bring up? 21:44:03 (i know of some but they are less optimal in terms of number of GETS to obj servers to piece together an object) 21:44:50 no, thanks 21:45:16 #topic else: 21:45:20 ok, so to sum up.. 21:45:28 we've got 4 patches to look at 21:45:39 oh, stuff like the right name for new headers etc in 215276 - please feedback opinions 21:45:40 cross-account copies with service tokens. hopefully will land today 21:45:58 ring validator from clayg. start with tests and go from there 21:46:24 the optimistic GETs (etc) from acoles 21:46:44 and the fix for ssync fast-post for the optimistic GETs patch 21:46:56 lots of stuff 21:47:10 If there is time, we should try to merge 196848 21:47:19 look for next versions on those last two patch 138498 patch 215276 21:47:20 acoles: https://review.openstack.org/#/c/138498/ - Add POST capability to ssync for .meta files 21:47:21 acoles: https://review.openstack.org/#/c/215276/ - Enable object server to return non-durable data 21:47:26 After the critical/high priority ones have been finished. 21:47:33 acoles: ack 21:47:43 thank you, everyone, for helping. and especially for acoles and clayg for the recent hard work on those big ones 21:48:04 patch 196848 21:48:04 notmyname: https://review.openstack.org/#/c/196848/ - Optimization of the reconstructor for handling of ... 21:48:32 minwoob: right! definitely want to get that one landed. extremely unlikely for liberty 21:49:09 anything else to bring up for this week? next week, we'll see where we are for liberty and start talking about summit planning 21:49:12 Okay, yeah, it might have to wait until the next release then. 21:49:33 oh, also as a teaser, I saw some pictures of tshirt proofs. I should have them on friday 21:49:38 they look good :-) 21:50:03 excited to see them 21:50:04 cool 21:50:15 if anyone's looking for some respite from the large, critical patches, i'd love to see mattoliverau's patch 120709, or kota_'s chain of three that ends with patch 198634 21:50:15 timburke: https://review.openstack.org/#/c/120709/ - Add container and account reverse listings 21:50:16 timburke: https://review.openstack.org/#/c/198634/ - Support last modified on listing containers 21:50:43 timburke: those would all be amazing 21:50:46 we can get respite from large patches next week ;-) 21:50:59 notmyname: good point 21:50:59 yeah, I like them too 21:51:08 notmyname: fair enough. it'll be like vacation :) 21:51:12 timburk: thanks for biringing up them. 21:51:12 heh 21:51:48 great. I think we all know what needs to happen. please be active in IRC. don't wait to ask, just ask 21:52:04 thanks for working on swift. you're great 21:52:07 #endmeeting