19:00:39 #startmeeting swift 19:00:40 thanks bye 19:00:41 Meeting started Wed Feb 5 19:00:39 2014 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:42 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:00:44 The meeting name has been set to 'swift' 19:00:59 hello world! 19:01:02 hello! 19:01:03 hola! 19:01:04 bonsoir! 19:01:08 hi 19:01:09 ehlo! 19:01:24 erf 19:01:24 hello 19:01:25 looks like we have a full agenda for this meeting 19:01:27 #link https://wiki.openstack.org/wiki/Meetings/Swift 19:01:44 but maybe some of it will go quickly (here's hoping ;-) 19:02:02 first, as a general update... 19:02:13 swift 1.12 was released last week. 19:02:18 sweet 19:02:23 a few days later than expected because of the gate issues 19:02:55 seems that gate issues have lessened somewhat, but that's (AFAIK) mostly because of the limitation on concurrent patches being tested 19:03:05 that means less gate resets and therefore faster processing 19:03:13 @notmyname, hi john. 19:03:24 notmyname: does it mean less stable code being committed? 19:03:25 there will be some trimming of the jobs that are run, so that should also help 19:03:38 still many rechecks 19:04:14 yes, there are many rechecks. but zuul is just testing the top XX patches in the gate instead of all of them. that means more nodes available for when resets happen in eg the top 10 patches 19:04:34 improvements, but IMO no fundamental systemic changes 19:04:47 anyway, that's all I got on that 19:05:23 seems like there are some discussions that may affect us to some degree in the TC and BoD about "what is openstack core", but I'll keep you up to date on that as new things happen 19:05:31 just stuff I'm keeping my eye on 19:06:10 so let's move on to the agenda items. anything else can go to open discussion at the end if we have time 19:06:20 #topic storage policy status 19:06:37 we've got about 8 weeks until the Icehouse RC is due 19:06:41 so the EC branch was recently rebased (thanks torgomatic) and 3 patches were marged since 19:06:49 good 19:06:49 there are now 5 policy patches ready for final review 19:07:03 status pon the account status rollup and the policy reconciler? 19:07:06 beyond that, I have one todo (account roll-up) and torgomatic can speak to SLO/DLO 19:07:18 So sorry not reviewing that. 19:07:37 ahh, acct rollup is WIP. torgomatic has the SLO/DLO and container reconciler 19:07:53 DLO was just approved I think 19:07:57 SLO is merged 19:08:00 torgomatic: right? 19:08:03 cool 19:08:31 so that would mean 5 in need of final review, 2 WIP and from there I think we are ready to make the call on merge to master 19:08:33 SLO is done and merged, DLO has had troubles merging, but we've been hitting it with the reverify plunger, and it'll go in eventually 19:08:37 well... docs too :) 19:08:53 it would be a good idea for people to start taking a look at the branch (feature/ec). after the rollup and reconciler land, the feature/ec branch needs to be proposed for merge into master 19:09:04 torgomatic: have you had a chance to look at the container reconciler (clayg said he thought you were doing it) 19:09:11 I think object versioning can live without being hoisted into middleware, FWIW... when I looked at the code I didn't see that it was necessary 19:09:25 peluse: yeah, I've started poking at it, but nothing interesting enough for Gerrit yet 19:09:25 I expect that merge process to take a few weeks, then we should be able to let it sit for a couple of weeks before the Icehouse RC 19:10:05 great, lets get those reviews going. the 5 that are up there are small compared to the last several 19:10:20 am I missing anything on storage policies here that are necessary before the merge to master? 19:10:46 i think Trello is up to date - all required are in doing or code review or done 19:11:08 well, docs... need to start on that. I'll do so after th 5 up there land 19:11:16 #link https://trello.com/b/LlvIFIQs/swift-erasure-codes-and-storage-policies 19:12:14 ok, so keep reviewing, finish up the patches for the last bit of functionality, and we're on track for storage policies in Icehouse. yay! 19:12:21 I'm pondering if I ought to ask peluse or someone else on EC crew to trade reviews with me. I was sitting on that 47713 since before last Hackathon! 19:12:24 rock n roll 19:12:35 yup, can do that zaitcev 19:12:40 thanks peluse 19:12:56 awesome, I always wanted to learn about EC more, too. 19:13:01 #topic EC status 19:13:17 a follow-on to storage policies 19:13:19 zaitcev: the time is coming.... EC has a few small things on gerrit but not up for review yet 19:13:21 peluse: anything here? 19:13:33 tushar and keving have been banging on it and should have PUT path ready for review soon.. Tushar? 19:13:48 notmyname: Kevin and I have been finalizing the pyeclib stuff 19:13:54 good 19:14:12 I will have stuff rebased and posted up on geritt first thing next week 19:14:13 note: for everyone that didn't catch it, the EC branch only has policy stuff now and we won't be landing EC work there until policies move to master 19:14:29 good point. thanks for mentioning that 19:14:47 yeah, in case folks were looking for ec code on the ec branch, go figure :) 19:15:04 also, when we get storage policies merged into master, I want to do a retrospecitve to see if the long-running feature branch is a good idea for us 19:15:06 zaitcev: if you could review ec/pyeclib discussion on trello, that will be cool 19:15:32 sound good 19:15:42 anything else on EC to report? 19:15:54 tgohad: I have account but do not quite understand how Trello works. I'll look into it today. 19:15:56 peluse: or tgohad? 19:15:56 I have started an rst for EC design 19:16:02 don't think so... 19:16:26 k 19:16:39 zaitcev: its just a bunch of semi-organized discussion. we're not using super wiz bang features 19:16:40 zaitcev: let one of us know if you have issues accessing 19:17:03 any questions on storage policies, EC, or the plans here before we move on? raise your hand so I know you have a question and don't move on too fast 19:17:54 ok, moving on 19:18:03 #topic python-swiftclient port to requests 19:18:13 tristanC has been working on this 19:18:28 the reason has started with wanting to do cert validation 19:18:29 what does it use currently? 19:18:32 What does it mean? 19:18:35 httplib 19:18:55 tristanC: want to give a summary here? 19:19:01 notmyname: yes sure 19:19:02 requests is also providing a much nicer (more pythonic) API 19:19:19 indeed 19:19:22 as of now, python-swiftclient does not verify ssl certificate, allowing mitm attacks 19:19:36 (also resulting in it being dropped from debian packaging) 19:19:45 They should've called it something that was obviously a name or symbol. Like bagels. Then you port to bagels. 19:19:57 an attempt have been made to fixes this within current implementation, but the validation wasn't done properly and the patch was not working when actually transfering data 19:20:20 #link https://review.openstack.org/#/c/69187 19:20:56 well, most openstack client are now using requests for http[s] requests, so here is the port to requests ^ 19:21:24 tristanC: can you list the current issues iwth ssl and 100 expect? 19:21:25 so there are 2 downsides to using requests. tristanC can you go over them? (ssl compression and 100-continue) 19:21:31 okay... I guess I'll review after the policies then 19:21:50 yes, as explain on the swift agenda wiki page, there are two remaining issue 19:22:19 First, ssl_compression. It isn't controllable anymore, as it should be disabled by default 19:22:31 and so it's always disabled now? 19:22:33 How can SSL compression be a "downside" when any expert you ask says just disable it and forget it ever existed. Bah, done. 19:22:56 zaitcev: right. just a change from existing swiftclient behavior 19:23:12 I think this issue should be noted, but isn't too concerning, from what I know of it 19:23:19 notmyname: not always sadly, old system like debian 7 does compress ssl traffic 19:23:42 tristanC: any system that has security updates installed from < 1 year ago should have ssl compression disabled 19:23:53 dirk: correct 19:23:56 tristanC: ah, so you get whatever the underlying system does? 19:24:07 "to disable SSL compression, please switch to a completely different operating system" <-- fun times 19:24:23 notmyname: as I said, ssl_compression is not controllable with requests as of now 19:24:31 * torgomatic actually isn't too worried about it 19:24:53 ok, so we lose functionality on older systems by moving to requests wrt ssl compression 19:25:10 cert validation >> toggling SSL compression IMO 19:25:22 debian 7 old? 19:26:16 does anything think this ssl compression issues is a showstopper for the port to requests? 19:26:45 maybe the HP guys if they are around since they were the one concerned hte most by the feature 19:26:56 (obviously the official vote there is in gerrit, but I wanted to bring it up here) 19:27:22 chmouel: good point. it would be good to get acoles to look at tit 19:27:25 and it 19:27:43 just to be clear -- is requests forcing it to be used? or is it providing the option to whereas that option was not present before? 19:27:45 tristanC: so 100-continue support 19:28:10 briancline: requests forces compression to be off all times, as it should be 19:28:12 briancline: the function is only available with python3.2+ version 19:28:17 chmouel: notmyname: i'll run it by the relevant hp folks 19:28:22 acoles: thanks 19:28:42 And the second issue, is that requests does not support the "except: 100-continue" header 19:28:50 dirk: well, tristanC said it ends up being on for debian 7 19:29:00 tristanC: that one concerns me a little more 19:29:13 tristanC: isn't there an unadressed issue open with requests to support it? 19:29:25 is expect-100-continue the sort of thing that we'd get for free later if requests ever did support it, or would we need extra plumbing in swiftclient? 19:30:05 torgomatic: I think we will get for free 19:30:12 notmyname: agree, and the patch that disable ssl_compression once and for all is good to go for python3.2+. For the other version there is solution in pyopenssl contrib module of requests. Though I haven't test this 19:30:24 100-continue support isn't something we can require clients to use, but it sure would be nice if the official client app actually supported it 19:30:38 rather than waiting around for it, perhaps it'd be a good idea to contribute towards it to make it happen in its rightful place (requests) 19:30:49 torgomatic: current implementation of httplib in swiftclient does not do any plumbing (as the 100-continue is controlled by requests header) 19:30:52 briancline: thanks for doing that ;-) 19:31:26 briancline: but yes, I agree :-) 19:31:33 as long as we don't end up with some annoying compatibility shim for old/new versions of requests, that's okay by me 19:31:41 I'll have to read on this more, I have no clue what we're talking about. It was my view that 100-continue was absolutely essential for PUT, because otherwise errors cannot be sanely reported. So it's very amazing that any HTTP library would not support it. 19:31:51 #link https://github.com/mikeal/request/issues/611 19:32:10 so same question on this point: does anyone see lack of support for 100-continue in requests is a showstopper for porting swiftclient to use requests? 19:32:11 or maybe that's not the one 19:32:27 #link https://github.com/kennethreitz/requests/issues/713 19:32:39 the good link ^ 19:32:52 zaitcev: it allows early failure, so yes it's very important for sane clients to use when they are uploading lots of data 19:32:57 zaitcev: 100-continue purpose is to avoid sending the full body before having an error. 19:33:19 notmyname, tristanC: exactly my recollection too 19:34:32 to me it's kinda like the boto default that was reported today (a default makes a bunch of extra requests to S3, resulting in higher bills). I'd like our client to do the Right Thing (tm) 19:35:06 tristanC: thanks for bringing these up 19:35:24 notmyname: you're very welcome 19:35:25 any more questions or discussions for this meeting on porting python-swiftclient to use the reuqests library? 19:35:37 notmyname: FWIWs, going forward with requests seems sane to me 19:35:50 we work on contributing 100-Continue to requests for the future 19:35:51 portante: +1 19:36:05 I agree 19:36:09 "We" being Tristan or...? 19:36:28 zaitcev: briancline volunteered I think ;-) 19:36:38 zaitcev: yes, the royal "we" of folks concerned about swift who want a client that uses 100-Continue 19:37:11 briancline: I would be willing to review the work on requests to get that 100-Continue support in, and/or help you as you see fit. 19:37:35 actually briancline was just the unlucky one to mention doing a patch upstream first 19:37:45 creiht: around for py3 discussion? 19:37:53 * portante continues to push him forward ... 19:38:04 #topic python3 ins wift 19:38:10 #topic python3 in swift 19:38:29 lots of talk here in the larger openstack community 19:39:03 we've held off on py3 support in swiftclient so far, but I think that would be the first place to support it 19:39:05 is a switch to asyncio (aka tulip the only game in town)? 19:39:24 * portante put the last paren in the wrong place 19:39:35 I switched a couple of simple apps to py3 in order to warm up. Seemed like painless enough -- unless you want to support 2.6 too. Then it's try:-expect: all over the place. 19:39:55 gevent is supposedly working on py3 compatibility, and it started life as a fork of eventlet, so that'd probably be our best bet for easy porting 19:39:56 portante: for python3? no there is gevents supporting it 19:40:12 AFAIK eventlet has made no moves toward py3 at all 19:40:27 * portante in favor of gevent w py3 19:40:35 "eventlet has made no moves" FTFY 19:40:38 Yeah, I had to drop eventlet for py3 and use the stock httpd. Swift can't do that though. 19:40:57 unless of, course we move to jython and just use os threads directly. :) 19:41:16 portante: recipe for success 19:41:22 =) 19:41:28 threads are not enough, we use the webserver parts of eventlet. 19:41:36 I remember creiht or redbo had reasons for not using gevent because of something that it didn't do. I don't remember the details (and it doesn't seem like they are here to help 19:42:02 I think swiftclient is the one we shoudl start focus 19:42:03 portante: zaitcev: notmyname: I can jump in on that upstream contribution as well, just gotta see what the state of their current issue on it is 19:42:09 chmouel: agreed 19:42:14 chmouel: agreed 19:42:16 cschwede_: do you know what's the status of removing the dependences from swiftclient on swift 19:42:22 notmyname: the advantage of gevent in my mind is that we can switch to it on py2 and work out the quirks without the py3 changes as well 19:42:46 portante: ya. because distros and install base we'll have to support py2 for quite a while 19:42:54 notmyname: gevent used to be libevent (which wasn't thread safe and we like threadpools) 19:43:06 dependencies on swiftclient: i think it's ready for review: https://review.openstack.org/#/c/65660/ 19:43:06 gevent is now libev which is threadsafe and should be all freaking gravy 19:43:07 it is now libev based, I belive 19:43:17 clayg: cool! 19:43:38 sounds good to me; so once gevent actually has py3 support finished, we port swiftclient and see how it goes? 19:43:39 swifterdarrell: what are your experiences with gevent and ssbench? 19:43:44 I'd love to see a tech session on this, in depth, in Atlanta 19:44:26 The zen of the gevents 19:44:26 torgomatic: and get cschwede_'s patch merged :-) 19:44:55 ok, timecheck, let's keep moving on. just a little more to discuss 19:44:57 I do love gravy 19:45:05 is it worth switching to gevent ahead of py3? 19:45:16 #topic moving bin/ to swift/cli/ for testing goodness 19:45:32 cschwede_: you're in progress here 19:45:32 * portante is in favor of bin to cli 19:45:35 the recently landed patch to test swift-recon moved bin/swift-recon to swift/cli/recon 19:45:36 I want to add more tests for scripts in bin/* and thus move more scripts to swift/cli/recon 19:45:37 two questions popped up doing this: 19:45:38 1. should i just submit a single patch for moving all the scripts from bin/ to swift/cli/recon (should be easy to review) or move each script separately when adding a test? the first one would clean up everything and things would not be scattered all around 19:45:39 2. when testing scripts, should I only test functionality or also output from print statements? testing print output requires redirecting sys.stdout and somehow this doesn't feel correct to me. 19:45:57 * portante thinks cschwede_ can type pretty darn fast 19:46:01 heh 19:46:04 =) 19:46:12 maybe he had that like pasted up and ready before hand? 19:46:16 cschwede_ is all the time ready 19:46:18 I'd rather see one thing at a time; if there's some 4000-line patch that moves everything, I'm gonna go find something else to review 19:46:33 there's not enough coffee in the world for that 19:46:37 +1 19:46:47 very large patch tend to sit around forever in the Q 19:47:06 cschwede_: there you go :-) 19:47:17 ok, so I'll will only move things when adding a test for it. 19:47:32 cschwede_: I don't think anything wrong with with getstdio(): do_stuff() assertEqual(mockio, expected) 19:48:05 cschwede_: so as you move them are you moving them from setuptools scripts to entry_scripts (or whatever it's called?) 19:48:10 sounds good; and if not everything moves (like bin/swift-object-server is maybe 2 LoC), then that doesn't bother me personally 19:48:43 does anyone have a link to the recent recon change that got merged? 19:48:45 clayg: yes, adding it to console_scripts 19:48:52 cschwede_: yeah that's the business! 19:48:56 cschwede_: thanks for tacking that task. those things have languished with no tests for too long 19:48:57 https://review.openstack.org/#/c/62584/ 19:49:14 https://github.com/openstack/swift/commit/cd4b4da8b6362e6621602a8b815bbeeea2000ff7 19:49:19 ok, quickly moving on. I think we have a direction here 19:49:28 thanks everyone! 19:49:34 #topic hacking 19:49:38 dirk: you 19:49:44 (this should be fast I think) 19:49:46 so the two sentence summary is.. 19:49:54 https://review.openstack.org/#/c/44757/ 19:49:59 you've had a patch for about 2 years now? ;- 19:50:03 ;-) 19:50:04 I sent this review about 4 months ago, then nothing happened 19:50:19 then I noticed that meanwhile apparently there was a decision to opt-in only 19:50:25 ya 19:50:31 regarding hacking warnings and somebody mentioned that I should bring up the topic in the meeting here 19:50:34 so here it is :-) 19:50:44 I think opt-out is overall the consensus in the wider range of openstack projects 19:51:08 turns out there isn't really a consensus, and there is strong support for the opt-in method 19:51:12 it might make sense for swift to go the same route as well. if there are new warnings that are bothering gating, they can be blacklisted until they can be resolved 19:51:18 based on in-person conversations I've had recently 19:51:32 * torgomatic likes the opt-in idea 19:51:38 I much prefer opt-in. I think it's much more sustainable 19:52:04 How do we know if new cool tests come about? 19:52:06 I prefer the opt-in too, some of the new rules are just weird (the dot in the commit message??) 19:52:13 but what if the add a "no bugs" hacking check and we forget to turn it on!? 19:52:15 those can be opted out 19:52:31 hacking is being release-managed pretty strictly 19:52:34 You know in Fedora we ruled that oneliners (such as RPM summary) MUST NOT end with a period. 19:52:36 no worries, we write bug-free code 19:52:42 new hacking checks are only introduced once per cycle 19:52:58 in a new minor release. patch level releases fixe bugs but do not add new checks 19:53:03 dirk: then we'll only have to check them once per cycle to see if we want new ones ;-) 19:53:05 just because someone else decides that they don't like backslashes for line continuations doesn't mean it needs to be "resolved" in Swift, IMO 19:53:18 torgomatic: +1 19:53:21 but I'm happy to have automated checking of things that I care about 19:53:37 wait, so some projects summary line DO end in a period? 19:53:38 notmyname: or check only once per cycle which ones should be blacklisted 19:53:39 hence my preference for use hacking, opt in to checks one at a time 19:53:49 ok, ruling from the bench: we stick with opt-in for now 19:53:59 and this is why we have a single leader :) 19:54:00 yay 19:54:09 instead of a Project Technical Committee 19:54:19 aw 19:54:22 #topic helpful gerrit queries (ie lets reduce the review queue) 19:54:31 torgomatic: you have some nice stuff here 19:54:31 #link https://gist.github.com/smerritt/8795447 19:54:49 basically, looking through the list of pending stuff sucks because it's huge 19:54:54 these queries let me narrow things down 19:55:07 I like the last one especially (patches with no core review yet) 19:55:12 and I can star reviews that I want to keep things moving on, and then find them easily later 19:55:35 that's about it; they're all pretty self-explanatory. Gerrit searches are pretty easy to read. 19:55:45 torgomatic: but those all say "torogomatic" 19:56:01 that is his way of getting others to work for him 19:56:05 clayg: yeah, our Gerrit is too old to support the magic "self" user that just references whoever uses it 19:56:15 we get his review queue down and then he can have a coffee 19:56:24 generalizing these helpful links.... 19:56:26 so s/torgomatic/$me/g and be happy :) 19:56:35 that's all I have on this 19:56:39 torgomatic: I think that breaks the last one 19:56:43 torgomatic: thanks 19:56:46 it turns out that patches seem to languish if they don't have a "core sponsor". I'm not saying that's good or bad, and we don't have time to go into depth now. but it's something that we should identify and either embrace or fix 19:57:19 I'd be happy to be a patch core sponsor 19:57:27 by the way i have a generated rss of reviews here http://rss.chmouel/swift.xml 19:57:33 but in the mean tim, using some of torgomatic's links will help with finding thing to review in order to reduce the review queue 19:57:37 (served from swift) 19:57:51 +1 for fixing. are there just not enough hours in the day for current core reviewers? 19:57:55 chmouel: I don't have a .chmouel TLD 19:57:57 so how many core devs do we have? 19:58:01 chmouel: http://rss.chmouel.com/swift.xml 19:58:06 yeah sorry :) 19:58:13 hehehe 19:58:15 but that's on my todo list (the TLD) 19:58:28 portante: 12 cores 19:58:30 I think there'd be enough time if every swift core did reviews full time :P 19:58:49 i haven't done any reviews *this week* 19:58:49 clayg: that is what I am afraid of, right now 19:58:58 I'd like to see a target of one review per day per core 19:59:12 can I just pick the one at the end of the list and -1 it 19:59:15 but not all reviews take the same time, so that's not simple to apply 19:59:16 core reviewers need hypethreading 19:59:16 i don't think all swift core are active 19:59:33 * torgomatic mostly just sits there 19:59:33 * clayg waves to redbo 19:59:39 notmyname: the priority review queue might be good to have a core sponsor assigned 19:59:41 http://russellbryant.net/openstack-stats/swift-reviewers-60.txt 19:59:47 portante: that's a good idea 20:00:10 portante: does stuff get on that list w/o some core at least like... idk sorta already keeping an eye on it? 20:00:12 unfortunately we're out of time 20:00:14 there is the swiftclient ones too as well 20:00:24 chmouel: ya 20:00:35 but this is something that we as a group need to be thinking about 20:00:45 thank you everyone for attending and participating today 20:00:48 #endmeeting