19:00:13 <notmyname> #startmeeting swift
19:00:14 <openstack> Meeting started Wed Dec 10 19:00:13 2014 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:15 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:00:17 <openstack> The meeting name has been set to 'swift'
19:00:26 <notmyname> who's here for the swift meeting?
19:00:35 <tdasilva> hello
19:00:36 <lpabon> aye
19:00:37 <gvernik> hello
19:00:43 <kota_> hi
19:00:45 <mattoliverau> o/
19:00:46 <peluse> yo
19:00:48 <acoles> here
19:01:04 <notmyname> welcome!
19:01:07 <cutforth> present
19:01:29 <notmyname> I gave a swift overview at a meetup last night and got to brag on all of you. easily my favorite part of telling people about swift
19:01:42 <notmyname> agenda for today's meeting
19:01:42 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift
19:02:09 <notmyname> ok, let's switch things up
19:02:20 <notmyname> I think gvernik's topic is pretty short (but a great question
19:02:27 <notmyname> #topic SWLite WAL
19:02:32 <notmyname> "WAL support for SQLite. This blueprint https://blueprints.launchpad.net/swift/+spec/cactus-sqlite-wal was closed by John, in 2011. Would like to hear what community thinks about it. What were the issues in Swift with WAL? Are they still exists? Recent SQLite versions are much better than of 2011 so I wonder if recent SQLlite version with WAL mode can provide better SQLlite response times for Swift."
19:02:47 <notmyname> gvernik: any specific question beyond that?
19:02:56 <gvernik> yes..here is the story:
19:03:16 <gvernik> I did series of benchmarks. One was to upload 500K objects of 15KB size to single, empty container.I saw that container server could process X OPS. If I put middleware on the container server that does nothing, (response with 200 OK), than container server can 'process'  6*X requests... Trying to understand what is wrong.
19:03:27 <gvernik> so i saw that WAL can greatly improve SQLite
19:03:39 <gvernik> and want to know your thoughts about
19:03:57 <peluse> I'm not super familiar with it, what the size/scope of the actual changes?
19:04:27 <notmyname> gvernik: we looked into it a long time ago, but I don't know if anyone has tried it recently
19:04:48 <gvernik> today WAL has great support in SQLite, much better than in 2011
19:05:09 * peluse wonders if his microphone is turned on
19:05:15 <gvernik> and it suppose to improve response times of SQLite
19:05:17 <notmyname> gvernik: but my opinion is that I'd love for someone to look in to it. peluse's question is pretty important when it get's down to the code
19:05:34 <notmyname> but as far as basic investigation goes, have at it
19:05:39 <gvernik> notmyname,:  what were the issues in 2011?
19:05:51 <notmyname> gvernik: have you had any chance to look into an implementation?
19:06:14 <notmyname> gvernik: I don't remember exactly. I don't know if it was replicating it or performance or what? I don't remember
19:06:41 <notmyname> gvernik: if you see him online, you might ask dfg. IIRC he's the one who looked at it back then
19:06:49 <notmyname> (doesn't look like he's on now)
19:07:39 <gvernik> notmyname: ok...
19:07:45 <peluse> gvernik, seems like if its easy to enable then a pre-req for real consideration would be to put it through its paces and see what if anything breaks perf wise or feature wise
19:08:08 <notmyname> peluse: ++
19:08:16 <mattoliverau> For those playing along at home, this is WAL (I think): https://www.sqlite.org/wal.html
19:08:28 <notmyname> mattoliverau: yup. thanks
19:08:50 <gvernik> peluse: I know that WAL will improve much response times of container servers, it looks like it will be able to handle more ops
19:09:21 <notmyname> gvernik: maybe a good first step is to write a toy app that compares WAL and not. then try it out in the container server. then figure out any durability/migration issues that have to be sovled
19:10:41 <gvernik> notmyname,: ok, sort of did it already... but than container data become corrupted at some point.. but performance was great :)
19:10:45 <notmyname> gvernik: are you able to look into it and work on Swift + WAL?
19:10:51 <notmyname> lol
19:11:03 <peluse> sweet :)
19:11:17 <acoles> gvernik: there's always a trade-off ;)
19:11:31 <gvernik> notmyname,: I will look at it, gladly. wanted to know if you have some strong objections, but seems you are open mind to check this feature
19:11:32 <kota_> lol
19:12:38 <notmyname> gvernik: AFAIK there's no "there be dragons here" danger. I'd love to have a better answer. either it won't work because X or it does work so let's use it
19:12:47 <notmyname> gvernik: thanks for bringing it up
19:13:19 <notmyname> gvernik: any other questions you have there?
19:13:44 <gvernik> notmyname,: no
19:13:52 <notmyname> ok
19:13:56 <notmyname> #topic EC status
19:14:02 <notmyname> and this will get into priority reviews
19:14:17 <peluse> so reconstructor is one thing... pretty good progress and should be ready for review within a week or so
19:14:20 <notmyname> peluse: what's up with EC right now? tons of (big) patches are there
19:14:32 <peluse> tsg and/or torgomatic will ahve to comment on PUT path (still needs .durable stuff)
19:14:33 <kota_> peluse: great!
19:14:54 <peluse> and clayg has started on GET but I'm not sure how much is in his head versus in the patch right now :)
19:15:02 <notmyname> and clayg has a GET path patch (admittedly "for the lolz")
19:15:05 <tsg> peluse, notmyname: I was finishing up some eventlet stuff so that can get released (a dependency we have for the .durable stuff)
19:15:08 <peluse> priority reviews should be up to date and there are a few that need review
19:15:10 <notmyname> good
19:15:35 <peluse> also trello should be up to date now as well.  there are some open questions up there for a few folks working on EC that I don't think are answered yet though
19:15:55 <peluse> tsg, ahh cool thanks.  ETA?
19:16:07 <tsg> eventlet - some time next
19:16:10 <tsg> notmyname: pyeclib stuff also has gone through some changes (that I pushed an hour ago) - will get back to the PUT stuff today
19:16:21 <tsg> peluse: ^^^
19:16:22 <notmyname> tsg: ok
19:16:30 <tsg> peluse: I need to also update the latest on Trello
19:16:30 <notmyname> tsg: some time next...?
19:16:38 <peluse> note:  when I say reconstructo patch it will have everything except the actual reconstruction.  Sounds funny but that's the easy part :)  Need PUT/GET to really be done before it has the final  touches
19:16:40 <tsg> sorry :) next week
19:16:48 <notmyname> :-)
19:16:51 <peluse> heh, could have been next year :)
19:16:55 <tsg> heh
19:17:27 <tsg> peluse: I am down to 2 eventlet issues - so next week is the best bet
19:17:29 <notmyname> peluse: ya, I'm getting the impression that there's a ton of foundational work that's been done and currently proposed (nearly done). then the visible stuff is relatively small to tie it all together
19:17:49 <notmyname> peluse: not saying we'll be done next week, but basically that there's light at the end of the tunnel
19:17:55 <notmyname> peluse: would you agree?
19:18:22 <peluse> notmyname, yes with the exception of probetests.... still kinda dark there at least for me
19:18:27 <notmyname> ok
19:18:56 <peluse> we can talk more about that offline also, there's a trello card there bringing up the issue(s)
19:19:09 <tsg> notmyname, peluse: can we get #link https://review.openstack.org/#/c/132389 reviewed?  simple patch - but this will be one thing out of the way
19:19:22 <torgomatic> hi
19:19:23 <clayg> whoops
19:19:27 <notmyname> tsg: ya. I'll put it on the priority reviews
19:19:35 <tsg> notmyname: thank you
19:20:06 <mattoliverau> clayg: welcome to the meeting :P
19:20:15 <notmyname> anything else peluse?
19:20:31 <notmyname> peluse: the biggest thign right now for those of us not coding a feature is to review, right?
19:20:37 <kota_> Sounds like the ec release target could be kilo-2 or kilo-3?
19:20:42 <peluse> right, reviews please
19:20:49 <peluse> is there a kilo-4?  :)
19:20:50 <notmyname> kota_: let's not get ahead of ourselves yet ;-)
19:21:01 <notmyname> peluse: there can be kilo-whateveryouwant
19:21:07 <kota_> I see
19:21:43 <notmyname> so...speaking of releases....
19:21:45 <clayg> mattoliverau: i have different idea of "early" than you do
19:21:51 <notmyname> #topic next release of swift
19:21:54 <notmyname> swift 2.2.1
19:22:01 <peluse> yeah, I think its too early to call... we need PUT/GET/recon done and line of sight on tests before we should make release statements
19:22:04 <mattoliverau> clayg: lol
19:22:22 <notmyname> final hurrah for the year: getting some of the landed bug fixes released
19:23:09 <notmyname> there's a few things listed under "stuff for 2.2.1" on https://wiki.openstack.org/wiki/Swift/PriorityReviews
19:23:17 <kota_> peluse: yup, thanks
19:23:18 <notmyname> none of these I feel are critical
19:23:27 <notmyname> but very nice to have in a bugfix release
19:23:39 <cschwede> hi
19:23:53 * cschwede is late too… timezones
19:23:58 <clayg> cschwede's thing with the ring warning looks interesting
19:23:58 <mattoliverau> cschwede: o/
19:24:00 <cschwede> notmyname: should i add https://review.openstack.org/#/c/140478/ too?
19:24:01 <notmyname> however, just yesterday there were some things discussed by cschwede and swifterdarrell that would be nice
19:24:11 <notmyname> clayg: cschwede: yes. you are typing faster than me
19:24:15 <notmyname> #link https://review.openstack.org/#/c/140478/
19:24:25 <notmyname> that one is important
19:25:07 <notmyname> there's nothing particularly broken about swift, but the current placement methodology can cause very imbalanced clusters to do something unexpected, and we need warnings for that
19:25:55 <cschwede> i added it to the important reviews
19:26:28 <notmyname> cschwede: thanks
19:26:38 <notmyname> tsg: and I added your EC one
19:26:53 <tsg> notmyname: great .. thx!
19:26:55 <notmyname> ok, so for 2.2.1, anything else?
19:27:08 <torgomatic> this one, maybe? https://review.openstack.org/#/c/140178/
19:27:09 <notmyname> I think it should mostly be a non-event (and that's good)
19:27:43 <clayg> torgomatic: self promoting much?
19:27:49 <notmyname> :-)
19:27:54 <notmyname> it's a good patch
19:28:00 * torgomatic has not yet begun to self-promote :)
19:28:01 <kota_> lol
19:28:20 <clayg> idk, there's lots of good patches up that need reviews, everytime I don't do a reviews for two days I feel like the entire thing gets away from me :'(
19:28:25 <notmyname> ya
19:28:51 <clayg> like, this bug really scares me -> https://bugs.launchpad.net/swift/+bug/1328735
19:28:53 <uvirtbot> Launchpad bug 1328735 in swift "Object-updater gives up updating container with no success if all containers are placed at handoff" [Undecided,In progress]
19:30:23 <notmyname> clayg: I'll look at it and see what we can do for progress there
19:30:27 <notmyname> https://review.openstack.org/#/c/135319/ has one +2 already (rework splice and tee)
19:30:30 <notmyname> needs one more
19:30:53 <notmyname> https://review.openstack.org/#/c/126923/ fsync on dirs has some more work to do, so I'm going to drop it from 2.2.1
19:31:15 <clayg> notmyname: a) I don't really know how to test that b) isn't that a newish feature anyway c) is anyone using it d) is it a bug or a footgun?
19:31:18 <notmyname> anything have any details on https://review.openstack.org/#/c/136258/ (fix GET if there is a missing segment in LOs)
19:31:24 <clayg> notmyname: re: splice and tee
19:31:44 <torgomatic> I like the missing-segment patch, but it now contains a significant portion of code written by me, so I've not +2ed it
19:31:55 <notmyname> torgomatic: ack
19:32:00 <clayg> bah, i disagree with everything today, ya'll review whatever :P
19:32:08 <torgomatic> we can argue over whether 409 is the right status code to return, but anything beats a 500 IMO
19:32:29 <notmyname> clayg: kota_: what's the current status of https://review.openstack.org/#/c/99824/ (global replication). IIRC clayg had some concerns
19:32:50 <notmyname> torgomatic: especially when you're incentivized (by SLA) to not return 5xx!
19:32:52 <clayg> notmyname: it's all rebased now - needs a good review - probably makes ssync better so it's high on my list!
19:33:01 <torgomatic> notmyname: srsly :)
19:33:01 <notmyname> clayg: ack. cool
19:33:33 <notmyname> ok, so for me, top of the list is cschwede's patch for reporting warnings on ring balance
19:33:37 <kota_> notmyname: I rebased according to clayg suggestion
19:33:50 <notmyname> kota_: thanks
19:33:56 <kota_> notmyname: I am waiting for another review, write now
19:34:10 <kota_> s/write/right/
19:34:16 <notmyname> then global cluster and splice each need one more +2. then GET on LOs.
19:34:25 <notmyname> I'll reorder the list to reflect that
19:34:33 <notmyname> any arguments with that order?
19:35:30 <notmyname> ok, I'll update after the meeting
19:35:47 <notmyname> #topic other reviews / other things
19:35:54 <notmyname> what else is on your mind?
19:36:06 <tdasilva> notmyname: should the swift-ring-builder warning go on top of that list?
19:36:19 <notmyname> tdasilva: ya, it will
19:37:16 <notmyname> I'm working on some possibilities for the next swift hackathon. I'll let you know as soon as I know something
19:37:27 <notmyname> I want to give you enough time to look at travel, etc.
19:37:49 <mattoliverau> +1
19:37:50 <tdasilva> notmyname: do you have a time in mind for when the hackathon would happen? even ballpark...
19:38:15 <notmyname> but the general thing is: yes, we'll do one. before the next summit (probably feb/march). probably (but not at all definite) in the eastern half of the US
19:38:43 <peluse> brrrr
19:38:59 <notmyname> peluse: maybe not ;-)
19:39:11 <cutforth> peluse: you act like your from AZ
19:39:22 <peluse> reminder:  weather in AZ in Jan/Feb can't be beat!
19:39:44 <mattoliverau> peluse: it can by being here :)
19:40:00 <mattoliverau> Thats the middle of summer
19:40:03 <tdasilva> mattoliverau: +1
19:40:03 <notmyname> ok, next meetings
19:40:16 <notmyname> yes to meeting next week. no to meeting on the 24th
19:40:33 <notmyname> also I propose no to meeting on the 31st
19:40:46 <cutforth> notmyname: +1
19:40:49 <tdasilva> sounds good
19:40:50 <clayg> sweet swift holiday break!
19:41:03 <notmyname> clayg: you still have to code/review. just no meeting ;-)
19:41:13 <cschwede> +1 :)
19:41:16 <clayg> shoot
19:41:22 <notmyname> merry christmas! I got you a code review!
19:41:30 <mattoliverau> lol
19:41:34 <clayg> lol
19:41:38 <mattoliverau> sounds like a meme
19:42:09 <notmyname> thanks everyone from coming. thanks for working on swift!
19:42:15 <notmyname> #endmeeting