19:00:15 #startmeeting swift 19:00:16 Meeting started Wed Oct 22 19:00:15 2014 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:17 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:00:19 The meeting name has been set to 'swift' 19:00:20 hello world 19:00:25 who's here for the swift meeting? 19:00:27 rock n roll 19:00:28 hola 19:00:32 o/ 19:00:32 bonjour 19:00:40 . 19:00:46 o/ 19:00:50 o/ 19:00:57 o/ 19:01:11 welcome :-) 19:01:22 here 19:01:30 I want to spend most of the time talking about the summit session topics 19:01:34 #link https://wiki.openstack.org/wiki/Meetings/Swift 19:01:39 acoles: here? 19:02:24 here 19:02:28 great 19:02:31 well, finally :) 19:02:38 I want to start with your topic :-) 19:02:50 #topic Proposed change to move devstack functional tests to use keystone v3 19:03:27 #link https://review.openstack.org/#/c/128888/ 19:03:38 #link https://review.openstack.org/#/c/130162/ 19:03:41 acoles: tell us more 19:04:11 ok, so the context is that some functional tests are skipped on jenkins because they need account in a non-default domain 19:04:20 and i would like them to not be skipped! 19:04:37 seems reasonable :-) 19:04:52 so the devstack changes (128888) put support in place for those tests 19:05:21 i.e. setup another account, and configure the tests to use v3 API 19:06:07 then the change in 130162 will flip the switch (devstack takes the value for authAPI version from test.conf) 19:06:29 acoles, so I assume the devstack side needs to land before we commit the swift patch and also, separate question since I haven't looked at the swift side yet, does that affect our local functional tests or saio env? 19:06:59 or is it that the swift one lands before devstack? 19:07:03 which one goes first? 19:07:17 peluse: yes devstack patch needs to land first and 130162 is failing anyway until 128888 lands 19:07:58 OK, yeah, looks like you have it workflow -1 awaiting just that thing to happen... what about saio env (local functional tests)? 19:08:06 i did it that way so that swift cores get to vote the test behaviour switch through rather than devstack 19:08:21 acoles: what do we need to do on the swift side to get the devstack patch landed? 19:09:34 peluse: re saio, the change in test.conf is to a comment so will only affect if you have local CI that uncomments (as devstack does) or are used to manually uncommenting and need to stick with v2 API 19:10:09 notmyname: so if you are happy it would be great to get some +1 form swift folks on the devstack patch - it has a +2 already 19:10:23 ok 19:10:26 I'd say swift guys go give a +1 to the devstack change if you agree with the patch, I think that would help the infra/qa guys know that we're in ageement. 19:10:47 sounds good 19:10:56 so devstack patch that is https://review.openstack.org/#/c/128888/ 19:11:07 acoles: anything else you need to bring up for those 2 patches? 19:11:11 acoles: thanks for working on it! 19:11:25 mr v3! 19:11:26 if/when that lands i will ping swift folks about 130162 19:11:40 peluse: roll on v4, i'm ready ;) 19:11:50 heh 19:11:51 lol 19:11:58 watch out, they might! ;-) 19:12:32 yeah careful what you wish for :\ 19:12:43 ok, let's move on to the paris scheduling 19:12:49 #topic paris summit scheduling 19:12:56 * clayg dusts off his devstack vm 19:13:00 who here is going to be in paris? 19:13:08 o/ 19:13:09 o/ 19:13:09 o/ 19:13:11 i am 19:13:13 me 19:13:14 oui 19:13:19 nice! 19:13:19 o/ 19:13:27 great! 19:13:28 kota_: !!!! 19:13:31 * cutforth wishes he was going to Paris, but is saddened he is not 19:13:42 I'll be there 19:13:42 clayg: :) 19:13:47 tdasilva is going also 19:13:49 o/ 19:14:01 kota you must be the king of frequent flyer miles :) 19:14:05 lol 19:14:13 so here's what I know about the design summit for swift 19:14:21 peluse: lol 19:14:23 1) we've got 6 slots to schedule 19:14:35 2) we've got a half-day of unscheduled time on Friday 19:14:36 lol (?) at devstack "World dumping..." 19:14:46 3) there are cross-project topics that we should look at 19:15:09 4) near-final schedule for swift sessions should be published by next tuesday 19:15:50 wrt cross project - meaning we should make sure that a pile of us attend those sessions whenever they are scheduled or that we should dicusss them in our free block? 19:15:52 so here's what I want to do now. I want to go over what has been proposed and prioritize them. then I will take that and schedule them later this week or this weekend and push it up for the formal schedule 19:16:29 for cross-project, meaning that we should look at what's scheduled and attend the ones that pertain to swift/storage 19:17:05 peluse: yeah like if glance wants to talk about the swift storage backend - we should consider going, if cinder wants to talk about volume archives to object storage we should go 19:17:12 peluse: meaning, don't just focus on the narrow swift things, but take advantage of the other in-person conversations while we're together ;-) 19:17:20 clayg: yup. exactly 19:17:27 peluse: if olso wants to talk about how they're nazi's - well.... notmyname should go 19:17:41 agreed, one of my co-workers (Reddy Chagham) has some cross project topics that, if scheduled, should draw some of us from the Swift side for sure 19:17:49 clayg: I think Ihave other commitments during that time block ;-) 19:17:52 well, I typed that "agreed" before clayg's comment 19:18:03 hehehehehhehe - i'm tricky like that 19:18:11 #link https://etherpad.openstack.org/p/kilo-swift-summit-topics 19:18:25 ^ there's the etherpad of our proposed sessions 19:18:36 so let's go through them one at a time, top to bottom 19:18:51 first, erasure code overview 19:19:06 I proposed this one, but I suspect peluse might have one or two things to say ;-) 19:19:20 so we can spend hours on that... we should narrow down to something more specific that people can get into 19:19:22 IMO this needs to happen since it's a big part of current work 19:19:29 peluse: agreed 19:19:36 like make the PUT path/MIME support or the reconstructor in detail (one of those) 19:19:48 and then maybe an overall ststus thingy for the open session in causal conversation 19:19:53 with an eye to an overview of the design? 19:20:02 * clayg cries at MIME (because he doesn't have any better ideas) 19:20:09 BTW... latest design spec is really fun to read for anyone that wants to see it; http://docs-draft.openstack.org/90/125190/8/gate/gate-swift-specs-docs/2d2cf67/doc/build/html/specs/swift/erasure_coding.html 19:20:11 yes 19:20:11 I wonder if there's any chance of having rudamentary PUT/GET patch support by then 19:20:24 there should be I think, tsg? 19:20:41 well, tsg and torgomatic 19:20:52 peluse: it's getting big - i'm having a hard time keeping up with drift w/o having to just re-read the thing from top to bottom 19:20:53 * peluse is making MIME signals to clayg right now :) 19:20:55 notmyname: yes, a first version of PUT+MIME patch should up 19:21:02 torgomatic is typing on PUT stuff right now I think 19:21:16 yeah, I was curious how it'd go, and I think I might have some stuff soonish 19:21:29 fantastic! 19:21:59 peluse: can you add some notes to the etherpad of what specifically you want to cover during the formal session? 19:22:32 back to clayg's comment wrt the design spec - yeah it is getting big so maybe we take the EC sessions and do 50% overview and 50% drill down on one topic (like PUT/MIME or something) 19:22:35 yes, I'll do that 19:22:41 peluse: thanks 19:22:48 ok, next topic proposed 19:22:57 queue integration and/or callbacks 19:23:17 this is something that's come up from time to time. I think it's pretty interesting, but IMO it's not a top priority for paris 19:23:30 mostly because there isn't actually anyone working on this in the near term that I know of 19:23:56 so I'd only schedule this one if there aren't less important ones. but I'd liek to have it on the whiteboard for the informal sessions 19:24:00 notmyname: maybe the zerovm guys? who knows? 19:24:07 yeah, maybe 19:24:22 you mean Swift emitting events or consuming them? 19:24:24 Why dont we push it to the 1/2 conversations, the if we have time :) 19:24:28 one of these things is reasonable, one is not :) 19:24:43 that's the thing. on the other hand paris might be the good place to talk with zerovm/zaqar/etc 19:25:15 torgomatic: swift putting stuff on a queue or otherwise emitting events when something happens 19:25:24 notmyname: okay, the good one then. got it. 19:25:27 :-) 19:25:36 +1 on this :9 19:25:45 ok :-) 19:26:11 I'll come back to it depending on how the other proposed go :-) 19:26:17 next up 19:26:19 service tokens 19:26:27 this is something we talked about in westford 19:26:54 it has the advantage of having integration with cinder/glance, so paris is a good time to talk about it 19:26:59 donagh and i have been doing more work on that since westford and will have a revised spec real soon 19:27:05 ok 19:27:10 acoles: both of you will be in paris? 19:27:19 just me unfortunately 19:27:28 ok 19:28:26 service tokens sounds like something I will probably not understand very well :\ 19:28:30 so higer priority (because of other projects) than queuing 19:28:42 clayg: 2 auth tokens at the same time 19:28:54 when 8k isn't enough - you've always got 16k 19:29:00 do we need to use service token to use the restrooms in Paris? 19:29:01 :-) 19:29:09 why have one when you can have two for only twice as much! 19:29:14 I guess if you've got a million dollars, that's a thing you can do 19:29:27 dfg!!!! 19:29:34 hey clayg 19:29:47 ok, next proposed session 19:30:01 performance improvements in all-in-one (PACO) deployments 19:30:14 this was brought up and discussed in westford 19:30:22 tdasilva: anything to add here 19:30:24 ? 19:30:39 I talked to Ignacio yesterday, he is already in Paris 19:30:45 both of us have been doing some work on it 19:30:48 ok, wow 19:30:52 and would like to discuss with the group about it 19:31:02 I think it's a really interesting idea, but could be done in the ad-hoc sessions. 19:31:07 tdasilva: argue with me :-) 19:31:21 * tdasilva is really bad at that 19:31:24 hehe 19:31:25 lol 19:31:40 not sure what the format of the ad-hoc sessions are 19:31:49 ok, what does everyone else think? better or worse than queue integration for a formal session? 19:31:57 but would definetely like to have some time to talk about it 19:31:57 better 19:32:00 tdasilva: the ad-hoc sessions will be very much like westford 19:32:03 better 19:32:15 doen :-) 19:32:17 it's better 19:32:37 next up 19:32:41 app developer tools 19:32:49 I proposed this one. here's my thinking 19:33:04 we've got a pretty cool storage engine, but we need to make sure people use it 19:33:27 that means working on producing stuff as the swift dev that make it easier for the ecosystem to grow. 19:33:52 notmyname: is this limited to a python client or other languages as well? 19:33:59 this session would be to disucss what those things are and find people to do it 19:34:25 cschwede: yes clients, but also docs, setup scripts, other guides 19:34:45 mostly I see it as a gap in the community right now, and I don't know who's going to fill it 19:34:58 notmyname: ok, sounds like „everything“ on the user/dev side. great 19:34:58 thoughts? 19:35:02 notmyname: does this involve ops tools such as deployment and monitoring? 19:35:07 maybe we can find a company that focuses its core business on Swift :) 19:35:12 tdasilva: no. app-dev 19:35:15 peluse: hmm ;-) 19:35:47 * cschwede thinks this topic is important… 19:36:12 cschwede: important as in we should take it upon ourselves? or important as in "yeah, somebody should do something about it" 19:36:17 +1 the idea, our documentation is good but samples etc should help adopt more usage. 19:36:25 agree... did the data migration middleware from Gil show up once as a topic on etherpad? Seems like a good example of something that falls in this category 19:36:57 we should, being the people that undertand the middle wares.. and hopefully foster app devs to get involved 19:37:01 do we have a Go client 19:37:07 notmyname: we should work on this (imo) - if „somebody“ needs to work on this we start again in 5-6 months 19:37:08 peluse: I added it there 19:37:18 ahh, great 19:37:38 peluse: and we could choose to combine sessions too, if that's best 19:37:42 was right in front of me, must be going blind... 19:37:54 peluse: go get a bigger monitor ;-) 19:38:17 bigger is indeed better... 19:38:25 so on a scale of 1-10 (10 being highest), where would rank this session topic? 19:38:46 (FWIW, we have 10 proposed topics for 6 slots) 19:39:06 ie is this one 1 of the 4 that gets moved to the ad-hoc discussions 19:39:29 that probably makes the most sense becaues it will be a lot of general ad-hoc discussion 19:39:39 5-6, it should be discussed.. but might be ok in the ad-hoc.. so long as it gets covered 19:39:43 i'd vote this is less than the previous one :P 19:39:55 ok 19:39:58 It seems like a good topic for ad-hoc 19:39:58 yeah, I'd say 6 19:39:58 notmyname: i’m fine with a ad-hoc for this, but then we should make sure it get’s covered by a few cores 19:40:10 ok 19:40:26 sounds like goal will be to generate a list of ideas, ad-hoc session sounds good 19:40:32 notmyname: it's pretty high for me - but I'd really like it to leverage some community participation - we have so much input from people who develop deploy and operate swift - be great to have some input from consumers beyond "I store the lolcatz" and "I upload the backups" 19:40:43 yeah 19:40:53 ok, next one 19:40:59 status of python-swiftclient 19:41:09 this is related, but separate, from the last one 19:41:10 getting less terrible with every commit! 19:41:19 clayg :) 19:41:35 I want to go over the current status (what's good, what's bad, etc) and make a set of goals for swiftclient 19:41:45 lol, this might need a session, so there is a defined end point to its dicsussions :P 19:41:51 $ git mv swiftly python-swiftclient 19:42:05 swiftclient is often the first impression people have of swift, and frankly it's not great IMO 19:42:35 I'd rank this one pretty high. what do you think? 19:42:46 yeah, 8 or 9 19:42:50 agreed 19:42:54 needs to be talked about 19:43:24 acoles: peluse: clayg: ? 19:43:43 notmyname: not great as in bugs or not great as in interface? 19:44:17 acoles: interface mostly. also not great from a maintenance point of view, although that's not often seen by users 19:44:24 yeah i get bummed out sometimes when I find broke stuff in swiftclient; but i try to just file bugs or fix things 19:44:32 acoles: where interface includes performance 19:44:34 I don't really understand what we can do except make it better 19:44:48 I don't have a huge amount of experience with it but always willing to listen/comment... 19:44:52 honestly my biggest rip is still pbr 19:45:06 * peluse thought torgotmatic solve it with his earlier git command already 19:45:08 like everytime someone says "I tried to install swiftlcient and pbr" my blood boils 19:45:32 ok, I'll put it down as an 8 19:45:36 next one 19:45:38 swift-on-file 19:45:46 this is tdasilva's stackforge project 19:45:54 this one we were just looking for some open time in case it is available 19:45:59 makes swift work on top of external durable filesystems 19:46:30 I'd love to hear about it 19:46:31 yeah, I'd rank it lower since it's not the codebase we officially manage. but supporting it is good 19:46:59 it will be a good opportunity to meet with other folks to are interested on the project, so whatever time we can get we appreciate 19:47:08 thoughts? above or below queues in swift? ;-) 19:47:16 above 19:47:44 above - there's like a whole thing that works - imporoving what we've got is ++ on new and shiny most of the time in my book 19:47:51 cool 19:47:59 above 19:48:03 * notmyname lowers priority for queues 19:48:08 let's do it :-) 19:48:18 when we get swift on file on fuse on rados then we win right? 19:48:38 clayg: something like that. but only when rados is running on netapps backed by tape, right? 19:48:45 next up 19:48:50 or s3 - we should throw aws a bone too 19:48:52 encryption in swift 19:48:58 clayg: especially then if you make rados store the file in the original Swift cluster, because then you get infinite free storage 19:49:03 on dis-encryption 19:49:10 on-disk encryption 19:49:17 Swift -> FS -> FUSE -> RADOS -> start again 19:49:20 DOCKER! 19:49:25 ok ok ok 19:49:26 lol 19:49:26 that was something I was interested in hearing about if anyone who is part of the effrt that Sam started to doc is around/wanting to discuss 19:49:30 oh... i meant ENCRYPTION! 19:49:33 let's do it! 19:49:34 needs a session, looking at the spec coments, this needs discussion 19:49:51 yeah, it's a touchy topic that a lot of people get really excited about 19:49:56 I think it needs a session 19:50:01 seems like you mention crypto and all the scope creepers come out of the woodwork 19:50:15 ok, 3 more (quickly) 19:50:17 oooh oooh, and then we can.... 19:50:18 there has got to be minecraft meme there 19:50:21 torgomatic: seagulls 19:50:21 you're just sitting there and you hear sssssssssssssSSSSSSSSSSSSSSo, we also need X! 19:50:24 and then your scope explodes 19:50:27 :-) 19:50:43 ok, migration middleware from gvernik 19:50:59 * notmyname is sad it's not landed yet 19:51:08 seems like we need a forcing function to get that landed 19:51:13 * notmyname has also not put a +2 on it and is part of the problem ;-) 19:51:19 peluse: no kidding 19:51:20 like lock some of us in a room in Paris until its done 19:51:36 http://cdn.meme.am/instances/500x/55541888.jpg 19:51:47 I only looked at Gil's thing once months ago. 19:51:47 could that be done during the ad-hoc time? or does it require a session? 19:52:19 zaitcev: it's nice, you should look again :-) 19:52:28 i do think its a must but ad-hoc is probably OK and there should a goal to get 2 cores signed up to attend and get all their questions answered so we can merge it 19:52:46 and I'll volunteer to be one of them... 19:53:07 so, are we all planning to be at the ad-hoc? can we structure it at all? 19:53:07 lets give it a mid range score, it might get a tail end session :) 19:53:23 acoles: yeah, I think we should all be there 19:53:27 peluse: what's the hold up - if it's awesome lets merge it - or if it's just middleware get it out on the githubs and referenced in the associated projects till we ready to eat it - or it's baked eough for core? 19:53:29 mattoliverau: yup 19:53:50 two more to review in 7 minutes 19:53:53 clayg, only hold up for me is a piss poor excuse of not having taken the time to go through it in detail 19:53:54 next 19:53:59 durability simulator 19:54:04 This is something we disscussed in Hackthon. 19:54:14 if we could only *simulate* durability then we wouldn't acctually need it! 19:54:18 yeah. kota_ and cschwede are kinda leading this one 19:54:19 " Discuss how swift durability should be calculated with showing some our sample tool design and how to make such tools for system architects to calculate their own durability when building Object Strage System" 19:54:27 I vote for ad-hoc 19:54:46 sounds good 19:54:47 fine with me 19:54:54 ok 19:54:57 peluse: clayg: i would be +2 on migration but got kinda involved so i'm hoping other cores will bless it 19:55:00 kota_: are you ok with that? 19:55:29 ok but I'd like to get some feedback also ops 19:55:47 last up 19:55:49 ops session 19:56:00 in the past, we've had a session for ops feedback 19:56:05 either we schedule this or we don't 19:56:11 no ad-hoc for this one 19:56:22 there is an "ops summit" or track at the summit 19:56:41 so we could punt to there, but that will be more general for all openstack (read: nova/neutron) 19:56:49 probably a good one to advertise, if no one turns up we could talk about app dev support for those ops :) 19:56:51 was it beneficial at the atlanta summit? 19:56:51 so do we want a session for just swift feedback? 19:57:24 * tdasilva wasn't there, but thinks it is always a good idea to hear how people are using software in the real-world 19:57:24 notmyname: seeing as how this is the first summit in euro - could turn up some interesting deployers we've not previously had a chance to talk to face to face? 19:57:35 I remember atlanta was not so good ops-session, hk was quite good 19:57:41 tdasilva: clayg ++ to both 19:58:25 cool, let's do it 19:58:36 lets shedule one then.. if its quiet we can always talk about a ad-hoc topic 19:58:38 awesome, we got through them all 19:58:39 thanks! 19:58:52 just in time :) 19:58:58 I'll work on scheduling these and then push the schedule 19:59:05 before tuesday :-) 19:59:13 ... and my devstack is running! 19:59:13 Cool, thanks notmyname 19:59:14 notmyname: thanks! 19:59:16 thanks everyone for coming and participating! 19:59:20 #endmeeting