21:00:31 #startmeeting swift 21:00:32 Meeting started Wed Apr 6 21:00:31 2016 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:33 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:35 The meeting name has been set to 'swift' 21:00:40 who's here for the swift meeting? 21:00:49 o/ 21:00:52 hey 21:00:53 hola 21:00:54 o/ 21:00:54 o/ 21:00:54 hello 21:00:55 hi 21:01:19 hi 21:01:47 o/ 21:01:54 here 21:01:59 wc 21:02:00 o/ 21:02:12 hi 21:02:13 welcome everyone 21:02:19 agenda is at 21:02:22 #link https://wiki.openstack.org/wiki/Meetings/Swift 21:02:35 * notmyname wishes the # commands would work mid-line 21:03:07 several interesting things have been proposed on the agenda this week 21:03:14 #topic rolling upgrade tests 21:03:24 first up, checking in on the rolling upgrade testing 21:03:31 #link https://etherpad.openstack.org/p/swift-rolling-upgrade-multinode-testing 21:03:33 cschwede: are you here? 21:03:48 the todo section at the bottom is what's needed, I think 21:04:00 acoles: timburke: do you have any other details or info? 21:04:34 notmyname: sorry no, i have been engrossed in feature/crypto stuff this week 21:04:41 that's ok :-) 21:04:48 over on swift3 stuff, myself 21:05:27 I haven't seen cschwede around much this week, bus I suspect that's because of the summer time zone. seems like I'm on relatively later in the mornings now 21:05:43 cd 21:05:43 I'll keep bugging people about this and helping where I can 21:06:10 although we got a reprieve on the tag thing, we still need the testing to be implemented relatively quickly 21:06:12 I'll try to check in with cschwede tomorrow Europe time 21:06:21 acoles: thanks 21:06:32 next up, a quick topic 21:06:33 notmyname: yeah i thought there was a timelimit component - if we're under the gun I might be able to steal some cycles if needed 21:07:01 clayg: we were spared this time, with a "we'll check back in a few weeks" vague comemnt 21:07:08 that's all I know 21:07:13 uh huh 21:07:22 #topic reverse listings bug follow-up 21:07:37 in the process of setting up some of this testing, we discovered https://bugs.launchpad.net/swift/+bug/1562083 21:07:39 Launchpad bug 1562083 in OpenStack Object Storage (swift) "mid-upgrade clusters can cause versioned write errors" [Medium,Fix released] - Assigned to Tim Burke (1-tim-z) 21:07:46 and timburke fixed it. yay! it's on master now 21:08:14 IMO it's a good candidate for a backport 21:08:25 agree 21:08:42 #topic idea: swift.callback.* namespace in wsgi environment 21:08:58 this is something tdasilva and acoles were talking about when i got online this morning. kinda interesting, i think 21:09:05 acoles: tdasilva: can you give a summary? 21:09:26 this came up while reviewing https://review.openstack.org/#/c/237393/25 21:09:36 patch 237393 21:09:49 sad patchbot :( 21:09:59 ah 21:10:15 patch 237393 21:10:16 notmyname: https://review.openstack.org/#/c/237393/ - swift (feature/crypto) - Support for http footers - Replication and EC 21:10:16 where we introduce a callback for middleware to get the chance to add footers to a multipart object PUT to backend 21:10:24 hey patchbot! 21:10:59 and tdasilva raised idea of starting to use some uniform naming of things swift puts into the req environ 21:11:15 specifically the callbacks, right? 21:11:26 so in this case swift.callback.update_footers or similar 21:11:29 notmyname: yes 21:11:44 what else is there? swift.authorize. others? 21:12:01 yeah, the name itself is not the point, just that we could have something uniform...swift.callback, swift.hook, etc... 21:12:03 the only existing callback i could think of is authorize which we probably need to stick with for 3rd party auth m/ware 21:12:30 and copy_hook which may go away soon anyway 21:12:40 may? 21:12:57 * tdasilva hopes that it *will* 21:13:13 notmyname: its not done til its done ;) 21:13:20 :-) 21:13:22 true :) 21:13:56 acoles: are you suggesting people should review copy middleware? 21:14:05 so the idea is swift.callback.authorize (and probably some migration plan), swift.authorize.update_footers, swift.authorize.do_what_i_mean, and antyhing else goes there 21:14:06 * tdasilva is done with shameless plug 21:14:15 tdasilva: subtlely yes 21:14:50 notmyname: yes but less of the authorize ;) 21:14:55 lol 21:14:56 yes 21:15:48 question is whether it has value/merit? IDK cos right now there's so few examples 21:15:52 so, not to be argumentative, but we've got 2 examples of callbacks right now, and one won't use this new thing. so why do we want to have a new convention for it? 21:16:10 notmyname: I see very little value in trying to change the name of the authorize callback 21:16:38 notmyname: it's one of those backwards compat crufts that's a bit out of our control - easier to just live with the old and have a better plan for the moving forward IMHO 21:16:56 clayg: do you think this idea is reasonable going forward? 21:17:10 agree, i think changing authorize is just making unnecessary work 21:17:44 ok. I'm not trying to argue that at all. 21:17:49 btw, technically there is a fetch_keys callback introduced in crypto too 21:18:17 jrichli: right! i'd missed that, so we have two new examples 21:18:21 notmyname: if we're going to change the interface for authorize we should do it for something functional - like lots of auth middleware have to do these silly closures of env data because of make_sub_request - last time I thought about it; i seem to recall thinking the callback should have some better args - something more contextextual about who & what - something like policy 21:19:24 acoles: jrichli: that's twice as many as before! ;-) 21:19:39 I think having a callback namespace (swift.callback.*) is fine. seems like something future me would appreciate 21:20:05 and since the only 2 examples we have right now are part of the crypto branch, then there's not yet any worries about migrations 21:20:11 notmyname: yeah having all new call backs under common name is a start - documentation is next - maintainace is forever :'( 21:20:14 so it's more of a "this seems reasonable so let's do it" 21:20:44 clayg: you told me yesterday "tests are the hard part". I think "docs are the hardest part" should be added there :-) 21:20:54 notmyname: nice 21:21:16 code is easy, tests are hard, words are near impossible 21:21:20 lol 21:21:38 anyone else have somethign to share about a swift.callback.* convention in the wsgi environment? 21:21:44 sounds like none of you have done GUI work 21:21:55 tdasilva: rofl! 21:21:56 :D 21:22:03 tdasilva: thank goodness! lol 21:22:05 I thought we were talking about reality ;-) 21:22:23 "...pixels are just right out" 21:22:58 so does the callback naming cause any major waves for the crypto branch at this point? 21:23:01 notmyname: it sounds like crypto is a good place to start 21:23:05 FWIW i think callbacks are an annoying interface to document and maintain - you put this here and the code will call it ... right ... *here* 21:23:36 I think sometimes it's obviously the simplest problem to address the challange at hand - but it doesn't always scale 21:23:38 no major waves really: just a patch to replace the strings being used now 21:23:42 so this is me having no better ideas :'( 21:23:44 DO IT! 21:23:48 notmyname: ok I/we will update our callback names accordingly 21:23:53 clayg: meh. but it's probably better than other sort of IPC like queues and shared memory? 21:24:27 ok, let's move on then 21:24:32 #topic copy middleware patch 21:24:36 https://review.openstack.org/#/c/156923/ 21:24:36 notmyname: patch 156923 - swift - Refactor server side copy as middleware 21:24:46 I should have added this to the agenda, but thanks for bringing it up tdasilva 21:24:49 sneaky 21:25:27 this is by far the most heavily starred patch in the swift community, for what that's worth. and beyond simplifying some existing code, it's critical for the crypto work 21:25:39 in other words, it's pretty much the number one thing that needs to get done 21:25:40 patch 260179 first though 21:25:40 jrichli: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY 21:26:06 I haven't reviewed it - I might have time around Friday - if we don't already have reviewers on deck? 21:26:12 what's the status for reviews? 21:26:42 there's been some reviews from kota_, torgomatic, Hisashi 21:26:42 kota_: thanks for reviewing this one 21:26:45 kota_: has done some reviews on 260179, thanks kota 21:27:12 yup, mostly I've been in final straight to add +2 I think. 21:27:24 I'm in testing in my lab env. 21:27:34 thanks kota_ 21:27:35 are any swift-core recused? tdasilva & acoles ? Can we let them have a vote? 21:27:35 tdasilva: you're effectively the owner on both of these 21:27:46 yeah, i can't vote 21:27:47 what's your sense of where it is? what do you need? 21:27:50 maybe I can make comments (or just +2) in this week. 21:28:01 i think it's good to go! 21:28:12 i know i need to leave a comment; SLO copies can fail under some circumstances. but maybe we don't care 21:28:31 can patch 156923 land without patch 260179 - or are they effectively a chain? 21:28:31 clayg: https://review.openstack.org/#/c/156923/ - swift - Refactor server side copy as middleware 21:28:32 clayg: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY 21:28:35 timburke: i'm curious 21:28:46 clayg: they are a chain 21:29:29 tdasilva: you'll have to wait for is review comment ;-) 21:29:33 *his 21:29:59 we might have to -2 the head of the chain head then? at least we've done that in the past when breaking up changesets that need to land together 21:30:24 is it bad if the first lands without the second? 21:30:27 if so, I agree 21:30:36 i think the first can land on its own 21:30:43 it still supports the COPY verb 21:30:50 decouple can land on its own 21:31:01 in fact once the second lands the COPY verb needs to be taken out of the decouple one IIRC 21:31:16 ok 21:32:15 therefore it doesn't need a -2 plug 21:32:19 notmyname: full ack 21:32:20 tdasilva: does the middleware patch remove COPY verb handling from versioned writes? 21:32:52 so if you're name is not tdasilva, then please review patch 260179 followed by patch 156923 21:32:52 notmyname: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY 21:32:53 notmyname: https://review.openstack.org/#/c/156923/ - swift - Refactor server side copy as middleware 21:32:56 acoles: no, COPY verb gets removed in patch 260179 21:32:56 tdasilva: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY 21:33:55 oh i had the chain backwards? gd new gerrit ui 21:34:23 clayg: I was afraid I got it backwards 21:34:41 clayg: sorry if I confused people, but basically copy middleware depends on decouple versioned writes from COPY 21:34:43 tdasilva: yes versioned writes won't issue COPY requests but still expects to receive COPY requests I think 21:35:24 acoles: only because it will land first... 21:35:27 tdasilva: line 501 https://review.openstack.org/#/c/260179/16/swift/common/middleware/versioned_writes.py 21:35:27 acoles: patch 260179 - swift - decouple versioned writes from COPY 21:35:28 acoles: tdasilva: that acctually makes pleanty of sense 21:35:36 tdasilva: yep, its cool, 21:36:59 acoles: once copy middleware lands, that code becomes useless, so we can remove it 21:37:19 notmyname: clayg: IIRC I got to point where I didn't feel I should +2 either of those patches cos I got involved 21:37:23 * cschwede arrives late to the party, sorry 21:37:26 tdasilva: yes thats what I was thinking 21:37:37 that means patch 260179 still expects incomming COPY but doesn't use COPY in itself. 21:37:37 kota_: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY 21:37:47 in my undestunding. 21:37:47 tdasilva: oh oh oh *third* patch! maybe a blocked series would be better 21:37:53 tdasilva: can you get this sort of info (the chain, what's removed where, and what happens after) into https://wiki.openstack.org/wiki/Swift/PriorityReviews? 21:38:01 tdasilva: I'll be happy to help, if you need it 21:38:18 tdasilva: I think having it written out there will help us all know first this, then that, etc 21:38:47 notmyname: honestly, i'd prefer if people just focused on this two patches...this 3rd one that acoles and I are talking about is not super important 21:38:49 ok, so al & thiago are out - maybe if tim can get his +2 on it in the next couple of days I can sweep the +A? 21:38:56 so i don't want to confuse people 21:38:57 oh - or Kota! 21:39:40 tdasilva: or we add that to 156923 - i.e. remove COPY handling from versioned writes at same time COPY mware lands 21:39:43 tdasilva: my thinking was more about if you have to load it all in your head to understand what's going on - and we're doing weird stuff to keep backwards compat during a transistion that only last from landing one patch to the next - a chain is a reasonable approach 21:39:57 yeah, I think we've got the bandwidth. with acoles and tdasilva writing, kota_ and timburke and torgomatic who have looked. maybe getting clayg myself or cschwede to take a look this week 21:40:16 tdasilva: yeah, I agree about staying focused 21:40:19 cschwede: will be busy with upgrade tests :P 21:40:35 :-) 21:40:37 is sam still sick? 21:40:43 tdasilva and i may sum our +1's ;) 21:40:46 I'm guessing so. haven't seen him today 21:40:52 lol 21:40:53 clayg: exactly 21:41:15 anythign else we need to cover on this topic in the meeting? 21:41:55 tdasilva: you think the rolling upgrade impact is covered? 21:42:21 clayg: mmm...now you got me 21:42:29 tdasilva: commit message mentioned snarfing post_as_copy from proxy-app section - which sounded pretty reasonable 21:42:43 clayg: right 21:43:47 clayg: very quickly thinking about it, I don't think rolling upgrade would be an issue 21:44:36 i bet it's going to be great - can't wait to look at it! 21:45:56 ok, let's move on 21:45:59 #topic summit prep status 21:46:03 mostly just a reminder 21:46:06 #link https://etherpad.openstack.org/p/swift-newton-summit-planning 21:46:08 tdasilva: acoles: if kota_ or timburke are feeling particuarlly strong about it i'm ok with you one or either of you guys +A'ing - since we're at the start of the cycle - just IMHO 21:46:09 fill that out. add your ideas 21:46:25 clayg: ack 21:46:35 and if you are giving a conference talk, please add it in the "conflicts" section 21:47:16 there's some good stuff there, but we've got room for more topics, i think 21:47:38 starting next week, I'll be organizing them into a schedule and pushign that to the scheduling system 21:47:40 notmyname: hrou told me he's short on round tuits for sysmlinks work - and matt's still hung out to dry on container sharding ... 21:48:08 ... I'm not sure when/how to deal with "XYZ will save us" only works out if we have infinite patience and people don't have lives/jobs 21:48:34 clayg: there's nothing (yet) on that etherpad about symlinks. if we have another champion, it should definitely be added 21:48:59 "But don't try to force dates or time frames; even if everyone is enthusiastic about a direction, it is unlikely that people can guarantee full-time effort for long enough to finish by a chosen date." 21:49:07 from the "lessons Learned" on http://www.aosabook.org/en/gdb.html 21:49:13 notmyname: yeah that's my point - i don't think it has a champion - timburke wants it in some nebulous future of amazing super duper data protection wonderment - and I think tiering was counting on it? 21:49:54 clayg: yeah 21:50:27 notmyname: well i guess ... hrmm ... you know maybe I need to default to your judgement 21:50:48 I'm thinking "hey no one is working on this we need to talk about that!" - maybe you're thinking "yeah, if someone is working on it - let's talk about it!" 21:51:08 at one point I wanted to raise my hand to work on that, but now I'm busy with crypto...maybe after crypto?? 21:51:30 it should be added to the etherpad. I don't want to overlook it when figuring out the schedule, but "someone who's working on it is present" is a heavy weighting factor in scheduling 21:51:57 Hey All, yea I'd love for symlink to make some progress though : ) - I think there are enough folks who care (maybe cared is the right tense as I haven't checked in lately) 21:52:34 hrou: yeah, I think we all want it to make progress. just not sure about actual ability to pound out code 21:53:23 notmyname, yea I think maybe what I could do to help is to put together a bit of status; There was an etherpad floating around a while back but its a bit of date 21:53:39 it really comes back to the handling of POST for fast-post (now default yey). 21:53:39 hrou: ok, thanks 21:53:54 I added a placeholder to the bottom of https://etherpad.openstack.org/p/swift-newton-summit-planning 21:53:58 sure its my pleasure, I'd love to see it land one day. 21:53:58 hrou: I don't think we quite went as far as changing the default FWIW 21:53:59 Thanks! 21:54:06 hrou: thank you! 21:54:21 hrou: any details/links you can throw in there would be good 21:54:30 ok, last topic 21:54:33 clayg, meh one can hope ;) I was being optimistic. But yea we def want it working with both 21:54:33 #topic swiftclient docs 21:54:39 joeljwright: what's up here? 21:54:47 I hear you've been making great progress! 21:54:53 well, we've made some really good progress 21:54:54 https://review.openstack.org/#/c/288566/6 21:54:55 joeljwright: patch 288566 - python-swiftclient - WIP: This patch adds a new doc structure for swift... 21:55:10 service-api.rst is now 'complete' 21:55:17 and has a full complement of examples 21:55:27 notmyname: I was *amazed* at what's there now - phenominal work - so many KUDOS to the people working on that 21:55:28 cli.rst just needs examples 21:55:44 connection-api.rst needs auth and examples 21:55:48 I'm still working on them 21:55:54 but would love people to start reviewing 21:56:00 yeah, that looks great 21:56:18 suggestions/comments are most welcome 21:56:22 quick question, do you want to keep this all as one big chunk like this? or can/should it be split to land in smaller pieces? 21:56:32 joeljwright: can the sections that are complete be broken out into a review that is ready to merge? 21:56:34 joeljwright: do you really need the feedback or just the merges? 21:56:42 I think we can get this one ready soon enough to land in one piece 21:56:50 suggestions will probably have to be follow on 21:56:59 joeljwright: we're going from 0 to %^&Uing awesome! How is that not a win? I'll go +A some stuff right now. 21:57:12 joeljwright: reall? that's great. when are you expecting it to be done, based on current progress? 21:57:18 only the listed reviewers have really read it 21:57:22 joeljwright: i know initially you guys were debating structure - but I *love* how things ended up layed out 21:57:25 so it'd be good to get eyes on it 21:57:35 you people are smart - asettle is owed beers for sure 21:57:36 clayg: thanks :) 21:57:40 definitely 21:57:47 * asettle appears 21:57:48 Ya what? 21:57:48 I'll definitely give this patch set a look 21:57:52 once she finally explained what 'active voice' meant 21:57:53 :) 21:58:06 asettle: the client docs patchset is looking great. thanks for your help 21:58:06 notmyname: thanks 21:58:07 asettle: you have highlights set for "beer"? ;p 21:58:12 lol 21:58:14 acoles: you know it. 21:58:19 I appear \o/ 21:58:19 rofl 21:58:24 I'm a lurker. Good meeting guys. 21:59:00 joeljwright: anything else on these docs? 21:59:05 I would like help with 'interesting' examples for the connection api if possible 21:59:16 ok 21:59:20 if not interesting, then at least useful 21:59:33 which file is that in? 21:59:33 acoles: foster 21:59:34 but that's it 21:59:52 joeljwright: they can get even more interesting after https://review.openstack.org/#/c/298968/ lands :) 21:59:52 client-api.rst 21:59:53 timburke: patch 298968 - python-swiftclient - Adding keystoneauth sessions support 22:00:11 joeljwright: ok 22:00:14 yeah, that will be interesting 22:00:17 bah, look at the time 22:00:22 #topic other 22:00:38 I'll be in san antonio early next week with the intel/rackspace people 22:00:55 flying out next wednesday, so if travel plans conflict with the meeting, I'll update. for now, the meeting is on 22:01:02 s/out/home/ 22:01:23 thanks for coming, everyone. thanks for working on swift 22:01:26 #endmeeting