21:00:23 #startmeeting swift 21:00:23 Meeting started Wed Feb 3 21:00:23 2016 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:24 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:27 The meeting name has been set to 'swift' 21:00:50 good morning everyone. 21:00:56 who'es here for the swift meeting? 21:00:59 o/ 21:01:01 yo 21:01:01 o/ 21:01:02 . 21:01:02 hi 21:01:03 hi 21:01:04 o/ 21:01:04 o/ 21:01:04 o/ 21:01:05 o/ 21:01:06 o/ 21:01:07 hi 21:01:09 o/ 21:01:09 o/ 21:01:12 o/ 21:01:19 Hi 21:01:20 akle 21:01:29 welcome 21:01:47 I'm currently visiting mattoliverau in australia 21:01:58 \o/ 21:02:03 gives a very real perspective of all the different time zones we work in :-) 21:02:09 here 21:02:33 o/ 21:02:42 whoa 21:02:52 so that being said, this week while I'm at a conference, I haven't kept up with IRC too well. 21:03:00 +1 21:03:04 so let's get started on the agenda :-) 21:03:08 #link https://wiki.openstack.org/wiki/Meetings/Swift 21:03:14 #topic hackathon 21:03:24 in a few weeks we've got the hackathon in bristol 21:03:29 I'm looking forward to seeing everyone 21:03:36 jrichli: you put together an etherpad 21:03:44 #link https://etherpad.openstack.org/p/swift-hackathon-feb-2016 21:04:12 jrichli: thanks. please share about it 21:04:23 yes - thought we could add topics and ideas similar to the last hackathon 21:04:56 that's really about it :-) 21:05:12 heh, ok :-) 21:05:19 I think it's great 21:05:20 can i add topics :P 21:05:33 mattoliverau: please do! 21:05:33 it's something we can use to seed our conversation topics in bristol 21:05:44 yes! everyone add topics you want to bring to the hackathon 21:06:28 'topic' is a candy bar in UK, so bring lots :) 21:06:40 nom nom 21:06:42 interesting 21:07:35 #topic auditor watchers^Hbug 21:07:41 #link https://bugs.launchpad.net/swift/+bug/1183656 21:07:42 Launchpad bug 1183656 in OpenStack Object Storage (swift) "object auditors don't finish" [High,Confirmed] 21:07:42 lol 21:08:04 clayg reminded us of this last week, and it seems that there is some comments on it 21:08:07 * clayg will endeavor to keep his trap shut 21:08:24 I don't remember...was someone starting to investigate a solution? 21:08:37 was it onovy? 21:09:20 hmm...doesn't seem to be around right now 21:09:27 anyone know anything more on this? 21:09:49 I asked Gerry to share some numbers from our public cloud just as a reference point, so he added a comment 21:10:00 cool, thanks 21:10:05 I hadn't seen Gerry's notes - that's amazing - you guys are the very best 21:10:46 shame those particular auditors won't be running too many more times :( 21:10:59 acoles: is that where the bugs at inside of hpe? "yup it's slow, here's numers" or is there any work on a patch? 21:11:06 acoles: isn't it already all turned off? 21:11:41 notmyname: my reading of the comment was just "here's some numbers" - not "we're working on it" or even "we care about this" 21:11:46 notmyname: no work on a patch sorry, I just remember last week there was some discussion of 'how much of a real problem is this in a largish cluster' 21:11:51 right 21:12:02 I mean, it told us what we needed to know. Is saving just the device enough? No, because audits take a couple weeks on real loads, so if we save the partition then we lose many fewer hours' progress on restart 21:12:04 clayg: I'm very optimistic ;-) 21:12:17 so now we know what level to save at; no pontificating needed. 21:12:28 :) 21:12:38 all over the but the typing 21:12:53 no now that pontification is done, we get to the real question... 21:13:07 who's doing the "think real hard. type in the answer" part? 21:13:32 onovy and peterlisak sorta said they might 21:13:52 but since the status of the bug is currently unassigned that's probably the most honest representation of reality 21:13:55 clayg: thanks for remembering that. :-) 21:14:14 yeah 21:14:40 so when they're back online, let's bring it up and see what's going on. if I don't, someone else can 21:14:49 k 21:14:55 ok, next candy bar 21:14:55 i'll try to think about it. finding typing bandwidth might be harder. 21:15:13 #topic patch 273073 21:15:13 notmyname: https://review.openstack.org/#/c/273073/ - swift - Insert versioned_writes in correct pipeline position 21:15:24 acoles: this one is your topic 21:15:40 * notmyname will now only think in terms of meeting candy bars 21:16:33 does this patch need discussion or reviews? 21:16:49 acoles: ? 21:16:52 if you dont specify it in config then versioned writes can end up in the wrong place 21:17:00 ok, just wanted to flag it up because its hopefully an easy review and also not sure who sets it explicitly or who is relying on the code to place it in the pipeline 21:17:19 * clayg pulls our our pipeline solver 21:17:33 and ti would be nice to get it fixed because other bugs vary according to where that middleware is in the pipeline :) 21:17:41 s/ti/it/ 21:17:43 * notmyname really wants to get the pipeline solver back as a thing for swift 21:18:30 ok. small review, important bug fix (or footgun removal). go review it 21:18:47 so I have a dependent patch to fix an SLO bug that can't land until we know that versioned writes will be in right place in a test cluster 21:19:12 notmyname: yeah, but also a headsup to check what you ship wrt config 21:19:18 yes 21:19:30 so it looks like we're still letting it flop in auto - although i'm currently working on a new swift release so that may change 21:19:33 clayg: just review i hope 21:19:47 #topic swift and high-latency media 21:19:53 cschwede: this is your topic. what's up? 21:19:59 there were some discussions at the last summit regarding tapes (or high latency media in general). ibm and bdt are working on this, and they just released a middleware prototype 21:20:01 #link http://lists.openstack.org/pipermail/openstack-dev/2016-February/085576.html 21:20:07 so i simply wanted to give a heads up on this topic and hopefully raise some interest 21:20:17 more details are in http://lists.openstack.org/pipermail/openstack-dev/2016-February/085576.html, and hseipp and akle are here today in case you have any questions 21:20:25 cschwede: i saw a notice on the ML - so sweet new stuff is out, yeah!? 21:20:33 cool. I love to see people building stuff around swift :-) 21:20:43 akle: hseipp: thanks for working on this :-) 21:20:50 ... i haven't really looked at it - does it add some new api goodness to do like "is it ready, is it ready, is it ready, ..." 21:20:51 cschwede: yes and thanks for getting us up here 21:21:29 you are welcome. would be great if we could talk about 21:21:33 clayg: well, kinda. the middleware adds some new verbs and triggers backend works 21:22:00 interesting 21:22:07 notmyname: yep, might be something completely new for swift. fresh ideas! 21:22:14 that would be good stuff to look at on a long plane ride home 21:22:25 new *VERBS* 21:22:34 is one of them GREP (cc redbo) 21:22:41 clayg: next up they'll have a grep one... yeha :-) 21:22:59 no grep. well, verbs is the wrong term in this case 21:23:12 hseipp: akle: would either of you possibly be at the hack-a-thon in a couple of weeks? 21:23:21 notmyname: do you think we could a call in something? 21:23:29 but doing a post and sending a MIGRATE, RECALL along with it 21:23:37 akle: hseipp: is there anything specific you need from us in the community right now? 21:23:41 or a GET STATUS 21:24:00 unfortunately can't attend this time 21:24:04 clayg: unfortunately I can't make it to this hackathon 21:24:15 GET STATUS!? 21:24:19 hrmmm... 21:24:27 i see using query parameter 21:24:29 clayg: send a GET, „get“ the STATUS of the RECALL 21:24:49 ho_away: yep. verb was the wrong term. it’s using POST & GET for now 21:24:55 a'ight 21:26:07 akle: hseipp: is there anything specific you need from us in the community right now? 21:26:10 so i think it would be nice to have a look at this, talk about this during the hackathon (maybe doing a conf call) and see if we can get to a common implementation that is usable with different backends/vendors 21:26:18 notmyname: at this stage we primarily see for comments and feedback - there is also a Wiki page linked to leave comments 21:26:27 s/see/seek/ 21:26:29 I had sort of imagined it might be more RESTful, maybe just adding a COPY sort of notion to move it into a storage policy that could do listings (maybe HEAD) but you had to "COPY" it somewhere to do a restore, with some sort of trigger to test if it was ready - I guess I need to dig in to understand some of the implementation considerations/trade-offs and history/alternatives 21:26:37 hseipp: ok, great 21:27:01 do we need to have HLM stub for testing? 21:27:04 cschwede: that's a great goal! you're the champ! 21:27:12 next up (testing!) 21:27:23 #topic in-process testing with fast-post 21:27:32 ho_away: this is one of the next things on our to-do list, for now we tried to put as much into the readme as possible 21:27:34 acoles: what's going on here? 21:27:53 I'd like to have a gate job that runs func tests against a fast-post configured service 21:27:55 ho_away: I think we all get our own LHC for testing 21:28:00 hseipp: thanks! 21:28:16 currently we have 3 func tests runs in the gate, all use post-as-copy 21:28:31 clayg: we should do the next hackathon there and ask for some resources… that would be fun! 21:28:31 acoles: *see* everyone uses post-as-copy! 21:28:33 acoles: that sounds like a great idea 21:28:43 so this patch adds a tox env to do in process testing with fast-post, patch 274086 21:28:43 acoles: https://review.openstack.org/#/c/274086/ - swift - Enable in-process func tests to optionally use fas... 21:29:01 didn't someone +A that yet? 21:29:04 optionally? 21:29:20 it just adds an env flag to make it so you can run the in-process functests with a different config option - and we already have some of those 21:29:24 question is do I swap gate-tox-func to run that new env, or add a new fourth func test run? 21:29:29 ... only difference with this one is that it's useful 21:29:35 acoles: ah 21:29:41 acoles: yeah that's a discussion 21:29:42 clayg: think its waiting for a +A 21:30:03 acoles: IMO replace the existing, since I don't think the current 2 different functests really do anything different anyway 21:30:12 and also, headsup that this means new patches will have to work with fast-post enabled! 21:30:17 acoles: don't care new job or change the other in-proc job - i don't really care about the in-proc job - i think it'd be fine to make it test fast-post 21:30:35 acoles: oh oh oh - that'll be wonderful! 21:30:47 can we get another core to commit to review this asap? 21:30:48 acoles: can the name of the job somehow reflect that it's using fast-post? 21:30:55 i have no idea how to openstack-ci 21:30:59 well, apart from patches to EC, but thats another story, and maybe another gate job :) 21:31:36 clayg: good idea, I will have to look at -infra again, maybe the job name simply takes the tox env name, so "yes" 21:31:48 -infra is not my strength 21:31:53 acoles: probably be nice with in-proc (since we have full control over the env) if we tested as many storage policies as we think we can get away with configuring - do functests still only run against a single policy (but it's configurable?) 21:31:56 (do I have any? :) 21:32:39 clayg: you can set the policy for func tests, SWIFT_TEST_POLICY IIRC 21:32:54 I'll take a look at the testing patch. Especially if it helps acoles. 21:32:56 clayg: and we could add an EC ring/policy to the in-process test setup 21:33:08 mattoliverau: thanks 21:33:14 mattoliverau: thanks, there are some notes in review comments about how to verify it 21:33:42 clayg: so we could even make func-in-process-fast-post use an EC policy! 21:33:47 acoles: awesome thanks.. I'll have a play with it during boring talks :) 21:34:01 mattoliverau: has notmyname not spoken yet then :D:D 21:34:03 mattoliverau: there's one at 2:15 today that will be terribly boring ;-) 21:34:08 acoles: right! 21:34:19 perfect chance for code reviews 21:34:33 notmyname: lol, I need to listen so I can heckle 21:34:43 lol 21:34:55 oh, so mattoliverau gave his first ever public presentation yesterday (on container sharding in swift). he did great 21:35:13 have the link yet? 21:35:14 Well first ever solo one 21:35:35 mattoliverau: congrats! 21:35:42 cool! 21:35:51 mattoliverau: also thanks for looking at fix-rings-again2 21:35:53 acoles: have you done anything yet to update the gate jobs in -infra yet for the fast post stuff? or just waiting for this patch to land first? 21:35:57 mattoliverau yay 21:36:32 notmyname: not yet, I will let y'all know when I have a patch up for -infra gate job change, and yeah it will depend on this one 21:36:43 ok 21:37:06 acoles: thanks for this. I'm very happy about our fast-post future 21:37:20 it's so bright I've gotta wear shades 21:37:26 #topic patch 202411 21:37:26 notmyname: https://review.openstack.org/#/c/202411/ - swift - Add functional test for access control (RBAC) with... 21:37:26 thanks for the feedback 21:37:40 * acoles goes to make reconciler fast-post proof... 21:37:41 crap 21:37:51 i got a +2 on it now so i want to have more reviews :-) 21:38:03 i sorta got a keystone-swift-all-in working (thanks ho_away !) but I haven't looped back to to that change yet 21:38:16 ok, who's volunteering to be another reviewer? 21:38:22 not me :) 21:38:25 well that was mostly just acoles getting exahusted 21:38:31 sounds like clayg is 21:38:35 s/mostly/partly/g 21:38:48 mattoliverau: I'm trying to get more keystone friendly if that's what your suggesting 21:39:19 ho_away: doesn't want *me* to review it - that's practiacally zero chance it makes it through that with a +A - clayg is stone cold heartless -1'er 21:39:35 clayg: cschwede: kota_: tdasilva: torgomatic: ? 21:39:57 clayg: lol 21:40:00 I'm in the middle of untangling get_*_info, but after I can take a look 21:40:03 notmyname: looking at it right now (but no rating this evening) 21:40:16 I also need to get more keystone friendly, once i tried it i got failed to set the keystone env :/ 21:40:17 clayg: ho_away took a bunch of -1s from me and is still standing 21:40:42 kota_: try the keystone branch on swiftstack/swift-all-in-one - it's getting better all the time! 21:40:57 acoles: ho_away is a rockstar 21:40:57 clayg: \o/ 21:41:03 ok, so between torgomatic cschwede clayg and kota_, it *should* be either landed or have other patch sets by next week ;-) 21:41:06 kota_: yeah that's also mostly ho_away 21:41:07 clayg: +1 21:41:34 torgomatic: cshwede: thanks! 21:41:37 ho_away: the keystone work on the vSAIO is really imporant, IMO. thanks 21:41:57 kota_: write a script to feed keystone with random combinations of options until you get a 200 ;) 21:42:04 #topic open discussion 21:42:12 anything else to bring up in the meeting this week? 21:42:25 what does everyone think about https://blueprints.launchpad.net/python-swiftclient/+spec/optional-query-strings-on-requests 21:42:27 anyone else noticed the 204 content-length thing!? 21:42:48 I was like "i'll be damned i guess we suck"; then I was like WTF this wasn't in 2616? 21:43:19 I love the tempest thing; there's tests in there that assert we have content-length in there, so we can't take it out to meet spec without first doing a ridiculous multi-repo song-and-dance 21:43:24 awelleck: yup. I think you'll also find a lot of agreement from timburke 21:44:09 clayg: that sounds like one of those things I marked as read this week. what's going on? 21:44:10 clayg: link? haven't seen it 21:44:14 awelleck: why is this a spec instead of a patch? 21:44:22 do it! standardize all the things! 21:44:27 notmyname: ok thanks, ill work on it 21:44:29 wait - that's not a spec - that's a blueprint - wtf are we supposed do with that? 21:44:32 awelleck: great thanks 21:44:45 convert to spec by using cp(1) 21:44:53 well, curl(1) 21:45:00 clayg: be nice. awelleck probably just found some openstack "here's how you're supposed to do it" docs :-) 21:45:07 this is the rfc 7230 tempest thing -> https://review.openstack.org/#/c/275652/ 21:45:22 I don't get notified of blueprints - is there a way for that to happen? 21:45:41 notmyname: awelleck: sry, if you remember where you found such a doc point me out it so I can hate on it 21:45:49 don't shoot the messenger and all that 21:46:02 clayg: not sure 21:46:09 clayg: oic. thaks 21:46:12 awelleck: anyway - sounds broken and annoying - not sure where to put my +A/please-fix - but thanks for point it out 21:46:30 i think we could make it an open bug - those we know how to deal with 21:46:42 acoles: no. there is no blueprint notification thing (for new ones. you have to subscribe to each one. yet another reason they are terrible) 21:46:55 clayg: so what do we do with the tempest thing? 21:47:01 notmyname: ok. awelleck ^^ take note 21:47:04 blueprint - more like BOOprint 21:47:35 notmyname: oh idk, i htink they're going to spin their wheels getting tempest team happy only to realize this is for swift api next 21:47:38 actually looks like ben martin wrote the blueprint 21:47:38 ... as we do 21:47:50 MOAR booprints!? 21:48:35 notmyname: at first i was feeling bad because the language is pretty clear - but then I remembered we're older than 7230 and they can go pound out a time machine 21:49:32 I don't know. maybe dropping content-length on 204 isn't a breaking thing. idk 21:50:01 we've made similar changes before with standard headers, IIRC 21:50:12 * notmyname doesn't remember specifics 21:51:19 clayg: how'd you find this? do you monitor tempest patches? was it on the ML? 21:52:04 notmyname: I think there's a bug associated with it? i get bugs 21:52:10 ah ok 21:52:34 anything else to bring up at the last minute this week? 21:52:39 from anyone 21:52:46 acoles: so what's the short and skinny on the bad with versioned_writes? 21:52:59 it should be... before/after dlo/slo? 21:53:18 i'd still like some reviews on adding delete markers to versioned_writes - https://review.openstack.org/#/c/214922/ 21:54:05 clayg: yeah, after both, apparently 21:54:31 clayg: should be after slo 21:54:56 patch 218490 has one +2, needs one more 21:54:57 jlhinson: https://review.openstack.org/#/c/218490/ - swift - Automatic refresh of memcache config settings 21:55:11 clayg: but if you have config'd' dlo/slo but not versioned-writes then it gets inserted before them 21:55:49 clayg: if you have not config'd dlo nor versioned-writes then it gets inserted after 21:56:02 clayg: so as always, not even a simple bug 21:56:17 conference stuff is starting here, so I need to close the meeting and physically move 21:56:33 thank you everyone for attending and working on swift. you are what makes swift awesome 21:56:36 #endmeeting