21:00:53 #startmeeting swift 21:00:54 Meeting started Wed Apr 13 21:00:53 2016 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:56 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:59 The meeting name has been set to 'swift' 21:01:08 hello everyone. who's here fro the swift meeting? 21:01:12 . 21:01:12 o/ 21:01:13 o/ 21:01:15 o/ 21:01:16 hello 21:01:17 hi 21:01:18 o/ 21:01:18 o/ 21:01:19 yo 21:01:28 o/ 21:01:35 o/ 21:01:43 o/ 21:01:44 here 21:01:45 Hi \o/ 21:01:52 o/ 21:01:59 welcome 21:02:08 I've been offline most of this week, so it's good to see everyone 21:02:18 agenda is at 21:02:20 #link https://wiki.openstack.org/wiki/Meetings/Swift 21:02:39 I'd like to go kinda fast today (but not too fast if a topic needs it) 21:02:53 immediately after the meeting I've got to drive to the airport and get on a plane to go home 21:03:08 #topic client docs 21:03:14 joeljwright: this is your topic. what's up? 21:03:23 very quickly then, review is here https://review.openstack.org/#/c/288566/ 21:03:40 I would like help with client-api.rst 21:03:41 patch 288566 21:03:41 notmyname: https://review.openstack.org/#/c/288566/ - python-swiftclient - WIP: This patch adds a new doc structure for swift... 21:03:46 but other than that it's 'complete' 21:03:54 joeljwright: great! 21:04:01 what do you need on client-api.rst? 21:04:07 it might be worth merging as is, and completing the connection api as a follow on 21:04:08 ie what does "help" mean? 21:04:28 it needs details of auth when using client.Connection() 21:04:40 and some examples 21:04:51 I wonder if hosanai could help with that 21:04:52 hello - i'm late 21:04:58 cutforth: welcome 21:04:58 mmotiani? 21:05:00 I have to be honest, I'm hoping someone out there has more experience of using the Connection API directly 21:05:08 notmyname: i will do it 21:05:15 hosanai: great, thanks 21:05:17 sorry, to be late. 21:05:22 kota___: no worries 21:05:25 hosanai: thanks 21:05:29 someone from osic swift team will help out 21:05:37 pdardeau: great, thanks 21:05:43 joeljwright: yeah, I saw the patch and the merge of examples to cli.rst looks great to me. 21:05:58 mmotiani: thanks 21:06:00 will gerrit comments work or would you prefer to have patch sets pushed over yours? 21:06:24 joeljwright: ^ 21:06:27 tbh, either is fine - I'm happy to be editor if needed 21:06:41 notmyname: I am already pushing my patches on his. 21:06:47 mmotiani: ah ok 21:07:03 so joeljwright and mmotiani and hosanai coordinate and it should be done soon! 21:07:04 joeljwright: I will look into the client-apt.rst as well. 21:07:16 looking at cli.rst or service-api.rst should give details of what sort of auth section we need 21:07:28 good news is that you are all on opposites sides of the world, so there's little change of pushing at the same time :-) 21:07:36 a translation of the examples from those pages would be good 21:08:25 that's all I need for now, and hopefully we can get it all tied up at the summit 21:08:36 I'm happy to see the progress here. thanks everyone for helping and reviewing, and especially to joeljwright for doing the lion's share 21:08:48 #topic rolling upgrade testing status update 21:09:00 unfortunately cschwede couldn't make it to the meeting 21:09:07 but he's left updates on the etherpad 21:09:12 #link https://etherpad.openstack.org/p/swift-rolling-upgrade-multinode-testing 21:09:45 it looks like he's got some patches (listed) that will set up the necessary things, but there are still some bugs to work through (IIRC) 21:10:03 any questions here? 21:10:31 ok, moving on :-) 21:10:34 #topic summit planning 21:10:47 I have less on this topic than i had hoped for by this point 21:10:51 #link https://etherpad.openstack.org/p/swift-newton-summit-planning 21:11:09 basically, I havne't had a chance to make a schedule yet, so I've got to do that tomorrow 21:11:25 the good news is that means you still have some time to add topics you're interested in 21:12:09 there were one or two that we talked about here with the OSIC team, so add those today/tonight, and I'll do the scheduling in about 24 hours 21:12:53 notmyname I will add mine tonight 21:12:59 hurricanerix: great, thanks 21:13:20 anythign I can help with as you prepare for the summit? 21:13:25 (to anyone) 21:14:08 the scheduling website has had some bugs knocked out, so it should be working 21:14:23 there's also phone apps available (TBH I find the phone app easier to use) 21:15:00 unfortunately, it doesn't seem that there's a good way to export it into a calendar, so you're left with the phone app probably during the week 21:15:32 thx, i will try the app 21:15:45 be sure to sign up for the openstack-wide party. you'll have to have a ticket to get in 21:16:18 as I understand it, there's just one big party that's goignt o be like a street fair instead of the separate sponsored parties this time 21:16:29 and an eventbrite app can help with the rsvp'd tickets 21:16:55 right 21:17:00 anything else on the summit from anyone? 21:17:28 ok. moving on, then 21:17:38 #topic searchlight integration 21:17:59 this is a follow-on from a conversation we had in bristol about searchlight and metadata integration 21:18:04 lakshmiS: are you here to talk about it? 21:18:04 Based on previous discussions, we have created a client library which can directly call elasticsearch behind Searchlight server and index swift data. 21:18:08 lakshmiS: looks like there's been some work on the searchlight side 21:18:15 There were some hesitancy on sending notifications using OSLO Messaging and RabbitMQ. 21:18:21 right 21:18:28 For reference here’s the spec and POC’s using both client library and notifications. As mentioned in spec link, notifications are the recomended option since Searchlight server can take care of upgrades which otherwise will be hard to support using client library. 21:18:41 Spec - https://review.openstack.org/305404 21:18:41 Client Library POC: https://review.openstack.org/#/c/305309/ 21:18:41 Notification POC: https://review.openstack.org/#/c/249471/ 21:18:41 lakshmiS: patch 305309 - swift - POC: Searchlight client library 21:18:42 lakshmiS: patch 249471 - swift - WIP Oslo.messaging middleware 21:19:40 Notifications is still the recommended option but we have created the client library poc to demonstrate direct call to elasticsearch 21:19:46 lakshmiS: the searchlight client would be an alternative to the oslo messaging one, right? 21:19:50 yes 21:19:56 That 305309 is very primitive and early 21:20:10 but enough to show the really barebones case 21:20:23 great 21:20:29 what do you see as next steps, then? 21:20:54 we have questions on what is the best place within swift to call this library 21:20:55 timburke_: can you point timur at that client library patch? 21:21:34 lakshmiS: that is a great question 21:21:45 we had some ideas as part of the conversation in bristol 21:22:14 including possibly modifying the container DB schema to have a better way to track what's been indexed and what hasn't (similar to container sync) 21:22:31 and it's something I want to discuss in more depth in austing 21:22:34 *austin 21:23:04 is there something we can look at before summit to prepare more on it. 21:23:15 so the short answer is the current idea sounds good, but it hasn't been written yet. and there might be a better idea too :-) 21:24:16 lakshmiS: this is a doc for container sync, but the basic idea (the high-water marks) is similar to what we discussed 21:24:18 #link http://docs.openstack.org/developer/swift/overview_container_sync.html#what-s-going-on-behind-the-scenes-in-the-cluster 21:24:33 so that would be a good starting point for background reading 21:24:50 oh, that's really good to have now 21:25:01 lakshmiS: imagine then an "indexer" background daemon of some sort that's similar to the existing container sync daemon 21:25:33 ok will read through that... 21:25:48 TravT: lakshmiS: is there anything else you need from us in the short term on this? 21:25:54 last question 21:26:05 I'm going to ping one of my coworkers about the indexer library patch you have 21:26:08 (timur) 21:26:23 can we use elasticsearch python client library as dependency for calling elasticsearch from swift? 21:26:28 he's done something already that works with ELK. I'm curious how much (if any) duplication there is 21:26:40 that would be great 21:26:54 since we only have a small poc 21:26:58 one other question, how current is the container sync process with reality? i mean is it near real time or behind by days? 21:27:02 lakshmiS: I'd want to see what other dependencies that brings in, but unless there's some obvious deficiency with it, probably yes 21:27:40 TravT: probably somewhere between there. actually, it's an active area of improvement right now (see the 3 patches listed on https://wiki.openstack.org/wiki/Swift/PriorityReviews for container sync) 21:27:59 okay, i'm just wondering if possibly there would be two integration points 21:28:09 one at actual operation time 21:28:14 right 21:28:15 and that one is just for consistency 21:28:23 ^ container sync 21:28:35 honestly, there probably will be 2 21:28:56 I'd prefer to only have the background one if possible, but it's likely we'll need/want one during the request time too 21:28:57 IIRC when we discussed in Bristol we identified that, but the container daemon one would always be needed as fallback so made sense to start there? 21:29:02 right 21:29:05 acoles: defintiely that 21:29:18 then look at write path integration after 21:29:21 thats even better for resync 21:29:35 (fwiw, elasticsearch dependencies seem minimal) 21:29:37 the container process sounds great for the consistency 21:29:50 timburke_: thanks for checking 21:30:24 needing to match client version to server version makes me a little nervous, though 21:30:41 not sure what server versions would be likely to be deployed 21:31:11 yes, we have started to look at the ramifications that will have for searchlight as well 21:31:13 currently we are at 1.4 but it will be moving towards 2.0 possibly in newton 21:31:17 right now just accumulating bug 21:31:31 lakshmiS 1.4 - 1.7.x 21:31:52 by "accumulating bugs" i mean we are actively identifying areas of concern and logging them 21:32:11 :-) 21:32:23 I accumulate bugs every time I open my code editor 21:32:39 lol 21:32:52 anything else on this project right now? 21:33:01 I'm excited to see it moving forward 21:33:04 thats all for now. thanks for the links 21:33:05 lakshmiS: thanks for your work 21:33:28 thanks! we look forward to working through this 21:33:39 great 21:33:42 we have a session for this on summit etherpad right? 21:33:47 yes 21:33:48 yes, there is one on there 21:33:50 good 21:33:54 i've added a bunch of links as well 21:34:08 thanks 21:34:12 #topic open discussion 21:34:17 TravT: ok thanks that's helpful 21:34:27 anything else to bring up this week in the meeting? 21:34:37 notmyname: crypto 21:34:56 TravT: when I do the scheduling this week, I'll make sure to also tag searchlight and not make it conflict with your working sessions 21:35:02 acoles: ok, crypto. go 21:35:06 ok, thanks! 21:35:08 what's up with crypto? 21:35:11 we're making great progress 21:35:15 great! 21:35:27 but copy middleware is still needed on master 21:35:50 ok 21:35:55 who's reviewing that? 21:35:59 jrichli: and I have both independently cherry picked it across to feature/crypto and had all func tests passing with EC and replication policy 21:36:14 kota___ and timburke_ have reviewed copy recently 21:36:18 many thanks!!! 21:36:28 yeah thanks timburke_ and kota___ 21:36:47 o/ 21:37:31 kota___: not a lot has changed since your last +2, so if you have a chance to look again, it would be great 21:37:46 tdasilva: ok, will do 21:38:00 notmyname: we had a call today to review our crypto trello board and status - tdasilva jrichli mahatic and myself 21:38:05 great 21:38:26 notmyname: and we have a call scheduled for tomorrow at 1400UTC to discuss bring your own key use case with a Barbican core 21:38:46 anyone who'd like to join in please ask me for details 21:39:43 notmyname: so that's where we're at with crypto 21:40:35 thanks for the update 21:40:53 I have something I would like to bring up. 21:41:06 twm2016: what's up 21:41:12 Is this something that should be marked as won't fix or should I continue working on it? https://review.openstack.org/#/c/291461/ 21:41:12 twm2016: patch 291461 - swift - Remove Content-Length from 204 No Content Response 21:42:32 it's won't fix (for now) 21:42:45 thanks for bringing it up 21:42:51 I'll -2 it to be clear 21:43:47 thanks! 21:44:44 twm2016: thanks for working on that, but clay's comments were right about the client impact 21:44:48 anything else this week? 21:45:28 ok, I'm calling it then 21:45:33 thank you for coming 21:45:38 thank you for working on swift! 21:45:41 #endmeeting