21:00:53 <notmyname> #startmeeting swift
21:00:54 <openstack> Meeting started Wed Apr 13 21:00:53 2016 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:56 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:59 <openstack> The meeting name has been set to 'swift'
21:01:08 <notmyname> hello everyone. who's here fro the swift meeting?
21:01:12 <torgomatic> .
21:01:12 <hurricanerix> o/
21:01:13 <timburke_> o/
21:01:15 <lakshmiS> o/
21:01:16 <joeljwright> hello
21:01:17 <jrichli> hi
21:01:18 <dmorita_> o/
21:01:18 <hosanai> o/
21:01:19 <tdasilva> yo
21:01:28 <TravT_> o/
21:01:35 <pdardeau> o/
21:01:43 <ntata> o/
21:01:44 <acoles> here
21:01:45 <mmotiani> Hi \o/
21:01:52 <gmmaha> o/
21:01:59 <notmyname> welcome
21:02:08 <notmyname> I've been offline most of this week, so it's good to see everyone
21:02:18 <notmyname> agenda is at
21:02:20 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift
21:02:39 <notmyname> I'd like to go kinda fast today (but not too fast if a topic needs it)
21:02:53 <notmyname> immediately after the meeting I've got to drive to the airport and get on a plane to go home
21:03:08 <notmyname> #topic client docs
21:03:14 <notmyname> joeljwright: this is your topic. what's up?
21:03:23 <joeljwright> very quickly then, review is here https://review.openstack.org/#/c/288566/
21:03:40 <joeljwright> I would like help with client-api.rst
21:03:41 <notmyname> patch 288566
21:03:41 <patchbot> notmyname: https://review.openstack.org/#/c/288566/ - python-swiftclient - WIP: This patch adds a new doc structure for swift...
21:03:46 <joeljwright> but other than that it's 'complete'
21:03:54 <notmyname> joeljwright: great!
21:04:01 <notmyname> what do you need on client-api.rst?
21:04:07 <joeljwright> it might be worth merging as is, and completing the connection api as a follow on
21:04:08 <notmyname> ie what does "help" mean?
21:04:28 <joeljwright> it needs details of auth when using client.Connection()
21:04:40 <joeljwright> and some examples
21:04:51 <notmyname> I wonder if hosanai could help with that
21:04:52 <cutforth> hello - i'm late
21:04:58 <notmyname> cutforth: welcome
21:04:58 <pdardeau> mmotiani?
21:05:00 <joeljwright> I have to be honest, I'm hoping someone out there has more experience of using the Connection API directly
21:05:08 <hosanai> notmyname: i will do it
21:05:15 <notmyname> hosanai: great, thanks
21:05:17 <kota___> sorry, to be late.
21:05:22 <notmyname> kota___: no worries
21:05:25 <joeljwright> hosanai: thanks
21:05:29 <pdardeau> someone from osic swift team will help out
21:05:37 <notmyname> pdardeau: great, thanks
21:05:43 <mmotiani> joeljwright: yeah, I saw the patch and the merge of examples to cli.rst looks great to me.
21:05:58 <joeljwright> mmotiani: thanks
21:06:00 <notmyname> will gerrit comments work or would you prefer to have patch sets pushed over yours?
21:06:24 <notmyname> joeljwright: ^
21:06:27 <joeljwright> tbh, either is fine - I'm happy to be editor if needed
21:06:41 <mmotiani> notmyname: I am already pushing my patches on his.
21:06:47 <notmyname> mmotiani: ah ok
21:07:03 <notmyname> so joeljwright and mmotiani and hosanai coordinate and it should be done soon!
21:07:04 <mmotiani> joeljwright: I will look into the client-apt.rst as well.
21:07:16 <joeljwright> looking at cli.rst or service-api.rst should give details of what sort of auth section we need
21:07:28 <notmyname> good news is that you are all on opposites sides of the world, so there's little change of pushing at the same time :-)
21:07:36 <joeljwright> a translation of the examples from those pages would be good
21:08:25 <joeljwright> that's all I need for now, and hopefully we can get it all tied up at the summit
21:08:36 <notmyname> I'm happy to see the progress here. thanks everyone for helping and reviewing, and especially to joeljwright for doing the lion's share
21:08:48 <notmyname> #topic rolling upgrade testing status update
21:09:00 <notmyname> unfortunately cschwede couldn't make it to the meeting
21:09:07 <notmyname> but he's left updates on the etherpad
21:09:12 <notmyname> #link https://etherpad.openstack.org/p/swift-rolling-upgrade-multinode-testing
21:09:45 <notmyname> it looks like he's got some patches (listed) that will set up the necessary things, but there are still some bugs to work through (IIRC)
21:10:03 <notmyname> any questions here?
21:10:31 <notmyname> ok, moving on :-)
21:10:34 <notmyname> #topic summit planning
21:10:47 <notmyname> I have less on this topic than i had hoped for by this point
21:10:51 <notmyname> #link https://etherpad.openstack.org/p/swift-newton-summit-planning
21:11:09 <notmyname> basically, I havne't had a chance to make a schedule yet, so I've got to do that tomorrow
21:11:25 <notmyname> the good news is that means you still have some time to add topics you're interested in
21:12:09 <notmyname> there were one or two that we talked about here with the OSIC team, so add those today/tonight, and I'll do the scheduling in about 24 hours
21:12:53 <hurricanerix> notmyname I will add mine tonight
21:12:59 <notmyname> hurricanerix: great, thanks
21:13:20 <notmyname> anythign I can help with as you prepare for the summit?
21:13:25 <notmyname> (to anyone)
21:14:08 <notmyname> the scheduling website has had some bugs knocked out, so it should be working
21:14:23 <notmyname> there's also phone apps available (TBH I find the phone app easier to use)
21:15:00 <notmyname> unfortunately, it doesn't seem that there's a good way to export it into a calendar, so you're left with the phone app probably during the week
21:15:32 <dmorita_> thx, i will try the app
21:15:45 <notmyname> be sure to sign up for the openstack-wide party. you'll have to have a ticket to get in
21:16:18 <notmyname> as I understand it, there's just one big party that's goignt o be like a street fair instead of the separate sponsored parties this time
21:16:29 <ntata> and an eventbrite app can help with the rsvp'd tickets
21:16:55 <notmyname> right
21:17:00 <notmyname> anything else on the summit from anyone?
21:17:28 <notmyname> ok. moving on, then
21:17:38 <notmyname> #topic searchlight integration
21:17:59 <notmyname> this is a follow-on from a conversation we had in bristol about searchlight and metadata integration
21:18:04 <TravT> lakshmiS: are you here to talk about it?
21:18:04 <lakshmiS> Based on previous discussions, we have created a client library which can directly call elasticsearch behind Searchlight server and index swift data.
21:18:08 <notmyname> lakshmiS: looks like there's been some work on the searchlight side
21:18:15 <lakshmiS> There were some hesitancy on sending notifications using OSLO Messaging and RabbitMQ.
21:18:21 <notmyname> right
21:18:28 <lakshmiS> For reference here’s the spec and POC’s using both client library and notifications. As mentioned in spec link, notifications are the recomended option since Searchlight server can take care of upgrades which otherwise will be hard to support using client library.
21:18:41 <lakshmiS> Spec - https://review.openstack.org/305404
21:18:41 <lakshmiS> Client Library POC: https://review.openstack.org/#/c/305309/
21:18:41 <lakshmiS> Notification POC: https://review.openstack.org/#/c/249471/
21:18:41 <patchbot> lakshmiS: patch 305309 - swift - POC: Searchlight client library
21:18:42 <patchbot> lakshmiS: patch 249471 - swift - WIP Oslo.messaging middleware
21:19:40 <lakshmiS> Notifications is still the recommended option but we have created the client library poc to demonstrate direct call to elasticsearch
21:19:46 <notmyname> lakshmiS: the searchlight client would be an alternative to the oslo messaging one, right?
21:19:50 <lakshmiS> yes
21:19:56 <TravT> That 305309 is very primitive and early
21:20:10 <TravT> but enough to show the really barebones case
21:20:23 <notmyname> great
21:20:29 <notmyname> what do you see as next steps, then?
21:20:54 <lakshmiS> we have questions on what is the best place within swift to call this library
21:20:55 <notmyname> timburke_: can you point timur at that client library patch?
21:21:34 <notmyname> lakshmiS: that is a great question
21:21:45 <notmyname> we had some ideas as part of the conversation in bristol
21:22:14 <notmyname> including possibly modifying the container DB schema to have a better way to track what's been indexed and what hasn't (similar to container sync)
21:22:31 <notmyname> and it's something I want to discuss in more depth in austing
21:22:34 <notmyname> *austin
21:23:04 <lakshmiS> is there something we can look at before summit to prepare more on it.
21:23:15 <notmyname> so the short answer is the current idea sounds good, but it hasn't been written yet. and there might be a better idea too :-)
21:24:16 <notmyname> lakshmiS: this is a doc for container sync, but the basic idea (the high-water marks) is similar to what we discussed
21:24:18 <notmyname> #link http://docs.openstack.org/developer/swift/overview_container_sync.html#what-s-going-on-behind-the-scenes-in-the-cluster
21:24:33 <notmyname> so that would be a good starting point for background reading
21:24:50 <TravT> oh, that's really good to have now
21:25:01 <notmyname> lakshmiS: imagine then an "indexer" background daemon of some sort that's similar to the existing container sync daemon
21:25:33 <lakshmiS> ok will read through that...
21:25:48 <notmyname> TravT: lakshmiS: is there anything else you need from us in the short term on this?
21:25:54 <lakshmiS> last question
21:26:05 <notmyname> I'm going to ping one of my coworkers about the indexer library patch you have
21:26:08 <notmyname> (timur)
21:26:23 <lakshmiS> can we use elasticsearch python client library as dependency for calling elasticsearch from swift?
21:26:28 <notmyname> he's done something already that works with ELK. I'm curious how much (if any) duplication there is
21:26:40 <lakshmiS> that would be great
21:26:54 <lakshmiS> since we only have a small poc
21:26:58 <TravT> one other question, how current is the container sync process with reality? i mean is it near real time or behind by days?
21:27:02 <notmyname> lakshmiS: I'd want to see what other dependencies that brings in, but unless there's some obvious deficiency with it, probably yes
21:27:40 <notmyname> TravT: probably somewhere between there. actually, it's an active area of improvement right now (see the 3 patches listed on https://wiki.openstack.org/wiki/Swift/PriorityReviews for container sync)
21:27:59 <TravT> okay, i'm just wondering if possibly there would be two integration points
21:28:09 <TravT> one at actual operation time
21:28:14 <notmyname> right
21:28:15 <TravT> and that one is just for consistency
21:28:23 <TravT> ^ container sync
21:28:35 <notmyname> honestly, there probably will be 2
21:28:56 <notmyname> I'd prefer to only have the background one if possible, but it's likely we'll need/want one during the request time too
21:28:57 <acoles> IIRC when we discussed in Bristol we identified that, but the container daemon one would always be needed as fallback so made sense to start there?
21:29:02 <notmyname> right
21:29:05 <notmyname> acoles: defintiely that
21:29:18 <acoles> then look at write path integration after
21:29:21 <lakshmiS> thats even better for resync
21:29:35 <timburke_> (fwiw, elasticsearch dependencies seem minimal)
21:29:37 <TravT> the container process sounds great for the consistency
21:29:50 <notmyname> timburke_: thanks for checking
21:30:24 <timburke_> needing to match client version to server version makes me a little nervous, though
21:30:41 <timburke_> not sure what server versions would be likely to be deployed
21:31:11 <TravT> yes, we have started to look at the ramifications that will have for searchlight as well
21:31:13 <lakshmiS> currently we are at 1.4 but it will be moving towards 2.0 possibly in newton
21:31:17 <TravT> right now just accumulating bug
21:31:31 <TravT> lakshmiS 1.4 - 1.7.x
21:31:52 <TravT> by "accumulating bugs" i mean we are actively identifying areas of concern and logging them
21:32:11 <notmyname> :-)
21:32:23 <notmyname> I accumulate bugs every time I open my code editor
21:32:39 <TravT> lol
21:32:52 <notmyname> anything else on this project right now?
21:33:01 <notmyname> I'm excited to see it moving forward
21:33:04 <lakshmiS> thats all for now. thanks for the links
21:33:05 <notmyname> lakshmiS: thanks for your work
21:33:28 <TravT> thanks! we look forward to working through this
21:33:39 <notmyname> great
21:33:42 <acoles> we have a session for this on summit etherpad right?
21:33:47 <lakshmiS> yes
21:33:48 <TravT> yes, there is one on there
21:33:50 <acoles> good
21:33:54 <TravT> i've added a bunch of links as well
21:34:08 <notmyname> thanks
21:34:12 <notmyname> #topic open discussion
21:34:17 <acoles> TravT: ok thanks that's helpful
21:34:27 <notmyname> anything else to bring up this week in the meeting?
21:34:37 <acoles> notmyname: crypto
21:34:56 <notmyname> TravT: when I do the scheduling this week, I'll make sure to also tag searchlight and not make it conflict with your working sessions
21:35:02 <notmyname> acoles: ok, crypto. go
21:35:06 <TravT> ok, thanks!
21:35:08 <notmyname> what's up with crypto?
21:35:11 <acoles> we're making great progress
21:35:15 <notmyname> great!
21:35:27 <acoles> but copy middleware is still needed on master
21:35:50 <notmyname> ok
21:35:55 <notmyname> who's reviewing that?
21:35:59 <acoles> jrichli: and I have both independently cherry picked it across to feature/crypto and had all func tests passing with EC and replication policy
21:36:14 <tdasilva> kota___ and timburke_ have reviewed copy recently
21:36:18 <tdasilva> many thanks!!!
21:36:28 <acoles> yeah thanks timburke_  and kota___
21:36:47 <kota___> o/
21:37:31 <tdasilva> kota___: not a lot has changed since your last +2, so if you have a chance to look again, it would be great
21:37:46 <kota___> tdasilva: ok, will do
21:38:00 <acoles> notmyname: we had a call today to review our crypto trello board and status - tdasilva jrichli mahatic and myself
21:38:05 <notmyname> great
21:38:26 <acoles> notmyname: and we have a call scheduled for tomorrow at 1400UTC to discuss bring your own key use case with a Barbican core
21:38:46 <acoles> anyone who'd like to join in please ask me for details
21:39:43 <acoles> notmyname: so that's where we're at with crypto
21:40:35 <notmyname> thanks for the update
21:40:53 <twm2016> I have something I would like to bring up.
21:41:06 <notmyname> twm2016: what's up
21:41:12 <twm2016> Is this something that should be marked as won't fix or should I continue working on it? https://review.openstack.org/#/c/291461/
21:41:12 <patchbot> twm2016: patch 291461 - swift - Remove Content-Length from 204 No Content Response
21:42:32 <notmyname> it's won't fix (for now)
21:42:45 <notmyname> thanks for bringing it up
21:42:51 <notmyname> I'll -2 it to be clear
21:43:47 <twm2016> thanks!
21:44:44 <notmyname> twm2016: thanks for working on that, but clay's comments were right about the client impact
21:44:48 <notmyname> anything else this week?
21:45:28 <notmyname> ok, I'm calling it then
21:45:33 <notmyname> thank you for coming
21:45:38 <notmyname> thank you for working on swift!
21:45:41 <notmyname> #endmeeting