15:01:35 <sjmc7> #startmeeting openstack search
15:01:36 <openstack> Meeting started Thu Jan  7 15:01:35 2016 UTC and is due to finish in 60 minutes.  The chair is sjmc7. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:37 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:39 <openstack> The meeting name has been set to 'openstack_search'
15:01:46 <sjmc7> Morning folks
15:01:51 <yingjun> Hi
15:01:54 <nikhil> Good morning!
15:01:56 <sjmc7> I’ll be MCing today, for a change
15:01:57 <rosmaita> o/
15:02:02 <sjmc7> regular service will be resumed soon
15:02:14 <GB21> hi everyone
15:02:21 <TravT> morning all.  i'm here, but asked sjmc7 to run the show today.
15:02:29 <nikhil> ++
15:02:38 <nikhil> GB21: glad to see you at this meeting
15:02:55 <GB21> likewise nikhil :D
15:02:55 <sjmc7> #topic General updates
15:02:57 <TravT> i'm still out recovering from knee surgery and just stepping in this morning
15:03:01 * david-lyle_ partially o/
15:03:03 <GB21> Happy new year to all
15:03:04 <GB21> :D
15:03:05 <TravT> thx nikhil
15:03:26 <GB21> gws TravT
15:03:35 <sjmc7> Meeting agenda at https://etherpad.openstack.org/p/search-team-meeting-agenda
15:04:00 <sjmc7> The only general update i’ve got is the mitaka-2 milestone in the third week of Jan
15:04:27 <sjmc7> so any outstanding blueprints/specs, especially the big ones, we should try to come to an agreement by then
15:04:42 <sjmc7> anyone have anything else they wanted to bring up?
15:05:18 <TravT> as sjmc7, we def need some review on the specs
15:05:54 <sjmc7> i’ll list some of the BPs later
15:05:57 <sjmc7> oh, i also noticed yesterday the elasticsearch devstack install was broken mid-December, but there was already a patch under review yesterday to fix it
15:06:22 <sjmc7> ok, moving on..
15:06:31 <sjmc7> #topic cinder and swift plugins
15:06:52 <sjmc7> not sure if lakshmi or duncan are here
15:07:13 <sjmc7> duncant? lakshmiS?
15:07:42 <TravT> duncant kept saying he had a patch ready to go.
15:07:54 <TravT> i think somebody needs to go buy him a beer to get him to upload it
15:08:05 <sjmc7> yeah. i’ll try and track him down later. ha, i need to negotiate that contract
15:08:34 <sjmc7> on the subject of plugins, i took a look at neutron’s notifications
15:09:31 <sjmc7> i added some notes to https://blueprints.launchpad.net/searchlight/+spec/neutron-plugins - i’m not sure how best to map neutron’s data. if anyone has any thoughts, please air them
15:10:35 <sjmc7> it looks like the notifications are enough to get all the data, which is definitely a positive
15:10:39 <TravT> i'll ping robcresswell on that as well
15:10:45 <TravT> see if he has any thoughts there
15:10:51 <sjmc7> ah yeah, that’s a good idea. i’ll track him down too
15:10:54 <TravT> since at summit he said he had interest
15:10:58 <sjmc7> this is gonna get expensive if i have to keep buying beers
15:11:15 <TravT> why do you think i wanted you to run the show today?
15:11:19 <TravT> ;-)
15:11:21 <sjmc7> cunning!
15:11:50 <sjmc7> the other plugin we spoke about at the summit was for Heat i think, which seems more straightforward
15:11:58 <sjmc7> but i’ll take a look at those notifications as well today
15:12:23 <sjmc7> ok, moving on…
15:12:38 <sjmc7> #topic python-searchlightclient
15:13:09 <sjmc7> li has done some really good work on this
15:13:28 <sjmc7> so thankyou yingjun!
15:13:31 <TravT> yingjun: \o/
15:13:50 <TravT> we definitely need to work through the commands a bit.
15:14:09 <sjmc7> yeah, i should’ve looked at your etherpad earlier
15:14:24 <yingjun> no problem, the patches need more feedback;)
15:14:26 <sjmc7> unlike most services which have more ‘RESTy’ looking openstackclient commands
15:14:32 <sjmc7> we’re a little bit different
15:14:38 <TravT> we probably could get some feedback from the OSC team and perhaps UX team on them before mitaka 3.
15:14:50 <sjmc7> since we don’t really have a proper noun to operate on, and we’re not doing CRUD type stuff
15:15:40 <sjmc7> yeah.. i’m sort of inclined to go with something now, and improve on it if we can get feedback from osc
15:15:48 <sjmc7> it’d be a shame for the patches to sit there for weeks
15:16:04 <sjmc7> if anyone wants to take a look at them, there are three patches - https://review.openstack.org/#/q/status:open+project:openstack/python-searchlightclient
15:16:36 <sjmc7> yingjun - i did run them all yesterday and they all work as expected, which is definitely good :)
15:16:55 <sjmc7> so i think if we can come to an agreement about naming then they’re in good shape
15:17:36 <yingjun> thanks
15:17:53 <TravT> yingjun
15:18:10 <TravT> we could potentially do some "eager" merging on them to make it easier to keep developing through
15:18:22 <TravT> and if needed change the commands a bit
15:18:40 <TravT> since this is still middle of initial release
15:19:25 <sjmc7> ok. next up, unless anyone else has anything on this topic
15:19:56 <sjmc7> #topic blueprint and spec review
15:20:00 <yingjun> TravT, seems good to me, I'll keep track on that
15:20:25 <sjmc7> on the subject of reviews, https://review.openstack.org/260504 is a real softball one to fix an ordering issue with the tox config
15:21:02 <sjmc7> there are three fairly big architectural specs floating aroudn at the moment
15:21:28 <sjmc7> https://blueprints.launchpad.net/searchlight/+spec/remove-index-api
15:22:10 <sjmc7> we have an /index API which seems kind of vestigial; it’s not documented anywhere, and of quite limited use
15:22:29 <sjmc7> i put up a review for this one while i was on a long plane flight - https://review.openstack.org/#/c/259551/
15:22:39 <sjmc7> i can’t see any benefit keeping it around in its current state
15:22:53 <TravT> yeah, i agree with ditching it
15:23:30 <TravT> it may have some value, but i don't think it has been fully thought through
15:23:56 <sjmc7> yeah, i think we can revive it to become “index a resource with this id"
15:23:56 <TravT> i just haven't had time to test through it since you put it up for potentially unintended consequences.
15:24:07 <sjmc7> rather than “index this raw document”
15:24:27 <sjmc7> all my consequences are intended!
15:24:53 <sjmc7> ok, a little more tricky - Zero down-time indexing, with a spec review at https://review.openstack.org/#/c/245222/
15:25:23 <sjmc7> this was one of the big things we decided on at the summit, so we need to get cracking with it
15:26:00 <TravT> yes, please give comments
15:26:11 <TravT> i'll put up another revision of it next week.
15:26:20 <TravT> i thought maybe rick was going to add something to it as well?
15:26:28 <sjmc7> yeah, i think if there are no further comments by mitaka-2 we’ll call it ready
15:26:39 <sjmc7> i’ll track him down too
15:26:42 <sjmc7> though no beers for him
15:26:51 <TravT> that seems wise
15:27:41 <sjmc7> ok, next up, reorganizing the publishing pipeline - https://review.openstack.org/#/c/246220/
15:28:20 <TravT> I'm good with all the concepts in there
15:28:35 <sjmc7> this was primarily to allow SL to resend notification data but after formatting it, and the specific use case being publishing notifications to tenant message queues
15:28:59 <TravT> i am fully aware that the coding will lead us to realize some things
15:29:16 <sjmc7> though more generally it would allow publishers to be plugged in (sort of middleware-like) in addition to the elasticsearch indexing operation we have now
15:29:22 <sjmc7> yeah, i think it’s potentially pretty gnarly
15:29:47 <TravT> especially as we get the zero downtime and role based stuff in.
15:29:57 <TravT> i think we should try to get zero downtime in and role based in
15:30:08 <TravT> and then build the code for this one on top of that
15:30:24 <sjmc7> yeah. i’ll take another look at this spec, and we can see if the folks from Intel are able/willing to work on it and give them help
15:30:25 <TravT> since they are higher priority.
15:31:04 <sjmc7> right, so the last big change (and this spec is already approved) is re-allowing search for admin fields
15:31:18 <sjmc7> there’s a patch up at https://review.openstack.org/#/c/257516/, but it will need some testing
15:31:43 <sjmc7> yingjun - since you’re doing a lot of client work it would be useful for you to run your server with this patch (you’ll need to reindex)
15:32:17 <sjmc7> the patchset looks scarily big but there’s not as much to it as the line count suggests
15:33:24 <TravT> thanks for getting that patch up sjmc7
15:33:28 <yingjun> ok, I'll do that tomorrow
15:33:30 <sjmc7> again, plane ride :)
15:33:37 <sjmc7> i should fly more often!
15:34:00 <TravT> it is funny how productive plane rides can be. :)
15:34:18 <sjmc7> especially when the movies are all 6 years old and on a screen 40 feet away
15:34:28 <sjmc7> i forgot my opera glasses
15:34:30 <TravT> old 747?
15:34:45 <sjmc7> smaller than that, but same vintage
15:35:07 <sjmc7> #topic bugs
15:35:13 <sjmc7> the good news is we don’t have many bugs filed
15:35:25 <sjmc7> the more serious one is that elasticsearch 2.0 causes us some problems
15:36:01 <sjmc7> lei-zh noted another one over the holiday (https://bugs.launchpad.net/searchlight/+bug/1529009)
15:36:02 <openstack> Launchpad bug 1529009 in OpenStack Search (Searchlight) "_id is no longer configurable in mappings of elasticsearch v2.0" [Medium,New]
15:36:12 <sjmc7> so we need to decide when we need to support 2.0
15:36:38 <sjmc7> i’m inclined to try to get things fixed for the mitaka release, or at least identify if we can support 1.0 and 2.0 together
15:36:51 <TravT> it seemed gordon on ceilometer wasn't looking at 2.0
15:37:18 <TravT> i thought monasca also was adding some elasticsearch stuff
15:37:20 <sjmc7> right. and i asked our internal team and they aren’t either
15:37:43 <TravT> the funny thing about this
15:37:44 <sjmc7> they’re looking at it. we’re sticking with 1.7 i think for the forseeable
15:38:09 <TravT> is that there is usually a lag in deploying releasees of openstack
15:38:14 <sjmc7> i don’t *think* there are that many changes; i’d at least like to identify what they are
15:38:18 <TravT> i know some deployers stay on top
15:38:30 <TravT> but i know a couple companies that lag by 6 - 12 months
15:38:49 <TravT> if that is true, then ES 2.0 will have been out for a year by the time they deploy searchlight
15:39:13 <TravT> so, not supporting 2.0 would be a bigger deal
15:39:45 <sjmc7> yeah.. although since debian’s dropping E-S from the package manegement, it’ll be more likely people will download and use the latest version
15:39:59 <TravT> which is a long way of saying yes, it would be great to at least identify 2.0 issues.
15:40:05 <sjmc7> i’ll make a note to track down any other issues so we at least know what we’re up against
15:40:17 <TravT> perhaps could be done as backport if needed post mitaka
15:40:19 <sjmc7> #action sjmc7 to identify any more elasticsearch 2.0 issues
15:40:39 <sjmc7> yeah. ok, that was all on the agenda
15:40:42 <sjmc7> #topic Open discussion
15:40:57 <sjmc7> anyone got anything else? otherwise we can talk about TravT’s knees
15:41:08 <GB21> I have small question
15:41:13 <sjmc7> uh oh
15:41:23 <sjmc7> what’s up? :)
15:41:51 <GB21> just work and trying to work with searchlight
15:41:56 <GB21> how about u?
15:41:57 <GB21> :D
15:42:09 <sjmc7> :)
15:42:32 <sjmc7> what’s the question?
15:43:17 <GB21> so, to index the nova flavors, what I understand is that I have to write a python script convert sql queries to elastic search
15:43:42 <sjmc7> well, we typically don’t have direct access to any of the service databases
15:43:45 <GB21> is that so, or do we already have the flavors indexed and I have to query them?
15:43:55 <sjmc7> so if you look at how nova servers are indexed, we make an API call to list them
15:44:35 <sjmc7> take a look at get_objects in the nova servers index class
15:45:21 <TravT> http://developer.openstack.org/api-ref-compute-v2.1.html#listDetailFlavors
15:45:51 <TravT> nova client will have a call to that api
15:46:13 <sjmc7> i’ll be around all the time in the searchlight IRC room, so feel free to get in touch if you need to
15:46:23 <GB21> ohk sure
15:46:24 <GB21> :D
15:46:27 <david-lyle_> Just making an offer, the horizon midcycle sprint is Feb 23-25 in Hillsboro, OR . We should have plenty of space if searchlight folks would like piggyback on it or just use as a chance to face to face. https://wiki.openstack.org/wiki/Sprints/HorizonMitakaSprint
15:46:44 <sjmc7> ah, thanks for the reminder, david-lyle_
15:46:55 <sjmc7> TravT - were you intending to go to that?
15:46:59 <TravT> thanks david-lyle_
15:47:19 <TravT> sjmc7: yes, i am planning to go
15:47:36 <TravT> i should be walking again by then
15:47:41 <sjmc7> ok. i’ll have a conflab with management and see if i can justify it
15:47:48 <sjmc7> ha, you’ll get more legroom in a brace!
15:47:57 <TravT> i'm still waiting for official approval as well.
15:48:00 <sjmc7> david-lyle_ can wheel you around
15:48:23 <david-lyle_> Finally something useful I can do
15:48:26 <TravT> nice!
15:48:57 <sjmc7> so yeah, if anyone’s around portland end of february, let us know
15:49:17 <sjmc7> anything else? otherwise i’ll call it here
15:49:48 <TravT> nothing for me.  thanks for running things today sjmc7
15:49:57 <TravT> i'll still mostly be offline the next two days
15:49:59 <david-lyle_> Thanks sjmc7
15:50:06 <sjmc7> np, heal up
15:50:13 <TravT> and trying to make a comeback next week
15:50:39 <sjmc7> don’t rush it
15:50:47 <sjmc7> ok, thanks folks for attending
15:51:58 <sjmc7> #endmeeting