15:01:35 #startmeeting openstack search 15:01:36 Meeting started Thu Jan 7 15:01:35 2016 UTC and is due to finish in 60 minutes. The chair is sjmc7. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:37 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:39 The meeting name has been set to 'openstack_search' 15:01:46 Morning folks 15:01:51 Hi 15:01:54 Good morning! 15:01:56 I’ll be MCing today, for a change 15:01:57 o/ 15:02:02 regular service will be resumed soon 15:02:14 hi everyone 15:02:21 morning all. i'm here, but asked sjmc7 to run the show today. 15:02:29 ++ 15:02:38 GB21: glad to see you at this meeting 15:02:55 likewise nikhil :D 15:02:55 #topic General updates 15:02:57 i'm still out recovering from knee surgery and just stepping in this morning 15:03:01 * david-lyle_ partially o/ 15:03:03 Happy new year to all 15:03:04 :D 15:03:05 thx nikhil 15:03:26 gws TravT 15:03:35 Meeting agenda at https://etherpad.openstack.org/p/search-team-meeting-agenda 15:04:00 The only general update i’ve got is the mitaka-2 milestone in the third week of Jan 15:04:27 so any outstanding blueprints/specs, especially the big ones, we should try to come to an agreement by then 15:04:42 anyone have anything else they wanted to bring up? 15:05:18 as sjmc7, we def need some review on the specs 15:05:54 i’ll list some of the BPs later 15:05:57 oh, i also noticed yesterday the elasticsearch devstack install was broken mid-December, but there was already a patch under review yesterday to fix it 15:06:22 ok, moving on.. 15:06:31 #topic cinder and swift plugins 15:06:52 not sure if lakshmi or duncan are here 15:07:13 duncant? lakshmiS? 15:07:42 duncant kept saying he had a patch ready to go. 15:07:54 i think somebody needs to go buy him a beer to get him to upload it 15:08:05 yeah. i’ll try and track him down later. ha, i need to negotiate that contract 15:08:34 on the subject of plugins, i took a look at neutron’s notifications 15:09:31 i added some notes to https://blueprints.launchpad.net/searchlight/+spec/neutron-plugins - i’m not sure how best to map neutron’s data. if anyone has any thoughts, please air them 15:10:35 it looks like the notifications are enough to get all the data, which is definitely a positive 15:10:39 i'll ping robcresswell on that as well 15:10:45 see if he has any thoughts there 15:10:51 ah yeah, that’s a good idea. i’ll track him down too 15:10:54 since at summit he said he had interest 15:10:58 this is gonna get expensive if i have to keep buying beers 15:11:15 why do you think i wanted you to run the show today? 15:11:19 ;-) 15:11:21 cunning! 15:11:50 the other plugin we spoke about at the summit was for Heat i think, which seems more straightforward 15:11:58 but i’ll take a look at those notifications as well today 15:12:23 ok, moving on… 15:12:38 #topic python-searchlightclient 15:13:09 li has done some really good work on this 15:13:28 so thankyou yingjun! 15:13:31 yingjun: \o/ 15:13:50 we definitely need to work through the commands a bit. 15:14:09 yeah, i should’ve looked at your etherpad earlier 15:14:24 no problem, the patches need more feedback;) 15:14:26 unlike most services which have more ‘RESTy’ looking openstackclient commands 15:14:32 we’re a little bit different 15:14:38 we probably could get some feedback from the OSC team and perhaps UX team on them before mitaka 3. 15:14:50 since we don’t really have a proper noun to operate on, and we’re not doing CRUD type stuff 15:15:40 yeah.. i’m sort of inclined to go with something now, and improve on it if we can get feedback from osc 15:15:48 it’d be a shame for the patches to sit there for weeks 15:16:04 if anyone wants to take a look at them, there are three patches - https://review.openstack.org/#/q/status:open+project:openstack/python-searchlightclient 15:16:36 yingjun - i did run them all yesterday and they all work as expected, which is definitely good :) 15:16:55 so i think if we can come to an agreement about naming then they’re in good shape 15:17:36 thanks 15:17:53 yingjun 15:18:10 we could potentially do some "eager" merging on them to make it easier to keep developing through 15:18:22 and if needed change the commands a bit 15:18:40 since this is still middle of initial release 15:19:25 ok. next up, unless anyone else has anything on this topic 15:19:56 #topic blueprint and spec review 15:20:00 TravT, seems good to me, I'll keep track on that 15:20:25 on the subject of reviews, https://review.openstack.org/260504 is a real softball one to fix an ordering issue with the tox config 15:21:02 there are three fairly big architectural specs floating aroudn at the moment 15:21:28 https://blueprints.launchpad.net/searchlight/+spec/remove-index-api 15:22:10 we have an /index API which seems kind of vestigial; it’s not documented anywhere, and of quite limited use 15:22:29 i put up a review for this one while i was on a long plane flight - https://review.openstack.org/#/c/259551/ 15:22:39 i can’t see any benefit keeping it around in its current state 15:22:53 yeah, i agree with ditching it 15:23:30 it may have some value, but i don't think it has been fully thought through 15:23:56 yeah, i think we can revive it to become “index a resource with this id" 15:23:56 i just haven't had time to test through it since you put it up for potentially unintended consequences. 15:24:07 rather than “index this raw document” 15:24:27 all my consequences are intended! 15:24:53 ok, a little more tricky - Zero down-time indexing, with a spec review at https://review.openstack.org/#/c/245222/ 15:25:23 this was one of the big things we decided on at the summit, so we need to get cracking with it 15:26:00 yes, please give comments 15:26:11 i'll put up another revision of it next week. 15:26:20 i thought maybe rick was going to add something to it as well? 15:26:28 yeah, i think if there are no further comments by mitaka-2 we’ll call it ready 15:26:39 i’ll track him down too 15:26:42 though no beers for him 15:26:51 that seems wise 15:27:41 ok, next up, reorganizing the publishing pipeline - https://review.openstack.org/#/c/246220/ 15:28:20 I'm good with all the concepts in there 15:28:35 this was primarily to allow SL to resend notification data but after formatting it, and the specific use case being publishing notifications to tenant message queues 15:28:59 i am fully aware that the coding will lead us to realize some things 15:29:16 though more generally it would allow publishers to be plugged in (sort of middleware-like) in addition to the elasticsearch indexing operation we have now 15:29:22 yeah, i think it’s potentially pretty gnarly 15:29:47 especially as we get the zero downtime and role based stuff in. 15:29:57 i think we should try to get zero downtime in and role based in 15:30:08 and then build the code for this one on top of that 15:30:24 yeah. i’ll take another look at this spec, and we can see if the folks from Intel are able/willing to work on it and give them help 15:30:25 since they are higher priority. 15:31:04 right, so the last big change (and this spec is already approved) is re-allowing search for admin fields 15:31:18 there’s a patch up at https://review.openstack.org/#/c/257516/, but it will need some testing 15:31:43 yingjun - since you’re doing a lot of client work it would be useful for you to run your server with this patch (you’ll need to reindex) 15:32:17 the patchset looks scarily big but there’s not as much to it as the line count suggests 15:33:24 thanks for getting that patch up sjmc7 15:33:28 ok, I'll do that tomorrow 15:33:30 again, plane ride :) 15:33:37 i should fly more often! 15:34:00 it is funny how productive plane rides can be. :) 15:34:18 especially when the movies are all 6 years old and on a screen 40 feet away 15:34:28 i forgot my opera glasses 15:34:30 old 747? 15:34:45 smaller than that, but same vintage 15:35:07 #topic bugs 15:35:13 the good news is we don’t have many bugs filed 15:35:25 the more serious one is that elasticsearch 2.0 causes us some problems 15:36:01 lei-zh noted another one over the holiday (https://bugs.launchpad.net/searchlight/+bug/1529009) 15:36:02 Launchpad bug 1529009 in OpenStack Search (Searchlight) "_id is no longer configurable in mappings of elasticsearch v2.0" [Medium,New] 15:36:12 so we need to decide when we need to support 2.0 15:36:38 i’m inclined to try to get things fixed for the mitaka release, or at least identify if we can support 1.0 and 2.0 together 15:36:51 it seemed gordon on ceilometer wasn't looking at 2.0 15:37:18 i thought monasca also was adding some elasticsearch stuff 15:37:20 right. and i asked our internal team and they aren’t either 15:37:43 the funny thing about this 15:37:44 they’re looking at it. we’re sticking with 1.7 i think for the forseeable 15:38:09 is that there is usually a lag in deploying releasees of openstack 15:38:14 i don’t *think* there are that many changes; i’d at least like to identify what they are 15:38:18 i know some deployers stay on top 15:38:30 but i know a couple companies that lag by 6 - 12 months 15:38:49 if that is true, then ES 2.0 will have been out for a year by the time they deploy searchlight 15:39:13 so, not supporting 2.0 would be a bigger deal 15:39:45 yeah.. although since debian’s dropping E-S from the package manegement, it’ll be more likely people will download and use the latest version 15:39:59 which is a long way of saying yes, it would be great to at least identify 2.0 issues. 15:40:05 i’ll make a note to track down any other issues so we at least know what we’re up against 15:40:17 perhaps could be done as backport if needed post mitaka 15:40:19 #action sjmc7 to identify any more elasticsearch 2.0 issues 15:40:39 yeah. ok, that was all on the agenda 15:40:42 #topic Open discussion 15:40:57 anyone got anything else? otherwise we can talk about TravT’s knees 15:41:08 I have small question 15:41:13 uh oh 15:41:23 what’s up? :) 15:41:51 just work and trying to work with searchlight 15:41:56 how about u? 15:41:57 :D 15:42:09 :) 15:42:32 what’s the question? 15:43:17 so, to index the nova flavors, what I understand is that I have to write a python script convert sql queries to elastic search 15:43:42 well, we typically don’t have direct access to any of the service databases 15:43:45 is that so, or do we already have the flavors indexed and I have to query them? 15:43:55 so if you look at how nova servers are indexed, we make an API call to list them 15:44:35 take a look at get_objects in the nova servers index class 15:45:21 http://developer.openstack.org/api-ref-compute-v2.1.html#listDetailFlavors 15:45:51 nova client will have a call to that api 15:46:13 i’ll be around all the time in the searchlight IRC room, so feel free to get in touch if you need to 15:46:23 ohk sure 15:46:24 :D 15:46:27 Just making an offer, the horizon midcycle sprint is Feb 23-25 in Hillsboro, OR . We should have plenty of space if searchlight folks would like piggyback on it or just use as a chance to face to face. https://wiki.openstack.org/wiki/Sprints/HorizonMitakaSprint 15:46:44 ah, thanks for the reminder, david-lyle_ 15:46:55 TravT - were you intending to go to that? 15:46:59 thanks david-lyle_ 15:47:19 sjmc7: yes, i am planning to go 15:47:36 i should be walking again by then 15:47:41 ok. i’ll have a conflab with management and see if i can justify it 15:47:48 ha, you’ll get more legroom in a brace! 15:47:57 i'm still waiting for official approval as well. 15:48:00 david-lyle_ can wheel you around 15:48:23 Finally something useful I can do 15:48:26 nice! 15:48:57 so yeah, if anyone’s around portland end of february, let us know 15:49:17 anything else? otherwise i’ll call it here 15:49:48 nothing for me. thanks for running things today sjmc7 15:49:57 i'll still mostly be offline the next two days 15:49:59 Thanks sjmc7 15:50:06 np, heal up 15:50:13 and trying to make a comeback next week 15:50:39 don’t rush it 15:50:47 ok, thanks folks for attending 15:51:58 #endmeeting