15:00:54 <TravT> #startmeeting openstack search
15:00:54 <openstack> Meeting started Thu Jan 28 15:00:54 2016 UTC and is due to finish in 60 minutes.  The chair is TravT. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:55 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:57 <openstack> The meeting name has been set to 'openstack_search'
15:01:41 <sjmc7> o/
15:01:45 <rosmaita> o/
15:01:47 <RickA-HP> o/
15:01:47 <TravT> o/
15:01:50 <yingjun> o/
15:01:58 <david-lyle_> o/
15:02:01 <lakshmiS> o/
15:02:04 <nikhil> o/
15:02:09 <TravT> hey, just a sec... my mouse just stopped working.  need to try something
15:02:28 <TravT> thx for coming today!
15:02:56 <TravT> brb
15:03:13 <RickA-HP> That was a quick meeting
15:04:39 <TravT> okay
15:04:42 <TravT> here's our agenda
15:04:43 <TravT> https://etherpad.openstack.org/p/search-team-meeting-agenda
15:04:57 <TravT> #topic Li Yingjun now core
15:05:06 <TravT> Congrats yingjun!
15:05:14 <RickA-HP> Great job!
15:05:15 <sjmc7> yay!
15:05:21 <rosmaita> congrats!
15:05:32 <lakshmiS> team is growing!
15:05:37 <yingjun> thanks, my honour to join the team!
15:05:39 <TravT> just for reference: http://osdir.com/ml/openstack-dev/2016-01/msg01544.html
15:05:56 <TravT> yingjun: I'll add you in gerrit and launchpad after this meeting.
15:06:18 <TravT> thanks for your hard work! we're very happy to have you!
15:06:22 <yingjun> :)
15:06:41 <david-lyle_> Change in can get behind
15:07:26 <TravT> #topic spec review day results
15:07:40 <TravT> Thanks for everybody that participated in spec review day
15:07:44 <TravT> it was very helpful
15:07:48 <TravT> we had some good discussion
15:07:58 <TravT> We approved the notification pipeline
15:08:09 <TravT> and it seem lei-zh already put up WIP patch
15:08:32 <TravT> we also had some good discussion on zero downtime and deletion journal
15:08:39 <TravT> with some follow up actions
15:09:00 <TravT> today, i was hoping that we could just continue live reviews / discussion on the specs
15:09:18 <TravT> i saw that RickA-HP has done a few modifications
15:09:23 <rosmaita> sounds good
15:09:34 <RickA-HP> Yes, I posted new updates for both specs
15:09:44 <TravT> so should we start with deletion journal or zero down time?
15:09:51 <RickA-HP> I hope I got everyones concenrs addresses :)
15:10:05 <RickA-HP> Let's start with Zero Downtime.
15:10:18 <TravT> #topic zero down time reindexing
15:10:26 <TravT> #link https://review.openstack.org/#/c/245222/
15:10:54 <TravT> i think we should all just take a minute to look through the changes
15:10:59 <TravT> and then we can discuss
15:12:03 <TravT> it is worth noting that you can read the rendered version with inline images from the build output here: http://docs-draft.openstack.org/22/245222/11/check/gate-searchlight-specs-docs/6038feb//doc/build/html/specs/mitaka/zero-downtime-reindexing.html
15:12:38 <rosmaita> TravT: how do you find that?
15:13:03 <TravT> gate-searchlight-specs-docs
15:13:12 <TravT> under the jenkins check
15:13:12 <rosmaita> ty
15:15:01 <sjmc7> one thing i mentioned to TravT last night, while i was being self pitying for being sick, is that in the situation of a change to code (like a new plugin being added, or an existing one’s data being changed in an incompatible fashion), the listener cannot pump data into both indices
15:15:38 <sjmc7> the choice there is to discontinue indexing into one index (or both), or run two listeners
15:16:21 <TravT> yes, sjmc7 brought up a very great point.  we still need to design out an overarching deployment workflow for when new code with incompatible changes needs to be rolled out.
15:16:32 <RickA-HP> sjmc7: In your latter case, do you mean the mapping has major changes to it or something else?
15:16:47 <sjmc7> yeah, something like that
15:17:20 <TravT> this spec primarily covers re-indexing without code rollout... which is okay i think. but something to consider
15:17:25 <RickA-HP> Should this use case be included in this spec, or should we create a new spec? Like we did for the deletion journal?
15:17:36 <sjmc7> i don’t think it needs including, but it’s worth bearing in mind
15:17:47 <RickA-HP> It's worth not forgetting!
15:17:56 <rosmaita> +1
15:17:56 <TravT> RickA-HP: I'm not 100% sure we need a spec for this, so much as we need documentation on deployment and upgrades.
15:18:12 <TravT> and based on that we might need specs for some feature to help support it
15:18:14 <sjmc7> in particular, that there will be cases when it doesn’t make sense to try to continue indexing into both, and if there are significant difficulties implementing that, it doesn’t necessarily need to be a blocker
15:18:28 <rosmaita> maybe add an "out of scope" section
15:18:58 <sjmc7> yeah, that’d be fine. or even in the docs as part of the change
15:19:23 <TravT> yes, agreed, that's what I meant by need documentation
15:19:48 <RickA-HP> sjmc7: Which change? Do you mean the implementation?
15:19:48 <TravT> but spending a little time thinking about that to ensure we don't run off and implement something that makes the upgrade rollout scenario impossible would be good
15:19:58 <sjmc7> yes
15:20:23 <RickA-HP> I'd rather document it now before we forget about it.
15:20:28 <TravT> RickA-HP: This could even be its own blueprint (document deployment and upgrade workflow)
15:20:38 <TravT> which wouldn't be a spec
15:20:53 <TravT> it would be https://github.com/openstack/searchlight/tree/master/doc/source
15:21:09 <TravT> which gets published here: http://docs.openstack.org/developer/searchlight/
15:21:14 <RickA-HP> Aah. I was using spec = blueprint.
15:23:30 <RickA-HP> TravtT: That github seems to be for users, not really for developers. Will it get lost if we place it there.
15:24:08 <RickA-HP> In any case, whereever the team thinks we should document this I'll go add it.
15:24:26 <sjmc7> maybe adda note to the spec
15:24:37 <sjmc7> and we’ll note in the documentation that this case isn’t covered
15:25:00 <TravT> i'm actually not 100% sure of all the intended audiences of our in tree documentation, but developers are definitely a primary target
15:25:21 <sjmc7> well, and admins
15:25:37 <TravT> david-lyle: rosmaita: do you know if there is some "what should be and what shouldn't be" in our in tree docs?
15:25:38 <sjmc7> i don’t want to keep holding up the spec for things i think of 2 months late though :)  let’s add a brief note to it
15:25:49 <RickA-HP> To stop the rat-holing I'll add it in both places Steve suggested :)
15:26:00 <TravT> yes, a brief note on it in spec is all that is needed for now
15:26:08 <TravT> and we can document in tree
15:26:21 <rosmaita> TravT: no, don't think there is
15:26:50 <TravT> that where people go afaik for base information.  There are operator guides, but i think they'd base anything from our in tree docs
15:27:04 <david-lyle_> No.
15:27:21 <david-lyle_> Up to us what it contains
15:31:55 <sjmc7> anything else on this one? i’m ok with the content
15:33:00 <TravT> I just added a minor nit and the note you mentioned, but did you do that as well?
15:33:18 <sjmc7> no, my laziness worked for me for once
15:33:20 <TravT> otherwise, i don't see anything else
15:34:31 <rosmaita> i have no objections
15:34:42 <TravT> david-lyle: nikhil: yingjun: lakshmiS: need more time or can we move on to the deletion journal?
15:34:59 <yingjun> no problem from me
15:35:00 <lakshmiS> i will spend time later on this
15:35:05 <TravT> this spec is an eye burner, that's for sure
15:35:53 <TravT> certainly feel free to continue on it as well.  we'll just move on to deletion journal
15:35:56 <TravT> #topic Deletion journaling spec
15:36:08 <TravT> #link https://review.openstack.org/#/c/269783/
15:36:45 <sjmc7> and both this, and the versioning one, are a pre-req for reindexing?
15:38:09 <TravT> versioning addresses a different bug... but i think is important to reindexing
15:38:14 <sjmc7> rosmaita: did rick clear up your questions with this one?
15:38:38 <rosmaita> sjmc7: need to look at latest changes
15:38:58 <rosmaita> i think the only question i had left is how the TTL will e managed
15:39:01 <rosmaita> *be
15:39:28 <sjmc7> as in how long?
15:39:33 <sjmc7> or when the reaper will run?
15:39:36 <RickA-HP> sjmc7: In terms of implementation we can still code the reindexing without the deletion, but we do not want to release the reindexing with the deletion.
15:39:54 <RickA-HP> So I don't know if the deletion is a strict pre-req for reindexing.
15:40:15 <rosmaita> sjmc7: both
15:40:41 <sjmc7> for how long, it’s really only as long as we might expect to process redundant notifications
15:40:52 <RickA-HP> rosmaita: Is that something that is needed  for the blueprint?
15:40:53 <sjmc7> so, for instance, if you shutdown a nova server, you get a load of notifications
15:41:05 <sjmc7> but there’s probably only a few seconds window where it’s a problem
15:41:26 <sjmc7> that’s a good point - it might be a documentation item rather than something needs to be in the spec
15:41:29 <rosmaita> RickA-HP: i may be wrong, but i see it as the key link between the 2 specs
15:42:34 <rosmaita> well, if the window is only a few sec, then i guess not a problem
15:42:40 <RickA-HP> rosmaitia. Ok. In the past when I've used TTL it seems that it gets designed with one value. But after coding it up and using it the value changes!
15:43:20 <rosmaita> i guess my question is really about using TTL in this way instead of controlling via the garbage collector
15:43:20 <RickA-HP> There was always an emperical-ness to TTLs.
15:43:22 <sjmc7> even if we let documents sit for ages, it shouldn’t be an issue. we wouldn’t expect notifications for those documents to arrive (since they’re meant to have disappeared from the cloud)
15:43:50 <sjmc7> the diff between the GC is that to indexing, the document has actually gone, even if it’s not been GCed
15:44:10 <sjmc7> so lucene marks the doc as deleted, but a reindexing operation will create a new one
15:44:19 <rosmaita> ok, i will just note that my ES knowledge is vastly exceeded here, and will rely on y'all
15:44:24 <sjmc7> with TTL, it’s still there, so we can detect that we’re processing a notification for an outdated document
15:44:51 <rosmaita> sjmc7: ok, that makes sense
15:45:09 <sjmc7> it’s not really e-s specific. the key thing is that instead of removing the doc for server ABC, we mark it such that notifications received from the past are ignored
15:46:34 <rosmaita> yeah, i forgot again about PUT creating a doc instead of 404ing
15:46:38 <sjmc7> i’m not trying to railroad you though - if it still doesn’t make sense, i may be missing something
15:46:47 <rosmaita> ok, i'm good
15:46:50 <sjmc7> yeah, that’s the heart of the issue
15:47:18 <sjmc7> it’s kind of a special case of ensuring we don’t process notifications of events older than ones we’ve already done
15:48:02 <rosmaita> i guess my only objection at this point is that latin and french phrases should be in italics
15:48:09 <rosmaita> :)
15:48:10 <sjmc7> yes.. or perhaps removed
15:48:20 <TravT> +1
15:48:37 <rosmaita> no, don't remove ... how many other specs refer to folie a deux?
15:48:39 <TravT> So, i did put up some comments right now
15:48:52 <sjmc7> :)
15:48:53 <TravT> I think specs and code could all use a little more humor
15:49:02 <sjmc7> we do use python, after all
15:49:11 <TravT> just humor at a 3rd grade reading level that somebody like me can understand. :)
15:49:29 <rosmaita> fart!
15:49:33 <sjmc7> ah, the benefits of a classical education
15:49:35 <RickA-HP> TravT: Feel free to have your daughter explain the spec to you
15:49:36 <TravT> LOL
15:50:21 <TravT> i did manage to get some star wars references into some test data in horizon.
15:50:23 <sjmc7> one other thing if we’re done with that one - there are a number of reviews (mostly mine) that are a little gnarly, but are important architecturally (and force rebases of other stuff)
15:50:42 <TravT> i don't think we're quite done with that topic...
15:50:46 <sjmc7> ok
15:51:26 <TravT> Is it safe to say that all that is potentially needed on deletion journal is the very minor clarification i just suggested on line 49?
15:51:29 <TravT> or are there others?
15:51:33 <sjmc7> i’m ok with the spec now, i think - it’s clear enough to start implementing
15:52:05 <sjmc7> no, i’m good
15:52:36 <TravT> rosmaita:
15:52:37 <TravT> ?
15:52:38 <rosmaita> i'm good too
15:52:58 <TravT> okay, well, if RickA-HP makes the minor change I'm happy to +2 as well.
15:53:17 <TravT> i just feel it is an important difference that could confuse somebody else coming in
15:53:33 <TravT> #topic sjmc7 needs some reviews!
15:53:36 <RickA-HP> Ok. I'll update both blueprints with your comments incorporated later today.
15:54:10 <sjmc7> yeah.. i get that some of these reviews are a bit intimidating (sorry) - if it would help people, i can talk through bits of them rahter than ping pong comments on gerrit
15:54:12 <RickA-HP> sjmc7: I'm in the middle of reviewing the user role code changes.
15:54:20 <RickA-HP> It's slow going!
15:54:43 <sjmc7> excellent, thanks. yeah, i know; i ended up refactoring some stuff because i was copy n pasting
15:54:53 <sjmc7> perhaps would’ve been better to do that separately
15:55:11 <TravT> was that to address the -1 i gave?
15:55:44 <TravT> sjmc7: i think https://review.openstack.org/257516, https://review.openstack.org/267864, https://review.openstack.org/236043 are your reviews we really should get through now, right?
15:56:16 <sjmc7> right. the -1.. let me look it up. i thought i’d replied
15:56:36 <TravT> you probably did
15:56:38 <sjmc7> ah, the try except
15:56:49 <TravT> i was up until 1 AM writing angular code.
15:56:51 <TravT> so maybe i missed it
15:57:11 <sjmc7> no , imust’ve not hit the reply button. i will reply, although i don’t think it was an issue, but have another look
15:57:27 <TravT> #topic open discussion
15:57:33 <TravT> anything else?
15:57:45 <RickA-HP> Nothing from me.
15:57:50 <sjmc7> i’m hungry
15:58:01 <rosmaita> i need more coffed
15:58:05 <rosmaita> *coffee
15:58:10 <TravT> yeah, i'm on energy drinks again today
15:58:13 <TravT> bad sign
15:58:15 <sjmc7> you need to cut those out
15:58:17 <rosmaita> i could be better coiffed, too
15:58:19 <sjmc7> hahahaha
15:58:25 <sjmc7> you’re always immaculately coiffed
15:58:56 <yingjun> nothing from me, i’m going to need some sleep now..
15:59:01 <TravT> allright friends, thanks for your time.
15:59:05 <sjmc7> :)  g'night
15:59:07 <sjmc7> thanks folks
15:59:11 <rosmaita> bye!
15:59:21 <TravT> #endmeeting