18:00:32 <SlickNik> #startmeeting trove
18:00:32 <openstack> Meeting started Wed Mar 18 18:00:32 2015 UTC and is due to finish in 60 minutes.  The chair is SlickNik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:36 <openstack> The meeting name has been set to 'trove'
18:00:48 <peterstac> o/
18:00:52 <georgelorch> o/
18:01:05 <sushilkm> hello @all
18:01:17 <SlickNik> Agenda for today's meeting is at:
18:01:20 <SlickNik> #link https://wiki.openstack.org/wiki/Meetings/TroveMeeting
18:01:23 <mattvd> o/
18:01:31 <schang> o/
18:01:33 <dougshelley66> o/
18:01:46 <vgnbkr> o/
18:02:18 <vkmc> o/
18:02:19 <pmalik> o/
18:02:59 <jodah> o/
18:03:20 <SlickNik> Pretty full agenda today, let's get started.
18:03:28 <SlickNik> #topic Trove pulse update
18:03:37 <SlickNik> #link https://etherpad.openstack.org/p/trove-pulse-update
18:04:42 <SlickNik> A lot more patchsets this week than last week.
18:04:59 <SlickNik> Understandable, given the feature freeze is this week.
18:05:32 <SlickNik> However, fewer reviews than last week.
18:06:54 <SlickNik> So we need to step up the reviews — especially over the next day to make sure that things that are in the pipeline for feature freeze make it through.
18:07:34 <SlickNik> Thanks to all the folks who are reviewing code!
18:08:16 <SlickNik> Any questions / comments?
18:08:51 <SlickNik> Let's move on.
18:09:09 <SlickNik> #topic Openstack Feature Freeze for kilo-3
18:10:27 <SlickNik> So ttx is planning to cut the kilo-3 release tomorrow which means that we will be in Feature Freeze after that.
18:11:01 <SlickNik> So no new features will be able to make it into the kilo release after that (without an exception).
18:11:19 <SlickNik> Currently this is what we have planned for release in kilo-3: https://launchpad.net/trove/+milestone/kilo-3
18:12:07 <SlickNik> #link https://launchpad.net/trove/+milestone/kilo-3
18:12:11 <SlickNik> There are a number of reviews for BPs that are still in the pipeline and need reviews.
18:12:20 <dougshelley66> SlickNik, so if the code for the 6 BPs isn't merged by tomorrow it will need an FFE to be put in after that?
18:12:20 <vgnbkr> So what about patchsets that are fine but can't get past the gate because it's too slow?
18:13:45 <SlickNik> Well, if it can't make it past the gate, it can't go in — so it won't make it through.
18:14:47 <johnma> isnt the gate-trove-functional-dsvm-mysql tests failing currently?
18:14:53 <dougshelley66> we need amrith's commit to merge in infra https://review.openstack.org/#/c/165161/
18:14:57 <dougshelley66> anyway to help that along?
18:15:06 <sushilkm> the tests are failing for the timeout
18:16:11 <vkmc> if there is a feature that cannot go through because of the gate
18:16:12 <SlickNik> Yes, it looks like occasionally we will get a slow instance causing the tests to timeout.
18:16:15 <vkmc> could we ask for a FFE?
18:17:00 <SlickNik> I pinged infra earlier this morning about getting some eyeballs on https://review.openstack.org/#/c/165161/
18:17:06 <amrith> ./
18:17:07 <dougshelley66> SlickNik, thx
18:17:09 <SlickNik> Will have to ping again after this meeting.
18:17:33 <amrith> sorry, what did I break?
18:17:45 * amrith reads scrollback quickly
18:17:50 <dougshelley66> amrith, nothing - just trying to get your timeout fix merged
18:18:09 <amrith> oh, I didn't break something. break out the champagne!
18:18:48 <SlickNik> vkmc: potentially yes, we could ask for a FFE. But we'll have to go through the FFE process and send out an email — and it's our / ttx's call at that point.
18:19:03 <dougshelley66> SlickNik, do you know when ttx is planning to cut kilo-3 (i.e. time of day)
18:19:05 <vkmc> SlickNik, makes sense
18:19:47 <SlickNik> dougshelley66: He was looking to do it before end of day UTC, so around noon PDT tomorrow.
18:20:00 <SlickNik> before*
18:20:18 <dougshelley66> ok thx
18:20:25 <dougshelley66> so we have about 24 hours
18:21:00 <SlickNik> Yes, that sounds about right — after which we can go the FFE route if needed.
18:21:05 <SlickNik> Hopefully we don't have to do that.
18:21:28 <SlickNik> So looking at https://etherpad.openstack.org/p/TroveKilo3Blueprints
18:21:57 <johnma> if we go the FFE route do we know when's the next code freeze going to be
18:23:24 <SlickNik> johnma: The earlier it is after the cut date (i.e. tomorrow) when the feature is ready to merge — the more likely that the exception will be granted.
18:23:32 <dougshelley66> looks like rc-1 is April 9
18:24:05 <SlickNik> If we get into mid-next week or so and the feature is not mergeable by then, it becomes more and more unlikely that we'll take the exception.
18:24:32 <johnma> ok, thanks SlickNik
18:24:41 <SlickNik> So at this point I have:
18:25:08 <SlickNik> Close to merge:
18:25:09 <SlickNik> - Guest Agent RPC Ping Pong Mgmt API
18:25:09 <SlickNik> - Create a CouchDB plugin for Trove
18:25:47 <SlickNik> Already has reviews and few hours to merge:
18:25:47 <SlickNik> - Add Vertica datastore support in trove
18:26:56 <SlickNik> Still needs reviews / gate issues and will take longer:
18:26:56 <SlickNik> - Replication V2
18:26:57 <SlickNik> - Implement Vertica Cluster provisioning for Vertica Datastore
18:28:32 <SlickNik> And then there's also a FFE already that has been asked by johnma wrt the DB2 patches
18:28:41 <SlickNik> #link https://blueprints.launchpad.net/openstack/?searchtext=db2-plugin-for-trove)
18:29:17 <SlickNik> I'd like to focus on getting the ones that are close to merge merged (shouldn't take too long).
18:29:29 <SlickNik> I'll go work with infra after this to try and iron out the gate issue.
18:29:50 <peterstac> I'll run through some tests with Vertica, if the dust has settled on that one
18:30:04 <peterstac> Plus I can probably do CouchDB at the same time
18:30:17 <johnma> greatly appreciate that Peter.
18:30:20 <sushilkm> thanks @peterstac
18:30:39 <sushilkm> though would like to suggest its the same more or less :)
18:30:41 <amrith> SlickNik, what's your thought re: replication v2?
18:30:41 <SlickNik> But we will need reviews and folks to look at the other patches.
18:30:59 <amrith> what do you propose re: that, and vertica cluster?
18:32:23 <SlickNik> amrith: I'd like to iron out the gate issues — however, I'm concerned that it only got pushed up 4-5 days ago and hasn't had enough folks look at it and review.
18:33:18 <amrith> what is the process for an FFE?
18:33:19 <SlickNik> So maybe the best course here is to apply for a FFE for it so that folks can get some time to review.
18:33:26 <amrith> who should do it etc?
18:33:52 <SlickNik> #link https://wiki.openstack.org/wiki/FeatureFreeze
18:34:47 <amrith> ok, maybe we can discuss this after the meeting, no point holding up mtg for this.
18:34:53 <SlickNik> The author of the patch sends out an email to the ML to get an exception.
18:35:00 <SlickNik> amrith: sounds good.
18:35:03 <johnma> I just did one yesterday. not sure if I wrote it correctly but I pretty much mentioned the link to the blueprint and the links to the patchset
18:35:36 <SlickNik> #topic Switch from oslo namespace packages to "oslo_" style modules
18:35:42 <johnma> and we need follow the instructions in the link SlickNik just sent
18:36:07 <SlickNik> So this came up on the ML, and we have some more information about it now.
18:36:30 <SlickNik> For context see: http://lists.openstack.org/pipermail/openstack-dev/2015-March/059031.html
18:36:30 <sushilkm> so every owner of patchset needs to write that
18:36:34 <SlickNik> #link http://lists.openstack.org/pipermail/openstack-dev/2015-March/059031.html
18:37:32 <amrith> SlickNik, some guy had a patch set about this.
18:37:38 <SlickNik> It seems like 1. The oslo namespace packages will definitely go away soon, and 2. using them means that we'd get "deprecated" error messages in our logs.
18:37:42 <amrith> and I'd suggested he also make a hacking rule to prevent creep
18:37:53 <amrith> and make sure these old references dont come in
18:38:00 <amrith> I think he abandoned his patches yesterday.
18:38:11 <amrith> do we want to do this for kilo or liberty?
18:38:35 <SlickNik> I think this seems like something we can take before RC1 — it's a bugfix with not much impact.
18:38:52 <amrith> and a hacking rule
18:39:04 <SlickNik> yes.
18:39:07 <amrith> or a test
18:39:10 <amrith> as the case may be
18:39:15 <amrith> either would work.
18:39:46 <SlickNik> Yes, that should be good.
18:39:59 <SlickNik> Folks okay with that?
18:40:42 <amrith> q?
18:40:47 <SlickNik> I'll take the silence as a yes :)
18:41:03 <amrith> is the move trove guest to its own module going to be in Kilo?
18:41:05 <amrith> schang?
18:41:16 <amrith> the only thing I'd wonder is about the order in which we do this
18:41:17 <schang> amrith: yes
18:41:24 <amrith> there's that change from schang.
18:41:37 <amrith> there are all the guest agents (new) that are gobs of new code
18:41:47 <amrith> and I have some oslo gyrations which also change imports
18:42:03 <schang> I'm still working on it, moved the common code out and now the int-test broke.
18:42:12 <amrith> if we can do these in some pre agreed order, we can minimize rebase nightmares
18:43:58 <SlickNik> So I'm hearing that this will most likely be Liberty then.
18:44:38 <amrith> SlickNik, please clarify what the *this* in your sentence above refers to.
18:44:40 <schang> amrith: sorry, my previous yes was responding to your ping. I think the guest agent refactoring will be for Liberty.
18:44:53 <SlickNik> this = move trove guest to its own module
18:44:58 <amrith> th
18:44:59 <amrith> thx
18:45:23 <SlickNik> np
18:45:27 <SlickNik> #topic Come up with a strategy to deal with Mock()ing utility methods
18:46:10 <SlickNik> #link https://review.openstack.org/#/c/156486/
18:47:08 <amrith> #link https://bugs.launchpad.net/trove/+bug/1398966
18:47:10 <openstack> Launchpad bug 1398966 in Trove "the test method of Mock()'ing system module entry points (like os.unlink) has thread safety issues" [Undecided,In progress] - Assigned to Amrith (amrith)
18:48:02 <peterstac> I'll give the background on this quickly
18:48:37 <peterstac> (it was important as it seemed to affect several patchsets for BPs)
18:48:54 <peterstac> (but it also seems to have been addressed somewhat)
18:49:04 <peterstac> mocking out certain calls, such as execute_with_timeout, can cause unwanted behaviour
18:49:20 <peterstac> as they are used in various places.  Since all calls now use the mock'ed method, things can get unpredictable
18:49:36 <peterstac> Amrith posted one way to fix this in #link https://review.openstack.org/#/c/138719/
18:49:51 <peterstac> This solves the problem, however necessitates changing production code just to facilitate testing
18:50:10 <peterstac> There may be other ways to reduce the impact:
18:50:21 <peterstac> 1. Having more utility helper methods that call execute_with_timeout for common actions (i.e. move_file, chmod)
18:50:28 <peterstac> 2. Putting the action into a private method (even if only called once)
18:50:37 <peterstac> Both these would allow mock'ing out just the method in question
18:51:27 <SlickNik> If you're mocking these calls if you make sure you mock it only for the current method (using a @patch header), or using a context manager ("with" statement) you should be safe, I think.
18:51:29 <peterstac> It looks like the Vertica patchset has used option #2
18:52:19 <amrith> SlickNik, NO!
18:52:24 <amrith> @patch is just a decorator
18:52:26 <amrith> it does the same thing
18:52:28 <amrith> process wide
18:52:30 <johnma> db2 patchset also uses #2, so my vote is for that. I did that precisely to help with testing
18:52:42 <SlickNik> yes, but we don't run more than one method at the same time in the process.
18:52:54 <amrith> Python supports no thread specific mock() capability.
18:53:04 <amrith> we don't run more than one, but the test harness can run multi-threaded.
18:53:19 <SlickNik> Yes, but each thread runs its own python process.
18:53:33 <amrith> SlickNik, remember the issue with mongodb rename?
18:53:35 <SlickNik> So mocking it out in one process cannot affect another.
18:53:38 <amrith> which failed intermittently
18:53:45 <amrith> that was this same mock() issue
18:54:02 <SlickNik> The problem there was that one method was replacing it with a Mock, and never putting back the original.
18:54:18 <peterstac> I've been doing some tests with mock and the behaviour seems limited to the thread (and child threads) of where the mock'ing occurs
18:54:28 <SlickNik> The tests were relying on an implicit ordering of the methods to run which was not true in that case.
18:54:45 <amrith> let me find the chat on #openstack-oslo about this.
18:55:14 <peterstac> So one thread mock'ing out a method isn't reflected in another thread
18:55:20 <amrith> SlickNik, the specific case mentioned in the bug with os.unlink() should never happen in that case.
18:55:58 <amrith> There (when I entered this bug) I made one call to a method that called os.unlink() once
18:56:05 <amrith> and a subsequent assert on calledOnce failed
18:56:12 <amrith> as it felt os.unlink() was called twice.
18:56:48 <peterstac> Either way - I didn't mean to have a discussion right now about this (we don't have the time)
18:57:00 <amrith> the change for write_config() was made based on doug's recommendation and others on #openstack-oslo (who know a ton more python than I do)
18:57:11 <peterstac> I just wanted to know what we should do wrt the patchsets in flight at the moment
18:57:22 <SlickNik> amrith: +1 would be good to find out more about this discussion and get to the bottom of this.
18:57:34 <peterstac> I.E. defer this until early Liberty and revist it?
18:59:13 <SlickNik> peterstac: sure we use mocks with context managers, and @patch decorators pretty much all over the code (in trove and other OpenStack projects) — so I think that if this were an issue, it would have a much bigger impact.
18:59:58 <SlickNik> But would be good to follow up on the discussions and figure out the details on what the underlying cause really is.
19:00:29 <SlickNik> It seems like the current BPs have mitigations for it, which is fine for the moment.
19:01:25 <peterstac> Ok, so for now "with patch.object(object, methoc, etc. )" is good
19:01:33 <peterstac> *method*
19:02:48 <SlickNik> That was my understanding — but I think amrith has some info that might suggest that's not the case.
19:03:16 <amrith> yes, that was my understanding and therefore the change for write_config() in earlier patch.
19:03:17 <SlickNik> Also, we're out of time — sushilkm can we move your topic to the next meeting?
19:03:35 <sushilkm> ok lets discuss it next meet
19:03:47 <SlickNik> sushilkm: thanks!
19:04:10 <SlickNik> Let's continue the discussions in #openstack-trove.
19:04:14 <SlickNik> Thanks all!
19:04:17 <SlickNik> #endmeeting