18:00:09 <cp16net> #startmeeting trove
18:00:10 <openstack> Meeting started Wed Jan  6 18:00:09 2016 UTC and is due to finish in 60 minutes.  The chair is cp16net. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:11 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:13 <openstack> The meeting name has been set to 'trove'
18:00:26 <amrith> hello y'all
18:00:35 <cp16net> howdy yall
18:00:59 <cp16net> i'll give a few minutes and update the pulse...
18:01:01 <peterstac> o/
18:01:13 <vkmc> o/
18:01:23 <imandhan> o/
18:01:33 <mvandijk> o/
18:01:34 <schang> o/
18:01:36 <atomic77> o//~~
18:01:51 <dloi> o/
18:02:35 <elmiko> hi
18:02:35 <cp16net> hmm looks like my script is failing atm for stats
18:02:42 <cp16net> guess i need to look at fixing it..
18:02:54 <cp16net> #action cp16net fix stats script
18:03:00 <amrith> please submit a blueprint
18:03:20 <amrith> in duplicate
18:03:22 <cp16net> i'll make sure to do that :-P
18:03:48 <cp16net> happy new year everyone
18:03:52 <atomic77> hmm i noticed one of my gerrit cli utilities broke as well
18:04:00 <cp16net> hope everyone is ready for an awesome new year! :)
18:04:20 <cp16net> oh...
18:04:30 <cp16net> i bet its related to the gerrit update then
18:04:42 <vgnbkr> o/
18:04:46 <peterstac> yeah, I'm not overly impressed with the latest gerrit version
18:04:47 <atomic77> yeah most likely
18:05:01 <cp16net> we'll skip the pulse update then this week and i'll have it for next week
18:05:17 <cp16net> i'll post it in the channel when it get fixed today
18:05:43 <cp16net> #topic trove-dashboard project
18:06:12 <cp16net> so fyi we have a separate repo now for our trove dashboard
18:06:19 <cp16net> its a plugin for horizon
18:06:39 <cp16net> this should allow us to move along a faster on making changes for trove's ui in horizon
18:07:12 <cp16net> dloi: brought up yesterday where we should post bp and bugs for the new dashboard project
18:07:25 <cp16net> i've found 2 options
18:07:40 <cp16net> 1. a new lp project for trove-dashboard
18:08:06 <cp16net> 2. use trove lp and mark the bugs/bp in there with a tag of dashboard
18:08:25 <cp16net> #2 is what a few of the other projects are doing
18:08:33 <cp16net> (sahara and designate)
18:09:09 <peterstac> Hmm, I would have thought a new lp project would make sense (like we have for python-troveclient)
18:09:10 <cp16net> from what i've seen they use the project "[ui]: bug/bp name"
18:09:19 <amrith> cp16net, is trove-dashboard not already a new repository?
18:09:28 <cp16net> it is already a new repo
18:09:29 <amrith> so wouldn't #1 be the logical choice?
18:09:41 <amrith> is there a benefit to 2? I don't see one.
18:09:45 <amrith> sounds like a hack to me.
18:09:48 <cp16net> https://github.com/openstack/trove-dashboard
18:09:53 <amrith> but I may not be fully informed
18:10:09 <cp16net> i was thinking about what the draw backs for the each were
18:11:10 <vkmc> I agree with amrith here... I don't see the point of tagging if we can have a new lp project
18:11:13 <cp16net> and i'm leaning on #2 because it makes sense to keep the bp that we create (i.e. grow cluster datastore x)
18:11:23 <cp16net> then it would have a task to update the ui
18:12:01 <cp16net> rather than creating a separate project and manage another proejct in lp for the dashboard
18:12:52 <cp16net> logicvally it might make sense to do option #1
18:12:59 <amrith> but don't we already do that for python-troveclient?
18:13:02 <vkmc> we already do that for trove-pythonclient and, IMO, you know that everything under trove-dashboard is for the UI
18:14:03 <cp16net> right the client is managed separately which i've seen how the client gets forgotten about
18:14:41 <cp16net> we manage most of the client bp stuff in trove as it is today
18:14:58 <cp16net> with the spec that is created for trove
18:15:33 <cp16net> i see the dashboard specs being similar in nature
18:15:37 <tellesnobrega> i agree with cp16net on this, it will be one more place to track, its easier to tag on the main project unless we have a major problem with that
18:15:42 <amrith> I guess to put the blunt question, how's dashboard different from CLI?
18:16:06 <amrith> I have a major problem with it.
18:16:16 <cp16net> amrith: i'd say not much
18:16:16 <amrith> we are relying on people putting text in a text field
18:16:28 <amrith> which can't be tracked/guaranteed/ensured
18:17:06 <amrith> and then there's the issue of control
18:17:17 <imandhan> Is there a built in tag on lp that we could use instead of the [ui] in front of the name of bug/bp?
18:17:18 <vkmc> that ^
18:17:18 <amrith> the dashboard has, in theory, joint control with horizon-core
18:17:37 <cp16net> imandhan: you can make up any tag you want
18:17:41 <amrith> therefore we would have to come up with a way to say that horizon-core can be bug managers for bugs with [dashboard] in the description.
18:17:52 <amrith> or [dashboard] as a tag
18:18:01 <amrith> seems kludgy to me.
18:18:07 <cp16net> tracking it with the dashboard tag would help aggregasting them
18:20:06 <vkmc> let's vote?
18:20:15 <cp16net> anyone else have an opinion?
18:20:24 <cp16net> i heard 2-2 right now
18:20:54 <pmackinn> seems like ui+impl changesets would be common
18:21:02 <pmackinn> unless we link them for #1
18:21:10 <amrith> let's hear from someone who would be making ui changes.
18:21:13 <amrith> dloi?
18:21:33 <dloi> I have a couple of questions first
18:22:03 <dloi> are we proposing putting UI bp descriptions in the trove specs as well?
18:22:35 <dloi> For example this is the minimal spec we have to do in horizon https://blueprints.launchpad.net/horizon/+spec/trove-support-cluster-grow-shrink
18:23:30 <cp16net> i think that would be ideal because to have a complete solution it would require the ui as well
18:24:02 <imandhan> Is it possible to have them in a separate repo (#2) and have them automatically show up with a [dashboard] tag in the trove lp page as well?
18:24:13 <cp16net> the ui changes can be done by someone else from the feature code in trove
18:24:48 <vgnbkr> The changes due to a trove spec can already be broken up amongst several people.
18:24:59 <cp16net> depeding on how you configure gerrit to understand the project bug number it would work
18:25:17 <vkmc> maybe the person implementing the logic is not the same as the one implementing the ui
18:25:29 <cp16net> right it can be done out of band
18:25:39 <vkmc> s/maybe/most of the times/
18:25:55 <vkmc> and having everything all together make it harder to keep track of all the changes you want to do
18:26:15 <vkmc> s/want/want and need/
18:27:30 <johnma> I think the question is whether we want to go through the spec process in Trove for the new Trove-dashboard features. I dont think horizon does that
18:28:03 <cp16net> no most proejcts dont have a separate spec for ui features
18:28:06 <amrith> just a time check, it is 27 past the hour
18:28:31 <dloi> ok so if we add a section for the UI user experience in the trove-spec then I'm ok with that.  I would prefer a separate LP for tracking bugs like the python-troveclient.
18:28:41 <cp16net> thanks amrith
18:29:10 <imandhan> Why don't we have a separate lp page for trove-dashboard and have those bugs/bps(no trove specs, just the way trove ui has been currently having short descriptions) show up in the trove lp page as well with a [dashbaord] tag that way it's getting visibility which will enable faster merges - satisfying the motivation behind creating a trove-dashboard repo
18:29:31 <peterstac> dloi, that makes sense to me (a trove-dashboard lp repo for bugs only and a section in the trove-spec doc)
18:29:34 <vgnbkr> +1 dloi
18:29:45 <dougshelley66> dloi +1
18:29:48 <cp16net> lets make a vote on it
18:29:52 <johnma> +1 to dloi's suggestion
18:30:01 <amrith> +1 to dloi
18:30:17 <vkmc> dloi, +1
18:30:31 <mvandijk> +11
18:30:36 <schang> dloi, +1
18:30:37 <SlickN1k> dloi: ++
18:30:47 <peterstac> I guess we just voted ;)
18:30:52 <dougshelley66> dloi for president
18:30:58 <cp16net> #startvote Should we use a separate lp project for trove-dashboard? Yes, No, Shrugs
18:30:58 <openstack> Begin voting on: Should we use a separate lp project for trove-dashboard? Valid vote options are Yes, No, Shrugs.
18:30:59 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
18:30:59 <johnma> :)
18:31:05 <amrith> #vote dloi
18:31:06 <openstack> amrith: dloi is not a valid option. Valid options are Yes, No, Shrugs.
18:31:21 <amrith> #vote Yes
18:31:21 <peterstac> #vote yes
18:31:21 <dougshelley66> #vote Yes
18:31:26 <imandhan> #vote yes
18:31:27 <schang> #vote yes
18:31:28 <vgnbkr> #vote yes
18:31:29 <mvandijk> #vote yes
18:31:29 <johnma> #vote yes
18:31:32 <dloi> #vote Yes
18:31:32 <pmackinn> #vote Shrugs
18:31:33 <SlickN1k> #vote yes
18:31:33 <tellesnobrega> #vote yes
18:31:35 <vkmc> #vote yes
18:31:44 <cp16net> #vote No
18:32:02 <cp16net> #endvote
18:32:03 <openstack> Voted on "Should we use a separate lp project for trove-dashboard?" Results are
18:32:04 <openstack> Yes (12): dloi, schang, vkmc, tellesnobrega, amrith, vgnbkr, peterstac, imandhan, johnma, mvandijk, SlickN1k, dougshelley66
18:32:05 <openstack> Shrugs (1): pmackinn
18:32:06 <openstack> No (1): cp16net
18:32:28 <cp16net> looks like like a landslide
18:32:48 <SlickN1k> cp16net: Is this code that we will be shipping as part of the openstack release?
18:33:15 <cp16net> SlickN1k: yes it will be part of the release
18:33:48 <atomic77> oh no i didn't get my shrug in
18:33:49 <atomic77> #vote Shrugs
18:33:53 <cp16net> :)
18:34:04 <dougshelley66> cp16net do we know if it is going to be packaged?
18:34:13 <cp16net> @action cp16net make new lp project to manage the trove-dashboard
18:34:36 <dougshelley66> do we also need to change the spec template to include dashboard impact?
18:34:40 <dougshelley66> should that be an action
18:35:04 <peterstac> I can push something up for that dougshelley66
18:35:07 <cp16net> dougshelley66: i'm not sure on all the details of how that will work but i expect it to be available as a plugin for horizon via pypi and pacakged for distros as well
18:35:20 <dougshelley66> cp16net thx
18:35:24 * amrith winces
18:35:35 <SlickN1k> dougshelley66: Yes, I think we should — I would shudder to think that we'd require a separate spec just for UI implementations of trove features.
18:36:07 <cp16net> yeah i dont think that detail was refuted
18:36:08 <dougshelley66> SlickN1k right - I think we all agreed on that above
18:36:39 <cp16net> #action cp16net trove spec include dashboard section
18:36:46 <cp16net> ok moving on :)
18:36:49 <SlickN1k> Awesome :)
18:36:53 <cp16net> #topic Announcements
18:37:03 <amrith> I have one.
18:37:08 <cp16net> i think amrith had an announcement :)
18:37:12 <amrith> Just a heads-up for all of you, and you will be seeing this on the ML shortly. I would like to nominate Mariam John (johnma) to the Trove core team. Mariam has been working on Trove for a while now, and did a lot of the early work on CouchDB and DB2. I look forward to working with her further on Trove, and in the future as a member of Trove core.
18:37:34 <amrith> that's it for me.
18:37:55 <vkmc> amrith, +1
18:38:13 <vkmc> johnma, \o/
18:38:13 <amrith> thanks vkmc, email to the ML just went out.
18:38:14 <cp16net> awesome :)
18:38:23 <vkmc> super
18:38:39 <johnma> Thank you.
18:38:41 <SlickN1k> amrith: I've had the pleasure of working with johnma on db2 / couchdb before — I'm ++ to that as well.
18:38:47 <SlickN1k> Will reply to the ML thread.
18:39:02 <amrith> Thx SlickN1k
18:39:10 <cp16net> i'm on board as well
18:39:11 <amrith> Please let SlickNik know as well ...
18:39:44 <cp16net> one of other announcements i have is that in 2 weeks m-2 will be cut
18:40:13 <cp16net> then m-3 feature freeze will be at the end of February
18:40:41 <cp16net> I know there are lots of specs in progress but i'd love to see code soon for them :)
18:41:13 <cp16net> ok moving on
18:41:28 <cp16net> #topic Looking for comments (Trove meets HBase)
18:41:33 <cp16net> i think this is amrith
18:41:37 <amrith> yup
18:41:49 <amrith> as it says in the meeting note, I'm looking for thoughts on how to proceed
18:41:59 <amrith> I am not looking for comments on the spec in the meeting. Rather there seems to be an issue about whether Trove should even go hear HBase or whether we should just say that's Sahara's turf. A question was raised in the spec review of whether this has been discussed on the ML. I'm happy to post the question on the ML but before I did that, I wanted to poll the team to see what people felt. It is my belief that
18:41:59 <amrith> HBase is a database and offering support for HBase through Trove, where Trove provisions and manages the instances independent of Sahara, is something of value. I realize that there are others who feel differently. The code is up there for review as well (see https://review.openstack.org/#/c/262048/ and https://review.openstack.org/#/c/262815).
18:42:08 <amrith> One thought it to put it on the ML
18:42:10 <amrith> I could do that.
18:42:34 <pmackinn> +1, reach out to sahara
18:42:45 * elmiko waves
18:43:06 <cp16net> while sahara focuses on hadoop
18:43:07 <pmackinn> elmiko aka mike mccune from sahara
18:43:22 <elmiko> hadoop/spark/storm/etc...
18:43:43 <cp16net> how does hbase work within sahara?
18:43:54 <cp16net> is it managed separately?
18:44:39 <elmiko> iirc, through the cloudera/hortonworks/mapr plugins, hbase is managed through those vendor provided solutions
18:44:45 <amrith> Is it the intent that Sahara is somehow different and therefore while all other databases can be managed and orchestrated by Trove, Sahara has exclusive control over Hadoop based databases?
18:44:50 <cp16net> i could see where a user would want to have hbase setup for them maybe independantly of a full hadoop deployment
18:44:51 <elmiko> sahara does not actively "manage" the hbase cluster
18:44:53 <cp16net> is that possible?
18:45:03 <elmiko> amrith: definitely not
18:45:03 <amrith> cp16net, yes
18:45:10 <amrith> that's the specific use case I'm targeting
18:46:28 <elmiko> i think as long as you are talking about smaller hbase deployments then it might make sense, but as you scale out hbase to interact with more hadoop technologies it can become quite complicated
18:46:30 <cp16net> so while there is overlap that habase could be "installed" from both projects but the managing of the service would greatly differ
18:47:13 <pmackinn> fundamentally, there is the hdfs cluster itself
18:47:18 <elmiko> i think there may even be a possibility for trove to use sahara as a provisioning engine for hbase clusters, but this obviously carries other baggage
18:47:35 <vkmc> don't know much about HBase, but I'm aware it is a big data store... in that sense, how much this overlaps with Sahara? are we expecting to fully support HBase? having backups, for example, for a big HBase deploy doesn't make much sense
18:47:55 <amrith> vkmc, how's that hbase specific
18:48:04 <amrith> what does a backup of a huge cassandra do for you?
18:48:12 <cp16net> (i've not worked with hadoop) it seems like maybe even the opposite could happen elmiko where sahara used trove to provision the hbase cluster?
18:49:07 <elmiko> cp16net: i suppose that's possible too, although it might become troublesome with the vendor provided plugins we have now (as they provision/manage hbase)
18:49:15 <amrith> this isn't a question of technologies; there is code in Sahara to manage hdfs and code in trove to provision databases. it is a matter of use-cases. People want a common management plane for their db's. that's the Trove value proposition.
18:49:20 <amrith> and hbase is a database
18:50:06 <cp16net> heres a dumb question... are there use cases outside of hadoop for hbase?
18:50:22 <cp16net> i assume yes...
18:50:26 <elmiko> good question, and that's what i was thinking too
18:50:37 <amrith> I don't understand the question
18:50:40 <cp16net> but i have not idea
18:51:02 <cp16net> amrith: so i know that hbase is used for hadoop
18:51:23 <cp16net> but what other use cases outside of hadoop is hbase used in?
18:51:48 <cp16net> if there are many other applications of hbase then i could see it being more benifical in trove
18:51:56 <cp16net> if not then not sure it makes as much sense
18:51:59 <amrith> hbase can run on plain file system, no hdfs
18:52:01 <elmiko> well, hdfs/hbase could be used by other apache based data processing frameworks
18:52:03 <amrith> that's called standalone
18:52:35 <pmackinn> pig, hive, spark, etc.
18:53:16 <elmiko> even in standalone mode though, you wouldn't be using hbase like say, mysql or mongo
18:53:20 <amrith> and as pmackinn says ... you can put tons of things above hbase
18:53:27 <amrith> elmiko, why?
18:53:30 <amrith> or rather why not?
18:53:31 <pmackinn> above hdfs
18:53:45 <amrith> how does one use mysql different from hbase?
18:54:10 <amrith> but anyway, cp16net I didn't think we'd be able to resolve this here
18:54:15 <amrith> I'm looking for next steps.
18:54:18 <elmiko> my understanding is that the coupling between hbase and the other hadoop technologies is much tighter than just using hbase as a generic database (but, i could be wrong about this)
18:54:29 <pmackinn> hadoop was invented to deal with the shortcomings of traditional rdbms :-)
18:54:42 <amrith> ouch
18:54:51 <amrith> we can start the sql-NoSQL battle here now.
18:54:57 <elmiko> lol
18:55:00 <amrith> I'm proposing that I put this on the ML
18:55:03 <amrith> the code is there
18:55:04 <atomic77> pmackinn, not exactly :)
18:55:12 <amrith> not just the spec
18:55:12 <elmiko> amrith: +1
18:55:14 <cp16net> amrith: i think that would makes sense
18:55:20 <pmackinn> atomic77, bring it on
18:55:21 <amrith> ok, done.
18:55:29 <cp16net> because i know others will question this as well
18:55:29 <elmiko> it would be nice to get some of our hbase experts to take a closer look
18:55:31 <amrith> pmackinn, you'll lose. I'll put money on that.
18:55:31 <vkmc> +1 for thread in the ML
18:56:01 <cp16net> +1 for ML
18:56:11 <pmackinn> faster queries over larger datasets
18:56:13 <amrith> cp16net, I'm done.
18:56:14 * pmackinn drops mic
18:56:18 <cp16net> #topic open discussion
18:56:23 <cp16net> anything else to bring up?
18:56:29 <pmackinn> https://etherpad.openstack.org/p/mitaka-trove-midcycle-rsvp as usual
18:56:38 <atomic77> pmackinn, hadoop was originally just an opensource impl of mapreduce which was more targeted as easier distributed processing
18:56:59 <pmackinn> not "easier"
18:57:06 <atomic77> i guess you could argue that that was necessary due to rdbms deficiencies but if you read the original paper that was very unlikely to have been cited as a motivation
18:57:36 <atomic77> now bigtable, on the other hand (which is what hbase is based on) is definitely
18:57:42 <cp16net> alright i'll let you guys fight it out....
18:57:44 <amrith> pmackinn, atomic77 take it to the parking lot ;)
18:57:52 <cp16net> but any thing else?
18:57:56 <pmackinn> hadoop is MR+HDFS, so not just processing
18:58:13 <cp16net> #endmeeting