18:00:38 <hub_cap> #startmeeting trove
18:00:39 <openstack> Meeting started Wed Nov 27 18:00:38 2013 UTC and is due to finish in 60 minutes.  The chair is hub_cap. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:41 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:43 <openstack> The meeting name has been set to 'trove'
18:00:46 <denis_makogon> o/
18:00:52 <imsplitbit> o7
18:01:02 <hub_cap> #link https://wiki.openstack.org/wiki/Meetings/TroveMeeting#Agenda_for_the_next_meeting
18:01:16 <cp16net> o^/
18:01:27 <hub_cap> cp16net: i still dont know what that is
18:01:38 <cp16net> its my virtual bicep
18:01:47 <cp16net> :-P
18:01:48 <vipul> o/
18:01:49 <hub_cap> o^>
18:01:50 <robertmyers> o/
18:01:57 <hub_cap> <^o^>
18:02:05 <amytron> o/
18:02:05 <isviridov> ))
18:02:06 <hub_cap> which way to the gun show?
18:02:24 <cp16net> o^7
18:02:24 <imsplitbit> we've met you, you're not foolin anyone
18:02:28 <hub_cap> ok so this should be a quick meeting since its close to the holiday
18:02:35 <hub_cap> imsplitbit: LOL i hate u
18:02:43 <cp16net> lol
18:02:47 <hub_cap> so lets get in to the jinja debate robertmyers denis_makogon
18:02:48 <amytron> 7^o   … that way hub_cap
18:03:08 <imsplitbit> ready?
18:03:08 <hub_cap> #topic jinja guest agent configuration
18:03:09 <imsplitbit> go!
18:03:11 <robertmyers> hub_cap: sweet
18:03:11 <juice> o/
18:03:12 <hub_cap> amytron: LOLLLLLL
18:03:29 <robertmyers> #link https://gist.github.com/rmyers/7678446
18:03:38 <denis_makogon> i don't what to relay on path
18:03:55 <robertmyers> Basically the idea is to move the guest conf in the templates
18:04:00 <esmute> \o/
18:04:02 <denis_makogon> if it's hard to misconfigure - it would be great
18:04:34 <robertmyers> it will work the same as other templates
18:04:45 <hub_cap> so at the end of the day we will likely need some templates
18:04:49 <robertmyers> you can override it in /etc/trove/templates
18:05:01 <denis_makogon> it would be bad if i want to use side guest_conf
18:05:15 <denis_makogon> with parameter for my personal environment
18:05:25 <hub_cap> the reason being is that we will provide 90% of configuration parameters that will not change
18:05:41 <hub_cap> take mysql for instance, those template files will not change, but there may be some values we need to tweak
18:05:51 <hub_cap> sorry, some values the _operator_ needs to tweak
18:05:56 <robertmyers> denis_makogon: turning into a template will help you do that
18:06:05 <denis_makogon> taskmanager had path to guest_conf
18:06:20 <vipul> insteaad of breaking it out as separate files.. how about just grouping options
18:06:23 <denis_makogon> i'd like to use it
18:06:24 <hub_cap> rather than having to keep their own copy of the template and merging back, we can do a _child_ conf for that
18:06:25 <vipul> [mysql]
18:06:29 <vipul> and have everything fall under that
18:06:42 <denis_makogon> vipul, oslo cannot read though multiple lines
18:07:02 <robertmyers> vipul: ++
18:07:06 <hub_cap> brb ~2 min while yall chat about this
18:07:12 <vipul> denis_makogon: not sure what you mean
18:07:26 <SlickNik> denis_makogon: can you explain what you mean by that? afaik, oslo configs can be broken out into sections.
18:07:26 <robertmyers> olso can to option groups
18:08:03 <denis_makogon> i dont want to keep everyting in one conf
18:08:11 <denis_makogon> because we have lots for overlaps
18:08:18 <denis_makogon> mount_points
18:08:27 <denis_makogon> backup_strategy
18:08:29 <denis_makogon> etc
18:08:49 <denis_makogon> dict option is worth idea
18:09:00 <robertmyers> denis_makogon: there is no overlap if you use different sections
18:09:19 <SlickNik> Options that you mention _have_ to be different per datastore type, and you can use a different section per datastore type.
18:09:26 <vipul> afaik, oslo also allows you to specify >1 config file
18:09:37 <SlickNik> vipul: that is also true.
18:09:41 <denis_makogon> vipul, that is what i'm doing
18:09:52 <denis_makogon> more then one config
18:09:59 <hub_cap> so hold up
18:10:03 <hub_cap> this is 2 conversations
18:10:03 <denis_makogon> #link https://review.openstack.org/#/c/58574/
18:10:04 <vipul> but then again.. are we at a point where need a config file for everything?
18:10:12 <robertmyers> denis_makogon: that is fine, but for the base I'm suggesting to use one
18:10:26 <hub_cap> the guest stuff in 2 config files vs [parameters]
18:10:30 <denis_makogon> robertmyers, one on VM side
18:10:34 <vipul> what you're trying to do is supported today in oslo via option groups..
18:10:37 <denis_makogon> robertmyers, i agree with that
18:10:39 <hub_cap> weve all agreed before the [params] is a good idea
18:10:49 <denis_makogon> but the way it formed is not clear
18:11:00 <hub_cap> its not the same thing though, right robertmyers ?
18:11:13 <hub_cap> you want separate files to be joined into one group [guest]
18:11:34 <robertmyers> well, sort of, I like using different config templates
18:11:40 <denis_makogon> robertmyers, ++
18:11:44 <robertmyers> but  using option groups is better
18:12:02 <robertmyers> if it can be done easily
18:12:20 <vipul> robertmyers: +1 let's at least give that a shot before we go crazy splitting everything up
18:12:29 * hub_cap is a fan of crazy
18:12:46 <vipul> i know you are hub_cap :D
18:12:53 <denis_makogon> vipul, could you point me how to use multiple groups ?
18:13:08 <juice> i'm in favor of fewer config files...unless something is needed by extension
18:13:08 <SlickNik> If we use option groups, we always have the option of splitting them into separate files later if we _need_ to...
18:13:14 <vipul> http://docs.openstack.org/developer/oslo.config/api/oslo.config.cfg.html
18:13:33 <juice> [GROUP1][....][GROUPn]
18:13:45 <juice> basically sections in an .ini file
18:13:53 <SlickNik> But I'd like to stick with option groups and the one config template for now.
18:13:54 <vipul> conf.register_group(OptGroup(name=’rabbit’)) conf.register_opt(rabbit_port_opt, group=’rabbit’)
18:14:17 <hub_cap> and then you have to do conf.rabbit.port
18:14:21 <isviridov> IMHO There are two things: the config as artifact on remote VM and the data in config to render the first one. Templates is robust way to prepare configs, but where to store the data for rendering depends on how it will be changed: one big config, or even db
18:14:30 <denis_makogon> seems to be easier to use jinja instead of oslo
18:15:00 <juice> oslo is for config. jinja is for templating
18:15:09 <juice> aren't these two separate concerns
18:15:14 <denis_makogon> jinja is for everthing
18:15:19 <vipul> LOL
18:15:27 <cweid> jinja does not parse the config tho...
18:15:43 <denis_makogon> cweid, and we don't need it do this
18:15:57 <hub_cap> lets rewrite trove in jinja
18:16:00 <hub_cap> :D
18:16:03 <SlickNik> lol
18:16:03 <vipul> hub_cap: +100
18:16:13 <juice> actually...
18:16:17 <cp16net> lets rename trove to jinja
18:16:18 <cp16net> :-P
18:16:22 <robertmyers> yes, well, my main point is to make the base config a template that we can change
18:16:22 <hub_cap> ok so do we have some sort of consensus? robertmyers denis_makogon u feel better?
18:16:23 <imsplitbit> jinja sounds awesome was it written by...
18:16:28 <juice> ok lets get back to constructive
18:16:33 <hub_cap> cp16net: no renaming sucks
18:16:37 <cp16net> haha
18:16:52 <denis_makogon> hub_cap, i will feel better to use jinja to merge separate configs into one, without oslo
18:17:01 <hub_cap> imsplitbit: im pretty sure obama wrote jinja
18:17:07 <hub_cap> ZING
18:17:27 <hub_cap> robertmyers: moving on?
18:17:28 <robertmyers> denis_makogon: I agree that is a easier path forward
18:17:38 <robertmyers> hub_cap: sure
18:17:45 <vipul> i don't know what decision was made :(
18:18:00 <SlickNik> I don't think one was made, in the interest of time.
18:18:02 <cp16net> vipul: you are not the only one
18:18:09 <denis_makogon> #action Decision - Jinja
18:18:16 <robertmyers> vipul: I like the idea of option groups, but that would be a major overhaul
18:18:26 <denis_makogon> robertmyers, ++
18:18:38 <hub_cap> [hehe] yes it would
18:18:47 <robertmyers> i think we could move to that later
18:18:54 <vipul> robertmyers: it will be more confusing to people used to deploying other openstack things when they look at trove
18:19:22 <vipul> we're not consistent
18:19:30 <robertmyers> vipul: true, but the config is not setup that way currently
18:19:32 <denis_makogon> vipul, there's no common policy,
18:19:40 <SlickNik> So hang on I have a question.
18:19:45 <robertmyers> so, we'd have to change alot of code
18:19:50 <SlickNik> (If we're sticking to this topic)
18:20:02 <hub_cap> lets keep w/ it SlickNik
18:20:26 <SlickNik> Are we talking about trove options per datastore type, or config options for the datastore itself (eg. my.cnf options)...
18:20:39 <denis_makogon> per datastore
18:20:53 <vipul> trove.conf option i believe
18:21:02 <denis_makogon> trove-guestagent.conf
18:21:24 <vipul> also.. i am not a fan of putting more things and injecting them..
18:21:44 <vipul> i think _some_ deployers will have alternate ways of getting static configs out to these things
18:22:44 <cp16net> so we are trying to make a guest config dynamic in nature so that it can support mutiple datastores
18:22:48 <denis_makogon> injection wins against generic configs
18:22:48 <vipul> the current patch basically requires this to happen via TaskManager
18:23:06 <vipul> cp16net: i don't think it should be
18:23:14 <denis_makogon> vipul, and it would happen only in taskmanager
18:23:16 <vipul> why couldn't you have 1 static config -- that has options for all datastores
18:23:35 <SlickNik> I tend to agree with vipul here, You need templating if you want to dynamically change the config (on the fly) based on certain inputs.
18:23:38 <vipul> why do i need to manage or need Trove to manage for me a datastore specific conf
18:23:46 <denis_makogon> does option groups handle parameters with same names ?
18:24:00 <vipul> sure.. rabbit.host is not == database.host
18:24:00 <isviridov> SlickNik, +1
18:24:22 <cp16net> yea i dont think there is a need for templating the guest config here
18:24:25 <robertmyers> SlickNik: we do for the guest_id
18:24:40 <cp16net> the guest should be able to run with its config
18:24:44 <vipul> robertmyers: i agree with that one.. that's the only dynamic thing that the guest shoudl be aware of
18:24:54 <denis_makogon> SlickNik, w need dynamic input because generic config look awful
18:25:11 <cp16net> and the injected /etc/guest_info should have the datastore type and other changable data
18:25:22 <cp16net> if anything you could make the /etc/guest_info a template if you like
18:25:25 <denis_makogon> to many parameters should be changed per provisioning
18:25:28 <cp16net> that would be fine
18:25:28 <vipul> denis_makogon: but nothing in the .conf you are proposing is actually dynamic
18:25:47 <SlickNik> denis_makogon: What config param (other than the guest_id) even makes sense to change dynamically from the task manager?
18:26:12 <denis_makogon> manager, mount_phttps://review.openstack.org/#/c/58574/3/trove/templates/mysql/guest_info.config
18:26:58 <vipul> ok that's the guest_info.. which actually has been dynamic.. we've moving it out from a hard coded string to a config file +1
18:27:36 <denis_makogon> vipul, do you agree with this idea ?
18:27:48 <denis_makogon> SlickNik, what do you think ?
18:28:04 <vipul> denis_makogon: i'm fine with that...
18:28:20 <vipul> denis_makogon: i'm not ok with https://review.openstack.org/#/c/58574/3/trove/taskmanager/models.py where you're pushing out trove-guestagent.conf
18:28:51 <denis_makogon> vipul, it would re-written to merge everything in one config
18:28:58 <SlickNik> denis_makogon: When would you want to change the mount point dynamically to cater to multiple entries across guest agents (i.e. for what set of input would a guestagent A have mount point A and guestagent B have mount point B).
18:29:41 <hub_cap> A.type != B.type ?
18:29:52 <denis_makogon> yes
18:30:00 <cweid> I think mount point should be set by the guest agent impl and not a config...
18:30:24 <denis_makogon> cweid, that is my words
18:30:26 <cweid> Is it even important to have the mount be dynamic?
18:30:36 <cweid> it is just a mapping to a device.
18:30:37 <SlickNik> hub_cap: you mean datastore type? Even then it would be consistent across a datastore type.
18:31:17 <hub_cap> ok we are 30 min in and have tangented a good bit :)
18:31:26 <denis_makogon> let's move on
18:31:30 <hub_cap> should we table this and move on?
18:31:35 <robertmyers> sure
18:31:48 <SlickNik> I'm good with that. We can bring it up later in #openstack-trove
18:32:04 <hub_cap> #topic custom test groups
18:32:07 <hub_cap> denis_makogon: this is u
18:32:29 <denis_makogon> hub_cap, i would like to bring trove reddwarf job into infra
18:32:59 <denis_makogon> to give us ability to change it as we need to support new test groups
18:33:11 <denis_makogon> without involving infra
18:33:15 <vipul> what is a test group
18:33:34 <cp16net> int-tests --group=mytest.group?
18:33:43 <denis_makogon> yes
18:34:45 <denis_makogon> hub_cap, is it possible to publish reddwarf job ?
18:34:58 <hub_cap> u should have access to view it denis_makogon
18:35:00 <cp16net> to make a new jekins job?
18:35:08 <vipul> you want to move teh int-test gate into openstack-infra's resources?
18:35:13 <denis_makogon> cp16net, update current
18:35:32 <denis_makogon> vipul, yes
18:35:40 <vipul> currently core can edit the job i believe
18:35:47 <hub_cap> denis_makogon: oh i was talking about this to the infra team
18:35:59 <denis_makogon> hub_cap, i know =/
18:36:00 <vipul> if you want to do that, +1 i think it will involve some puppetry
18:36:16 <SlickNik> that's right, currently only core can edit the job.
18:36:26 <hub_cap> they said they do not want us to move forward with this job until the tempest tests are working
18:36:39 <denis_makogon> since i'm not in -core, that's impossible for me
18:36:53 <vipul> if you want something edited.. just reach out to someone
18:37:00 <SlickNik> denis_makogon: I can send you a list of what the job does, it's really simple.
18:37:24 <SlickNik> But, like hub_cap says, there's other things that need to fall into place before we this can happen.
18:37:27 <vipul> while on this topic... we should really think about requiring some form of 'donated' resources when adding new datastores
18:37:33 <hub_cap> denis_makogon: i wouldnt do more work on this task.. i was told yesterday late that we should first do the tempest tests
18:37:33 <vipul> we add them.. but have no clue if they still work
18:37:35 <denis_makogon> SlickNik, i know what the job does, i update it to support tests for mongo, cassandra, redis, postgresql
18:38:16 <SlickNik> denis_makogon: A good first step would be to add the tests first and to get them to pass _outside_ the gate.
18:38:18 <hub_cap> vipul: the tempest tests, imho, should be spinning up instance/clusters of these and running similar and unique to a _type tests
18:38:30 <denis_makogon> SlickNik, of course
18:38:51 <vipul> i don't like the idea of one job doing all datastore types
18:39:05 <vipul> it's likely to fail always -- and hard to figure out why
18:39:25 <denis_makogon> waiting for tempest sounds like huge delay for any new database
18:39:52 <denis_makogon> first review with new databases were published almost two month ago
18:39:53 <hub_cap> there will be a point where we need to make databases wait
18:40:09 <hub_cap> i think we can let these in once we verify they are clean
18:40:20 <hub_cap> and redis, since that is a work in progess
18:40:28 <hub_cap> but for new datastores, lets wait till tempest
18:40:33 <cp16net> well that makes tempest tests a higher prio
18:40:44 <denis_makogon> cp16net, even more then higher
18:40:53 <cp16net> super high
18:41:03 <denis_makogon> zero-priority
18:41:31 <vipul> are you guys describing how you felt last night
18:41:38 <SlickNik> LOL
18:41:56 <cp16net> how did you know?
18:42:03 <denis_makogon> ok
18:42:05 <cp16net> lol
18:42:09 <hub_cap> LOL
18:42:12 <denis_makogon> what the status with tempest tests ?
18:42:18 <SlickNik> vipul has zero-intuition, cp16net
18:42:18 <hub_cap> pineapple express priority
18:42:25 <vipul> lol
18:42:26 <denis_makogon> is it possible to make any estimates ?
18:43:01 <cp16net> denis_makogon: good question. what is the status?
18:43:32 <cp16net> #link https://blueprints.launchpad.net/trove/+spec/trove-tempest
18:43:50 <hub_cap> heh
18:43:54 <cp16net> i think the prio should be uped... != undefined
18:43:57 <denis_makogon> cp16net, nice one
18:43:57 <hub_cap> im starting work on that today
18:44:03 <SlickNik> I'm only getting started working on some of it since I got back.
18:44:12 <hub_cap> heh SlickNik and i are in the same boat
18:44:27 <imsplitbit> you guys are on a boat????
18:44:41 <SlickNik> #link https://etherpad.openstack.org/p/TroveTempestTesting
18:44:47 <cp16net> with flippy floppys
18:46:12 <vipul> http://theawesomer.com/photos/2010/07/070810_old_spice_boat_t.jpg
18:46:25 <denis_makogon> any other topics ?
18:46:27 <hub_cap> hahah
18:46:32 <hub_cap> time to move on then?
18:46:45 <denis_makogon> yes
18:46:47 <cp16net> sure
18:46:54 <hub_cap> #topic config template per version
18:46:55 <cp16net> oh...
18:47:03 <denis_makogon> mine
18:47:14 <cp16net> #action someone update the priority of https://blueprints.launchpad.net/trove/+spec/trove-tempest
18:47:25 <denis_makogon> #link https://blueprints.launchpad.net/trove/+spec/config-template-per-package
18:48:00 <denis_makogon> there's need to provide config.template per datastore_version
18:48:39 <cp16net> denis_makogon: i've worked on addressing this already in the configurations
18:48:51 <cp16net> unless i am misunderstanding what you mean
18:49:10 <hub_cap> aww ya sweet :P
18:49:15 <denis_makogon> i mean that my.cnf for mysql 5.5 would not work with mysql 4
18:49:19 <denis_makogon> etc
18:49:24 <cp16net> yeah
18:49:26 <cp16net> exactly
18:49:41 <cp16net> i'm about to have that addressed in my new review
18:50:05 <cp16net> i'm working on addressing the comments from denis_makogon and amcrn right now
18:50:22 <denis_makogon> cp16net, which one ?
18:50:22 <cp16net> #link https://review.openstack.org/#/c/53168
18:50:24 <SlickNik> cp16net: link please?
18:50:27 <SlickNik> thanks!
18:50:45 <denis_makogon> cp16net, ah
18:50:54 <cp16net> i'll update the ML thread when i have it finished
18:51:07 <denis_makogon> but your case covers another scope
18:51:17 <denis_makogon> mine covers different versions
18:51:29 <denis_makogon> your covering configuration overall
18:52:03 <cp16net> oh....
18:52:10 <denis_makogon> common thins between our reviews is word "configuration"
18:52:16 <cp16net> i think i know what you are getting at
18:52:21 <cp16net> and no i didnt adderss that...
18:52:34 <denis_makogon> ok, nevermind
18:52:37 <cp16net> :)
18:52:44 <denis_makogon> i'd like to get approve on this BP
18:52:49 <robertmyers> templates/{datastore}/{version}/config ?
18:52:52 <cp16net> trove/template/mysql/config.template
18:53:05 <denis_makogon> {version}.config.template
18:53:06 <robertmyers> templates/{datastore}/{version}.config ?
18:53:12 <robertmyers> ok
18:53:14 <cp16net> but you want something like trove/template/mysql/4/config.template
18:53:22 <cp16net> trove/template/mysql/5.5/config.template
18:53:28 <cp16net> trove/template/mysql/5.1/config.template
18:53:30 <denis_makogon> cp16net, to complicated hierarchy
18:53:42 <denis_makogon> *too
18:53:52 <denis_makogon> trove/template/mysql/5.1.config.template
18:54:01 <robertmyers> denis_makogon: +1
18:54:05 <cp16net> ok sound sgood
18:54:18 <cp16net> will this still have a "default"
18:54:22 <denis_makogon> yes
18:54:29 <denis_makogon> trove/template/mysql/config.template
18:54:41 <denis_makogon> trove/template/mysql/default.config.template
18:54:47 <cp16net> so the version needs to match what is setup in the datastore version list?
18:54:52 <denis_makogon> yes
18:55:00 <denis_makogon> it's important
18:55:05 <cp16net> yeah +1
18:55:26 <denis_makogon> only problem - duplicating
18:55:50 <robertmyers> denis_makogon: jinja extend is your friend
18:55:51 <cp16net> #agree we need versions on template configs and a default
18:55:57 <denis_makogon> you could name version like mysql-5.5-lol and mysql-5.5-bark-bark
18:56:14 <denis_makogon> two different version, but same db
18:56:45 <robertmyers> denis_makogon: mysql-5.5-bark-bark extends  mysql-5.5-lol
18:56:49 <robertmyers> done
18:56:51 <cp16net> its different packages
18:57:00 <SlickNik> denis_makogon: why mysql-5.5-bark-bark, and not mysql-5.5-bark… DRY.
18:57:07 <denis_makogon> robertmyers, extending would not work at this time, too complicated
18:57:08 <cp16net> so technially they are different and should be treated as such
18:57:41 <robertmyers> denis_makogon: no, it is not
18:57:44 <hub_cap> i would totally buy mysql-5.5-bark-bark service if i could
18:58:16 <cp16net> the extentions need to be handled and made by the deployer anyways as well as manging the datastores
18:58:33 <denis_makogon> robertmyers, lets see if it would work without extensions, and then discuss it again
18:58:43 <robertmyers> as long as it has a default it doesn't mater
18:58:56 <imsplitbit> you're a cat person hub_cap you know you'd rather have mysql-5.5-meow-mewo
18:59:04 <robertmyers> if you set up 5 of the same that is your mess to clean up
18:59:46 <robertmyers> I'm just saying you *can* extend a template and it will be the same.
18:59:48 <hub_cap> imsplitbit: meow-bark-meow
18:59:55 <denis_makogon> robertmyers, suppose we need to change one parameter and add another block ? does jinja handle such cases ?
19:00:04 <robertmyers> denis_makogon: yes
19:00:10 <denis_makogon> robertmyers, at the same time >
19:00:11 <denis_makogon> ?
19:00:47 <robertmyers> Jinja is pretty powerful and you can do a lot with it
19:00:55 <cp16net> jinja sounds magically
19:01:06 <denis_makogon> i mean that child block override several parameters and add new ?
19:01:06 <robertmyers> try it you might like it
19:01:07 <cp16net> over meeting time
19:01:16 <cp16net> need to end
19:01:25 <hub_cap> cp16net: magically delicious
19:01:28 <hub_cap> #endmeeting