21:01:26 <jd__> #startmeeting ceilometer
21:01:27 <openstack> Meeting started Wed Jul  3 21:01:26 2013 UTC.  The chair is jd__. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:01:28 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:01:30 <openstack> The meeting name has been set to 'ceilometer'
21:01:34 <sandywalsh> o/
21:01:39 <nealph> o/
21:01:42 <eglynn> o/
21:01:47 <litong> o/
21:01:52 <dhellmann> o/
21:01:54 <gordc> o/
21:01:56 <jd__> hi everybody
21:03:20 <jd__> #topic Review Havana-2 milestone
21:03:25 <jd__> #link https://launchpad.net/ceilometer/+milestone/havana-2
21:03:37 <jd__> I'm afraid we are getting late
21:04:04 <jd__> there's a lot of things started and needing reviews
21:04:07 * dhellmann submitted a bunch of changes for review for one-meter-per-plugin
21:04:20 <eglynn> are we looking at Friday July 12th as the effective cutoff dat for merges?
21:04:23 <jd__> dhellmann: yeah I already +2'ed most/all of them
21:04:31 * dhellmann has a few more to go
21:04:35 <gordc> will take a look at patches.
21:04:37 <sandywalsh> I should have my event collector stuff up shortly
21:04:45 <dhellmann> and I will spend some time reviewing this week, too
21:04:53 <nealph> I've signed up for several...sandywalsh: especially interested in those.
21:04:54 <jd__> eglynn: in theory I guess, though that's not fixed AFAIK
21:05:01 <eglynn> jd__: k
21:05:06 <sandywalsh> I'll throw another review day in the mix this week
21:05:20 <jd__> thanks sandywalsh
21:05:32 <gordc> still working on auditing events. i think we're going to post on mailing list to get feedback on that item.
21:06:08 <dhellmann> jd__: pipeline-configuration-cleanup is small, but since it relies on this other bigger one it should probably get bumped to h3
21:06:15 <jd__> so in the end I don't have more to say than review! review! review! code! code! review! :)
21:06:46 <jd__> dhellmann: ack, though it's low so it's not my main source of anxiety
21:06:48 <eglynn> yep, noses to the grindstone on both reviews and completing BPs
21:06:55 <dhellmann> jd__: ok
21:07:26 <nealph> tomorrow is a US holiday...that doesn't help. :-)
21:07:28 * jd__ . O O (damn, every day discussing with eglynn is a way to learn more words and expressions!)
21:07:36 <jd__> ah indeed :)
21:07:45 <eglynn> LOL
21:07:50 <jd__> #topic Release python-ceilometerclient?
21:07:57 <jd__> I think we don't need that right?
21:08:06 <eglynn> 1.0.1 went out last week
21:08:14 <eglynn> so no, not needed
21:08:18 <jd__> ok that should be good enough for now :)
21:08:22 <eglynn> yep
21:08:24 <jd__> #topic Multi dispatcher enablement blueprint
21:08:28 <jd__> #link https://blueprints.launchpad.net/ceilometer/+spec/multi-dispatcher-enablement
21:08:33 <jd__> I think that's for litong
21:08:45 <litong> @jd__, yes,
21:08:52 <litong> as I indicated in the agenda,
21:09:25 <litong> the point of the blueprint is to allow one to configure multiple outlets for further data processing.
21:10:27 <ekarlso> what's a outlet ?
21:10:37 <jd__> a database
21:10:39 <eglynn> whoops we lost litong there I think ...
21:10:52 <jd__> litong: I didn't read the whole code but I think I prefer https://review.openstack.org/#/c/34301/
21:10:52 <eglynn> k, back again
21:11:07 <dhellmann> the plugins I remember seeing logged or wrote to our database using the storage driver
21:11:18 <jd__> litong: though it might requires a mechanism close to the one we have in the pipelinen to be implemented correctly
21:11:25 <litong> lost connection.
21:11:58 <litong> the second pipeline implementation has problem since the data has to be converted into Counter.
21:12:06 <litong> which actually lose some information.
21:12:12 <litong> it is also limiting.
21:12:23 <jd__> you need to build a mix of both
21:12:36 <litong> jd__, I prefer the first patch as well. but Doug wanted to use pipeline.
21:12:43 <jd__> a pipeline but not publishing, a pipeline calling record_metering_data
21:12:46 <dhellmann> well, if that's what we're going to do, we should just go with the first patch
21:13:07 <dhellmann> we don't *need* a pipeline, I just thought it would be better than having a second thing that was so similar
21:13:24 <jd__> dhellmann: we need if we want to dispatch counter recording in different "outlet"
21:13:32 <jd__> with the same collector
21:13:38 <dhellmann> these aren't counters though, right?
21:13:47 <jd__> these are counters
21:13:47 <sandywalsh> we're running into a similar problem with event data ... it's going to have to go in the pipeline at some point. The payload of the pipeline should be more generic
21:13:56 <litong> right, not counters, almost like a raw data.
21:13:57 <jd__> sandywalsh: agreed
21:14:01 <dhellmann> jd__: litong said that converting the data to counters lost information
21:14:15 <jd__> well the collector records counters, nothing else
21:14:37 <litong> yes. such as message id. though it may be generated along the way, but it may be important for some.
21:14:39 <dhellmann> it records event data that includes the fields of a counter, plus some, doesn't it?
21:14:46 <dhellmann> message id, signature, etc.
21:14:48 <dhellmann> those are not counter fields
21:14:50 <jd__> litong: are you recording counters or events?
21:15:01 <jd__> or notifications
21:15:07 <dhellmann> events
21:15:15 <dhellmann> not notifications, the rpc events we send as output from the pipeline now
21:15:21 <litong> jd__, any thing that passed into record_metering_data without any loss.
21:15:37 * jd__ thinks the major problem in this project is terminology :)
21:15:53 <dhellmann> heh
21:15:54 <jd__> litong: ok so that's counters/meters
21:15:56 <eglynn> amen to that! ;)
21:16:16 <litong> @jd__, totally agree. the point is that anything goes to db, should have to a chance to go to other outlet. (wished I have a better term for outlet)
21:16:20 <dhellmann> jd__: this is a new layer between the pipeline RPC output and the storage driver
21:16:22 <jd__> so I don't do understand why you'd lose information at first, but anyhow the pipeline could/should be data unaware
21:16:42 <dhellmann> the RPC payload includes values that are not in the counter: message signature is the most important
21:16:55 <jd__> dhellmann: ah ok I get it!
21:16:57 <dhellmann> because a Counter object, which is taken as input to the pipeline, is not the same thing as a message
21:17:07 <dhellmann> which is the output of the pipeline
21:17:11 <dhellmann> ok
21:17:15 <litong> jd__, but pipeline realy on data being Count since it does the filgering based on the configuration.
21:17:19 <jd__> indeed, so the pipeline should be fixed to be data agnostic, that's for sure
21:17:40 <jd__> so we can use it in a way it could be used as a dispatcher for record_metering_data
21:17:44 <litong> I mean filtering
21:17:48 <jd__> how does that sound? do I miss something? :)
21:17:51 <dhellmann> we could make a generic pipeline, but that will make it possible for users to configure it with transformers that are "wrong" for the type of data being processed
21:18:15 <jd__> dhellmann: the way we build pipeline allows us to specify different namespaces for everything
21:18:26 <dhellmann> ok
21:18:32 <jd__> so we should have a different transformer namespace for different purpopses
21:18:35 <dhellmann> do we need transformations in this dispatcher?
21:18:47 <jd__> or not, but we can have it and not use it
21:18:48 <dhellmann> jd__: good point
21:18:51 <litong> I would think not.
21:19:11 <jd__> does that answer your questions litong?
21:19:16 <dhellmann> so it could be as simple as a stevedore.NamedExtensionManager().map() call?
21:19:37 <jd__> dhellmann: no because we may want to route 'meters' based on meter name like we do in pipeline?
21:19:43 <litong> since the point for this blueprint is to have the data handed to the plugins and it is up to the plugins to do whatever it wants to.
21:19:45 <jd__> I mean, like we do in 'publishing pipeline'
21:20:11 <jd__> but we can blind routing with NamedExtensionManager.map() if that is enough for litong
21:20:14 <jd__> that's a first step
21:20:27 <dhellmann> jd__: eventually, maybe. I think making the pipeline more flexible is more work than we have time to do between now and the deadline, so I'm looking for a solution litong can have now
21:20:32 <dhellmann> right
21:20:33 <dhellmann> :-)
21:20:37 <jd__> agreed
21:20:54 <litong> so making changes based on the first patch?
21:20:59 <jd__> litong: yes
21:21:05 * dhellmann is getting frustrated with colloquy and needs a new irc client
21:21:10 <litong> it already uses NamedExtensionManager.
21:21:15 <dhellmann> litong: yes, I think so
21:21:22 <jd__> dhellmann: erc :-)
21:21:24 <litong> all right. thanks guys.
21:21:29 <litong> I feel a lot better now.
21:21:39 <dhellmann> litong: sorry to put you through that!
21:21:46 <jd__> litong: your first patch might be a really good start indeed, I just didn't review it :) but I will
21:21:52 <litong> @dhellmann, doug, np.
21:22:04 <litong> that is ok jd__.
21:22:09 <litong> whenever you have time.
21:22:11 <litong> it is still there.
21:22:14 <jd__> bah it always ends well anyway ;-)
21:22:19 <dhellmann> right, I'll go back and re-review it in more detail
21:22:33 <litong> thanks so much.
21:22:41 <dhellmann> I think we'll want to change some of the names. "dispatcher" is a little generic
21:23:07 <dhellmann> we lost sandywalsh, and he had the same question about using the pipeline
21:23:33 <litong> ok, what name would you guys like to use?
21:23:33 <jd__> we'll tell him to go through the log, I imagine that'll solve his question?
21:23:45 * dhellmann nods
21:24:02 <jd__> #topic Open discussion
21:24:23 <jd__> #action jd__ Write a terminology page in the documentation
21:24:37 <jd__> I think everybody will thank me if I do it before the next summit
21:24:47 <eglynn> +1, that'll really help with on-boarding
21:24:55 <litong> totally.
21:25:17 <sandywalsh> bloody vpn sucks :)
21:25:18 <jd__> i'll send a patch, you'll review, and we'll have a good chat about this once and for all :)
21:25:21 <jd__> re sandywalsh
21:25:36 <jd__> you missed all the fun, you may want to go through the log
21:25:46 <sandywalsh> will do ... got dropped after
21:25:47 <sandywalsh> <jd__> so I don't do understand why you'd lose information at first, but anyhow the pipeline could/should be data unaware
21:25:47 <sandywalsh> <sandywalsh> jd__, +1 ... at the very least, some sort of mimetype check to ensure a fit between pipeline blocks
21:25:47 <sandywalsh> <sandywalsh> (vs depending on python object types, since they'll be leaving the system at some point)
21:26:02 <jd__> ack :)
21:26:19 <gordc> got a question: are ceilometer.tests.publisher.test_rpc_publisher.py tests suppose to run?
21:26:23 <jd__> we did talked about pipeline and dhellmann was saying you could be interested
21:26:36 <jd__> gordc: I don't like this question
21:26:44 <gordc> :)
21:26:45 <sandywalsh> jd__, yup, definitely
21:26:59 <jd__> gordc: did we break it?
21:27:04 <gordc> jd__, there's quite a few tests that are sitting around but never run.
21:27:33 <jd__> gordc: why so? :(
21:27:53 <dhellmann> are they skipping explicitly, or is the test discovery failing to find them?
21:28:00 <gordc> jd__, could be when we shifted from nose to testr? but they never run locally for me
21:28:13 <gordc> dhellmann, the latter
21:28:22 <dhellmann> gordc: open a bug ticket?
21:29:08 <gordc> will do. just wanted to check if they were suppose to run... they're named like valid tests but it's not set up to run.
21:30:07 <jd__> ok
21:30:13 <jd__> anything else guys before I wrap up?
21:30:43 <gordc> nothing from me.
21:30:43 <nealph> I put a question out on the dev mail...
21:30:56 <nealph> dipping toes into config options for glance.
21:31:04 <nealph> any response appreciated. :)
21:31:24 <eglynn> nealph: I'll have a look ...
21:31:44 <nealph> pollster config stuff....eglynn: thanks!
21:32:07 <jd__> ack
21:32:24 <jd__> see you on -metering then, and use that last 28 minutes to review some code :p
21:32:37 <jd__> #endmeeting