15:00:17 <jd__> #startmeeting ceilometer
15:00:19 <openstack> Meeting started Thu Jun 13 15:00:17 2013 UTC.  The chair is jd__. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:20 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:22 <openstack> The meeting name has been set to 'ceilometer'
15:00:28 <jd__> #link https://wiki.openstack.org/wiki/Meetings/MeteringAgenda
15:00:28 <dragondm> o/
15:00:30 <eglynn> o/
15:00:31 <llu-laptop> o/
15:00:32 <terriyu> o/
15:00:35 <dhellmann> o/
15:00:37 <jd__> hi everybody
15:00:45 <gordc> o/
15:01:31 <sandywalsh> o/
15:01:48 <xingzhou> o/
15:02:31 <apmelton> o/
15:02:38 <jd__> #topic Review Havana-2 milestone
15:02:47 <jd__> #link https://launchpad.net/ceilometer/+milestone/havana-2
15:03:10 <jd__> we did some progress, so congrats
15:03:38 <jd__> don't forget to update your blueprint statut to make me happy
15:03:51 <jd__> anyone wants to add something about these bp/bugs?
15:04:22 <eglynn> might be slightly off-topic, but just wondering about non-mongodb support for metadata-query
15:04:47 <eglynn> related to the approach around instance metadata to identify autoscaling groups etc.
15:04:57 <eglynn> that we discussed the other day ...
15:05:22 <eglynn> requires that the alarm threshold eval logic is querying something like ...
15:05:30 <sandywalsh> eglynn, perhaps the schema work I'm doing around events might be applicable to that?
15:05:30 <eglynn> GET /v2/meters/cpu_util?q.op=eq&q.value=foobar&q.field=metadata.autoscaling_group
15:05:49 <jd__> sandywalsh: I think it will
15:05:52 <eglynn> sandywalsh: could be
15:05:52 <sandywalsh> Events have Traits ... where traits are essentially metadata
15:06:06 <jd__> but nobody's working on that actually eg
15:06:07 <jd__> but nobody's working on that actually eglynn
15:06:18 <jd__> I'm saying just to worry you more
15:06:25 <eglynn> :)
15:06:32 <thomasem> o/
15:06:43 * eglynn just had a wobble about building in a dependency on mongo only ...
15:06:55 <sandywalsh> yep, rightfully so :)
15:07:04 <gordc> it's just sql that needs metadata support right? i vaguely remember seeing a patch for hbase.
15:07:30 <eglynn> gordc: yep, it was mysql I had in mind
15:07:51 <eglynn> anyhoo, I've dragged the meeting off-topic
15:08:01 <eglynn> (not in scope for h2 I'd imagine)
15:08:30 <eglynn> (unless someone steps up with a sqlalchmey metadata-query patch)
15:08:48 <gordc> eglynn, i think we had some ppl reviewing how to support sql metadata query but i'm not sure about the status on that. might be taking another path.
15:08:59 <gordc> i'll let you know if i hear anything.
15:09:02 <eglynn> gordc: thanks!
15:09:02 <sandywalsh> eglynn, the event-based approach is a bit of a radical change since the meters wouldn't duplicate the metadata, but rather, point back to the underlying event. So, that might affect your efforts in many other ways.
15:09:55 <sandywalsh> eglynn, the meters may add new metadata (traits) though
15:10:12 <eglynn> sandywalsh: right, but still possible/efficient to do metadata/trait matching in meter queries?
15:10:18 <sandywalsh> (then the whole debate comes up about modeling meters as Events, but I'll leave that :)
15:10:47 <sandywalsh> eglynn, I think so. Once I wrap up this event collector, we're going to pound it to check performance
15:10:55 <eglynn> sandywalsh: cool
15:10:56 <sandywalsh> early tests are promising
15:11:02 <eglynn> nice!
15:13:12 <jd__> #topic Open discussion
15:13:22 <jd__> not many things to discuss anyway so go on :)
15:13:51 <sandywalsh> when's the Havana 2 deadline again?
15:14:06 <eglynn> release expected on July 18th IIRC
15:14:14 <sandywalsh> cool, thx
15:14:38 <sandywalsh> jd__, how are we looking on code reviews?
15:14:47 <jd__> pretty good I think
15:14:54 <sandywalsh> good to hear
15:14:55 <jd__> the backlog isn't giant
15:15:26 <eglynn> release expected on July 18th IIRC
15:15:32 <eglynn> doh
15:16:11 <eglynn> meh, serious lag on IRC today ...
15:16:12 <llu-laptop> sandy, about https://review.openstack.org/32714, is string of 255 large enough?
15:16:40 <llu-laptop> what the performance penalty of string 1024 compared to 255?
15:16:49 <sandywalsh> llu-laptop, million dollar question. But I think it's a good line in the sand. Don't want the Trait strings to be a dumping ground.
15:17:08 <sandywalsh> I'll be adding an external json table next so large strings should go there
15:17:34 <llu-laptop> sandywalsh: ok, got that
15:18:10 <sandywalsh> my fear is URL's right now ... but 255 should be ok for that (I hope)
15:19:00 <thomasem> sandywalsh: There would never be a larger URL than that?
15:19:35 <sandywalsh> thomasem, depends if someone does some crazy GET parameters
15:20:28 <sandywalsh> but, that gets into a json vs. string issue again. Don't want to try and figure that all out beforehand. Adjust accordingly later.
15:21:39 <thomasem> sandywalsh: Sure, hmmm?
15:23:03 <thomasem> sandywalsh: First thought that comes to mind, have the possibility of a long string if needed as a data type?
15:23:36 <sandywalsh> the problem is mysql and managing the page size
15:24:01 <sandywalsh> so they need to be normalized out of the trait table. And in that case, might as well use the text field for large strings (like json)
15:24:23 <sandywalsh> I think newer versions of mysql (or postgres) treat json as a first class citizen too.
15:25:05 <thomasem> sandywalsh: I don't know much about this, so I'll poke around and bring it up once I have better context.
15:25:38 <sandywalsh> I'm still figuring it out too ... so please share your insights :)
15:26:06 <thomasem> sandywalsh: Absolutely. :) Definitely something I want to learn about.
15:30:17 <xingzhou> one question on https://bugs.launchpad.net/ceilometer/+bug/1188649, have we considered to support mongodb replication set before? as it's a famous feature of mongo
15:30:18 <uvirtbot> Launchpad bug 1188649 in ceilometer "connect to mongodb with pymongo's mongo_client" [Undecided,Confirmed]
15:31:28 <sandywalsh> xingzhou, I think that's a deployment choice ... now the shard key choice is a very serious one we should consider.
15:31:56 <sandywalsh> xingzhou, but yeah, that would be the way to address that bug
15:32:20 <dhellmann> does the shard key need to be configurable, or is that something we would be able to set based on our knowledge of the data?
15:32:49 <sandywalsh> I think we need to decide on it, since it's so critical to balancing and query performance
15:32:57 <dhellmann> makes sense
15:33:07 <eglynn> is it possible to re-partition after the fact if unbalanced?
15:33:19 <sandywalsh> we've been talking about it a lot internally and have some opinions ... we should send it to the ml
15:33:34 <sandywalsh> eglynn, it is, but it's very expensive
15:33:43 <jd__> FWIW #1188649 is about HA, not sharding
15:33:48 <sandywalsh> and if you get it wrong mongo will spend all it's time rebalancing
15:33:59 <eglynn> k
15:34:06 <sandywalsh> jd__, yes, sorry, replica sets are the way to address that
15:34:09 <xingzhou> re jd__, its replica set
15:36:08 <jd__> closing in 2 minutes if nobody speaks :)
15:45:36 <jd__> #endmeeting