16:00:06 <nijaba> #startmeeting
16:00:07 <openstack> Meeting started Thu Jun 14 16:00:06 2012 UTC.  The chair is nijaba. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:16 <nijaba> #chair nijaba dachary
16:00:16 <nijaba> #link http://wiki.openstack.org/Meetings/MeteringAgenda
16:00:17 <openstack> Current chairs: dachary nijaba
16:00:38 <nijaba> Hello everyone!
16:00:45 <nijaba> #topic actions from previous meetings
16:00:57 <nijaba> #topic  dhellmann: submit plugin branch for review and merging
16:01:10 <nijaba> dhellmann: any news on this?
16:02:42 <nijaba> ok, looks like doung went to get a coffee ;)
16:02:45 * ttx lurks
16:03:01 * dachary drinks coffee
16:03:10 <nijaba> we'll get back to it in a moment
16:03:12 <nijaba> #topic nijaba: to propose a google spreadsheet calculator to estimate volume of metering message (including nova, swift, cinder, quantum)
16:03:28 <nijaba> so here is what I came up with:
16:03:29 <nijaba> #link https://docs.google.com/spreadsheet/ccc?key=0AtziNGvs-uPudDhRbEJJOHFXV3d0ZGc1WE9NLTVPX0E
16:03:41 <dachary> I tried it and it worked for me.
16:03:53 <nijaba> did not get that many comments about it
16:03:57 <dachary> Also I agree with the assumption related to the size of the metadata payload.
16:04:06 <dachary> That was my primary concern.
16:04:13 * dhellmann sorry, noisy office today
16:04:22 <dachary> Although the size is large, I don't think it's out of proportion.
16:04:40 <nijaba> well, to me it shows that being able to fine tune frequency could have a huge impact on vilume
16:04:51 <nijaba> volume even
16:05:26 <dhellmann> it also shows that we probably won't want to store everything long term, but aggregate the data
16:05:30 <nijaba> so, should I link this gdoc in the blueprint?
16:05:39 <dhellmann> +1
16:05:54 <dachary> +1
16:05:57 <nijaba> +1 (obviously)
16:06:21 <nijaba> #action nijaba to point to the calculator in the blueprint
16:06:36 <nijaba> #topic  dhellmann: submit plugin branch for review and merging
16:06:43 <dhellmann> that's done and merged
16:06:51 <nijaba> ok cool
16:06:56 <dhellmann> with the renaming (next action?)
16:07:03 <nijaba> #topic dhellmann rename plugin to engine for storage backend ex-plugin-now-engine system
16:07:11 <nijaba> so that's done too!
16:07:11 <dhellmann> also done
16:07:16 <nijaba> thanks
16:07:22 <nijaba> #topic dhellmann: start mapping API queries to database engine methods
16:07:34 <dhellmann> I haven't made any progress on that. :-/
16:07:40 <dhellmann> I plan to start this afternoon, after this meeting
16:07:49 <nijaba> should we forward it to next week?
16:08:00 <dhellmann> yes, please
16:08:01 <nijaba> #action dhellmann: start mapping API queries to database engine methods
16:08:08 <nijaba> thanks!
16:08:21 <nijaba> #topic Discuss and hopefully agree on meter configuration mechanism
16:08:33 <nijaba> this was my latest proposal:
16:08:35 <nijaba> #link http://www.mail-archive.com/openstack@lists.launchpad.net/msg12859.html
16:08:55 <nijaba> any comments?
16:09:05 <dachary> I spent the day talking about puppet & mongodb and I'm convinced your proposal makes sense ;-)
16:09:09 * dhellmann has not changed his mind about leaving this for the ops team to deal with
16:09:37 <dachary> If mongodb left all for the ops to do, I'm sure they would have a really hard time.
16:09:50 <flacoste> dhellmann: that means leaving the configuration local to each agents and ops deals with that problem?
16:10:07 <flacoste> that: pushing out and syncing changes
16:10:23 <dhellmann> flacoste, right. put the config in a text file, like everything else, and use the same tool to push it out that we use for the other files
16:10:26 <nijaba> flacoste: yes, that's what we discussed last week
16:10:48 <nijaba> but I am claiming that we are going to have data consistency issues
16:10:50 <dhellmann> I won't block work on this, but it feels like the wrong thing to be focusing on right now
16:11:13 <flacoste> nijaba: i think dhellmann argument is that this is a sovled ops problem
16:11:17 <flacoste> with tools like puppet and checf
16:11:25 <flacoste> you don't have syncing issue
16:11:27 <nijaba> dhellmann: would you agree to put it in our todo, and see if someone picks it up?
16:11:34 <flacoste> puppet espeically will make sure that everything is synced
16:11:41 <nijaba> flacoste: yes you do
16:12:02 <flacoste> nijaba: care to explain which syncing problems remain?
16:12:06 <dhellmann> nijaba, that's fine, I guess
16:12:10 <flacoste> assuming puppet is used?
16:12:32 <flacoste> i agree that if you aren't using puppet, you have a problem to solve
16:12:51 <nijaba> well, experience shows that sometime classes are missused, and then you fall into a data consistency problem that cannot be reversed synced
16:13:14 <nijaba> so I am advocating that we neeed to care where data consistency is crucial
16:14:04 <nijaba> anyway, I propose that we put it in a todo list, and see if we can have someone eventuall do it
16:14:08 <dachary> flacoste: how do you broadcast a change to all agents using puppet so that they all agree at a given point in time ?
16:14:12 <dhellmann> there are lots of other parts of the config that have to be consistent in order for OS to work. message bus credentials, for example
16:14:49 <nijaba> dhellmann: yes, but this translate to immediate operational disfunction, not woopies that you discover in your next billing cycle
16:15:10 <nijaba> say a month later
16:15:35 <flacoste> nijaba: using cast won't solve that problem fwiw
16:15:50 <dachary> dhellmann: yes, and that's also true of any mongodb deployment. They *partly* rely on puppet / chef. And also communicate with each other to ensure a consistent setup. Such as voting in replica sets and what not.
16:15:53 <flacoste> nijaba: it offers no garantee on when all agents are going to update their config
16:16:15 <jd___> and that's still hypothetical
16:16:18 <nijaba> flacoste: ok, what would you recommend then?
16:16:28 <flacoste> YAGNI
16:16:32 <flacoste> for now :-)
16:16:33 <dhellmann> dachary: realtime voting on HA/failover feels like a different sort of problem than this
16:16:45 <flacoste> the stringent requirement that all agents be synced at the same time is overkill i think
16:17:22 <flacoste> who cares that there is 30s window where different agents have different configs?
16:17:31 <nijaba> ok.  shall we vote?
16:17:33 <flacoste> if that's a must-have, your solution doesn't cut it
16:18:02 <dachary> dhellmann: true. Think about the master / slave relationship between mysql servers. then. It does not go only in one way. The slave communicates with the master and vice versa. Both are deployed using puppet / chef. But if that was their *only* mean of ensuring configuration consistency, they would be in trouble.
16:18:43 <dhellmann> do we have a requirement that all agents be configured exactly the same way?
16:19:25 <nijaba> dhellmann: for the data to be useable, I would think so
16:19:26 <jd___> dhellmann: not for now
16:19:33 * dhellmann smiles
16:19:38 <nijaba> Vote on: shall we put meter configuration mechanism in our todo?
16:19:41 <flacoste> yeah, i'd say that's too early to know
16:19:43 <jd___> and if we have, maybe it's a problem about the accounting format
16:19:44 <nijaba> +1
16:19:47 <flacoste> -1
16:19:49 <dhellmann> -1
16:19:56 <jd___> -1
16:20:25 <nijaba> #agreed push meter configuration to a future version
16:20:35 <dachary> +1
16:20:48 <dhellmann> nijaba, maybe the thing to do next on this is to prepare a more detailed problem description and some requirements? maybe I'm just not seeing something...
16:20:52 * nijaba wonders if he should use #agree or #agreed ?
16:21:12 <dhellmann> the bot instructions say agreed
16:21:30 <dhellmann> also, startvote appears to be a command
16:21:35 <nijaba> dhellmann: as flacoste said, let's wait until we see if there is a problem or not
16:21:44 <dhellmann> nijaba: that works for me
16:21:49 <nijaba> #agree push meter configuration to a future version
16:22:02 * nijaba will use startvote next time :)
16:22:16 <nijaba> #topic Discuss proposing ceilometer as an incubated project
16:22:29 <nijaba> I asked ttx to lurk for this part
16:22:43 <nijaba> as I think his advice could be usefull
16:22:52 <nijaba> so we now have a good skeleton
16:23:00 <nijaba> a defined architecture
16:23:19 <nijaba> is it the right time to propose our project for incubation in folsom?
16:23:47 <dachary> I think it is, indeed.
16:23:56 <dhellmann> is there a document that describes what we need to do to have it considered?
16:24:19 <nijaba> ttx: ^ I think you would be best placed to answer that one
16:25:41 <nijaba> dhellmann: IIRC, there is no other requirement than having something the tb can look at and send a formal request via the mailing list
16:25:50 <dhellmann> FWIW, I agree we should try, I just want to make sure we've met the "qualifications" if there are any defined
16:25:57 <dhellmann> ok, well, I think we've got that :-)
16:26:25 * nijaba thinks so too, thanks to you an jd___
16:26:26 <dhellmann> do we need to document more of the plans for future work?
16:27:02 <nijaba> well, it woudl be a good idea in any case
16:27:08 <dhellmann> true
16:27:49 <nijaba> I guess ttx is doing something else
16:27:55 <nijaba> should we vote?
16:28:06 <dachary> yes, please
16:28:28 <nijaba> #startvote propose ceilometer as an incubated project
16:28:32 <nijaba> +1
16:28:32 <dhellmann> +1
16:28:34 <flacoste> +1
16:28:41 <jd___> +1
16:29:17 <dhellmann> oops: http://ci.openstack.org/meetbot.html#voting
16:29:19 <dhellmann> yes
16:29:22 <dachary> +1
16:29:28 <dhellmann> #vote yes
16:29:32 <nijaba> #endvote
16:30:13 <nijaba> #agree  propose ceilometer as an incubated project
16:30:29 <nijaba> #action nijaba to send a proposal to the mailing list
16:30:45 <nijaba> #topic Prepare documentation and framework for plugin contributors
16:31:13 <nijaba> so, I think it is time that we lay the ground for plugin/agent contributor
16:31:18 <dhellmann> +1
16:31:23 <nijaba> and this requires to have:
16:31:33 <nijaba> a sample agent implementation
16:31:36 <nijaba> and documentation
16:31:45 <nijaba> I'll be happy to work on the doc
16:31:59 <nijaba> but I'll have to rely on someone to work on the sample
16:32:01 <dhellmann> I can get us set up on readthedocs.org
16:32:17 <dhellmann> we could use the existing code as examples, couldn't we?
16:32:23 <ttx> back
16:32:43 <ttx> document. incubation, looking.
16:32:49 <nijaba> ttx: can you read the backlog and tell what you think about incubation for ceilometer?
16:33:20 <nijaba> dhellmann: it would be nicer if we had a proper "sample/start you project here"
16:33:36 <dhellmann> ah, I see what you mean
16:33:49 <nijaba> atm, you have to know where to look
16:33:53 <dhellmann> I thought the existing code would be useful as a walk-through example because it actually does something
16:34:03 <dachary> dhellmann: I agree.
16:34:13 <dhellmann> oh, sure, we need to put it in the docs and write about it, I just meant instead of making up a "fake" simple example, use a real one
16:34:39 <nijaba> dhellmann: ok.  if you tell me which one to use, I'll start the doc if you want
16:34:49 <dachary> it might be enough to say : look at this one, it is well commented & up to date. Take a look at the tests, they are good and extensive.
16:34:54 <dhellmann> we need separate examples for each of the plugin types
16:35:12 <dhellmann> #action dhellmann: look at existing plugins and pick one of each for examples in docs
16:35:20 <nijaba> cool
16:35:34 <dhellmann> dachary, we probably want to "freeze" the example so the text does not get out of date with the line numbers in the code
16:35:43 <dhellmann> we can copy a version into the doc tree for that purpose
16:35:43 <nijaba> I'll pick up the doc action next week then :)
16:35:52 <nijaba> agreed
16:35:59 <dachary> dhellmann: good idea indeed
16:36:04 <dhellmann> #action dhellmann: email info on examples to nijaba
16:36:34 <nijaba> #action nijaba to prime the doc once info received from dhellmann
16:36:57 <dhellmann> we should probably include an "ops" section, even though that will eventually move into the regular admin guides
16:37:05 <nijaba> should we shortly move back to the incubation topic?
16:37:21 <dhellmann> yes, let's get ttx's input on that
16:37:30 <ttx> Project types = http://wiki.openstack.org/ProjectTypes
16:37:31 <nijaba> #topic incubation
16:37:42 <ttx> the form for incubation is...
16:37:45 <nijaba> #link http://wiki.openstack.org/ProjectTypes
16:37:54 <ttx> http://wiki.openstack.org/Governance/Approved/Incubation
16:38:24 <nijaba> #link http://wiki.openstack.org/Governance/Approved/Incubation
16:38:25 <ttx> some pieces of it might be a bit outdated, but that's still official procedure :)
16:38:52 <nijaba> ttx: so, do you have some gut feeling about readiness of ceilomet for application?
16:39:20 <dachary> ttx: is there an example of a recently accepted application to get inspiration on the wording etc. ?
16:39:29 <nijaba> in other words, would you advice us to apply at this stage?
16:41:07 <ttx> I think it's ready to be incubated. The main issue is whether it belongs to the scope of what we call "OpenStack"
16:41:22 <ttx> I expect the discussion to revolve around that
16:41:24 <nijaba> I personally would think so
16:41:32 <nijaba> but that's q good time to know
16:41:43 <ttx> The PPB is still in cahrge of that, until the foundation is finally set up
16:42:09 <flacoste> ttx: what aspect of ceilometer is bringing that question?
16:42:13 <dachary> ttx: why wouldn't it be in the scope of OpenStack ?
16:42:14 <dhellmann> do we need to put together an argument in favor?
16:42:18 <ttx> so depending on whether you think you'll get more support from the future TC and Foundation board, or from the current PPB, it mught be a good idea to hold :)
16:42:48 * nijaba has absolutely no idea why one woulld be more favorable than the other...
16:42:59 <ttx> OpenStack is the basic blocks of IaaS. You can argue that metring is as critical vbase piece as .. say.. common auth
16:43:29 <nijaba> IaaS without metering = crapware ;)
16:43:30 <ttx> So good luck :)
16:43:34 <nijaba> k
16:43:50 <nijaba> so, anyone think we should hold?
16:44:00 <dachary> ttx: I see your point now.
16:44:01 <dhellmann> if we're rejected now we can try again, right?
16:44:39 <dachary> nijaba: I think there are good arguments to advocate that ceilometer is an essential part of openstack. We should not hold, in my opinion.
16:44:43 <nijaba> well, if the ppb says it is out of scope, then I think it will be tough for future TC to say otherwise
16:45:06 <dhellmann> it's likely to be the same people, isn't it?
16:45:06 <nijaba> but I think now is a good time to test our arguments
16:45:34 <nijaba> nope, TC will be composed of the paying foundation menbers + some devs
16:45:42 <nijaba> while today it is only the project leads
16:45:47 <dhellmann> we should probably ask for supporting comments from some ops folks, too, to include in the proposal
16:46:15 <nijaba> dhellmann: would seem like a good idea
16:46:22 <ttx> nijaba: no. TC is all elected
16:46:28 <dhellmann> oh, I thought the TC was still going to be mostly project leads, but I haven't followed that too closely
16:46:50 * nijaba also need to dig on that one
16:47:00 <ttx> dhellmann: all elected, PTLs included.
16:47:13 <ttx> http://wiki.openstack.org/Governance/Foundation/TechnicalCommittee
16:47:19 <dhellmann> ok, so possibly different makeup
16:47:23 <nijaba> anyway, I propose to revise my action to "prepare the application for review at the next meeting"
16:47:26 <ttx> anyway, no way to tell if waiting increases your chances :)
16:47:37 <dhellmann> nijaba, that sounds like a good adjustment
16:47:41 <dachary> nijaba: ok
16:48:01 <dhellmann> ttx, has anyone made an argument that metering shouldn't be included?
16:48:02 <nijaba> #action nijaba to prepare incubation application for review at the next meeting
16:48:30 * nijaba bets no one has made an argument either way
16:48:56 <nijaba> shall we move on?
16:48:59 <dachary> dhellmann: I think the strongest argument against would be "Why is it bad that ceilometer is not part of OpenStack ?".
16:49:10 <nijaba> #topic Establish communication with swift/quantum/cinder on best points of interaction for our agents
16:49:34 <dhellmann> dachary, that's something to think about
16:49:39 <dhellmann> nijaba, nova, too?
16:49:39 <nijaba> so it seems that we need to start discussing with other projects how to best integrate with them
16:49:49 <ttx> dachary: ++
16:50:17 <dachary> and that's probably be the best advocacy. If swift / quantum / nova etc. all agree on ceilometer one way or the other, it will become part of their roadmap.
16:51:14 <nijaba> so, shall we divide the task among ouselves and join their next meetings to introduce ceilometer and what we would need?
16:51:36 <dachary> nijaba: that's a good idea actually
16:51:59 <nijaba> anyone care about some project in particular?
16:52:02 <dachary> I've not had much success in communicating with Dragon but I can try again. I've not been very persistent.
16:52:15 <nijaba> I bet dhellmann does not care much for swift, for example
16:52:22 <dhellmann> :-)
16:52:50 <dachary> #action dachary talk to Dragon about SystemData / ceilometer and try to create cooperation
16:52:58 <dhellmann> I think we're most interested in quantum at this point
16:53:06 <dhellmann> (by "we're" I mean DreamHost)
16:53:14 <nijaba> feel free to take the action then
16:53:26 <dhellmann> #action dhellmann to talk to Quantum devs about integration with ceilometer
16:53:39 <nijaba> flacoste: do you want to take one?
16:53:56 <flacoste> nijaba: i'd rather not for the moment tbh
16:54:04 <nijaba> ok
16:54:09 <flacoste> i don't have relationships with any projects
16:54:15 <dachary> nijaba: I'll take swift
16:54:17 <flacoste> and hiring a new squad is keeping me real busy :-)
16:54:28 <nijaba> ok, I'll take cinder then
16:54:28 <dachary> unless someone else wants to ;-)
16:54:45 <nijaba> flacoste: please concentrate on that, for sure!
16:54:47 <dachary> #action dachary to talk to swift devs about integration with ceilometer
16:55:01 <nijaba> #action nijaba to talk to cinder devs about integration with ceilometer
16:55:21 <nijaba> nova, anyone?
16:55:30 <dachary> nijaba: that would be me (Dragon)
16:55:45 <dachary> #action dachary talk to Dragon about SystemData / ceilometer and try to create cooperation (i.e. nova)
16:55:55 <nijaba> duh....
16:56:10 <dhellmann> dachary, I think I'm going to end up working on a patch to add more details to notification messages coming from nova
16:56:15 <nijaba> ok, so the missing one is glance then
16:56:41 <nijaba> I can take it if noone wants it
16:56:51 <nijaba> ...
16:56:54 <jd___> i'll do
16:57:00 <nijaba> thanks jd!
16:57:24 <nijaba> #topic Status of the essex compatibility effort that jd is leading
16:57:26 <dachary> dhellmann: I'll submit messages to you for approval before actually sending them. Review is good.
16:57:59 <nijaba> jd___: 2 minutes left if you want to give us some update
16:58:00 <dhellmann> dachary, sounds good. I thought you might want to mention that we're willing to contribute code, not just asking for them to do stuff. :-)
16:58:30 <dachary> dhellmann: good idea indeed. The code you wrote will be our best ambassador ;-)
16:58:41 <jd___> I've send review requests to os-infra to have some tests automated on essex, and waiting for approval
16:58:44 <nijaba> dhellmann: ++
16:59:07 <nijaba> jd___: great.  so we should go do some reviews?
16:59:15 <nijaba> jd___: any blockers found?
16:59:21 <jd___> it's at https://review.openstack.org/#/c/8222/ FYI
16:59:27 <nijaba> thanks
16:59:52 <dhellmann> I'll take a look after lunch and give a +1
17:00:03 <nijaba> #action jd___ to talk to glance devs about integration with ceilometer
17:00:03 <jd___> so far the tests passes, so it's good, it should work for essex for most (tested) parts
17:00:26 <jd___> I know that the daemon won't actually run but we don't have test for that so far :)
17:00:31 <dachary> thank's all !
17:00:38 <nijaba> neat!!!  I think we should keep that topic for next weel, as we are out of time...
17:00:45 <nijaba> thanks all!
17:00:47 <jd___> ack
17:00:51 <jd___> thanks
17:00:51 <nijaba> #endmeeting