21:00:07 #startmeeting Ceilometer 21:00:07 #meetingtopic Ceilometer 21:00:07 #chair nijaba 21:00:07 #link http://wiki.openstack.org/Meetings/MeteringAgenda 21:00:07 ATTENTION: please keep discussion focused on topic until we reach the open discussion topic 21:00:08 Meeting started Wed Jan 30 21:00:07 2013 UTC. The chair is nijaba. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:09 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:11 The meeting name has been set to 'ceilometer' 21:00:13 Current chairs: nijaba 21:00:17 Hello everyone! Show of hands, who is around for the ceilometer meeting? 21:00:17 o/ 21:00:19 o/ 21:00:20 o/ 21:00:21 o/ 21:00:25 o/ 21:00:25 o/ 21:00:28 o/ 21:00:59 o/ 21:01:18 o/ 21:01:26 nice showup! thanks. asalkeld is at at linuxconf.au and may join us if he can. 21:01:34 #topic actions from previous meeting 21:01:42 #topic nijaba to specify draft policy on wiki for units 21:01:42 My deepest apologies, I still haven't been able to work on this 21:01:42 #action nijaba to specify draft policy on wiki for units 21:02:11 I won't leave to much time for others to blame me :) 21:02:14 #topic nijaba to untarget synaps bp for now 21:02:14 #info That was done. 21:02:34 #nijaba to propose a doodle for helthmon meeting 21:02:34 #info Done as well. The meeting occured today and allowed good progress. More to come at the summit. 21:02:39 comments on this? 21:02:47 o/ 21:03:12 I guess not! 21:03:15 the meeting went well, and I'm looking forward to seeing the blueprints that come from it 21:03:18 my gut says that healthmon, as proposed today, steps on a lot of other projects toes 21:03:26 so getting them in the fold is key 21:03:35 sandywalsh: +1 21:03:35 and I think they'll span many projects 21:03:43 +1 again 21:04:00 Good to see agreement! 21:04:19 sounds like some interesting sessions for the summit 21:04:22 +1 too fwiw 21:04:42 yeah, the summit should be interesting ... lots of whiteboards :) 21:04:49 hehe 21:04:50 good progress indeed...I was impressed by the open conversation. Look forward to hearing more. 21:05:42 moving on... 21:05:45 #topic nijaba to formaly start ml thread about core dev startus for yjiang5_home and llu-laptop 21:05:45 #info This was done. We'll discuss this is a moment 21:05:56 #topic sandywalsh to start a thread about adopting https://github.com/openstack/nova/blob/master/HACKING.rst 21:05:56 #info This was done and we'll vote on the adoption in a moment 21:06:08 That's it for last week's actions 21:06:22 #topic yjiang5 and llu-laptop are now core dev for ceilometer 21:06:28 I would like to formally congratulate both of them, even though they are certainly asleep at this time. I just added them to the ceilometer driver's group. 21:06:28 #info congratulations 21:06:44 +1 21:06:46 really thanks for your trust and will contribute more to the project :) 21:06:48 it's nice to see the team continuing to grow 21:06:52 (or +2 I guess) 21:06:59 that's good news 21:07:00 yjiang5_home: llu-laptop: good to see you around at such a late time for you 21:07:02 absolutely, welcome guys! 21:07:12 thanks, looking forward to contirubte more. 21:07:20 * jd__ needs more +2 on his patches :-) 21:07:33 * dhellmann needs more hours in the day 21:07:40 * nijaba too 21:07:47 * jd__ pats dhellmann 21:07:55 nijaba: has the technical side of adding them been done already? 21:08:02 jd__: just did 21:08:07 awesome! 21:08:49 llu-laptop: yjiang5_home: feel free to turn your past +1 in +2 then :) 21:08:58 jd__: :) 21:09:16 llu-laptop: yjiang5_home: please remember not to use the approve status unless you are the second one applying a +2 to a patch 21:09:32 yeah, that can be tricky 21:09:46 as soon as you set 'Approve' the patch is merged 21:09:54 right, no matter what the other setting is 21:10:09 nijaba: sure, thanks for reminding. we will learn the gerrit doc firstly. 21:10:09 llu-laptop: yjiang5_home: you know where to find us if you have any questions 21:11:04 yes 21:11:11 looking in the gerrit, not find the +2 options :-( 21:11:40 llu-laptop: it may need some time for the group membership to propagate 21:11:47 unless I missed something 21:11:49 I think there's a cron job that has to update that configuration setting. 21:12:22 or maybe we were joking about adding you to the core team, you'll see 21:12:24 llu-laptop: you did receive and email with your new group membership, right? 21:12:43 s/and/an 21:12:57 nijaba: yes 21:13:14 llu-laptop: then it should propagate soon 21:13:28 let's move on? 21:13:34 nijaba: ok, i'll wait for that then :-) 21:14:02 #topic Vote on adopting the hacking guidelines 21:14:02 #link https://github.com/openstack/nova/blob/master/HACKING.rst 21:14:02 Any comments before we vote on the subject 21:14:23 yep 21:14:23 HACKING requires translation, I'm not sure if we are ready for that. 21:14:38 others should be ok 21:14:54 I'm really not confortable having rules that should be detected automagically by rules 21:15:09 isn't there a tool for checking? 21:15:11 how much of the HACKING guidelines are enforcable via the pep8 checks? 21:15:22 dhellmann: what I'm thinking is typically the import alphabetical order 21:15:31 this is a PITA, there's no tool for that currently 21:15:38 there is hacking.py for checking 21:15:39 and nitpicking in the review for that is going to make me cry 21:15:49 sandywalsh: can we put it with tox? 21:15:56 jd__: I thought hacking.py checked that 21:16:04 #link https://github.com/openstack/nova/blob/master/tools/hacking.py 21:16:10 jd__: I don't see why not 21:16:10 I don't know hacking.py, I only saw HACKING.rst :) 21:16:28 ok 21:16:28 +1 for enforcement 21:16:33 jiang5_home: the transaltion framework is already done. What we need to do is to annotate those strings to be translated with _(), and did the actual translation on transifex.com. All others will be done by jenkins. 21:16:37 +1 21:16:41 (it's the openstack way) 21:16:45 if we do adopt it, we should only have hacking.py turned on non-voting in jenkins until we get it passing 21:16:53 dhellmann: +1 21:16:55 well as long as we can have tools helping us follow any hacking guide and not having humans nitpicking manually, I'm all good 21:17:00 I don't want to stop everything else just to fix style issues 21:17:05 dhellmann: +1 21:17:18 dhellmann: +1 21:17:21 ready to formally vote then? 21:17:27 yes 21:17:42 #startvote adopt the hacking guidelines? yes, no, abstain 21:17:43 Begin voting on: adopt the hacking guidelines? Valid vote options are yes, no, abstain. 21:17:44 Vote using '#vote OPTION'. Only your last vote counts. 21:17:51 #vote yes 21:17:52 #vote yes 21:17:53 #vote yes 21:17:53 #vote yes 21:17:54 #vote yes 21:17:55 #vote yes 21:17:59 #vote abstain 21:18:00 #vote yes 21:18:02 #vote yes 21:18:25 look like a done deal 21:18:33 #vote no 21:18:37 just because I can 21:18:40 haha 21:18:44 anyone wants to take #action for importing hacking.rst and hacking.py into our git and add this to a non-voting check in jenkins? (can be 2 #action) 21:18:53 #vote yes 21:18:58 thanks guys, it'll make code reviews cause less neck twitches 21:19:00 #endvote 21:19:00 Voted on "adopt the hacking guidelines?" Results are 21:19:01 yes (9): apmelton, n0ano, sandywalsh, jd__, eglynn, llu-laptop, dhellmann, danspraggins, yjiang5_home 21:19:02 no (1): nijaba 21:19:35 #action sandy will integrate hacking.py 21:19:42 thanks sandywalsh 21:19:49 (since he caused this mess :) 21:20:01 thanks sandywalsh :) 21:20:11 yes, thanks! 21:20:19 #topic Approve http://wiki.openstack.org/NewCeilometerAgent ? 21:20:19 SandyWalsh would like to see this blueprint approved. 21:20:39 That seems a big change for me, late in the cycle 21:20:58 We can approve without targetting at G, right? 21:21:03 nijaba: the proposal is to do it in parallel, without upsetting the current agent 21:21:17 before we schedule it, let's discuss whether we want it at all 21:21:27 branches may land, but have no effect (it's like a experimental branch) 21:21:35 *an 21:21:38 dhellmann: to me, we want it if Nova offers the mechanism to do it 21:21:40 sandywalsh: do you have the ressources to deliver it before g3? 21:21:47 nijaba: I'm working on it now 21:22:00 sandywalsh: will it get into Nova for g3? 21:22:04 jd__: I do have a couple of issues 21:22:11 jd__: we'll make sure nova will have what it needs 21:22:14 dhellmann: raise, raise then :) 21:22:20 sandywalsh: so, your proposal is to add this as a new mechanism, not to replace? 21:22:27 nijaba: correct 21:22:34 Yup, I'm working on nova for this too. 21:22:36 nijaba: we can decide later on the best approach 21:22:40 well, for starters, the blueprint assumes that the only information we are interested in collecting from the hypervisor is information that nova is (or will) collect 21:22:43 mmmh... that removes some of my objections then 21:22:57 when we tried to extend the hypervisor API in nova before to collect more data, we were rebuffed 21:23:18 so I'm not sure what will be different about that this time 21:23:23 dhellmann: myself and dragon have done a lot of stuff in those layers before 21:23:32 yes. 21:23:41 sandywalsh: the objection was in principle, not over any specific patch or implementer 21:23:44 dhellmann: yeah that's the point I raised to sandywalsh, but he seems confident to change things on Nova side :) 21:23:47 dhellmann: Will ceilometer monitor more host information like host utilization etc? 21:23:54 that is, they didn't want nova collecting the data 21:24:05 yjiang5_home: that is a long-term goal, yes 21:24:42 dhellmann: the precedent is already set on the xen side 21:24:53 dhellmann: then we have two point for this bp: a) will nova provide the instance information, b) will ceilometer need more information that possibly out of nova scope? 21:24:56 dhellman: in that case, I dont think nova will provide enough information in a long-term goal. 21:25:00 sandywalsh: you mean xcp? 21:25:02 i mean, there is a chance we'll be rebuffed again, but it will make for a wonderful debate :) 21:25:10 sandywalsh: I would have to go look at that email thread again to see what was being added before. eglynn may remember. 21:25:25 * nijaba takes it that sandywalsh loves a good debate 21:25:37 hi 21:25:43 (sorry late) 21:25:45 hey asalkeld 21:25:53 dhellmann: I need to refresh my memory also 21:25:56 asalkeld: hope the conf is good 21:26:01 if we do win the debate over the nova changes, that still leaves the issue of collecting data not related to instances 21:26:05 yea, thx 21:26:22 nova does some of that for the scheduler, and we've discussed the idea of moving that into ceilometer 21:26:34 I think we've already determined that there is still a need for another agent. This proposal is the 90% we can cover ... the most common deploys 21:26:48 we could leave it in nova, except that then we have 2 projects doing data collection still 21:27:05 if there are situations where we can't get everything we need we may have to use an agent again, but I think they are very special cases 21:27:09 and deployment specific 21:27:23 I'm not sure how collecting scheduler data is deployment specific. 21:27:56 it's not related to the scheduler 21:28:09 the stuff the scheduler needs is what's there now (ram + disk, cpus, etc) 21:28:15 (I put it there :) 21:28:31 we were going to be the data collector for scheduler at some point 21:28:41 or some suggested it 21:28:50 this can be an argument for sandywalsh 21:28:51 sure, no reason why you still can't be ... depends on latency really 21:29:01 asalkeld: vishy did 21:29:06 the scheduler needs very timely data 21:29:15 CM could be too slow for it 21:29:16 right, vishy and I discussed it briefly at the grizzly summit 21:29:37 sandywalsh, just how timely 21:29:46 asalkeld: near real-time 21:29:49 second/minute? 21:30:07 the longer the delay the more over allocations will occur 21:30:12 collecting the data in a separate process elimintes issues with nova operations affecting the timed process 21:30:25 although some of that may be fixed with belliot's allocation code 21:30:55 that's why we were using the fanout queues from the compute nodes to all the schedulers 21:31:16 broadcast updates 21:31:46 that change is not a simple one 21:31:47 but, perhaps this is a discussion for another topic ... the core premise I think is still sound 21:31:55 one issue, are we talking about going from a push (nodes send data to scheduler) to a pull model (scheduler asks CM for info)? 21:32:06 n0ano: not necessarily 21:32:14 being: we can do most all we need with notifications and a simpler agent model 21:32:24 the mandate for this project is to collect and distribute this sort of data 21:32:47 n0ano: there can be different schedulers do things different ways 21:33:00 in some respects I like CM pushing data as it could send data about multiple nodes in a single message 21:33:01 n0ano: it's just how they update their HostInfo records 21:33:25 sandywalsh: does this plan also include some alternate solution for the instance delete event notifier plugin (mentioned in point 1 of the blueprint)? 21:34:07 dhellmann: it's just another event in the stream, shouldn't be any different than any other message 21:34:30 receiving that event is not, but the reason for the notifier is the race condition between generating it and the instance being deleted 21:34:42 we want to collect final stats on the instance *before* it is deleted 21:34:51 and we don't want to charge users for the time it takes us to delete the thing 21:34:57 we need to capture the last dying breath of the instance ;) 21:35:13 so the plugin blocks the operation, collects stats and sends them to the bus, then continues 21:35:35 dhellmann: in that case, nova should send the additional information in the delete notification. This would require a patch to nova, which we are willing to add. 21:35:44 (in the .start event) 21:35:53 sounds great 21:36:04 yep, reasonable solution 21:36:17 I think that's basically already there, if we want to change only up until the user tries to delete it, then timestamp on the start should suffice 21:36:18 yah, we can always add data to a notification. 21:36:20 that would be fine. again, we had some of those sorts of changes rejected early on 21:36:56 apmelton is working on adding this to stacktach currently :) 21:37:21 I think this is a low-impact BP overall 21:37:26 with great upside 21:37:50 I am not sure we have a consensus though 21:38:10 should we hava a core-dev only vote to check? 21:38:12 I'm happy to ease your troubled minds :) 21:38:12 I think we agree that Nova needs to have some code before this blueprint is doable 21:38:19 dhellmann: sandywalsh: can we take a step-by-step method ? If anything changed successfully in nova, we can remove the corresponding code in ceilometer, and in the end possibly the compute agent will have nothing to do. 21:38:25 jd__: I agree with that 21:38:31 yjiang5_home: yeah totally. 21:38:36 sounds good. 21:38:36 jd__: they can be done in parallel 21:38:38 +1 21:38:50 now that sounds more like one 21:38:51 yjiang5_home: that makes sense 21:38:56 yjiang5_home: the new agent is so small it hardly makes sense. it's easier to cut-and-run 21:38:56 sandywalsh: sure, but we won't merge anything in Ceilo until Nova has the parts we need though 21:39:17 nijaba: see how I can build a consensus by stating the obvious! :-) 21:39:26 jd__: great job! 21:39:32 we used to support older versions 21:39:42 sandywalsh: can current notification code be used for your agent temply? 21:39:43 is that not an issue anymore 21:39:45 asalkeld: until g2 21:39:52 this is sort of how we got zones/cells going, we merged the new functionality in but it was all disabled by default 21:39:58 people could play with it if they wanted 21:40:05 but otherwise had no idea it was there 21:40:07 asalkeld: that's whart we agreed 21:40:13 ok 21:40:20 yjiang5_home: for a lot of it yes 21:40:29 yjiang5_home: the delete scenario might need some fix 21:40:48 yjiang5_home: and the libvirt side might need some tweaks (dragondm is looking into that) 21:40:56 jd__: care to write up the (#)agreed ? 21:41:03 nijaba: sure 21:41:37 sandywalsh: so would it be possible that: a) change nove b) add to current notification code, c) in the end, remove compute agent, d) update the notification code as the type you suggested? 21:41:42 #agreed sandywalsh & cie to work on implementing feature in Nova in order to deprecates the Ceilometer compute agent in favor of notifications consuming 21:42:06 perfect, thanks 21:42:07 does that cover enough for you sandywalsh? 21:42:56 yjiang5_home: how about I get an agent working that stuffs all the current events into CM as they stand today (as a proof of concept) and in parallel we work on the nova changes? 21:42:57 "_ 21:42:58 :) 21:43:04 jd__: cie? 21:43:11 & co 21:43:17 ah :) 21:43:19 so we can still use the compute agent, just not needed? 21:43:19 yeah sorry 21:43:20 :-> 21:43:26 french... 21:43:28 cie is probably French :D 21:43:34 (thinking of montoring) 21:43:36 * jd__ mixed up 21:43:45 asalkeld: yes, everything stays the same, this will be an alternative agent 21:43:49 sandywalsh: if nova generates the notifications, the existing collector should receive them without any major changes 21:43:57 although we need to make it listen to the error events 21:43:57 yup 21:44:01 ok, cool with that 21:44:25 next topic? 21:44:29 yup 21:44:40 sandywalsh: dhellmann: yes 21:44:44 jd__: your summary looks good to me 21:44:45 #topic A note about https://blueprints.launchpad.net/ceilometer/+spec/use-new-rpc-messsage 21:44:45 jd__ wanted to bring up somethinkg about this: you have the floor. 21:44:53 (I need to head off to keynote) 21:44:58 later guys 21:45:01 asalkeld: have fun 21:45:05 asalkeld: see ya 21:45:06 yeah, I've removed the target on this blueprint actually 21:45:14 later asalkeld ... good questions 21:45:17 it seems to me that the implementation for nova has been retargeted too 21:45:19 * dhellmann waves to asalkeld 21:45:47 https://blueprints.launchpad.net/nova/+spec/trusted-messaging has no goal/milestone target anymore 21:45:56 so I think we won't have this for G 21:46:07 yes, that seems unlikely 21:46:18 mmmm.. We should talk with Eric Windish.... 21:46:25 so I just wanted you guys and our dear PTL to be aware 21:46:46 I'm going to try to finally get to the notifications listener api next week, but I don't see us moving our meter messages away from rpc in time 21:46:49 nijaba: I can take care of that 21:47:04 jd__: sure. care to action yourself? 21:47:07 do we know the reason of untarget for G? 21:47:13 in nova 21:47:32 llu-laptop: no, this is why we should talk to Eric 21:47:34 oh, wait, I'm confused, this is just about the new signing code? 21:47:36 #action jd__ contact Eric Windisch about nova's trusted-messaging blueprint status 21:47:48 dhellmann: yes that's it 21:47:59 dhellmann: though it's not really far of what you talk about anyway 21:48:07 I think they're still trying to figure out which part of the library is supposed to do the serialization 21:48:29 there was some discussion about that today, maybe in a bug report 21:48:37 iirc, there was issues with double serialization 21:49:10 ok, we'll clear it up with Eric I guess, anyway 21:49:25 jd__: yep, that sounds like the right thing to do 21:49:40 #link http://lists.openstack.org/pipermail/openstack-dev/2013-January/005200.html 21:50:04 jd__: +1 21:50:27 let's move to open discussion? 21:50:36 yup 21:50:40 #topic Open discussion 21:50:59 * eglynn wonders whether we/I should refer to the new emerging "agent-less" architecture at FOSDEM this weekend? 21:51:14 (or stick to the existing architecture?) 21:51:15 eglynn: let's not "sell futures" 21:51:23 eglynn: stick to current :) 21:51:29 dhellmann: that's a good way of putting it 21:51:36 fair enough, sounds reasonable 21:52:00 * sandywalsh is still waiting for his flying car 21:52:30 * eglynn never got his hover-bike from Santa ... 21:52:40 anyone working on Hyperv/VMWare support? 21:52:50 * dragondm thinks if sandywalsh had a flying car, he'd have to spend all his time de-icing it... 21:53:04 hah! no kidding 21:53:25 * nijaba does have his folding bike that only flies when on a plane 21:53:40 oh, and thanks to all you guys for putting up with my questions over the last week ... thanks for being patient! 21:53:50 yup. ditto. 21:54:12 you're welcome :) 21:54:43 I have submit the patch for https://blueprints.launchpad.net/ceilometer/+spec/publisher-counters-frequency and hope more feedback on the implementation. 21:55:22 yeah we need more review 21:55:34 I'd be happy to look. 21:55:42 though I think multi-publisher is going to be merged soon now that we've more +2-able folks on board 21:56:21 jd__: :-) 21:57:06 dragondm: thanks 21:57:26 any final words before I close the meeting? 21:57:33 banana 21:58:04 ok. thanks everyone for another great meeting 21:58:11 thanks y'all! 21:58:14 thx 21:58:14 o/ 21:58:20 tty next week, and see some of you this we at fosdem! 21:58:34 thx & bye! 21:58:36 #endmeeting