15:00:07 <eglynn> #startmeeting ceilometer
15:00:08 <openstack> Meeting started Thu Aug  7 15:00:07 2014 UTC and is due to finish in 60 minutes.  The chair is eglynn. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:12 <openstack> The meeting name has been set to 'ceilometer'
15:00:20 <eglynn> hey y'all!
15:00:27 <tongli> hello.
15:00:29 <ildikov> o/
15:00:33 <fabiog_> o/
15:00:46 <cdent> hola
15:00:49 <llu-laptop> o/
15:01:01 <_nadya_> hi!
15:01:16 <eglynn> #topic juno-3 planning
15:01:23 <eglynn> #link https://launchpad.net/ceilometer/+milestone/juno-3
15:01:32 <eglynn> let's focus in on the 2 blocked BPs
15:01:43 <eglynn> first the bigger-data-sql ... seems there's a new patchset pending, but converging on agreement?
15:01:50 <eglynn> #link https://review.openstack.org/#/c/101009
15:01:53 <DinaBelova> o/
15:02:16 <eglynn> gordc, cdent: does it feel like convergence?
15:02:40 <sileht> o/
15:02:56 <cdent> gordc just posted some new comment saying it seems pretty okay, but may have some issues with meter reads, so a bit more poking to come, but it is heading in the right direction
15:03:11 <cdent> (specially meter-list performance)
15:03:25 <eglynn> #link https://etherpad.openstack.org/p/ceilometer-big-data-pt2
15:03:26 <nealph> o.
15:03:37 <eglynn> that's the latest proposal basically ^^^ ?
15:03:52 <cdent> there's active code here:  https://review.openstack.org/111313
15:04:25 <cdent> the decision was that there was no way to reach the ideal schema by thinking along, real code was required
15:04:41 <cdent> what's implemented there is very close to what's in the etherpad
15:04:49 <eglynn> cool, so we need to get the spec lined up the etherpad and the PoC code in order to get it landed, amiright?
15:05:06 <cdent> reckon so, yes
15:06:04 <eglynn> cdent: cool, I'll follow up with gordc after the meeting
15:06:13 <eglynn> next central-agent-partitioning ... we have too alternative proposals
15:06:24 <eglynn> we've done a preso/hangout on each this week
15:06:28 <eglynn> so it really is decision time now
15:06:34 * eglynn pastes links for each approach ...
15:06:39 <eglynn> (prepare for flood!)
15:06:49 <eglynn> #link https://review.openstack.org/101282
15:06:50 <eglynn> #link http://www.slideshare.net/FabioGiannetti/ceilometer-central-agent-activeactive-ha-proposal
15:06:52 <eglynn> #link https://review.openstack.org/111978
15:06:53 <eglynn> #link http://www.slideshare.net/EoghanGlynn/hash-based-central-agent-workload-partitioning-37760440
15:07:26 <fabiog_> eglynn: can we have some clarifications on the Tooz future support?
15:07:33 <eglynn> can everyone who cares about this feature (e.g. attended the hangouts or participated in the reviews already) please "nail their colours to the mast" on gerrit before EoD today?
15:07:57 <DinaBelova> eglynn, ok, will be done :)
15:08:14 <DinaBelova> eglynn, actually I wonder what will be the answer on the fabiog_ quesiton here
15:08:15 <ildikov> eglynn: I'm kinda waiting for the answer of that question that Fabio has just asked and also asked by you on gerrit
15:08:16 <eglynn> fabiog_: since you posed the question on gerrit already, I'll ask jd__ to respond there
15:08:23 <DinaBelova> ildikov, ++
15:08:32 <DinaBelova> eglynn, thank you sir
15:08:33 <jd__> o/
15:08:41 <DinaBelova> jd__, good evening :)
15:09:00 <eglynn> jd__: question on https://review.openstack.org/101282 about future maintainership of tooz
15:09:19 <eglynn> jd__: sorry wrong link! ... https://review.openstack.org/111978
15:10:11 <eglynn> let's complete that discussion on gerrit
15:10:12 <jd__> it has support for harlowj and it's going to be put under the Oslo maintenance
15:10:18 <jd__> s/for/from/
15:10:45 <jd__> same as stevedore, etc…
15:11:09 <eglynn> jd__: a-ha, cool, that's good news about the oslo aspect
15:11:11 <DinaBelova> jd__, thanks for the clarification
15:11:26 <jd__> we're just waiting for some projects to use it before moving it under Oslo
15:11:29 <fabiog_> jd__: thanks, this helps
15:11:31 <jd__> and I'm working as we speak as using it with Nova
15:11:43 <jd__> (to replace their custome implementation of memcached/zookeeper)
15:11:46 <_nadya_> jd__: what projects? :)
15:12:03 <jd__> _nadya_: Nova and Neutron are first in sight last time I checked
15:12:04 <eglynn> _nadya_: nova
15:12:10 <DinaBelova> jd__, I remember kind of this mailing thread - am I right it was already started?
15:12:16 <jd__> there might be more I'm not aware of
15:12:29 <_nadya_> cool
15:12:42 <jd__> it's likely that a lot of project have crafted custom solutions that are not so robust and would have a better usage of tooz, but we'll see
15:12:55 <jd__> and Ceilometer obviously :)
15:13:32 <eglynn> jd__: and on the cards also, possibly new tooz drivers? (e.g. based on olso-messaging)
15:13:47 <jd__> sure thing, we just need someone to write them
15:13:56 <jd__> it's not on my priority list, though if I get bored, you never know…
15:14:01 <jd__> :)
15:14:31 <eglynn> fabiog_: it that sufficient information about tooz future status?
15:14:58 <fabiog_> eglynn: sure. I think it responds to my concern
15:15:06 <eglynn> fabiog_: excellent!
15:15:11 <eglynn> jd__: thank you sir!
15:15:29 <jd__> you're welcome ladies and gentlmen
15:15:58 <eglynn> ok, apart from those 2 blocked BPs, does anyone have any concerns about any other stuff target'd for juno-3?
15:16:14 <jd__> Gnocchi dispatcher?
15:16:20 * jd__ whispers
15:16:42 <ildikov> jd__: I have several excuses, none of them is good enough for mentioning it here...
15:16:53 <jd__> :-)
15:17:22 <eglynn> ildikov: are you still confident that it's do-able in the j-3 timeframe?
15:17:30 <ildikov> if anyone would like to jump in, I will not kill him/her, otherwise, I'm on it, just had some environmental issues to fix
15:17:42 <eglynn> ildikov: cool, understood! :)
15:18:03 <ildikov> eglynn: it has to be regardless my concerns...
15:18:17 <ildikov> eglynn: still 4 weeks, right?
15:18:28 <eglynn> ildikov: yep 4 weeks from today
15:18:34 <ildikov> eglynn: cool, tnx
15:18:46 <eglynn> OK, shall we move on?
15:18:55 <fabiog_> eglynn: I responded already in Gerrit ;-)
15:19:06 <eglynn> fabiog_: thank you sir!
15:19:10 <eglynn> #topic Ironic conductor ipmi -> ceilometer saga (punted from last week)
15:19:19 <eglynn> cdent: the floor is your's sir!
15:20:03 <cdent> I think we can punt this one further down, unless any is gasping for it.
15:20:13 <eglynn> cdent: k ... Haomeng's patch has now landed amiright?
15:20:19 <cdent> yes
15:20:21 <eglynn> #link https://blueprints.launchpad.net/ironic/+spec/send-data-to-ceilometer
15:20:30 <eglynn> cool, let's return to that so
15:20:39 <eglynn> #topic Tempest status
15:21:10 <eglynn> DinaBelova: your mongodb experimental job landed, nice work!
15:21:17 <DinaBelova> thank you sir :)
15:21:25 <DinaBelova> eglynn, it behaves quite nice :)
15:21:46 <DinaBelova> I encourage everyone who have ceilo changes to comment "check experimental" on your patches
15:21:52 <DinaBelova> and see what will happen
15:22:14 <DinaBelova> currently this job has experimental status and needs to be run manually
15:22:21 <eglynn> great! ... so maybe we could consider switching the main tempest job to mongodb if it continues to behave nicely for the next say week or so?
15:22:34 <DinaBelova> eglynn, I hope so, yes
15:22:55 <llu-laptop> DinaBelova: so the magic cast is 'check experimental', literally?
15:23:02 <DinaBelova> llu-laptop, yes
15:23:05 <llu-laptop> good
15:23:16 <eglynn> DinaBelova: so se could potentially swap the status of sql-a and mongodb in the gate?
15:23:35 <DinaBelova> eglynn, well, if TC and over community won't be agians - why not, actually
15:23:46 <DinaBelova> that will work faster and more stable I guess
15:23:51 <eglynn> i.e. mongodb becomes the main "blessed" job while sql-a becomes the second-tier
15:24:09 <eglynn> since that the reality in terms of what is recommended by distros for production
15:24:16 <DinaBelova> eglynn, yeah, I got your idea - that lgtm actually, but we need QA team blessing here :)
15:24:34 <eglynn> DinaBelova: cool :) ... I'll raise at the project meeting next Tuesday
15:24:57 <DinaBelova> eglynnm thank you sir - I'll try to attend - but it's late a little bit for me, so can't promise :)
15:24:59 <eglynn> #action eglynn raise blessing the mongodb-based tempest variant at PTL meeting
15:25:09 <eglynn> DinaBelova: np!
15:25:18 <DinaBelova> :)
15:25:27 <eglynn> cool, this feels like real progress :)
15:25:40 <eglynn> k, anything else on Tempest?
15:25:45 <DinaBelova> also I've raised this question yesterday, but not all of us was aware, so I'll say one more thing here
15:25:54 <eglynn> shoot!
15:26:08 <DinaBelova> we have notification tempest patches -2'ed -  https://review.openstack.org/67164 and https://review.openstack.org/70998
15:26:10 <DinaBelova> that
15:26:23 <DinaBelova> that is because of unstable gate tests passing here
15:26:38 <_nadya_> nice :)
15:26:39 <DinaBelova> sometimes notifications are just not comint
15:26:42 <DinaBelova> coming*
15:26:47 <DinaBelova> _nadya_, hehe
15:27:06 <DinaBelova> eglynn - as I remember, you even disabled nova notifications now to keep gate more or less stable
15:27:11 <eglynn> DinaBelova: yes, I think these will have to be moved to the new in-tree functional tests
15:27:16 <_nadya_> not comming or not sent?
15:27:32 <DinaBelova> _nadya_, that actually is a good quesiton
15:27:45 <DinaBelova> vrovachev could not reprouce it locally
15:27:47 <eglynn> _nadya_: not arriving and persistent in the metering store in time I thought?
15:28:03 <DinaBelova> eglynn, yes, at least it looks so after the logs surfing
15:28:38 <DinaBelova> but we have no actual data to say - 'yes, it is so' - as it was not actually reproduced
15:28:43 <DinaBelova> locally
15:29:15 <eglynn> k, I think we're kind of blocked on that one unless we it's simply collector latency that would not be such an impact if the main tempest job were re-based on mongodb?
15:29:29 <DinaBelova> eglynn, yes, exactly
15:29:58 <ildikov> sounds more like an environmental or load related issue tho'
15:30:33 <gordc> DinaBelova: have you tried just bumping up the number of collector workers?
15:30:36 <eglynn> it's very hard to know without fully reproducing the Jenkins slave
15:31:03 <DinaBelova> gordc, don't remember actually - I thought Vadim had tried different combinations...
15:31:06 <_nadya_> ildikov: yep
15:31:09 <eglynn> gordc: that could be tried, tho' the tests would also have to unskipped to confirm
15:31:10 <ildikov> eglynn: I know that is why I said sounds like :(
15:31:13 <DinaBelova> ildikov, +1
15:31:17 <llu-laptop> in that case, should we postpone the patch https://review.openstack.org/#/c/80225/ until we're sure that the notification won't get lost, at least not change the default publisher until then
15:31:59 <gordc> eglynn: yeah, i'm not sure if its' helpful much either... might just be masking issue by incresaing workers.
15:32:02 <eglynn> llu-laptop: can't we verify that wil local tempest runs?
15:32:08 <gordc> would be good if there were logs for mq
15:32:23 <eglynn> gordc: k, worth experimenting with in any case I think
15:32:23 <DinaBelova> gordc, YES, for sure
15:32:40 <eglynn> k, let's move on with the agenda
15:32:45 <DinaBelova> eglynn, ok
15:32:46 <eglynn> #topic TSDaaS/gnocchi status
15:33:06 * eglynn passes the conch to jd__ ...
15:33:36 <jd__> nothing really knew on that side for this week
15:33:40 <jd__> s/knew/new/
15:33:51 * DinaBelova whispers that initial python-opentsdbclient version was merged
15:33:57 <jd__> I've started to work on the archiving policy we discussed at mid-cycle
15:34:04 <jd__> cool DinaBelova
15:34:04 <eglynn> jd__: cool
15:34:16 <jd__> and I'm also working on tooz wrt the file driver we have in Gnocchi
15:34:20 <eglynn> DinaBelova: I also had a quick peek at the opentsdb driver
15:34:22 <DinaBelova> so I rewritten opentsdb driver change to fit it
15:34:28 <DinaBelova> eglynn, oh, cool
15:34:43 <jd__> finger crossed I'll have a first implementation of the archive policy patch for next week
15:34:52 <jd__> s/finger/fingers/
15:34:59 <eglynn> jd__: excellent!
15:35:27 <eglynn> DinaBelova: so one thought that occurred is that we should think again out about what we consider the set of "standard aggregates" to be
15:35:41 <DinaBelova> eglynn, also I found one really interesting guy who used opentsdb much with ceilo - they have rewritten ceilo actually - so I'm trying to reach him for some experience here
15:35:42 <eglynn> DinaBelova: (given that opentsdb doesn't support median etc.)
15:35:52 <DinaBelova> eglynn, yeah, I found this moment too
15:35:53 <eglynn> cool
15:36:32 <eglynn> FYI on a more general point, I was asked to explain the background on gnocchi to the TC
15:36:38 <eglynn> ... this ML thread is a precursor to that conversation
15:36:46 <eglynn> #link http://lists.openstack.org/pipermail/openstack-dev/2014-August/042080.html
15:36:47 <DinaBelova> eglynn, yes, I saw the e-mail
15:37:24 <eglynn> that'll prolly be at the TC meeting next week, if there's space on the agenda
15:38:07 <eglynn> k, anything else to discuss on gnocchi folks?
15:38:45 <eglynn> cdent: shall we un-punt IPMI in that case?
15:39:02 <eglynn> #topic Ironic conductor ipmi -> ceilometer saga (punted from earlier in the meeting)
15:39:26 * eglynn passes conch to cdent ...
15:40:05 <cdent> The basic gist there is looking for advice on what to do when code on the ceilo side has to work with code on another project's side and that other side has insufficient testing such that simple runtime bugs are being shown up by my own manual testing.
15:40:18 <cdent> I wonder if people have good ways of "managing" that.
15:40:39 <cdent> (that's runtime bugs in the other side's code)
15:40:51 <eglynn> so I guess the obvious one is to draw attention to this testing deficit on gerrit
15:40:59 <eglynn> (which I think you've already done)
15:41:30 <eglynn> the less obvious approach is to jump in and start the ball rolling on the other project
15:41:47 <eglynn> i.e. by raising it on their meeting agenda
15:42:02 <eglynn> or even by authoring and proposing patches with tests
15:42:20 <eglynn> though that involves obviously a big investment of time
15:42:21 <jd__> send patches with unit tests + fixes to the other project, pointing how they suck? ;)
15:42:36 <cdent> you're so graceful jd__ :)
15:42:52 <jd__> at least reporting a bug with a basic test in it on how to reproduce is a good thing
15:43:00 <jd__> because sometimes you don't have the time or knowledge to fix it
15:43:22 <jd__> but most devs will feel the shame and will try to fix it since you provided a good way to reproduce it
15:43:29 <jd__> :)
15:44:03 <eglynn> so I think the telling them, and then telling them again, will get you so far ...
15:44:10 <eglynn> ... but jumping in and actually leading by example can be a more powerful motivator for the other project
15:44:21 <jd__> yes
15:44:23 <cdent> ✓
15:44:32 * cdent worries about spread
15:44:33 <jd__> the good strategy is to give everything you can and terminate by "how can I help more?"
15:44:49 <jd__> that pushes the other dev to do something so you can help, and things move on :)
15:45:43 <eglynn> jd__: good point ... a helpful tone tends to be welcomed, whereas snark tends to be counter-productive
15:45:52 * jd__ nods
15:46:18 <eglynn> cdent: by "spread", you mean spreading cdent too thinly?
15:46:33 <eglynn> cdent: yeah, I hear ya ... it's a balancing act to be sure
15:46:48 <cdent> (yes, on spread) In the particular case the response I got was the it was basically impossible to unit test the particular path that was (multiple times) causing problems.
15:47:16 <cdent> Which suggests there's some kind of infrastructure problem that perhaps in-tree functional testing could help with, eventually.
15:47:23 <eglynn> cdent: you were unsatisified with that answer? ... i.e. felt it would/might be possible?
15:47:33 <eglynn> cdent: a-ha, yes, got it
15:48:29 * eglynn hopes we as a community are not starting to cargo-cult the new in-tree functional testing ...
15:48:36 * eglynn meant that half in jest :)
15:48:49 <cdent> it's a mythical savior, coming soon, probably around christmas
15:49:16 <DinaBelova> cdent, :D
15:49:32 <DinaBelova> yeah that looks quite mystical :) and mythical :)
15:49:34 <eglynn> cdent: let's codename the project "santa" :)
15:49:39 <DinaBelova> eglynn, lol
15:51:50 <eglynn> cdent: do you feel you've enough to go on with the above discussion?
15:52:14 <cdent> yup, just wanted to get a feel for people's approaches
15:52:27 <eglynn> cool
15:52:31 <eglynn> #topic open discussion
15:52:52 <ityaptin> Hi! One little update for performance tests.
15:52:58 <eglynn> ityaptin: cool
15:52:59 <ityaptin> I'm prepare doc for getting start tests with oslo messaging
15:53:04 <ityaptin> https://docs.google.com/document/d/1MXhZRXm8UoEN1kYt2NdZe7qyMb7pM9aW1kq5TuGz5lc/edit?usp=sharing
15:53:36 <ityaptin> In doc there is plan for prepare for test and repository url
15:53:57 <eglynn> ityaptin: a-ha, so this would it make it possible for anyone to run that load test?
15:54:05 <eglynn> (as requested by gordc last week)
15:54:16 <ityaptin> If someone wants to trying perf tests and have questions or run into issues - please comment or edit)
15:54:25 <ityaptin> threoreticall yes)
15:54:29 <eglynn> ityaptin: excellent, thanks!
15:54:43 <eglynn> it would be great if someone had time to try that
15:54:58 <eglynn> once it's proven out, maybe we could put the content up on the wiki?
15:56:03 <gordc> eglynn: i'm going to try to run it locally. of course it can't be compared to previous numbers since it's a diferent machine.
15:56:25 <gordc> ityaptin: i may ping you tomorrow. haven't tried to run it yet.
15:56:33 <ityaptin> yes, we can) Also we can add  profiling graphs to wiki. They are very nice)
15:56:42 <ityaptin> gordc - ok)
15:56:48 <eglynn> gordc: excellent! :)
15:57:27 <eglynn> wow, we've 3 whole minutes left in our slot
15:57:31 <eglynn> ... that's unusual :)
15:57:34 <llu-laptop> ityaptin: in step2, which file to create?
15:58:13 <ityaptin> file with db urls, like hbase://localhost:9090
15:58:16 <ityaptin> one at line
15:58:47 <ityaptin> It's used for configuration backend runtime)
15:58:55 <llu-laptop> ityaptin: got that, thanks
15:59:43 <eglynn> ok, thanks folks as always for a productive meeting
15:59:49 <eglynn> ... let's call it a wrap!
15:59:54 <eglynn> #endmeeting