15:01:27 <eglynn-office> #startmeeting ceilometer
15:01:27 <openstack> Meeting started Thu Jan 22 15:01:27 2015 UTC and is due to finish in 60 minutes.  The chair is eglynn-office. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:29 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:32 <openstack> The meeting name has been set to 'ceilometer'
15:01:47 <ildikov> o/
15:01:52 <_elena_> o/
15:02:04 <cdent> o/
15:02:18 <gordc> o/
15:02:34 <_nadya_> o/
15:02:50 <fabiog> hi
15:03:00 <eglynn-office> hi y'all
15:03:25 <eglynn-office> #topic kilo-2
15:03:30 <eglynn-office> #link https://launchpad.net/ceilometer/+milestone/kilo-2
15:03:37 <idegtiarov> o?
15:03:48 <idegtiarov> o/
15:04:28 <eglynn-office> does the status of all those BPs look accurate?
15:04:55 <eglynn-office> I bumped the instance-restart, as I don't Einav is working actively on it
15:05:07 <eglynn-office> don't think
15:05:17 <gordc> i'll probably need to push elasticsearch support to k-3 (i haven't tried looking at how to test in gate yet)
15:05:54 <eglynn-office> gordc: cool enough
15:06:11 <eglynn-office> gordc: what does a local install of elasticsearch look like?
15:06:18 <eglynn-office> (in terms of complexity)
15:06:48 <gordc> eglynn-office: it isn't that bad. basically the code i posted back in november.
15:07:05 <eglynn-office> h-ha, k ... just wondering about potential push-back from infra about installing on CI nodes
15:07:10 <eglynn-office> *a-ha
15:07:11 <gordc> the queries we have in our api aren't that complex so building the queries weren't hard.
15:07:40 <sileht> o/
15:07:41 <gordc> i think the complaint is that actually running elasticsearch eats RAM like crazy.
15:07:59 <gordc> it was fine for just doing testing locally on my thinkpad though.
15:08:07 <eglynn-office> is it tunable I wonder, to dial down cache sizes etc.
15:08:48 <gordc> eglynn-office: possibly... i believe there is a memory based backend which is typically used for testing
15:08:54 <ityaptin> o/
15:09:20 * gordc haven't really looked at it recently... just been glancing at it during downtime
15:09:40 <eglynn-office> gordc: cool enough, all that'll prolly come out in wash anyway in the discussion with infra about installing for CI
15:10:10 <gordc> eglynn-office: yeah... i'll need to talk with clarkb later once i've dug into it.
15:10:24 <eglynn-office> coolness
15:10:42 <eglynn-office> any got anything else to flag for kilo-2?
15:11:04 <eglynn-office> feb 5 is the magic date ... https://wiki.openstack.org/wiki/Kilo_Release_Schedule
15:11:36 <sileht> eglynn-office, I have that  https://review.openstack.org/#/c/149063/
15:12:12 <sileht> eglynn-office, it's fresh it's allow to write gnocchi alarm stuff on gnocchi side
15:12:30 <jd__> o/
15:12:32 <eglynn-office> sileht: a-ha cool, looks very reasonable on a first quick scan
15:13:36 <eglynn-office> sileht: ... I'll review in more detail after meeting, but looks like we can get it in quickly
15:14:46 <sileht> eglynn-office, sure
15:16:43 <eglynn-office> fabiog: quick dumb question about the RBAC stuff landed in kilo-1
15:16:53 <eglynn-office> fabiog: ... in the absence of a matching rule in the policy.json, default is to allow amiright?
15:17:15 <fabiog> eglynn-office: no, still apply the rule of admin or not
15:17:50 <fabiog> eglynn-office: so if there is no rule and you are admin, you have total control, otherwise you are bound to the project
15:18:29 <eglynn-office> fabiog: for non-admin ... in the absence of match rule for API operation, then apply segregation rule as always, but allow the API operation to proceed?
15:18:42 <eglynn-office> *matching rule
15:19:08 <fabiog> eglynn-office: yes
15:19:32 <eglynn-office> coolness, just as I thought, needed a sanity check :)
15:19:39 <fabiog> eglynn-office: unless it has a @admin-required
15:19:55 <eglynn-office> yeap
15:20:02 <eglynn-office> k, let's move on from kilo-2?
15:20:45 <eglynn-office> #topic gnocchi
15:21:36 <eglynn-office> just a heads-up that jd__ and I were discussing options around where to move the gnocchi code
15:21:50 <eglynn-office> (we touched on this in last week's meeting also)
15:23:04 <eglynn-office> jd__: so the idea is to propose gnocchi for big-tenting under the openstack/ namespace, right?
15:23:51 <jd__> yup eglynn-office
15:24:04 <jd__> if the rest of the Ceilometer folks are interested obviously
15:24:28 <jd__> but since it sounds like we are going to leverage Gnocchi heavily in the future I guess it'd be a good idea to bring them closer
15:24:33 <cdent> I think more repos is better than less repos
15:24:34 <DinaBelova> jd__, well, we are :)
15:24:38 <sileht> I'm ;)
15:25:15 <eglynn-office> the advantage of doing that as opposed moving directly into the ceilo repo would be to avoid continueing to add directly to the existing monolith
15:25:16 <idegtiarov> I'm too :)
15:25:55 <ildikov> eglynn-office: sounds good to me :)
15:25:56 <eglynn-office> also advantageous in terms of supporting the "standalone gnocchi" usecase
15:26:04 <gordc> the same way we hand oslo.* libs? still under Ceilometer/telemetry umbrella?
15:26:06 <eglynn-office> i.e. as a general purpose metrics store
15:26:24 <jd__> gordc: yes
15:26:36 <gordc> jd__: cool cool. works for me
15:26:42 <jd__> a lot of projects have more than one repository FWIW already
15:27:16 <eglynn-office> gordc: ... still with a separate (but expanded) gnocchi-core group
15:27:19 <ityaptin> jd__, gordc: Agree, it looks cool for me too)
15:28:15 <jd__> eglynn-office: gordc: yeah matching Oslo practice
15:28:31 <eglynn-office> one further advantage would be gnocchi independently attracting contributors
15:28:55 <eglynn-office> ... i.e. folks interested in metrics storage on its own merits
15:29:04 <eglynn-office> independent of the data collection pipeline
15:29:23 <fabiog> eglynn-office: so in the long term gnocchi will replace Ceilometer? I thought there would have been some sort of "merging" ...
15:29:35 <DinaBelova> fabiog, not 'replace'
15:29:41 <DinaBelova> ceilometer will use gnocchi
15:29:45 <eglynn-office> fabiog: gnocchi will be used by ceilo, not replace
15:30:24 <DinaBelova> that's basically kind of API to write/query datapoints
15:30:38 <fabiog> eglynn-office: but now you have part of the API in 1 service and part in a second one ... do you need to deploy two API servers?
15:31:28 <eglynn-office> fabiog: or treat gnocchi as a "backend service"
15:31:43 <fabiog> eglynn-office: ok, that makes sense
15:32:49 <fabiog> eglynn-office: so Ceilometer API v3 would be the place where other Openstack services interact and Gnocchi is internal to Ceilometer, did I get it right?
15:33:08 <fabiog> eglynn-office: Gnocchi API I mean
15:33:22 <jd__> if someone wants to build a Ceilometer API v3 on top of Gnocchi probably
15:33:28 <jd__> I'm not sure it makes more sense right now
15:33:54 <jd__> we have 3 different APIs for 3 different purposes that are not tied together (events/samples/alarms)
15:33:56 <fabiog> jd__: but if it is backend service you are not going to expose its API to generic clients ...
15:34:09 <eglynn-office> fabiog: the ceilometer v3 API could be partailly layered over the gnocchi API (specifically an equivalent of the statistics API implemented as a pass-thru' to gnocci)
15:34:29 <jd__> eglynn-office: yeah what's not clear is the value the Ceilo APIv3 would add over Gnocchi's API yet :)
15:35:40 <eglynn-office> jd__: yeap ... say if instead gnocchi API is called direct, the python-ceiloclient would need to be aware of two endpoints from the ks service catalog, amiright?
15:35:47 <fabiog> eglynn-office: maybe we should have a deeper discussion on how to integrate/leverage the two projects. It was one of the mid-cycle topics
15:36:08 <eglynn-office> ... i.e. gnocchi would expose it's own publicURL endpoint in the service catalog
15:36:15 <jd__> eglynn-office: yes, though likely that'd be gnocchi client I guess
15:36:17 <gordc> jd__: is there a place where we can see response gnocchi will return (to compare how different it is from current ceilometer api v2 responses?)
15:36:38 <jd__> eglynn-office: I wouldn't be offended by seeing 3 endpoints in the long term
15:36:56 <jd__> gordc: the documentation is pretty good in this regard
15:36:56 <eglynn-office> gordc: http://gnocchi.readthedocs.org/en/latest/rest.html
15:37:00 <jd__> gordc: tox -e docs and open it
15:37:19 <jd__> eglynn-office: yeah that one is not up to date anymore since we started building dynamic doc :(
15:37:19 <gordc> cool cool. i was too lazy to look. :)
15:37:19 <sileht> gordc, http://gnocchi.readthedocs.org/en/latest/rest.html
15:37:38 <jd__> I wish we could build the doc on OS CI but I don't think we can until we are off stackforge
15:37:51 <jd__> gordc: so yeah better doc is tox -e docs
15:37:52 <DinaBelova> jd__, btw am I right that right now when we query gnocchi we grab list of datapoints basically, although some of Ceilo requests may return one concrete value?
15:38:07 <sileht> jd__, does gnocchi.readthedocs.org is updated manually ?
15:38:09 <DinaBelova> like avg for all period of time VM is working?
15:38:27 <jd__> sileht: no it's automatic but you need a complete env (SQL) to build the doc now so RTFD fails
15:38:45 <sileht> jd__, oh ok
15:38:45 <eglynn-office> DinaBelova: there is no direct analogue of that on-the-fly aggregation in gnocchi
15:39:01 <jd__> DinaBelova: hum I think so yes, you'd have to do it yourself
15:39:23 <DinaBelova> eglynn-office, jd__ - a-ha, ok, I'll take a look on it
15:39:29 <DinaBelova> will add to todos :D
15:39:29 <jd__> DinaBelova: because I think it'd only work with some aggregation methods (like min/max/mean) but not all
15:39:48 <jd__> I think we have a much better understanding of statistics in Gnocchi :->
15:39:53 <jd__> thanks to eglynn-office
15:40:03 <eglynn-office> DinaBelova: so if the v3 API is based on gnocchi, then on-demand periodization is replaced by up-front declaration of the needed granularities via the archive policy
15:40:54 <DinaBelova> eglynn-office, yeah, thanks
15:41:33 <DinaBelova> also I hope to make this chain https://review.openstack.org/#/c/148271/ working today
15:41:45 <DinaBelova> will send changes on review :)
15:42:30 <eglynn-office> so the other thing I wanted to mention about gnocchi is that we've a bunch of integration tasks that we need to ensure are covered
15:42:54 <eglynn-office> e.g. alarm evaluation (that sileht has starting working on :))
15:43:35 <sileht> eglynn-office, I have a working PoC to help me to design the Alarm Rule description
15:43:44 <eglynn-office> figuring out the v3 API within the restrictions of the gnocchi pre-aggregation
15:44:08 <jd__> ✔︎ alarm evaluation
15:44:31 <jd__> eglynn-office: not sure what that's mean but OK :D
15:44:31 <eglynn-office> sileht: yep, I saw https://review.openstack.org/149182 , nice! (... I left a quick suggestion on gerrit)
15:45:16 <eglynn-office> also figuring out the CLI side (i.e. extend python-ceiloclient or write a separate python-gnocchiclient)
15:45:24 <ityaptin> Also I want to tests gnocchi dispatcher via ceilometer test tools for different storages.  if someone interesting, i create etherpad with test plan. https://etherpad.openstack.org/p/gnocchi_dispatcher_test_plan
15:45:28 <eglynn-office> also ensuring full resource type coverage
15:46:13 <DinaBelova> idegtiarov, were you going to send volumes resources change on review today?
15:46:28 <DinaBelova> due to the moment eglynn-office mentioned?
15:46:55 <idegtiarov> DinaBelova: already done
15:47:17 <DinaBelova> idegtiarov, may you please cover all this task?
15:47:26 <DinaBelova> I mean adding all these resources/meters to gnocchi
15:47:31 <idegtiarov> DinaBelova: yep, will do
15:47:41 <eglynn-office> I'll put together a list of these integration tasks for next week, will be looking for volunteers to get them all squared away for kilo-3
15:47:58 <DinaBelova> eglynn-office, cool, really nice thing
15:48:09 <jd__> I'm eager to see that
15:48:12 <_nadya_> we will add alarm evaluation to gnocchi? It seems to be ceilometer responsibility... Ok then, looks like conversation about Gnocchi may continue forever :)
15:48:55 <jd__> _nadya_: read the backlog :)
15:49:17 <eglynn-office> _nadya_: silehthas proposed an alarm eval PoC to gnocchi, but this could optionally replace "classic" ceilo alarm eval
15:49:35 <idegtiarov> DinaBelova: actually I would like to propose some refactoring according to resource adding in gnocchi, hope will propose WIP patch early next week
15:49:48 <DinaBelova> idegtiarov, stevedore thing?
15:50:10 <idegtiarov> DinaBelova:  yes it is :)
15:50:14 <DinaBelova> cool-cool
15:50:42 <sileht> eglynn-office, both can be used together !
15:50:49 <_nadya_> eglynn-office: yep, I see. Just thought that fabiog's question about replacing may appear again :)
15:51:28 <eglynn-office> sileht: not an mutually exclusive proposition?
15:52:10 <eglynn-office> sileht: ... I thought you'd choose to load the classic threhsold evaluator, or the gnocchi-based one, but never both in the same deployment?
15:52:17 <fabiog> eglynn-office and _nadya_ I would like to see a longer term picture of how these two components will co-exist in the future, does someone have such a picture?
15:52:39 <sileht> eglynn-office, I'm thinking about combining ceilometer and gnocchi alarm with the CombinationAlarmRule
15:52:57 <fabiog> and if that has not been defined yet, I think it would be a good think to work on
15:53:09 <fabiog> thing
15:53:14 <eglynn-office> fabiog: yes, it's a work-in-progress
15:53:20 <sileht> eglynn-office, this allows to smoothly migrate to gnocchi
15:53:37 <fabiog> eglynn-office: is there a wiki or some docs about it?
15:53:53 <fabiog> eglynn-office: please :-)
15:54:21 <_nadya_> fabiog: I'm absolutely agree with you. I think that we need more time for it ...
15:54:22 <eglynn-office> fabiog: jd__ had a wiki but it's not up-to-date ... let's add that as a task
15:55:17 <fabiog> eglynn-office: I have a doodle for the Batching suggestion
15:55:26 <fabiog> eglynn-office: I added to the agenda topic
15:55:38 <eglynn-office> fabiog: yep, let's move on that in open discussion
15:55:44 <eglynn-office> #topic open discussion
15:55:45 <jd__> eglynn-office: if we write something it'd be cool to have that in one of the ceilo or gnocchi doc, not a wiki, so it's actually "useful"
15:56:10 <eglynn-office> jd__: fair point
15:56:15 <eglynn-office> jd__: ... a spec perhaps?
15:56:32 <eglynn-office> jd__: (... so as to make it a discussion)
15:56:47 <eglynn-office> fabiog: do you want to mention your doodle now?
15:56:51 <jd__> eglynn-office: at least
15:57:20 <fabiog> eglynn-office:yes http://doodle.com/cwv5nmv3hxadvreu
15:57:42 <eglynn-office> folks pls vote on that ^^^
15:57:51 <eglynn-office> fabiog: what's the native TZ for the poll?
15:57:53 <fabiog> so please people subscribe for the slot, I will then set-up a Google Hangout and I will present the idea, if that has traction I will prepare a BP
15:57:53 <eglynn-office> PST?
15:58:01 <fabiog> it is PDT
15:58:05 <fabiog> PST
15:58:11 <fabiog> Pacific Standard Time
15:58:21 <fabiog> so the first slot is the same time as the Ceilo meeting
15:58:23 <eglynn-office> UTC-8:00
15:58:28 <fabiog> yes
15:58:33 <eglynn-office> cool\
15:58:49 <fabiog> I will check the votes EOD friday
15:59:39 <eglynn-office> cool thanks fabiog
15:59:43 <_nadya_> fabiog: thanks a lot! It would be great to attend
15:59:52 <eglynn-office> anything else folks wanted to raise?
16:00:27 <eglynn-office> the shot-clock has beaten us again
16:00:29 <eglynn-office> ... thanks for your time folks
16:00:34 <eglynn-office> #endmeeting ceilometer