Monday, 2015-07-27

*** alexpilotti has quit IRC00:00
*** promulo_ has quit IRC00:03
*** ankita_wagh has quit IRC00:08
*** ddieterly has joined #openstack-ceilometer00:17
*** ddieterly has quit IRC00:22
*** ddieterly has joined #openstack-ceilometer00:24
*** lexholden has quit IRC00:26
*** r-daneel has joined #openstack-ceilometer00:38
*** r-daneel has quit IRC00:44
*** Kennan has quit IRC01:13
*** htruta has quit IRC01:14
*** htruta has joined #openstack-ceilometer01:15
*** Kennan has joined #openstack-ceilometer01:16
*** krotsche_vaca is now known as krotscheck01:28
*** htruta_ has quit IRC01:34
*** pauldix has quit IRC01:53
openstackgerritRyota MIBU proposed openstack/aodh: Refactor alarm scenario tests (RuleThreshold)  https://review.openstack.org/20443902:07
openstackgerritRyota MIBU proposed openstack/aodh: Refactor alarm scenario tests (RuleThreshold)  https://review.openstack.org/20443902:11
*** ddieterly has quit IRC02:17
*** liusheng has joined #openstack-ceilometer02:25
openstackgerritliusheng proposed openstack/aodh: Use user_id/project_id from service_credentials in alarm_change  https://review.openstack.org/20578202:27
openstackgerritliusheng proposed openstack/aodh: Avoid getting alarm change notifier repeatedly  https://review.openstack.org/20587502:27
*** ddieterly has joined #openstack-ceilometer02:56
*** ddieterly has quit IRC03:01
*** ddieterly has joined #openstack-ceilometer03:10
*** changbl has quit IRC03:19
openstackgerritJie Li proposed openstack/ceilometer: Replace isotime() with utcnow() and isoformat()  https://review.openstack.org/20588403:24
*** changbl has joined #openstack-ceilometer03:32
*** pauldix has joined #openstack-ceilometer03:41
*** pauldix has quit IRC03:46
openstackgerritliusheng proposed openstack/aodh: void getting alarm change notifier repeatedly  https://review.openstack.org/20587503:54
*** liusheng has quit IRC04:00
openstackgerritliusheng proposed openstack/aodh: Avoid getting alarm change notifier repeatedly  https://review.openstack.org/20587504:03
*** liusheng has joined #openstack-ceilometer04:03
*** deepthi has joined #openstack-ceilometer04:13
*** fawadkhaliq has joined #openstack-ceilometer04:17
*** Kennan2 has joined #openstack-ceilometer04:20
*** Kennan has quit IRC04:20
*** ankita_wagh has joined #openstack-ceilometer04:25
*** yprokule has joined #openstack-ceilometer04:28
openstackgerritliusheng proposed openstack/aodh: Use user_id/project_id from service_credentials in alarm_change  https://review.openstack.org/20578205:01
*** fawadkhaliq has quit IRC05:13
*** fawadkhaliq has joined #openstack-ceilometer05:13
*** _nadya_ has joined #openstack-ceilometer05:29
*** _nadya_ has quit IRC05:33
*** _nadya_ has joined #openstack-ceilometer05:35
*** _nadya_ has quit IRC05:36
*** _nadya_ has joined #openstack-ceilometer05:37
*** _nadya_ has quit IRC05:41
*** _nadya_ has joined #openstack-ceilometer05:42
*** jwcroppe has joined #openstack-ceilometer06:00
*** _nadya_ has quit IRC06:01
*** _nadya_ has joined #openstack-ceilometer06:01
*** pauldix has joined #openstack-ceilometer06:02
*** _nadya_ has quit IRC06:06
*** fawadk has joined #openstack-ceilometer06:07
*** fawadkhaliq has quit IRC06:08
*** lsmola has joined #openstack-ceilometer06:12
openstackgerritJie Li proposed openstack/ceilometer: Replace isotime() with utcnow() and isoformat()  https://review.openstack.org/20588406:18
*** alexpilotti has joined #openstack-ceilometer06:26
*** lan has quit IRC06:27
*** fawadk has quit IRC06:30
*** fawadkhaliq has joined #openstack-ceilometer06:30
*** r-mibu has joined #openstack-ceilometer06:39
openstackgerritliusheng proposed openstack/aodh: Use user_id/project_id from service_credentials in alarm_change  https://review.openstack.org/20578206:39
*** ddieterly has quit IRC06:51
*** fawadk has joined #openstack-ceilometer07:13
*** fawadkhaliq has quit IRC07:13
*** lsmola has quit IRC07:21
*** lsmola has joined #openstack-ceilometer07:21
*** jwcroppe has quit IRC07:30
*** jwcroppe has joined #openstack-ceilometer07:31
*** safchain has joined #openstack-ceilometer07:34
openstackgerritKamil Rykowski proposed openstack/ceilometer: Versioned object for Event model  https://review.openstack.org/18133807:36
*** ankita_wagh has quit IRC07:41
*** cmyster has joined #openstack-ceilometer07:42
*** fawadkhaliq has joined #openstack-ceilometer07:44
*** fawadk has quit IRC07:44
*** ankita_wagh has joined #openstack-ceilometer07:49
*** Ala has joined #openstack-ceilometer08:01
*** jwcroppe has quit IRC08:01
*** jwcroppe has joined #openstack-ceilometer08:02
*** jwcroppe has quit IRC08:06
*** ankita_wagh has quit IRC08:13
*** jwcroppe has joined #openstack-ceilometer08:15
*** fawadkhaliq has quit IRC08:20
*** _nadya_ has joined #openstack-ceilometer08:28
*** promulo_ has joined #openstack-ceilometer08:33
*** pauldix has quit IRC08:34
*** lexholden has joined #openstack-ceilometer08:53
*** promulo_ has quit IRC09:06
openstackgerritJie Li proposed openstack/ceilometer: Replace isotime() with utcnow() and isoformat()  https://review.openstack.org/20588409:23
*** afazekas has joined #openstack-ceilometer09:28
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: Improves send_test_data tools  https://review.openstack.org/20336609:30
*** _nadya_ has quit IRC09:33
*** _nadya_ has joined #openstack-ceilometer09:36
openstackgerritJulien Danjou proposed openstack/aodh: tests.api: remove unused argument/config option  https://review.openstack.org/20596909:41
openstackgerritJulien Danjou proposed openstack/aodh: api: remove force_canonical option  https://review.openstack.org/20597009:41
openstackgerritJulien Danjou proposed openstack/aodh: api: remove global conf and local pecan config  https://review.openstack.org/20597109:41
openstackgerritJulien Danjou proposed openstack/aodh: rpc: remove global conf usage from notifier  https://review.openstack.org/20597209:41
openstackgerritJulien Danjou proposed openstack/aodh: evaluator: remove global conf usage from threshold evaluator  https://review.openstack.org/20597309:41
openstackgerritJulien Danjou proposed openstack/aodh: mongodb: replace custom retry code by retrying  https://review.openstack.org/20597409:41
openstackgerritJulien Danjou proposed openstack/aodh: mongodb: stop relying on global conf object in utils  https://review.openstack.org/20597509:41
jd__yeah I was bored yesterday on the train hihi09:41
openstackgerritMerged openstack/aodh: Use user_id/project_id from service_credentials in alarm_change  https://review.openstack.org/20578209:48
*** jwcroppe has quit IRC09:54
*** jwcroppe has joined #openstack-ceilometer09:56
*** dobson` has quit IRC10:21
*** Ala has quit IRC10:21
*** ildikov has quit IRC10:21
*** ildikov has joined #openstack-ceilometer10:22
openstackgerritIlya Tyaptin proposed openstack/ceilometer: [MongoDB] Use a aggregate pipeline in statistics  https://review.openstack.org/17234010:25
*** dobson has joined #openstack-ceilometer10:25
*** promulo_ has joined #openstack-ceilometer10:29
*** promulo_ has quit IRC10:29
*** _nadya_ has quit IRC10:31
*** _nadya_ has joined #openstack-ceilometer10:32
*** Ala has joined #openstack-ceilometer10:34
*** _nadya_ has quit IRC10:36
*** cdent has joined #openstack-ceilometer10:56
*** fawadkhaliq has joined #openstack-ceilometer10:59
openstackgerritliusheng proposed openstack/aodh: Keep alarm other attrs constantly after evaluating an alarm  https://review.openstack.org/20573211:08
openstackgerritliusheng proposed openstack/aodh: Avoid getting alarm change notifier repeatedly  https://review.openstack.org/20587511:11
*** fawadkhaliq has quit IRC11:21
*** fawadk has joined #openstack-ceilometer11:21
*** thorst1 has joined #openstack-ceilometer11:22
*** rbak has joined #openstack-ceilometer11:23
openstackgerritliusheng proposed openstack/aodh: Delete its corresponding history data when deleting an alarm  https://review.openstack.org/20482811:27
*** rbak has quit IRC11:28
*** jwcroppe_ has joined #openstack-ceilometer11:30
*** _nadya_ has joined #openstack-ceilometer11:30
*** deepthi has quit IRC11:33
*** jwcroppe has quit IRC11:33
openstackgerritMerged openstack/ceilometer: Replace isotime() with utcnow() and isoformat()  https://review.openstack.org/20588411:41
*** jwcroppe_ has quit IRC11:46
*** dikonoor has joined #openstack-ceilometer11:52
openstackgerritMerged openstack/aodh: Avoid getting alarm change notifier repeatedly  https://review.openstack.org/20587512:00
*** jaosorior has joined #openstack-ceilometer12:03
openstackgerritIlya Tyaptin proposed openstack/ceilometer: Initial separating unit and functional tests  https://review.openstack.org/20412812:07
openstackgerritIlya Tyaptin proposed openstack/ceilometer: Initial separating unit and functional tests  https://review.openstack.org/20412812:09
*** fawadk has quit IRC12:13
jasonamyersgood morning12:23
* cdent waves at jasonamyers 12:24
*** gordc has joined #openstack-ceilometer12:26
*** edmondsw has joined #openstack-ceilometer12:30
*** pauldix has joined #openstack-ceilometer12:43
*** jruano has joined #openstack-ceilometer12:50
*** ddieterly has joined #openstack-ceilometer12:53
*** ityaptin has quit IRC12:56
*** ddieterly has quit IRC12:56
*** yprokule has quit IRC12:59
*** yprokule has joined #openstack-ceilometer13:00
*** eglynn has joined #openstack-ceilometer13:02
*** fawadkhaliq has joined #openstack-ceilometer13:03
*** julim has joined #openstack-ceilometer13:05
*** edmondsw has quit IRC13:08
*** tongli has joined #openstack-ceilometer13:08
*** pauldix has quit IRC13:12
*** thorst1 has quit IRC13:16
*** thorst1 has joined #openstack-ceilometer13:16
*** r-mibu has quit IRC13:22
*** r-mibu has joined #openstack-ceilometer13:22
openstackgerritgordon chung proposed openstack/ceilometer: add flexible grouping key  https://review.openstack.org/20414013:29
openstackgerritMerged openstack/aodh: Use stevedore directive to document plugins  https://review.openstack.org/20569313:45
cdentgordc: recall last week me saying the collector was using a fair bit of cpu while idling?13:45
cdenta) it is just under mongo as we thought b) it appears to be in pymongo not ceilo code13:46
openstackgerritMerged openstack/aodh: Fix the service entry of evaluator and notifier  https://review.openstack.org/20574613:46
*** alexpilotti has quit IRC13:46
cdentgordc: can you remind me of the pollster notifier combo we talked about investigating?13:47
cdent(for performance)13:47
*** pauldix has joined #openstack-ceilometer13:50
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: storage: deprecates mongodb_replica_set option  https://review.openstack.org/20607313:51
gordcpollster notifier combo?13:51
cdentgordc: yeah, you said something was using more cpu and I said I'd look into it13:52
cdentbut I don't recall the details13:52
gordci think the two things i was found was having pollster spiked the CPU load when sending individual samples vs batch13:52
gordc~2x the CPU load on compute agent and a longer CPU load on agent (the latter would be expected)13:53
gordcand the other item was that notification agent would never idle on my machine13:53
cdentand the assumption is that this is o.messaging overhead, yeah?13:54
openstackgerritOpenStack Proposal Bot proposed openstack/ceilometer: Updated from global requirements  https://review.openstack.org/20565013:54
gordccdent: first item or second?13:54
gordcbrb. meeting13:54
cdentboth?13:55
*** julim_ has joined #openstack-ceilometer13:57
*** edmondsw has joined #openstack-ceilometer13:57
*** julim has quit IRC14:00
*** boris-42 has joined #openstack-ceilometer14:02
*** exploreshaifali has joined #openstack-ceilometer14:03
* sileht ETOOMUCH_JD_PATCHES14:05
*** yprokule has quit IRC14:05
*** yprokule has joined #openstack-ceilometer14:06
*** prad has joined #openstack-ceilometer14:11
jd__lol sileht14:15
*** ityaptin has joined #openstack-ceilometer14:23
openstackgerritMichael Bayer proposed openstack/gnocchi: Switch to facade  https://review.openstack.org/20466914:24
*** jruano has quit IRC14:25
*** titilambert is now known as Taytay14:25
*** Taytay is now known as titilambert14:25
*** exploreshaifali_ has joined #openstack-ceilometer14:26
*** exploreshaifali has quit IRC14:28
*** changbl has quit IRC14:32
gordccdent: i have no idea why notification agent doesn't idle for me.14:38
gordccdent: i think the first one is just the added loaded of pushing x times the number of discrete messages just causing load.14:39
gordcdebating whether it's better to move back to batch... or at the very least default to batch and have an option to send individual samples14:39
*** fawadkhaliq has quit IRC14:43
*** r-daneel has joined #openstack-ceilometer14:46
*** jaypipes has joined #openstack-ceilometer14:48
*** shaifali__ has joined #openstack-ceilometer14:49
*** dikonoo has joined #openstack-ceilometer14:50
*** dikonoor has quit IRC14:50
*** exploreshaifali_ has quit IRC14:52
*** kbyrne has quit IRC14:56
cdentgordc: a raise load/raised cpu usage doesn't necessarily mean something bad14:57
cdentit can mean we are doing more, more quickly14:57
cdentand thus making more efficient use of the cpu14:57
cdentin theory, if there is a sufficiently sized threadpool would should be able to accomodate for i/o latency better when _not_ batching14:58
cdenthowever, theory and reality rarely map14:58
cdents/would/we/14:58
cdentif you're seeing raised load but _not_ raised cpu usage (user or sys) and perhaps with raised "wa" usage then that could be a sign of a problem15:00
cdentif load is higher and cpu (real and sys) usage is higher, but there is still idle cpu and wa is not much changed, then what we've got is faster delivery of messages (at the cost of using more cpu)15:01
*** alexpilotti has joined #openstack-ceilometer15:02
*** ddaskal has joined #openstack-ceilometer15:03
*** exploreshaifali_ has joined #openstack-ceilometer15:04
openstackgerritIlya Tyaptin proposed openstack/ceilometer: Initial separating unit and functional tests  https://review.openstack.org/20412815:04
*** shaifali__ has quit IRC15:04
*** exploreshaifali_ has quit IRC15:06
gordccdent: what's 'wa'?15:09
*** kbyrne has joined #openstack-ceilometer15:10
cdentthat's "wait": it means the percentage of process that are paused waiting on i/o (if I recall correctly)15:10
gordccdent: got it15:10
cdentsorry it is perce of cpu time that is waiting on i/o (indepdendent of number of processes)15:11
gordci guess my concern would be that the compute agent is already putting quite a bit of load onto nova api... if users notice compute agent itself consuming load on the compute node, they'll start b*tching again15:11
*** ildikov has quit IRC15:12
jasonamyersgordc: that's they're secret they never stop complaining :P15:13
cdentgordc: we should make CONF.shuffle_time_before_polling_task default to the same value as the polling cycle15:14
gordcyeah, that's all our secrets15:14
cdentso that all the polling tasks are spread out15:14
cdentright now the default is to wake up and do everything15:14
cdentwhich is a _horrible_ idea15:14
gordccdent: yeah, i don't mind implementing mandatory shuffle15:14
openstackgerritPradeep Kilambi proposed openstack/ceilometer: Update install docs with gnocchi dispatcher info  https://review.openstack.org/20611015:16
cdent(jd__: what you are and mike trying to accomplish with this enginefacade stuff. It feels like you and he aren't talking entirely clearly about the end goal. Or at least it is non-obvious to observers15:16
gordcregarding the previous load stuff, we need a way to measure faster. is it fair to say throughput the time it takes for all samples in poll cycle to reach collector15:16
gordcas oppose to time it takes for a single sample to reach collector?15:17
cdentthat's going to vary hugely on whether or not shuffle is being used...15:17
cdentso a potential measurement is coming up with a way to describe latency in any given situation15:18
cdenthow long it takes any given measure to traverse the system under different scernarios15:18
cdentbut even then you need to know what matters15:19
cdentyou'd tune differently for whether you are trying to maximize for any single measure15:19
cdentor all the measures15:19
gordccdent: agreed. just want to know criteria for faster/better/etc15:19
gordcrough criteria15:19
jasonamyerswith shuffle = one, without shuffle = all15:20
cdentI don't know well enough what a normal scenario/environment is for the pollster to give a target15:20
cdentwithout it feels a bit like making stuff up15:21
gordcwe'll define one for our talk.15:21
cdentyeah, we'll need to, but it would be good to make sure it is somewhere associated with the real world15:22
gordcyeah, right now my criteria is high CPU load bad, low CPU load good... which as you've mention can be flawed.15:22
gordcain't no real world.15:22
cdentdo we have any data on what people expect or can tolerate? that is are people okay if a datapoint traverses the system in >60s (to make an extreme example)?15:23
cdentIf not, why not?15:23
gordccdent: we use to do anomoly/threat detection with some of the audit events... i'd assume that case you'd want "as quick as possible"15:24
cdentthat then would be an argument in favor of not batching15:25
gordcyep.15:26
cdent(at the cost of increased load)15:26
* gordc is leaning more to configurable option15:28
jasonamyersgordc: +1 config15:28
cdentmakes sense15:28
jasonamyersdefault to lower load15:28
cdentshould be pretty easy to add15:28
cdentshall I go ahead?15:29
gordci hate options but it is what it is.15:29
cdenthah15:29
*** dobson has quit IRC15:29
gordccdent: yeah, if you have time.15:29
gordcagree with jasonamyers that we should default to 'batch'.15:29
*** dikonoo has quit IRC15:30
cdentyeah, I think so15:30
cdentIt will keep me from getting distracted on much more hairbrained schemes15:30
gordcmy attempt to reign in the craziness.15:31
cdent(I was thinking about ways to spread the polling smoothly over the entire cycle)15:32
cdent(not randomly)15:32
gordc... i will let you go down that rabbit hole alone.15:32
cdentyour lack of support is noted for future reprisals15:33
gordcha15:33
*** dobson has joined #openstack-ceilometer15:34
gordci actually have zero advice on your plans. i have no idea how to do that.15:34
gordcityaptin: hey, were you able to sync up with pauldix on what we need to get influxdb working in gnocchi?15:35
openstackgerritMerged openstack/ceilometer: Improves send_test_data tools  https://review.openstack.org/20336615:35
*** afazekas has quit IRC15:36
openstackgerritMerged openstack/ceilometer: Close and dispose test database setup connections  https://review.openstack.org/20513815:40
*** ildikov has joined #openstack-ceilometer15:48
openstackgerritgordon chung proposed openstack/ceilometer: add flexible grouping key  https://review.openstack.org/20414015:56
*** thorst1 is now known as thorst1_away15:56
*** r-mibu has quit IRC15:58
*** mgarza_ has joined #openstack-ceilometer16:00
*** vishwanathj has joined #openstack-ceilometer16:01
*** dikonoor has joined #openstack-ceilometer16:03
*** fawadkhaliq has joined #openstack-ceilometer16:05
*** _nadya_ has quit IRC16:08
*** _nadya_ has joined #openstack-ceilometer16:08
*** lexholden has quit IRC16:12
*** _nadya_ has quit IRC16:13
*** Ala has quit IRC16:20
*** fawadkhaliq has quit IRC16:21
*** afazekas has joined #openstack-ceilometer16:36
*** _cjones_ has joined #openstack-ceilometer16:38
ityaptingordc: Hi! At Friday I checked most issues which I met during the gnocchi driver and these issues have been already fixed. Most problems were in GROUP BY requests and it have been changed to better side.16:39
ityaptingordc: in influxdb 0.9.216:39
*** lexholden has joined #openstack-ceilometer16:49
*** jwcroppe has joined #openstack-ceilometer16:49
*** afazekas has quit IRC16:58
openstackgerritChris Dent proposed openstack/ceilometer: Add a batch_polled_samples configuration item  https://review.openstack.org/20614816:59
cdentgordc: that ^ took me 5 minutes to write the code and hours to fail to write a useful test16:59
*** afazekas has joined #openstack-ceilometer17:00
openstackgerritChris Dent proposed openstack/ceilometer: Add a batch_polled_samples configuration item  https://review.openstack.org/20614817:04
*** yprokule has quit IRC17:05
*** thorst1_away is now known as thorst117:08
*** ankita_wagh has joined #openstack-ceilometer17:17
*** prad has quit IRC17:20
*** afazekas has quit IRC17:22
openstackgerritChris Dent proposed openstack/ceilometer: Add a batch_polled_samples configuration item  https://review.openstack.org/20614817:24
cdentmanaged a resonable test ^17:24
*** vishwanathj has quit IRC17:32
*** eglynn has quit IRC17:34
*** prad has joined #openstack-ceilometer17:37
*** boris-42 has quit IRC17:40
*** pauldix_ has joined #openstack-ceilometer17:44
*** pauldix has quit IRC17:44
*** mgarza_ has quit IRC17:44
*** pauldix_ has quit IRC17:44
gordcityaptin: that's good to hear.17:48
openstackgerritIlya Tyaptin proposed openstack/ceilometer: Initial separating unit and functional tests  https://review.openstack.org/20412817:49
ityaptingordc: Also I send to review a separating of unittests and functional tests17:49
ityaptingordc: ^^^ Please, take a look, if you have a minute17:49
*** afazekas has joined #openstack-ceilometer17:49
*** dikonoor has quit IRC17:51
gordcityaptin: is there a way to enable functional tests in gate right now?17:51
gordcso we can keep testing both?17:51
*** pauldix has joined #openstack-ceilometer17:52
*** harlowja has joined #openstack-ceilometer17:52
openstackgerritOpenStack Proposal Bot proposed openstack/ceilometer: Updated from global requirements  https://review.openstack.org/20565017:54
ityaptingordc: yep, in current test we should comment  'check experimental' for running functional job on these context and if all works good I send to review patch with moving functional from experimental to regular17:54
gordcyeah. that would be good.17:55
cdentprad you back yet?17:55
pradcdent, in 1:117:56
cdentroger17:56
ityaptincdent, jd__: Hi! I fixed notes and some issues in aggregation pipeline patch. Could you take a look https://review.openstack.org/#/c/172340/ ???17:58
* cdent looks17:58
*** vishwanathj has joined #openstack-ceilometer17:58
ityaptingordc: I will glad if you take a look too :) Thanks!17:58
gordcityaptin: sure. will give it a try today18:01
pradcdent, i'm here now18:04
cdenti just wanted to see if gordc had his ears on enough now to talk abou the grenade thing, cuz I'll be gone shortly18:05
pradcdent, dint have time to explore what we spoke yet..18:05
pradcool yea18:05
pradgordc, so here is the situation ..18:05
cdentyeah, I just wanted to bounce it off him in case he had something to say (no point trying the idea if he or someone else hates it)18:05
pradgordc, the grenade change i made to copy the meter.yaml wont work because, as per grenade the code should work with old existing config18:06
pradthough in our case yaml is not really a config18:06
* cdent kicks gordc18:07
pradso one of the suggestions was to use pkg_resources and load the yaml from the code and override via config18:08
pradhehe18:08
pradhe's probably ignoring us18:08
cdentwell I know he hates me so that's cool, but it's rude to ignore you18:08
pradhe hates me too, so i dont mind18:09
cdentah well, then, no problem18:09
pradcdent, but yea, i see we do soemthing similar in sphinx18:10
pradcdent, to load translations18:10
*** changbl has joined #openstack-ceilometer18:12
pradcdent, if you need to leave thats fine, i can update you what he says.. meanwhile i'll give it a try this afternoon and see  if it helps18:16
cdentnot quite gone yet, I started late today18:16
pradcool18:16
*** afazekas has quit IRC18:19
*** fawadkhaliq has joined #openstack-ceilometer18:25
* gordc wakes18:27
gordcisn't the whole point of the grenade patch to handle the exceptional case of adding new file?18:28
gordcprad: ^18:28
cdentgordc: the grenade patch will be rejected because it violates "the theory of upgrading"18:28
gordcprad: when we added the api paste file we had teh same thing.18:29
pradgordc, evidently not, the point is to abide by https://github.com/openstack-dev/grenade#theory-of-upgrade18:29
cdentbut there are additional reasons to consider using pkg_resources in this case18:29
pradgordc, atleast thats what sdague told us18:29
cdent(and probably in a few others): We should put config in /etc/ceilometer, not defaults18:30
cdentdefaults should be more permanent18:30
gordccdent: so i'll be perfectly honest, i don't know difference between /etc/ceilometer and defaults.18:30
cdentexactly!18:30
cdentthat's a significant problem18:30
cdentthere are set of the things that the system must have to run18:31
cdentat the moment those things are required in /etc/ceilometer but ideally everything could run (but maybe do nothing) without any /etc/ceilometer18:32
cdentI was going to make this point long ago, but there was no point disrupting the way things are18:32
cdentbut now this issue with grenade opens the door18:32
gordccan't we run now without /etc/ceilometer?... err except for ceilometer.conf18:32
cdentwe maybe can18:33
cdentbut once meters.yaml exists, we'll have some issues18:33
gordcthe issue being tempest expects the meters.18:34
gordci should clarify the above, what i meant was i don't know what "/etc/ceilometer/" is referring to and what "defaults" is referring to.18:36
gordcbased on that, i still may or may not know the difference :)18:37
cdentlet me try it from another angle18:37
pradgordc, anything that we're migrating to yaml as part of the existing code is essentially the defaults(for the issue we have in hand)18:37
cdentprad++18:38
pradand custom meters user would want to add from liberty on will be in the usual config location as override18:38
gordcso we want to keep existing code and only make new meters declarative?18:41
cdentgordc, no18:41
cdentwe want to treat the new yaml _as_ code18:41
*** liusheng has quit IRC18:41
gordchow would users disable it?18:41
*** liusheng has joined #openstack-ceilometer18:41
*** boris-42 has joined #openstack-ceilometer18:42
cdentsetting parts to null, none, whatever the yaml is?18:42
gordcdisable the 'coded' yaml i meant.18:42
cdentyeah: in the custom file in /etc/ceillometer (which merges with the coded yaml) they would declare something off18:43
gordcfrom what i undertand you want to hardcode existing meters in the yaml context.18:43
cdent(by a mechanism to be determined018:43
cdentthat's what was already going to be the case: a _gigantic_ yaml file in /etc/ceilometer18:43
gordchow would they know what's on?.. or what the defaults are?18:43
cdentnow it'll be a smaller one18:43
cdentgordc: how do they now?18:43
gordccdent: they don't. which is why the yaml makes it more obvious18:44
cdentonly if it is short18:44
gordcwell if we ignore ildikov and we create yamls per project it would be :)18:44
cdentI agree with you that it is a pain in the butt18:44
cdentbut the grenade thing is a hoop we need to get through and this is a way to do it that also has some semantic sense18:45
gordcfor me, i'm not really sure how the meter.yaml is different from this: https://review.openstack.org/#/c/114278/18:45
cdentI'm not 100% sold on the idea, its just something that I've had percolating for a while, so prad and I decided to bring it up18:45
pradso18:46
cdentgordc: I think it is just a matter of time passing18:46
pradwhen we say yaml as python code, would we ahve yaml file as is and just load it via pkg_resources18:46
cdentprad, that's what I think, yeah18:46
ildikovgordc: you mean create files for projects and go for the dricetion of storing them at those projects repo?18:46
pradcdent, something like pkg_resources.resource_string(__name__, "meters.yaml")  and move meter.yaml under ceilo/meter or soemthing18:47
pradcdent, essentially move it out of etc18:47
gordcildikov: lol yeah. not plannig on it now. but just to explain multiple smaller files item18:48
gordcprad: what would happen from user perspective? they have to edit package?18:48
cdentsomething like that yeah. I don't know the exact details; it seems likely that there's some kind of obscurationist way of using pkg_resources which is de rigueur in openstack18:49
pradgordc, yea from users perspective, its not very ideal.. they would expect it to be in some *usual* config location18:49
ildikovgordc: ok :)18:49
cdentno18:49
cdentthe user would not edit the pkg_resource file18:49
cdentthey would override it18:49
cdentwouldn't they?18:50
*** ankita_wagh has quit IRC18:50
pradcdent, how would you disable something then? .. so say i want to disable sahara metrics18:50
pradwhich are part of the pkg resources file18:50
cdentcan you link me to the current file so I can have a look18:50
pradcdent, https://github.com/openstack/ceilometer/blob/master/etc/ceilometer/meters.yaml18:51
* cdent hadn't realized it had already merged18:51
*** ankita_wagh has joined #openstack-ceilometer18:51
gordcprad: yeah, that'd be very strange workflow.18:51
cdentprad I would think disabling comes in two forms:18:52
*** _nadya_ has joined #openstack-ceilometer18:52
cdenta) the notifications never happen in the first place, so no worries18:53
cdentb) the override.yaml (not its real name) would have the same structure, and the sequence on metric: would be merged with the existing one. to disable you could do something like: {name: "image.size", event_type: null}18:54
cdentyes, they need to look at the pkg resources file to know what the various names are, but:18:55
pradcdent, yea so we'll have to duplicate the meter definitions in both files?18:55
cdenta) that's how it works now, you have to edit the entry points18:55
prador yea look at the pkg resources file18:55
cdentb) we can make a tool that does that18:55
cdentprad, no not duplicate18:55
cdentthe override file only changes stuff18:56
cdentit can add meters18:56
cdentor disable existing meters18:56
pradcdent, my main worry is how would i as a user know for example say i have sahara meters defined .. without looking at pkg resources file18:56
cdentceilometer-config-inspector-tool --list-meters ?18:56
cdent(again not its real name)18:56
pradif i have to look at this file.. thats a extra step we're introducing in the work flow.. which is may be ok, i'm not quite sure18:56
gordcthis seems like we're moving the hassle of coding meters to not the user needs to do a lot more stff.18:57
gordci use 'a lot' loosely18:57
cdentgordc: no, becuae right now the user has no control over anything18:57
cdent(unless they go digging in entry_points.txt)18:57
cdentyou guys keep talking about a work flow that doesn't currently exist18:58
pradcorrect, which is why we're moving to yaml and making it easier for user to edit.. if we move the meters back to some place else arent we ending up with the same?18:59
cdentno, because we're provide an override/custom file18:59
cdentwhich allows them control they currently don't have: add custome metrics, disabled ones they don't like18:59
cdentthe interface may not be perfect yet, but it is an improvement19:00
pradyea understand the override and i'm with you on that. I like the overall idea, just one piece concerns me which is the pkg resources file the user has to go inspect to see whats already there and come override it in another file.19:01
gordccdent: different... arguably an improvement. i think it sort of worries me that we needed so many lines to explain how to override/disable stuff.19:01
pradis that too much distraction for the user vs just going in and changing the meter.yaml as we have now19:01
cdenteven if the meters.yaml file were in /etc/ceiloemter i'd still argue that there should be an override file instead of edit in place19:01
cdentgordc, we can probably do much better than my suggestion, I came up with that in the space of this conversation19:02
cdentyou could also have a 'disable:' key in the file that just takes a list of meters19:02
gordccdent: understood. for me, the story went from: "to enable, a meter. add fields x, y, z to meters.yaml and your good"19:03
gordcnow it's that, and "to disable something, do this. to override something do this, etc..."19:04
cdentI'm not sure that's a function of just this idea, I think that's a function of actually talking through the use cases more19:04
gordc"etc..." is to play up the hyperbole and anger you :)19:04
cdentyou'll have to try harder than that!19:04
cdentBesides I wouldn't get angry with you, I'd get angry with the system known as openstack for which you have stockhold syndrome ;)19:05
cdents/hold/holm/19:05
gordclol, it's so nice though. it locked me in this hellhole but pays me.19:05
*** r-mibu has joined #openstack-ceilometer19:05
cdentyou get paid!19:06
* cdent is getting a raw deal19:06
gordclol yeah, your masters don't play the confusion game.19:06
gordcfor me, i just want to make sure we're not jumping through hoops to satify grenade.19:07
pradi guess there are not many options.. other than somehow convince grenade guys to include this change19:07
cdentI think even without grenade it is good that we talk about all three of enable, disable and modify19:07
cdentbecause we hadn't really though much beyond "add another one"19:08
gordcwhat's to stop us from just hard coding it a cycle and then removing it next cycle.19:08
cdentand apparently it is more complex than we had been thinking19:08
cdentgordc: that assume that people upgrade in a clean fashion19:09
gordci think the original story is still pretty simple. you want a meter, it needs to be in this file.19:09
cdentthat begs the question about how to bug diagnosis19:10
gordcin the case of api-paste file. is the idea that if there's no workaround, only then can you add grenade exception?19:10
cdentmy system will never be like their system if they are editing the only file that has meter.yaml info in it19:10
cdentit would be better if I could tell them: mv your custom file out so that the core file takes over19:10
cdentis the bug still there?19:10
cdentgordc: it's not clear, it kind of seemed like a back port would be needed19:10
pradwhat i dont get is, we should have a situation where a new config file was introduced in one release and needs to be included.. if they dont allow this how would it work?19:11
cdentI gotta go19:11
pradin our case especially when existing tempest tests depend on it19:11
gordccdent: kk i'll think some more i guess.19:11
cdentif you guys get anywhere may you could summarize to the mailing list and make it visible to the [qa] gang too so that we can get input from more than just us19:11
cdentwe are working on a lot of assumptions19:12
cdenthave a good eve all19:12
*** cdent has quit IRC19:12
pradi mean, i get their point that code should work with old config as you upgrade.. but in some cases, you introduce new functionality that requires new config19:12
pradonly way to upgrade is include the new file?19:13
gordci will get in trouble if i talk more.19:13
gordclol19:13
pradlol19:13
gordcprad: what if we had two copies.19:13
gordcthe pkg_resources version19:13
gordcand we'd only use iff we couldn't find meter.yal19:13
gordcand we never ever promote the internal version19:14
gordcand then we kill it19:14
pradyea we could, its just ugly to have to maintain two versions and it is bound to bite us at some point19:14
gordcnext cycle...19:14
gordclike right away19:14
gordcor you can maintain the internal version enough to trick grenade.19:15
gordcjust have cpu_util and glance notification19:15
gordcor whatever tempest tests19:15
pradhmm yea just to satisfy grenade.. i guess19:16
gordcthe disable logic is not something i want to support.19:16
gordci don't understand the argument really.19:16
pradi agree, the disable approach seems a bit backwards to me, a lot more user has to do to figure out19:16
pradfor code pov its definitely cleaner impl19:16
pradbut end user will have to go a extra step to do what they want19:17
gordcif we split services there's no way you're going to hardcode a fallback service just in case the person didn't start the new service.19:17
pradif i understood correctly19:17
pradright19:17
pradlemme play around with the pkg_resource approach and see if grenade is happy19:18
pradat least something to get this moving19:18
gordcwe should just create dummy.yaml files so we can use them in future cycles19:18
pradgordc, you mean per project files?19:19
gordcprad: no i mean just blank files that are never referenced.19:24
gordcprad: but we can just use them later so grenade never complains :)19:24
*** safchain has quit IRC19:26
*** ildikov has quit IRC19:28
*** thorst1 has quit IRC19:36
*** vishwanathj has quit IRC19:37
*** fawadkhaliq has quit IRC19:44
*** harlowja has quit IRC19:45
*** thorst1 has joined #openstack-ceilometer19:55
*** harlowja has joined #openstack-ceilometer20:01
*** _nadya_ has quit IRC20:15
pradgordc, so the duplicate(backup) yaml seems to be working.. the grenade script dint fail on my vm.  though i left the yaml in ceilometer/meter/ dir.. is there a better place to put it?20:16
*** _nadya_ has joined #openstack-ceilometer20:16
gordcprad: somewhere hidden.20:17
gordcprad: i'm just wondering if we want to start handling it as multiple files now.20:17
pradgordc, like just have glance ones separate and just duplicate that file?20:18
gordci guess just hide it somewhere for now. i don't really have a preference where tbh20:18
gordcprad: nah, i thought about it a bit more. i don't think we gain anything20:18
gordci think it's cool to hide the single file somewhere.20:19
pradk cool20:19
gordcprad: while i have your attention, do you know why we don't default to generic resource when we don't find matching resource.20:19
pradgordc, in gnocchi?20:20
gordcthe dispatcher20:20
*** _nadya_ has quit IRC20:20
gordcright now we try to find specific resource. if we find no match, we'll drop it20:21
gordcmakes sense to just store it as generic no?20:21
pradgordc, perhaps we havent gotten to adding that support.. we have generic resource in gnocchi, so should be doable20:21
gordckk20:22
gordcseems to be dropping a good chunk of data right now20:22
*** yassine has joined #openstack-ceilometer20:23
*** vishwanathj has joined #openstack-ceilometer20:29
*** ankita_w_ has joined #openstack-ceilometer20:32
*** ankita_wagh has quit IRC20:33
*** ankita_wagh has joined #openstack-ceilometer20:34
*** ankita_w_ has quit IRC20:34
*** kfox1111_ has joined #openstack-ceilometer20:37
*** julim_ has quit IRC20:38
kfox1111_project_id in the meter's doesn't seem to be an openstack uuid. what is it?20:38
gordckfox1111_: we discovered this recently.20:40
gordchttp://lists.openstack.org/pipermail/openstack-dev/2015-June/068104.html20:40
gordcceilometer doesn't generate ids... we just pull ids from payloads.20:41
gordcit's really two cases: ceilometer is pulling the wrong field or more likely, the service is generating some made up id.20:42
gordcit's probably the latter... that said, which meter are you referring to?20:42
kfox1111_confusing since, I'm pulling 'instance' metters by instance id and seeing stuff like:20:42
kfox1111_dad318e0-a2fb-4663-ab43-18c595818faa bf07b33d835e41e8b076843e291c111120:42
kfox1111_where the first is the instance id, adn the second the project_id.20:43
gordcwhich release are you using? i can take a look at code20:43
kfox1111_juno.20:44
kfox1111_looks too short to be the uuid without the dashes.20:44
pradgordc, so was thinking, instead of duplicating the file via pkg_resources, why cant we add a data_files in setup.cfg ? wouldnt that work?20:47
pradgordc, something like https://gist.github.com/pkilambi/07bfa6250327729e638120:47
*** harlowja_ has joined #openstack-ceilometer20:47
kfox1111_is it a md5sum of the uuid or something?20:49
gordckfox1111_: hmm. i'm not sure instance meters has anything special done to it. we seem to be just pulling a value.20:49
kfox1111_how strange.20:49
gordcprad: possibly. tbh i'm not sure how that works.20:50
*** harlowja has quit IRC20:51
pradgordc, though i dont know if this will reflect at upgrade time, which is what grenade complains20:52
kfox1111_something's mutated the value somehow...20:53
gordcand it appears fine in nova?20:53
kfox1111_I think so. I don't have notification messages to verify.20:54
*** tongli has quit IRC20:57
openstackgerritPradeep Kilambi proposed openstack/ceilometer: Enable entry points for new declarative meters  https://review.openstack.org/20303220:58
gordckfox1111_: for the instance meter, is the metadata extremely verbose?20:58
gordctrying to figure out if it's coming from notification or pollster20:58
kfox1111_let me see if I can get that...20:58
gordcmore specifically does it contain more than the attributes defined here: https://github.com/openstack/ceilometer/blob/stable/juno/ceilometer/compute/pollsters/util.py#L38-L7220:59
kfox1111_khow off hand what the field's named?21:01
kfox1111_resource_metadata?21:03
gordcdoes your metadata have 'launched_at'?21:03
kfox1111_keys() are: [u'node', u'state_description', u'image_meta.base_image_ref', u'vcpus', u'event_type', u'disk_gb', u'availability_zone', u'terminated_at', u'ephemeral_gb', u'instance_type_id', u'instance_flavor_id', u'image_meta.disk_format', u'image_meta.container_format', u'user_id', u'deleted_at', u'reservation_id', u'os_type', u'image_ref_url', u'access_ip_v6', u'audit_period_beginning', u'root_gb', u'display_name', u'instance_type', u'tenant_i21:04
gordckfox1111_: that's coming from notification21:04
kfox1111_yeah.... and inside: u'tenant_id': u'f5018fb2c0304fea9283d7f92e790db9'21:05
kfox1111_so that one's short too.21:05
gordcit doesn't look like we modify resource_id in ceilometer. unless there's a hidden bug somewhere.21:06
gordcdo you have resources where it doesn't have the 'project_id' appended?21:07
kfox1111_not that I've found.21:07
kfox1111_haven't looked too hard though.21:07
kfox1111_I'm trying to do a query to show all instances associated with a project.21:07
kfox1111_its not helping when the project id isnt an id. :/21:07
gordci'm inclined to say it's something in nova but you can open a bug if you'd like21:08
gordckfox1111_: we tried to normalise it in ceilometer but when we tried we realised it would break other projects.21:08
kfox1111_k. thanks. I've started asking in the nova channel.21:09
gordci didn't get much feedback but if you look at the model of all the projects in openstack, the ids are set at varchar255 (which is way beyond uuid)21:09
gordcso i'm guessing it's a big fix.21:09
kfox1111_I'm seeing the strange projectid's in nova.instance.project_id. so I think it is for sure an issue there.21:16
kfox1111_is ceilometer going to even allow a project to fetch their own meters for a project_id that looks this strange?21:16
gordckfox1111_: i'm not sure. it depends on what the role user has in keystone... i'd guess no though.21:19
gordcif you can track down the issue in nova i think it'd be good to bring it up on mailing list21:19
gordcno one seemed very concern when i brought it up but maybe if others start complaining we can start standarding the ids we have in OpenStack21:21
kfox1111_yeah. the lack of a horizon interface for ceilometer for users probably means users haven't actually ran acorss this yet. :/21:22
*** ddaskal has quit IRC21:25
*** htruta_ has joined #openstack-ceilometer21:36
*** yassine has quit IRC21:42
*** ccrouch has quit IRC21:42
*** ankita_w_ has joined #openstack-ceilometer21:43
*** ankita_wagh has quit IRC21:45
*** julim has joined #openstack-ceilometer21:46
openstackgerritgordon chung proposed openstack/ceilometer: fix gnocchi resources yaml  https://review.openstack.org/20623521:53
gordcprad: what do you guys plan to do in gnocchi with bad ids?21:56
*** ddaskal has joined #openstack-ceilometer22:02
kfox1111_odd. I see the same strange id's in keystone's db.22:13
kfox1111_a different string then what keystone tenant-list shows for the tenant...22:14
*** edmondsw has quit IRC22:14
gordckfox1111_: that's quite the rabbit hole22:14
kfox1111_this one's totally new to me. I've had no idea the db's have stored a different version of the uuid all this time.22:17
*** tongli has joined #openstack-ceilometer22:18
gordckfox1111_: heading home. sorry can't help debug. feel free to create a mailing list item or bug... laters.22:20
*** gordc has quit IRC22:21
*** harlowja_ has quit IRC22:21
*** harlowja has joined #openstack-ceilometer22:21
*** prad has quit IRC22:28
*** alexpilotti has quit IRC22:54
*** r-daneel has quit IRC23:02
*** jwcroppe has quit IRC23:02
*** jwcroppe has joined #openstack-ceilometer23:03
*** arahal has joined #openstack-ceilometer23:06
*** jaosorior has quit IRC23:11
*** thorst1 has quit IRC23:12
*** jaypipes has quit IRC23:19
*** arahal has quit IRC23:37
*** arahal has joined #openstack-ceilometer23:37
*** arahal has quit IRC23:37
*** thorst has joined #openstack-ceilometer23:51
*** tongli has quit IRC23:54
*** flwang1 has joined #openstack-ceilometer23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!