| *** oanson has quit IRC | 00:22 | |
| *** oanson has joined #openstack-telemetry | 00:24 | |
| *** tovin07_ has joined #openstack-telemetry | 00:36 | |
| *** zhurong has joined #openstack-telemetry | 00:55 | |
| *** AlexeyAbashkin has joined #openstack-telemetry | 01:09 | |
| *** AlexeyAbashkin has quit IRC | 01:13 | |
| *** itlinux has joined #openstack-telemetry | 01:32 | |
| *** psachin has joined #openstack-telemetry | 01:37 | |
| *** oanson has quit IRC | 01:52 | |
| *** BoJin has joined #openstack-telemetry | 01:52 | |
| *** oanson has joined #openstack-telemetry | 01:52 | |
| *** itlinux has quit IRC | 02:03 | |
| openstackgerrit | Wenyan Zhang proposed openstack/ceilometer master: Fix bug for ceilometer polling generates an exception https://review.openstack.org/517191 | 02:25 |
|---|---|---|
| *** zhurong has quit IRC | 02:30 | |
| *** zhurong has joined #openstack-telemetry | 02:39 | |
| *** Tom__ has joined #openstack-telemetry | 02:51 | |
| *** thorst has joined #openstack-telemetry | 02:55 | |
| *** thorst has quit IRC | 02:58 | |
| *** thorst has joined #openstack-telemetry | 02:58 | |
| *** Tommy-Tom has joined #openstack-telemetry | 03:01 | |
| *** Tom__ has quit IRC | 03:04 | |
| *** Tom__ has joined #openstack-telemetry | 03:06 | |
| *** Tommy-Tom has quit IRC | 03:07 | |
| *** Tommy-Tom has joined #openstack-telemetry | 03:08 | |
| openstackgerrit | Shaoman Zhang proposed openstack/panko master: Support admin to get all events https://review.openstack.org/494382 | 03:10 |
| *** AlexeyAbashkin has joined #openstack-telemetry | 03:10 | |
| *** AlexeyAbashkin has quit IRC | 03:10 | |
| *** Tommy-Tom has quit IRC | 03:10 | |
| *** Tom___ has joined #openstack-telemetry | 03:11 | |
| *** Tommy-Tom has joined #openstack-telemetry | 03:11 | |
| *** Tom__ has quit IRC | 03:12 | |
| *** thorst has quit IRC | 03:13 | |
| *** Tommy-Tom has quit IRC | 03:15 | |
| *** Tommy-Tom has joined #openstack-telemetry | 03:20 | |
| *** lhx__ has joined #openstack-telemetry | 03:20 | |
| *** Tommy-Tom has quit IRC | 03:39 | |
| *** Tommy-Tom has joined #openstack-telemetry | 03:39 | |
| *** Tom___ has quit IRC | 03:42 | |
| *** Tom__ has joined #openstack-telemetry | 03:43 | |
| *** Tommy-Tom has quit IRC | 03:44 | |
| *** Tom__ has quit IRC | 03:47 | |
| *** Tommy-Tom has joined #openstack-telemetry | 04:16 | |
| *** lhx__ has quit IRC | 04:18 | |
| *** lhx__ has joined #openstack-telemetry | 04:19 | |
| *** Tommy-Tom has quit IRC | 04:20 | |
| *** zhurong has quit IRC | 04:43 | |
| *** Tommy-Tom has joined #openstack-telemetry | 05:10 | |
| *** Tommy-Tom has quit IRC | 05:14 | |
| *** oanson has quit IRC | 05:34 | |
| *** oanson has joined #openstack-telemetry | 05:34 | |
| *** Tommy-Tom has joined #openstack-telemetry | 05:37 | |
| *** Tommy-Tom has quit IRC | 05:42 | |
| *** Tom__ has joined #openstack-telemetry | 05:56 | |
| *** Tommy-Tom has joined #openstack-telemetry | 05:56 | |
| *** zhurong has joined #openstack-telemetry | 05:58 | |
| *** yprokule has joined #openstack-telemetry | 06:14 | |
| *** Tommy-Tom has quit IRC | 06:15 | |
| *** Tommy-Tom has joined #openstack-telemetry | 06:15 | |
| *** Tommy-Tom has quit IRC | 06:18 | |
| *** Tommy-Tom has joined #openstack-telemetry | 06:18 | |
| *** maestropandy has joined #openstack-telemetry | 06:33 | |
| *** maestropandy1 has joined #openstack-telemetry | 06:48 | |
| *** maestropandy has quit IRC | 06:50 | |
| *** thorst has joined #openstack-telemetry | 06:54 | |
| *** spectr has joined #openstack-telemetry | 07:05 | |
| *** pcaruana has joined #openstack-telemetry | 07:24 | |
| *** Tom__ has quit IRC | 07:34 | |
| *** Tom__ has joined #openstack-telemetry | 07:34 | |
| *** Tom__ has quit IRC | 07:34 | |
| *** Tom__ has joined #openstack-telemetry | 07:35 | |
| *** zhurong has quit IRC | 07:36 | |
| *** Tommy-Tom has quit IRC | 07:53 | |
| *** Tommy-Tom has joined #openstack-telemetry | 07:53 | |
| *** Tom__ has quit IRC | 07:55 | |
| *** Tommy-Tom has quit IRC | 07:58 | |
| *** Tom__ has joined #openstack-telemetry | 08:04 | |
| *** AlexeyAbashkin has joined #openstack-telemetry | 08:06 | |
| *** Tom__ has quit IRC | 08:09 | |
| *** tesseract has joined #openstack-telemetry | 08:10 | |
| *** alexchadin has joined #openstack-telemetry | 08:13 | |
| *** maestropandy1 has quit IRC | 08:16 | |
| *** gkadam has joined #openstack-telemetry | 08:17 | |
| *** Tom__ has joined #openstack-telemetry | 08:27 | |
| *** Tommy-Tom has joined #openstack-telemetry | 08:34 | |
| *** therve has left #openstack-telemetry | 08:39 | |
| *** Tommy-Tom has quit IRC | 08:42 | |
| *** Tommy-Tom has joined #openstack-telemetry | 08:43 | |
| *** Tom__ has quit IRC | 08:44 | |
| *** Tom__ has joined #openstack-telemetry | 08:45 | |
| *** Tommy-To_ has joined #openstack-telemetry | 08:46 | |
| *** dims has quit IRC | 08:46 | |
| *** Tommy-Tom has quit IRC | 08:47 | |
| *** dims has joined #openstack-telemetry | 08:48 | |
| *** Tommy-To_ has quit IRC | 08:50 | |
| *** Tom__ has quit IRC | 08:50 | |
| *** Tommy-Tom has joined #openstack-telemetry | 09:00 | |
| *** Tom__ has joined #openstack-telemetry | 09:01 | |
| *** lpigueiras has joined #openstack-telemetry | 09:10 | |
| *** Tom__ has quit IRC | 09:10 | |
| *** Tom__ has joined #openstack-telemetry | 09:10 | |
| *** Tom__ has quit IRC | 09:13 | |
| *** Tommy-Tom has quit IRC | 09:13 | |
| lpigueiras | hello, when enabling workload partitioning in ceilometer notification agents, do I have to configure something special? I realized that I was losing systematically events with workload_partitioning=True | 09:14 |
| *** Tom__ has joined #openstack-telemetry | 09:15 | |
| lpigueiras | if not, any help to debug this problem I have would be appreciated :) | 09:16 |
| *** Tom__ has quit IRC | 09:19 | |
| *** Tom___ has joined #openstack-telemetry | 09:21 | |
| lpigueiras | i noticed it because of this: https://pasteboard.co/GRUCR9A.png | 09:24 |
| *** Tom___ has quit IRC | 09:25 | |
| openstackgerrit | xianbin xie proposed openstack/ceilometer master: fix ceilometer-compute invoke libvirt exception error https://review.openstack.org/517588 | 09:31 |
| *** maestropandy has joined #openstack-telemetry | 09:35 | |
| *** hoonetorg has quit IRC | 09:50 | |
| *** pradk has joined #openstack-telemetry | 09:57 | |
| *** maestropandy has quit IRC | 09:58 | |
| *** links has quit IRC | 09:58 | |
| *** vint_bra has joined #openstack-telemetry | 09:58 | |
| *** hoonetorg has joined #openstack-telemetry | 10:07 | |
| *** daidv has quit IRC | 10:10 | |
| *** rabel has joined #openstack-telemetry | 10:10 | |
| *** tovin07_ has quit IRC | 10:12 | |
| *** rcernin has joined #openstack-telemetry | 10:21 | |
| *** rabel has quit IRC | 11:00 | |
| *** psachin has quit IRC | 11:07 | |
| *** alexchadin has quit IRC | 11:11 | |
| *** alexchadin has joined #openstack-telemetry | 11:11 | |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: test why pipeline queues filling https://review.openstack.org/517066 | 11:28 |
| *** vint_bra has quit IRC | 11:37 | |
| *** BoJin has quit IRC | 11:46 | |
| openstackgerrit | Hanxi Liu proposed openstack/ceilometer master: Make Doc links point to the latest branch https://review.openstack.org/517611 | 11:55 |
| openstackgerrit | Hanxi Liu proposed openstack/ceilometer master: Make Doc links point to the latest branch https://review.openstack.org/517611 | 12:00 |
| *** zhenguo has quit IRC | 12:05 | |
| *** leitan has joined #openstack-telemetry | 12:19 | |
| *** leitan_ has joined #openstack-telemetry | 12:20 | |
| *** leitan has quit IRC | 12:20 | |
| *** thorst has quit IRC | 12:20 | |
| *** thorst has joined #openstack-telemetry | 12:20 | |
| *** alexchadin has quit IRC | 12:26 | |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: test why pipeline queues filling https://review.openstack.org/517066 | 12:27 |
| *** BoJin has joined #openstack-telemetry | 12:27 | |
| *** vint_bra has joined #openstack-telemetry | 12:29 | |
| *** vint_bra has quit IRC | 12:29 | |
| *** vint_bra has joined #openstack-telemetry | 12:29 | |
| *** vint_bra has quit IRC | 12:29 | |
| *** leitan_ has quit IRC | 12:37 | |
| *** catintheroof has joined #openstack-telemetry | 12:37 | |
| *** catintheroof has quit IRC | 12:37 | |
| *** catintheroof has joined #openstack-telemetry | 12:37 | |
| *** lhx__ has quit IRC | 12:37 | |
| *** BoJin has quit IRC | 12:37 | |
| *** gkadam has quit IRC | 12:47 | |
| *** rcernin has quit IRC | 12:54 | |
| *** gordc has joined #openstack-telemetry | 13:01 | |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: common notification endpoint https://review.openstack.org/509612 | 13:44 |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: rename sample handler https://review.openstack.org/509648 | 13:44 |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: move listener targets to notification agent https://review.openstack.org/509668 | 13:44 |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: separate base manager from pipeline https://review.openstack.org/509826 | 13:44 |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: separate polling code https://review.openstack.org/510124 | 13:44 |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: move sample/event specifc pipeline models to own module https://review.openstack.org/516719 | 13:44 |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: remove agent setup_* helper functions https://review.openstack.org/517308 | 13:44 |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: nearly pluggable notification agent https://review.openstack.org/517309 | 13:44 |
| gordc | lpigueiras: when you lose events, do you see errors in your notification agent? | 13:59 |
| gordc | or messages in your ceilometer-pipe queues? | 13:59 |
| gordc | i don't really understand the chart... is the green line the number of create events and red line number of delete events? | 14:01 |
| lpigueiras | i don't see any errors in the logs | 14:01 |
| gordc | lpigueiras: so i do know there is one bug with partitioning... this would affect master/pike. | 14:02 |
| lpigueiras | well, i agree it's a bit cryptic without context :P | 14:02 |
| lpigueiras | basically i'm comparing compute.instance.create.end and compute.instance.delete.end | 14:03 |
| gordc | https://bugs.launchpad.net/ceilometer/+bug/1729617 basically, it's not distributing properly when agents join. | 14:03 |
| openstack | Launchpad bug 1729617 in Ceilometer "notification agent does not refresh" [Critical,In progress] - Assigned to gordon chung (chungg) | 14:03 |
| lpigueiras | rally is running tests creating vms that are immediately deleted, that's why the two lines should be the same | 14:03 |
| gordc | lpigueiras: kk. that's what i guessed. | 14:04 |
| lpigueiras | and we were missing systematically deletions (that's how we realized that something wasn't ok...) | 14:04 |
| gordc | what happens if you have one notification agent, with partitioning on. | 14:04 |
| gordc | did you always have partitioning on? or you just started using it? | 14:05 |
| lpigueiras | i always had yes | 14:05 |
| lpigueiras | i'm using ocata, I'm affected by that bug? | 14:06 |
| gordc | ok. so it was working with partitioning but it stopped? | 14:06 |
| gordc | the bug i'm fixing shouldn't affect ocata | 14:06 |
| gordc | i do notice in the gate that for some reason the partitioning queues aren't being read | 14:06 |
| gordc | i wonder if something changed in oslo.messaging recently | 14:07 |
| lpigueiras | well, i can't tell if this was happening before, maybe we didn't realize before about this | 14:07 |
| gordc | i'm still debuggin that issue here https://review.openstack.org/#/c/517066/ | 14:07 |
| gordc | i see | 14:07 |
| gordc | so your ceilometer-pipe-* queues are empty as well? | 14:08 |
| lpigueiras | let me check | 14:09 |
| lpigueiras | there is something https://pasteboard.co/GRWv9cx.png, not sure if they are leftovers when the partitioning was on? | 14:11 |
| gordc | yes, only partitioning will push to those queues | 14:11 |
| gordc | that said, it is strange that you are missing the delete events more than create | 14:12 |
| gordc | hmmm.. actually we don't hash on event_type so maybe it's only picking up the queues with create event | 14:13 |
| lpigueiras | that's why my first idea was to check the hypervisors, but the notifications were arriving fine to rabbit | 14:13 |
| lpigueiras | what do you mean? | 14:14 |
| gordc | which one of my messages are you asking about? | 14:14 |
| gordc | the hashing? | 14:15 |
| lpigueiras | yeah | 14:15 |
| gordc | oh. so the way it works is that since all messages arrive in one queue. all the agents will pull from that queue and then hash it into another queue so that similar events are together (in case you need to do transformations) | 14:16 |
| gordc | in your case, since you're using events, you don't have transformations so you don't need partitioning (and can launch as many agents as youw ant) | 14:16 |
| gordc | but yeah, i'm not sure why some interim queues are read and others aren't.... it could be related to: https://github.com/openstack/ceilometer/commit/726b2d4d67ada3df07f36ecfd81b0cf72881e159 | 14:17 |
| gordc | but i'm guessing... i haven't found the reason. just noticed it started to happen in gate recently (maybe past month). | 14:17 |
| gordc | might be worth checking if something is wrong with oslo.messaging relesaes in last month. | 14:18 |
| lpigueiras | so if I spawn multiple agents with workload_partitioning=False it will work for the events? | 14:20 |
| gordc | yes | 14:21 |
| gordc | partitioning is only need to group messages together if you have transformations | 14:22 |
| *** sbezverk has quit IRC | 14:23 | |
| lpigueiras | nice! i'll disable it then | 14:23 |
| lpigueiras | thanks a lot gordc | 14:23 |
| gordc | np. let me know if you fix the real problem :P | 14:27 |
| *** itlinux has joined #openstack-telemetry | 14:52 | |
| *** itlinux has quit IRC | 15:20 | |
| *** itlinux has joined #openstack-telemetry | 15:28 | |
| *** links has joined #openstack-telemetry | 15:36 | |
| *** spectr has quit IRC | 15:49 | |
| *** AlexeyAbashkin has quit IRC | 15:53 | |
| *** itlinux has quit IRC | 16:39 | |
| *** itlinux has joined #openstack-telemetry | 16:43 | |
| *** tesseract has quit IRC | 16:54 | |
| *** links has quit IRC | 16:55 | |
| *** jmlowe has quit IRC | 17:20 | |
| *** yprokule has quit IRC | 17:20 | |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: nearly pluggable notification agent https://review.openstack.org/517309 | 17:25 |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: set models as part of pipeline manager class https://review.openstack.org/517660 | 17:25 |
| *** catintheroof has quit IRC | 17:31 | |
| *** catintheroof has joined #openstack-telemetry | 17:33 | |
| *** catintheroof has quit IRC | 17:45 | |
| *** AlexeyAbashkin has joined #openstack-telemetry | 17:45 | |
| *** AlexeyAbashkin has quit IRC | 17:50 | |
| *** itlinux has quit IRC | 17:51 | |
| *** catintheroof has joined #openstack-telemetry | 17:53 | |
| *** itlinux has joined #openstack-telemetry | 18:00 | |
| *** catintheroof has quit IRC | 18:11 | |
| *** catintheroof has joined #openstack-telemetry | 18:17 | |
| *** weshay is now known as weshay_brb | 18:30 | |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: move pipeline out of __init__ https://review.openstack.org/517693 | 18:31 |
| *** jmlowe has joined #openstack-telemetry | 18:43 | |
| *** itlinux has quit IRC | 18:44 | |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: remove redundant filter check https://review.openstack.org/517707 | 18:51 |
| *** itlinux has joined #openstack-telemetry | 19:14 | |
| *** rwsu has quit IRC | 19:18 | |
| *** weshay_brb is now known as weshay | 19:20 | |
| *** pcaruana has quit IRC | 19:32 | |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: pluggable notification agent https://review.openstack.org/517715 | 20:03 |
| *** catintheroof has quit IRC | 20:04 | |
| *** rwsu has joined #openstack-telemetry | 20:14 | |
| *** catintheroof has joined #openstack-telemetry | 20:15 | |
| *** itlinux has quit IRC | 20:31 | |
| *** itlinux has joined #openstack-telemetry | 20:37 | |
| *** jmlowe has quit IRC | 20:43 | |
| *** catintheroof has quit IRC | 21:04 | |
| *** AlexeyAbashkin has joined #openstack-telemetry | 21:06 | |
| *** AlexeyAbashkin has quit IRC | 21:10 | |
| *** thorst has quit IRC | 21:19 | |
| *** gordc has quit IRC | 21:23 | |
| *** jmlowe has joined #openstack-telemetry | 21:43 | |
| *** itlinux has quit IRC | 21:49 | |
| *** harlowja has quit IRC | 22:12 | |
| *** itlinux has joined #openstack-telemetry | 22:50 | |
| *** itlinux has quit IRC | 22:53 | |
| *** thorst has joined #openstack-telemetry | 22:59 | |
| *** thorst has quit IRC | 23:00 | |
| *** itlinux has joined #openstack-telemetry | 23:01 | |
| *** harlowja has joined #openstack-telemetry | 23:04 | |
| *** itlinux has quit IRC | 23:04 | |
| *** AlexeyAbashkin has joined #openstack-telemetry | 23:07 | |
| *** AlexeyAbashkin has quit IRC | 23:11 | |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: nearly pluggable notification agent https://review.openstack.org/517309 | 23:13 |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: move pipeline out of __init__ https://review.openstack.org/517693 | 23:13 |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: remove redundant filter check https://review.openstack.org/517707 | 23:13 |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: pluggable notification agent https://review.openstack.org/517715 | 23:13 |
| openstackgerrit | gordon chung proposed openstack/ceilometer master: queues per manager rather than per pipeline https://review.openstack.org/517734 | 23:13 |
| *** links has joined #openstack-telemetry | 23:33 | |
| *** Guest63936 has quit IRC | 23:46 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
