Wednesday, 2016-06-22

*** julim has joined #openstack-telemetry00:17
*** thorst_ has joined #openstack-telemetry00:47
*** catintheroof has joined #openstack-telemetry00:47
*** thorst_ has quit IRC00:51
openstackgerritShaoman Zhang proposed openstack/ceilometer: Add install description for Neutron  https://review.openstack.org/33192400:53
catintheroofgordc, one quick but important question, i read you told leitan that on mitaka you offloaded the normalization of RPC data to notification-agent so that the ceilometer-collector now reads only the data processed from the notification00:54
catintheroofgordc, im kinda mad that i didnt saw those changes comming, are those documented?00:55
catintheroofgordc, and moreover, is there a "place" where i can follow those decitions, even after the release is made00:56
*** cheneydc has joined #openstack-telemetry00:59
gordchmm... didn't realise i was still online.00:59
catintheroofgordc, oh, sorry00:59
gordccatintheroof: you can track details here starting in mitaka: http://releases.openstack.org/mitaka/01:00
gordcthey were in a wiki previously: https://wiki.openstack.org/wiki/ReleaseNotes/Liberty01:00
catintheroofgordc, and there are THOSE CHANGES01:00
gordcadmittedly, there may be stuff we miss... but not intentionally.01:00
gordcsorry, going to drop off. feel free to ping me tomorrow if anything. lates.01:03
*** gordc has quit IRC01:03
*** catintheroof has quit IRC01:07
*** rbak has joined #openstack-telemetry01:10
*** rbak has quit IRC01:12
*** rbak has joined #openstack-telemetry01:13
*** rbak has quit IRC01:17
*** thorst_ has joined #openstack-telemetry01:20
*** khushbu_ has joined #openstack-telemetry01:27
*** thorst_ has quit IRC01:27
*** toddnni_ has joined #openstack-telemetry01:50
*** toddnni has quit IRC01:51
*** toddnni_ is now known as toddnni01:51
openstackgerritMerged openstack/ceilometer: add l3 cache usage meter  https://review.openstack.org/32563801:54
*** thorst_ has joined #openstack-telemetry01:58
*** rbak has joined #openstack-telemetry02:03
*** thorst_ has quit IRC02:22
*** thorst_ has joined #openstack-telemetry02:23
*** rbak has quit IRC02:29
openstackgerritMerged openstack/ceilometer: Add install description for Neutron  https://review.openstack.org/33192402:29
*** thorst_ has quit IRC02:31
openstackgerritMerged openstack/ceilometer: Merge tag '6.1.0'  https://review.openstack.org/33241702:32
*** khushbu_ has quit IRC02:37
*** khushbu__ has joined #openstack-telemetry02:38
khushbu__I am facing this problem http://pastebin.com/eEAKmeBu03:27
*** links has joined #openstack-telemetry03:29
*** thorst_ has joined #openstack-telemetry03:29
*** thorst_ has quit IRC03:37
*** peristeri has quit IRC03:41
*** catintheroof has joined #openstack-telemetry03:44
*** dave-mcc_ has quit IRC03:48
*** anush__ has joined #openstack-telemetry03:58
*** anush__ has quit IRC04:00
*** anush__ has joined #openstack-telemetry04:00
*** catintheroof has quit IRC04:02
*** ljxiash has joined #openstack-telemetry04:22
*** sheel has joined #openstack-telemetry04:30
*** thorst_ has joined #openstack-telemetry04:34
*** thorst_ has quit IRC04:41
*** rcernin has joined #openstack-telemetry04:57
*** yprokule has joined #openstack-telemetry05:11
*** rcernin has quit IRC05:16
*** shardy has joined #openstack-telemetry05:20
*** jwcroppe has quit IRC05:21
*** chlong has quit IRC05:34
*** achatterjee has joined #openstack-telemetry05:38
*** jkraj has joined #openstack-telemetry05:38
*** shardy has quit IRC05:38
*** thorst_ has joined #openstack-telemetry05:39
*** jwcroppe has joined #openstack-telemetry05:46
*** thorst_ has quit IRC05:46
*** chlong has joined #openstack-telemetry05:47
*** rcernin has joined #openstack-telemetry05:55
*** jwcroppe has quit IRC05:56
*** _nadya_ has joined #openstack-telemetry06:10
*** tomoiaga has joined #openstack-telemetry06:38
*** jwcroppe has joined #openstack-telemetry06:39
*** jmccarthy has joined #openstack-telemetry06:42
*** thorst_ has joined #openstack-telemetry06:44
*** jwcroppe has quit IRC06:45
*** tasdomas` has joined #openstack-telemetry06:47
*** llu__ has joined #openstack-telemetry06:47
*** tesseract- has joined #openstack-telemetry06:47
*** thorst_ has quit IRC06:51
*** dewanee has joined #openstack-telemetry06:54
*** khushbu__ has quit IRC06:55
*** khushbu_ has joined #openstack-telemetry06:56
*** shardy has joined #openstack-telemetry06:56
*** anush__ has quit IRC07:04
*** dewanee_ has quit IRC07:04
*** jd__ has quit IRC07:04
*** llu_ has quit IRC07:04
*** tasdomas has quit IRC07:04
*** llu__ is now known as llu_07:05
*** permalac_ has quit IRC07:06
*** _nadya_ has quit IRC07:09
*** jd__ has joined #openstack-telemetry07:14
*** pcaruana has joined #openstack-telemetry07:22
*** chlong has quit IRC07:25
*** amoralej|off is now known as amoralej07:28
*** jwcroppe has joined #openstack-telemetry07:35
*** jwcroppe has quit IRC07:41
openstackgerritliusheng proposed openstack/ceilometer: Add 'task_state' attribute to meter metadata  https://review.openstack.org/33201007:42
*** jmccarthy has left #openstack-telemetry07:44
*** sheeprine has quit IRC07:45
*** sheeprine_ is now known as sheeprine07:45
*** thorst_ has joined #openstack-telemetry07:49
*** thorst_ has quit IRC07:56
*** khushbu_ has quit IRC07:56
*** cllewellyn_ has joined #openstack-telemetry08:07
*** cllewellyn_ has quit IRC08:12
*** M00nr41n has joined #openstack-telemetry08:18
*** cllewellyn_ has joined #openstack-telemetry08:19
*** jkraj has quit IRC08:25
openstackgerritMerged openstack/ceilometer: Remove oslo_utils.timeutils.total_seconds() usage + catch DriverLoadFailure for get_transport optional + fix CI failure due to oslo.messaging 5.0.0 + correct concurrency of gabbi  https://review.openstack.org/32707308:28
*** khushbu has joined #openstack-telemetry08:31
*** jwcroppe has joined #openstack-telemetry08:33
*** _nadya_ has joined #openstack-telemetry08:35
*** bapalm has quit IRC08:37
*** jwcroppe has quit IRC08:38
*** _nadya_ has quit IRC08:39
*** _nadya_ has joined #openstack-telemetry08:40
*** bapalm has joined #openstack-telemetry08:42
*** yassine__ has joined #openstack-telemetry08:45
openstackgerritliusheng proposed openstack/ceilometer: Switch to use glanceclient to get image  https://review.openstack.org/33150008:47
*** thorst_ has joined #openstack-telemetry08:54
thervejd__, So I didn't get what you said about https://review.openstack.org/327641 yesterday08:54
*** thorst_ has quit IRC09:02
*** tomoiaga has quit IRC09:25
*** jkraj has joined #openstack-telemetry09:37
*** tomoiaga has joined #openstack-telemetry09:37
*** jwcroppe has joined #openstack-telemetry09:41
*** jwcroppe has quit IRC09:46
*** bapalm has quit IRC09:48
openstackgerritliusheng proposed openstack/ceilometer: add support of batch recording metering data for mongodb  https://review.openstack.org/32364109:49
*** bapalm has joined #openstack-telemetry09:54
*** thorst_ has joined #openstack-telemetry09:59
*** cheneydc has quit IRC10:00
openstackgerritgaryk proposed openstack/ceilometer: Add HACKING rules for translations  https://review.openstack.org/33272010:00
openstackgerritgaryk proposed openstack/ceilometer: Add HACKING rules for translations  https://review.openstack.org/33272010:04
*** thorst_ has quit IRC10:07
jd__therve: ah I checked the other one I guess10:29
khushbum facing this issue m http://pastebin.com/eEAKmeBu10:33
*** khushbu has quit IRC10:34
*** jwcroppe has joined #openstack-telemetry10:35
*** jwcroppe has quit IRC10:40
*** p01nt3r75 has joined #openstack-telemetry10:49
*** thorst_ has joined #openstack-telemetry11:04
*** thorst_ has quit IRC11:12
*** cllewellyn_ has quit IRC11:18
thervejd__, I think if you unapprove and approve again that should unstuck it11:18
jd__therve: just did11:19
jd__next step is kick in the nuts11:19
*** cllewellyn__ has joined #openstack-telemetry11:20
therveIt's in the gate queue, looks good :)11:20
*** chlong has joined #openstack-telemetry11:21
*** jwcroppe has joined #openstack-telemetry11:30
*** ljxiash has quit IRC11:33
*** ljxiash has joined #openstack-telemetry11:34
*** jwcroppe has quit IRC11:35
*** ljxiash has quit IRC11:38
*** ljxiash has joined #openstack-telemetry11:42
*** amoralej is now known as amoralej|lunch11:44
*** ljxiash has quit IRC11:47
*** leitan has joined #openstack-telemetry11:51
*** chlong has quit IRC11:51
*** fwdit has joined #openstack-telemetry11:57
*** fwdit has quit IRC11:59
*** thorst_ has joined #openstack-telemetry11:59
openstackgerritMerged openstack/ceilometer: Use trusts in Heat integration test  https://review.openstack.org/32764112:07
*** peristeri has joined #openstack-telemetry12:10
*** ljxiash has joined #openstack-telemetry12:11
*** ljxiash has quit IRC12:15
*** dave-mccowan has joined #openstack-telemetry12:19
*** fwdit has joined #openstack-telemetry12:29
*** fwdit has quit IRC12:31
*** julim has quit IRC12:39
*** c64cosmin has joined #openstack-telemetry12:44
*** KrishR has joined #openstack-telemetry12:56
*** chlong has joined #openstack-telemetry13:06
*** julim has joined #openstack-telemetry13:06
*** permalac has joined #openstack-telemetry13:07
*** links has quit IRC13:09
*** jwcroppe has joined #openstack-telemetry13:17
*** jwcroppe has quit IRC13:23
*** BigWillie has joined #openstack-telemetry13:24
*** KrishR has quit IRC13:33
*** ljxiash has joined #openstack-telemetry13:34
*** amoralej|lunch is now known as amoralej13:38
*** pradk has joined #openstack-telemetry13:39
*** jefrite has joined #openstack-telemetry13:39
*** sheeprine has quit IRC13:48
*** sheeprine has joined #openstack-telemetry13:48
*** chlong has quit IRC13:49
*** gzhai2 has quit IRC13:49
*** gzhai2 has joined #openstack-telemetry13:51
*** ametts has joined #openstack-telemetry14:01
*** jkraj has quit IRC14:04
*** chlong has joined #openstack-telemetry14:05
*** gordc has joined #openstack-telemetry14:06
*** ljxiash has quit IRC14:15
*** anush__ has joined #openstack-telemetry14:15
*** rita has joined #openstack-telemetry14:24
*** ljxiash has joined #openstack-telemetry14:27
*** KrishR has joined #openstack-telemetry14:27
openstackgerritliusheng proposed openstack/ceilometer: add support of batch recording metering data for mongodb  https://review.openstack.org/32364114:30
*** jwcroppe has joined #openstack-telemetry14:33
*** jwcroppe has quit IRC14:38
*** liamji has joined #openstack-telemetry14:40
*** tomoiaga has quit IRC14:41
*** tasdomas` is now known as tasdomas14:42
*** rbak has joined #openstack-telemetry14:50
*** anush__ has quit IRC14:54
*** permalac has quit IRC15:08
*** rcernin has quit IRC15:09
*** bapalm has quit IRC15:10
*** bapalm has joined #openstack-telemetry15:10
*** agireud has quit IRC15:10
*** yarkot1 has quit IRC15:10
*** tesseract- has quit IRC15:10
*** yarkot1 has joined #openstack-telemetry15:11
*** agireud has joined #openstack-telemetry15:11
*** jwcroppe has joined #openstack-telemetry15:13
*** _nadya_ has quit IRC15:17
*** rbak has quit IRC15:21
*** rita has quit IRC15:21
*** pcaruana has quit IRC15:22
*** anush__ has joined #openstack-telemetry15:23
*** anush__ has quit IRC15:23
*** rcernin has joined #openstack-telemetry15:24
*** jwcroppe has quit IRC15:31
*** KrishR has quit IRC15:35
*** pcaruana has joined #openstack-telemetry15:36
openstackgerritgordon chung proposed openstack/gnocchi: separate cleanup into own worker  https://review.openstack.org/31938315:37
*** KrishR has joined #openstack-telemetry15:38
*** anush__ has joined #openstack-telemetry15:41
*** thorst_ has quit IRC15:42
*** thorst_ has joined #openstack-telemetry15:43
*** thorst__ has joined #openstack-telemetry15:44
*** thorst_ has quit IRC15:47
*** thorst__ has quit IRC15:49
*** catintheroof has joined #openstack-telemetry15:49
*** chlong has quit IRC15:50
gordcjd__:  we have a problem. gnocchi isn't writing aggregate files properly15:57
jd__gordc: ?16:00
gordcjd__: all my aggregate files (ie: gnocchi_d9b6da76-49f9-47b4-91d1-abb8372e16a8_1462752000.0_min_60) serialised using https://github.com/openstack/gnocchi/blob/master/gnocchi/carbonara.py#L84-L8516:02
gordcit's just a bunch of uncompressed points... is that expected?16:02
jd__uncompressed?16:02
gordcit's only msgpacks encoded... not lz416:03
jd__it's not expected16:03
gordcit's also using https://github.com/openstack/gnocchi/blob/master/gnocchi/carbonara.py#L152-L157 format rather than https://github.com/openstack/gnocchi/blob/master/gnocchi/carbonara.py#L402-L43116:03
jd__not for  AggregatedTimeSerie16:03
jd__right16:04
jd__that's because once split an AggregatedTimeSerie is a list of TimeSerie16:04
jd__since we don't serialize AggregatedTimeSerie anymore the serialize is maybe misplaced16:05
jd__though https://github.com/openstack/gnocchi/blob/master/gnocchi/carbonara.py#L402 might be used somewhere IIRC16:05
jd__don't have everything in mimd right now but feel free to dig16:06
jd__mind*16:06
gordcwe don't serialise aggregatedtimeseries?16:06
jd__we split them IIRC16:06
jd__https://github.com/openstack/gnocchi/blob/master/gnocchi/carbonara.py#L34116:06
*** jwcroppe has joined #openstack-telemetry16:06
gordcso the lz4 compression stuff is never used?16:06
jd__yes looks like it16:08
jd__because the function is in the wrong place16:08
jd__and nobody noticed I guess16:08
jd__s/function/method/16:08
jd__though that'd need the whole to_dict/from_dict is also useless and could be deleted16:09
gordc... so i guess the lz4 stuff was never used?16:10
gordcon purpose i mean16:10
jd__probably16:10
jd__looking at the commit back then I think we just commited on the side16:10
jd__on the wrong class basically16:10
jd__i'm running the tests to see what breaks if we remove that16:11
jd__ - Failed: 416:11
jd__so yes it's barely used16:11
gordci was under impression it worked... does that mean the size of datapoint it wrong too? not 9KB max?16:11
jd__currently yes, it's just msgpacked16:12
jd__though we do need to fix that now lol16:12
jd__it should be hard16:12
jd__it should not be hard16:12
gordcfix to use lz4 or fix to remove mention of lz4?16:13
gordcright now the biggest issue is that there's a crazy amound of io... specifically from updating/writing aggregates.16:13
jd__fix to use lz416:13
jd__that won't change anything, except the size of I/O16:13
jd__is the IOPS the problem or their size?16:13
gordcit actually might be better to not use lz4 and start looking at appending vs read/decompress/compress/write16:14
gordciops i believe. it seems like you need a larger ceph cluster than your capacity requirements to keep up with io16:14
*** yprokule has quit IRC16:15
gordcso i may only need 5TB of space, but i need a cluster x times bigger to handle the iops16:15
*** links has joined #openstack-telemetry16:16
openstackgerritJulien Danjou proposed openstack/gnocchi: carbonara: compress all TimeSerie classe using LZ4  https://review.openstack.org/33289616:16
jd__gordc: try that ^^^^16:16
jd__gordc: no happen with Swift16:16
jd__s/happen/append/16:17
jd__lol16:17
jd__so it's a possibility to go toward that direction but it'd need to rewrite the drivers in a different way16:17
*** p01nt3r75 has quit IRC16:20
*** liamji has quit IRC16:21
*** liamji has joined #openstack-telemetry16:21
gordcjd__: yeah... that code will work. it'd need a v3.0 though since i imagine all existing aggregates need to be cleaned.16:22
*** liamji has quit IRC16:23
*** liamji has joined #openstack-telemetry16:24
gordcright now, about ~80-85% of time is spent on read/write/clean IO.16:25
gordcand it makes everythign slower because of number iops (for ceph, i imagine others as well)16:26
*** ljxiash has quit IRC16:39
*** links has quit IRC16:40
*** rcernin has quit IRC16:40
*** pcaruana has quit IRC16:43
*** jwcroppe has quit IRC16:45
*** shardy has quit IRC16:56
jd__gordc: you keep saying that but it's not clear what your plan is? :)17:20
*** rcernin has joined #openstack-telemetry17:21
*** _nadya_ has joined #openstack-telemetry17:30
jd__gordc: about LZ4, having Grenade support would help to see that'd be broken17:33
jd__I'd take the lazy approach of reading (un)compressed and serializing compressed17:33
*** marcin1234 has quit IRC17:43
*** jwcroppe has joined #openstack-telemetry17:44
*** jkraj has joined #openstack-telemetry17:45
openstackgerritJulien Danjou proposed openstack/gnocchi: carbonara: compress all TimeSerie classes using LZ4  https://review.openstack.org/33289617:47
jd__gordc: I've updated it in a way we can backport it to 2.117:48
jd__gordc: ^^^17:48
gordcjd__: cool cool. seems legit17:49
jd__it's kinda lazy but I think it's good enough17:50
jd__considering the resource, testing and deployment we have17:50
gordci dont know what we can do for swift driver. based on testing it seems we should look at minimising IO as it's taking the bulk of the time and my ceph cluster is arguably much larger than the data i'm putting into it. will try to think of some plans.17:55
jd__ack17:57
catintheroofjd__, gordc quick question, what does this syntax means exactly when parsing resource metadata attributes on the collector side ? > resource_metadata.(instance_flavor_id|(flavor.id))18:07
catintheroofjd__, gordc on the gnocchi_resource.yml ?18:08
gordcit's yaml parsing. it means try to get instance_flavor_id and if not, try flavor.id18:11
gordcbiab. forgot to eatch lunch18:12
*** stupidni` is now known as stupidnic18:13
*** marcin1234 has joined #openstack-telemetry18:13
*** c64cosmin has quit IRC18:13
marcin1234gordc how was your test?18:14
catintheroofgordc, thanks !18:14
*** amoralej is now known as amoralej|off18:21
*** anush__ has quit IRC18:22
*** anush__ has joined #openstack-telemetry18:23
*** anush__ has quit IRC18:25
openstackgerritMerged openstack/gnocchi: separate cleanup into own worker  https://review.openstack.org/31938318:34
*** cllewellyn__ has quit IRC18:38
*** cllewellyn__ has joined #openstack-telemetry18:39
*** jkraj has quit IRC18:42
*** _nadya_ has quit IRC18:44
*** r-mibu has quit IRC18:45
*** r-mibu has joined #openstack-telemetry18:45
*** cdent has joined #openstack-telemetry18:46
*** liamji has quit IRC18:52
*** jwcroppe has quit IRC19:22
*** sheel has quit IRC19:25
*** jwcroppe has joined #openstack-telemetry19:32
*** thumpba has joined #openstack-telemetry19:43
*** _nadya_ has joined #openstack-telemetry19:45
*** _nadya_ has quit IRC19:50
*** pradk_ has joined #openstack-telemetry19:56
*** pradk has quit IRC20:17
*** pradk_ is now known as pradk20:17
*** jwcroppe has quit IRC20:37
*** BigWillie has quit IRC20:54
*** thumpba has quit IRC20:55
*** thorst_ has joined #openstack-telemetry21:05
*** julim has quit IRC21:05
*** cdent has quit IRC21:11
catintheroofjd__, gordc i have a master gnocchi that was processing metrics ok, but now its been one hours since its only telling this : http://paste.openstack.org/raw/521288/21:25
catintheroofjd__, gordc is gnocchi/master from github with ceph21:25
*** cllewellyn__ has quit IRC21:26
*** cllewellyn__ has joined #openstack-telemetry21:30
*** khushbu_ has joined #openstack-telemetry21:33
catintheroofjd__, gordc we had a single metricd using redis, we commented the redis line, and started metricd again and now is working21:35
*** anush__ has joined #openstack-telemetry21:45
*** ametts has quit IRC21:50
*** anush__ has quit IRC21:52
*** anush__ has joined #openstack-telemetry21:53
*** anush__ has quit IRC22:05
*** KrishR has quit IRC22:24
*** rcernin has quit IRC23:00
*** pradk has quit IRC23:17
*** catintheroof has quit IRC23:35
*** leitan has quit IRC23:39
*** thorst_ has quit IRC23:41
*** thorst has joined #openstack-telemetry23:42
*** peristeri has quit IRC23:47
*** thorst has quit IRC23:51
*** sheel has joined #openstack-telemetry23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!