Friday, 2014-01-31

*** SergeyLukjanov is now known as SergeyLukjanov_00:00
*** lsmola has quit IRC00:01
*** jhurlbert has quit IRC00:35
*** mst89 has joined #openstack-ceilometer00:37
*** xmltok has quit IRC00:41
*** mst89 has quit IRC01:03
*** luisg has quit IRC01:48
*** luisg has joined #openstack-ceilometer01:49
*** ildikov_ has quit IRC04:39
*** tongli has quit IRC04:49
*** julienvey_ has quit IRC04:50
openstackgerritA change was merged to openstack/ceilometer: Fix some typos in architecture doc  https://review.openstack.org/6871405:50
*** rwsu has quit IRC05:59
openstackgerritJenkins proposed a change to openstack/ceilometer: Imported Translations from Transifex  https://review.openstack.org/6280806:03
*** doug-fish has quit IRC06:19
*** sayali_ has joined #openstack-ceilometer06:58
*** sayali has joined #openstack-ceilometer06:58
*** ildikov_ has joined #openstack-ceilometer07:19
openstackgerritMehdi Abaakouk proposed a change to openstack/ceilometer: Replace oslo.rpc by oslo.messaging  https://review.openstack.org/5745708:17
*** sileht has quit IRC08:42
*** sileht has joined #openstack-ceilometer08:47
*** julienvey_ has joined #openstack-ceilometer08:54
*** Alexei_987 has joined #openstack-ceilometer08:55
*** eglynn has joined #openstack-ceilometer09:04
openstackgerritMehdi Abaakouk proposed a change to openstack/ceilometer: Replace oslo.rpc by oslo.messaging  https://review.openstack.org/5745709:07
*** yassine has joined #openstack-ceilometer09:12
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: storage: store recording timestamp  https://review.openstack.org/7016609:12
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: Update oslo  https://review.openstack.org/6990309:14
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: Update oslo  https://review.openstack.org/6990309:18
silehtjd__, I just realize the database dispatcher pass the sample (named meter...) from the rpc directly to the database backend without convert it into storage.models.Sample09:25
jd__not surprised indeed09:25
jd__I think it'd be correct to do so OTOH it kind of add overhead09:31
* jd__ going to debug ceilometer eventlet loop, good bye friends09:31
silehtjd__, if you need some help...09:34
ekarlsojd__: hey, is there a simple way of adding support to PaaS alike services like Designate in Ceilo ?09:34
jd__ekarlso: depends on what you mean by simple09:40
jd__but yes we can09:41
*** SergeyLukjanov_ is now known as SergeyLukjanov09:43
jd__sileht: this looks bad09:44
jd__sileht: the collector get stucks in the impl_kombu consume() method from the RPC since it blocks on socket.recv() now09:46
jd__I wonder if oslo.messaging is going to help us on that09:46
ekarlsojd__: any pointers ?09:46
jd__or if it's as badly 'designed'09:46
silehtjd__, if ceilometer use oslo.messaging with blocking mode, we can build our Service class that handle only thread (ceilometer use only one thread) and use normal timer signal for timed tasks09:46
jd__ekarlso: 0xd3adb33f09:47
silehtlol09:47
ekarlsojd__: HAH! :p09:47
jd__sileht: I'm not sure a timer signal would work correctly if you mean SIGALRM but we can build an event loop with some other lib09:48
silehtjd__, agreed over eventloop will be better09:48
jd__the thing is I don't know what to do09:48
silehtjd__, but what about python2.6/2.7/3.309:49
jd__sileht: http://paste.openstack.org/show/62226/ FWIW09:49
silehtjd__, I have speak to quickly collector use two green thread, one for rpc, one for udp09:49
jd__ah right, that tooo09:49
jd__this is getting me nuts09:50
*** sayali__ has joined #openstack-ceilometer09:55
*** sayali_ has quit IRC09:58
jd__sileht: you know if in oslo.messaging there's a polling function?10:13
silehtjd__, no evernever thing is think async10:16
silehtevernever/everything10:16
openstackgerritMehdi Abaakouk proposed a change to openstack/ceilometer: Replace oslo.rpc by oslo.messaging  https://review.openstack.org/5745710:17
*** SergeyLukjanov is now known as SergeyLukjanov_10:20
jd__sileht: ok so you have to call something to consume() and it blocks I imagine10:21
jd__sounds like the same crap10:21
*** yassine has quit IRC10:22
*** yassine has joined #openstack-ceilometer10:22
silehtjd__, no you need to register callback to get the notification/rpc message10:23
silehtjd__, the oslo.messaging executor, must fit in your eventloop10:23
jd__sileht: how the read() handled then?10:23
jd__ah ok10:23
jd__that's part of the executor too, I though it was just for request10:23
silehtjd__, blocking block on read, eventlet spawn a greenlet,10:23
jd__sileht: ah I see, perfect10:24
jd__so that would solve our problem in this case10:24
silehtjd__, but (because their always have a but)10:25
jd__:(10:25
silehtjd__, if we create a asyncio executor for example10:25
silehtthe executor can retreive the event loop and add the read/write task, but , kombu is a blocking library, so how to deal with taht10:26
silehteventlet monkeypatch, but what for asyncio ?10:26
jd__sileht: well if you do a write()/read() operation it'll block yeah10:27
jd__e.g. if komby sends a request and wait for the reply10:27
jd__that's the main problem with library not designed in a async mannter10:27
jd__that's also why I keep explaining that async and event loop are different things10:28
jd__so the executor is about integrating in the event loop, not doing async10:28
silehtjd__, yep10:28
jd__in the end I don't care if kombu blocks, I prefer to have something safe that something with concurrency where it has not been designed for10:28
silehtjd__, latest kombu version have an exprimental async API but the oslo.messaging don't use it10:29
jd__in the end I'm pretty sure we could replace the current eventlet greenthread by real threads + a big lock to be sure only one thread run at the same time10:29
jd__sileht: yeah oslo.messaging has not been designed in async manner so anyway we're good to keep working on that10:29
jd__(unless I'm wrong and I'd be happy to be :))10:30
silehtjd__, asyncio can deal a bit with block io, by starting a new (real) thread for block IO10:33
silehtjd__, I think the oslo.messaging executor can use that, but I need to PoC the thing10:33
openstackgerritA change was merged to openstack/ceilometer: Update oslo  https://review.openstack.org/6990310:38
Alexei_987sileht: > new (real) thread for block IO This won't work10:42
Alexei_987it depends on libraries that do blocking IO10:42
Alexei_987sileht: not all of the release GIL10:42
Alexei_987sileht: I guess it's better to try something like nova approach - spawn some number of worker processes10:43
Alexei_987s/of the/of them/10:43
*** SergeyLukjanov_ is now known as SergeyLukjanov10:50
openstackgerritMehdi Abaakouk proposed a change to openstack/ceilometer: Replace oslo.rpc by oslo.messaging  https://review.openstack.org/5745710:52
*** SergeyLukjanov is now known as SergeyLukjanov_a10:53
*** SergeyLukjanov_a is now known as SergeyLukjanov_10:54
*** SergeyLukjanov_ is now known as SergeyLukjanov10:59
openstackgerritA change was merged to openstack/ceilometer: Correct a misuse of RestController in the Event API  https://review.openstack.org/6988511:08
*** SergeyLukjanov is now known as SergeyLukjanov_11:11
jd__sileht: I don't think it'll help, since if you call a function like requests.get() that block because it does write()/read(), there's no way having a thread here is going to going to help since you expect the result as a return value :)11:27
silehtjd__, in oslo.messaging you don't want a return value, but passing the value into a callback (ran in the main thread)11:33
jd__sileht: which is exactly the same actually11:34
jd__I mean it's going to block11:34
jd__unless you spawn a thread, and now you program is multithreaded11:34
jd__which is NOT WHAT I WANT11:34
jd__:)11:35
silehtjd__, ack, so I have no clue11:35
jd__sileht: oh I'm happy with your answer about oslo.messaging, so you have a clue11:35
jd__I mean the impl_blocking is exactly what I'd want11:35
jd__my main concern is that we can't use oslo.messaging yet11:36
jd__I'm trying to find a solution but it's like playing with explosives11:36
openstackgerritMehdi Abaakouk proposed a change to openstack/ceilometer: Replace oslo.rpc by oslo.messaging  https://review.openstack.org/5745711:43
eglynnhttp://en.wikipedia.org/wiki/File:HLposterUSA2.jpg11:43
sileht:)11:44
openstackgerritMehdi Abaakouk proposed a change to openstack/ceilometer: Replace oslo.rpc by oslo.messaging  https://review.openstack.org/5745711:50
*** llu has quit IRC11:58
*** tian has quit IRC11:58
*** llu has joined #openstack-ceilometer11:58
*** tian has joined #openstack-ceilometer11:58
openstackgerritMehdi Abaakouk proposed a change to openstack/ceilometer: Replace oslo.rpc by oslo.messaging  https://review.openstack.org/5745711:58
openstackgerritMehdi Abaakouk proposed a change to openstack/ceilometer: Replace oslo.rpc by oslo.messaging  https://review.openstack.org/5745712:04
eglynnsuper-dumb question on the conversion logic from DB model -> API representation12:17
eglynnhttps://github.com/openstack/ceilometer/blob/master/ceilometer/api/controllers/v2.py#L16012:17
*** vrovachev has joined #openstack-ceilometer12:17
eglynnsurely using wsme.Unset as the sentinel requires that the storage layer knows about WSME in order to supress an attribute?12:17
eglynn... that smells wrong to me, why not just use None for the same purpose?12:18
eglynn... maybe /me just hasn't had enough coffee yet this morning12:18
silehteglynn, I think wsme.Unset means user haven't set it through API, None means user have set the an attribut to None12:20
eglynnsileht: yeah, still seems odd to leak wsme.Unset into the storage layer12:21
silehtso if the attribut is None we pass it to the storage, if it is wsme.Unset we don't pass it to the storage12:21
eglynnsileht: I'm talking about the conversion on the return path, i.e. the opposite direction (storage->API)12:22
silehteglynn, ok ok I see12:22
silehteglynn, this is because as_dict() is used for both direction12:23
silehteglynn, but wsme.Unset is only valid for one direction12:23
eglynnsileht: yeah, maybe just need to make the DB model constructors more flexible12:24
eglynn... so as to allow some attributes to remain unset (i.e. as opposed to being explicitly set to None)12:24
* sileht loose between as_dict() from the storage model and as_dict wsme base type12:26
silehteglynn, I confirm the method as_dict() L160 is for "API -> storage" (that removes wsme.Unset attributes), opposite direction is done by the storage model object as_dict() method12:30
openstackgerritMehdi Abaakouk proposed a change to openstack/ceilometer: Replace oslo.rpc by oslo.messaging  https://review.openstack.org/5745712:35
eglynnsileht: yeah, looking at the code again, you seem to be correct12:36
eglynnsileht: though weirdly, I found that setting the unwanted attributes to wsme.Unset within the response storage model did have the result I expected originally12:37
eglynn(i.e. those attributes were dropped from the API representation in the response)12:37
eglynn... meh, /me has gone off-piste somewhere12:37
andreykurilinHi everyone, I'm working on bp db-sql-tests-migrations-refactoring and and I have question about it.12:43
andreykurilinWhen all the unit tests are run at a real MySQL and a real PostgreSQL servers, MySQL creates Indexes for each Foreign key because it can't create fk without index, PosgreSQL can create Foreign keys without Index, so it doesn't create indexies. Therefore, we have different db structure on different backends.12:43
andreykurilin One of the ways to avoid failing test due to difference between DBs on backends is to add validation of relation Index to Foreign key in MySQL, other way is to add Indexes to PostgreSQL via migrations.12:43
andreykurilinwhat are your thoughts on this?12:44
vrovachevhi all12:44
*** ildikov_ has quit IRC12:44
vrovachevhelp me, i have this problem with ceilo: http://paste.openstack.org/show/62235/12:45
jd__andreykurilin: I think we should simply ignore things we didn't create, no?12:47
andreykurilinjd__: I think so too. but I do not like the fact that we will have different databases in MySQL and PostgreSQL12:50
*** SergeyLukjanov_ is now known as SergeyLukjanov12:54
*** SergeyLukjanov is now known as SergeyLukjanov_a12:54
*** SergeyLukjanov_a is now known as SergeyLukjanov12:54
jd__andreykurilin: you will always have different things because they are different engine12:55
jd__with different types, etc12:55
*** SergeyLukjanov is now known as SergeyLukjanov_a12:55
jd__learning to deal with it is better than layering everything to the bottom12:55
andreykurilinok, thanks12:55
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: Revert "Revert "Ensure we are not exhausting the sqlalchemy pool""  https://review.openstack.org/7033712:56
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: service: re-enable eventlet just for sockets  https://review.openstack.org/7033812:56
*** SergeyLukjanov_a is now known as SergeyLukjanov12:56
jd__sileht: if you have an opinion ^12:56
silehtjd__, socket only ? CTRL+C work with that  ? you don't need to patch threading too ?12:56
jd__sileht: no, C-c is not fixed by that12:57
jd__I think we need to revert my patch in oslo12:57
silehtjd__, or patch threading too12:57
jd__sileht: I don't want to patch threading because it breaks libraries using threading12:57
silehtjd__, yep, I agree, the entire service.py and deps must be rewritte with an other event loop12:58
* sileht is sad to see again its sqlalchemy pool hack :(13:00
*** SergeyLukjanov is now known as SergeyLukjanov_13:00
jd__me too13:00
*** AMike has quit IRC13:01
*** SergeyLukjanov_ is now known as SergeyLukjanov13:03
*** ruhe_ has joined #openstack-ceilometer13:09
*** ildikov_ has joined #openstack-ceilometer13:10
jd__sileht: https://review.openstack.org/#/c/70339/13:16
silehtjd__, for the first I think if we patch socket we can patch everything, that don't change so much :(13:19
jd__sileht: everything except threading?13:20
silehtI'm talking about https://review.openstack.org/7033813:20
jd__sileht: I feel better knowing that's the only place infected by greenlet13:20
*** SergeyLukjanov is now known as SergeyLukjanov_13:28
*** jdob has joined #openstack-ceilometer13:38
jd__sileht: if you got 5 minutes I'd be happy to have your opinion on https://review.openstack.org/#/c/70166/ failure wrt PostgreSQL, as I think you wrote the PreciseTimestamp in the first place13:43
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: storage: store recording timestamp  https://review.openstack.org/7016613:46
silehtjd__, DATETIME is a alias of Datetime in sqlalchemy13:48
jd__lol13:50
jd__dunnow why I though it was different13:50
jd__though that's not the problem sileht, that's my trying to fix the failure13:50
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: storage: store recording timestamp  https://review.openstack.org/7016613:51
silehtjd__, lol I have looked into the code they are a small difference, the case !13:53
jd__sileht: http://logs.openstack.org/66/70166/1/check/check-tempest-dsvm-postgres-full/7bd1c1e/console.html13:54
silehtjd__, all other PreciseTimestamp are created with Datetime and then converted in PreciseTimestamp for mysql only13:56
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: service: re-enable eventlet just for sockets  https://review.openstack.org/7033813:56
jd__sileht: except that 'datetime' does not exists in pg13:56
silehtit should be convert in timestamp in postgres13:57
vrovachevGuys, please help me with this problem: http://paste.openstack.org/show/62235/13:57
*** eglynn is now known as eglynn-afk13:58
vrovachevI installed devstack of master branch with this localrc: http://paste.openstack.org/show/62239/13:59
*** SergeyLukjanov_ is now known as SergeyLukjanov14:00
jd__sileht: you're not helping ;-)14:01
silehtjd__, sqlalchemy but I think: https://github.com/zzzeek/sqlalchemy/blob/master/lib/sqlalchemy/dialects/postgresql/base.py#L1267 datetime is convert but not DATETIME14:02
silehtbut/bug/14:02
jd__not sure it's a bug14:02
silehtjd__, in all case, you have right with your first tought, we must use Datetime and not DATETIME14:04
jd__ack14:04
jd__let's try that14:05
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: storage: store recording timestamp  https://review.openstack.org/7016614:05
*** jmckind has joined #openstack-ceilometer14:11
*** SergeyLukjanov is now known as SergeyLukjanov_a14:26
*** SergeyLukjanov_a is now known as SergeyLukjanov_14:26
*** tian has quit IRC14:36
*** llu has quit IRC14:37
*** ruhe_ has quit IRC14:38
*** ruhe has quit IRC14:41
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: storage: store recording timestamp  https://review.openstack.org/7016614:43
*** SergeyLukjanov_ is now known as SergeyLukjanov14:45
*** tian has joined #openstack-ceilometer14:52
*** llu has joined #openstack-ceilometer14:52
*** SergeyLukjanov is now known as SergeyLukjanov_15:04
eglynn-afksileht: d'oh, I see my confusion now ... apols for the noise earlier15:12
*** eglynn-afk is now known as eglynn15:13
eglynnturns out Statistics is a special case, we don't even use the v2._Base.from_db_model() to create it15:13
eglynn(unlike the other API response representations)15:13
eglynnso the Unset is just passed straight thru and then dropped by WSME when marshalling up the response15:13
*** rwsu has joined #openstack-ceilometer15:13
eglynn... anyhoo, alls that's needed is to make the Statistics model ctor a tad more flexible in terms of missing args15:14
*** SergeyLukjanov_ is now known as SergeyLukjanov15:14
*** SergeyLukjanov is now known as SergeyLukjanov_15:15
*** kwhitney has joined #openstack-ceilometer15:17
*** SergeyLukjanov_ is now known as SergeyLukjanov15:20
*** sayali__ has quit IRC15:24
*** SergeyLukjanov is now known as SergeyLukjanov_15:29
*** prad_ has joined #openstack-ceilometer15:29
*** ruhe has joined #openstack-ceilometer15:31
*** ruhe is now known as ruhe_15:31
*** SergeyLukjanov_ is now known as SergeyLukjanov15:34
*** ruhe_ has quit IRC15:36
*** SergeyLukjanov is now known as SergeyLukjanov_a15:43
*** SergeyLukjanov_a is now known as SergeyLukjanov_15:44
silehtjd__, what do you think about https://review.openstack.org/#/c/68939/15:47
*** SergeyLukjanov_ is now known as SergeyLukjanov15:54
*** SergeyLukjanov is now known as SergeyLukjanov_15:56
*** SergeyLukjanov_ is now known as SergeyLukjanov16:04
*** SergeyLukjanov is now known as SergeyLukjanov_a16:19
*** SergeyLukjanov_a is now known as SergeyLukjanov_16:20
openstackgerritMehdi Abaakouk proposed a change to openstack/python-ceilometerclient: Ensure url sent to proxy don't have redondant /  https://review.openstack.org/7036816:21
*** ildikov_ has quit IRC16:21
*** SergeyLukjanov_ is now known as SergeyLukjanov16:24
openstackgerritAndreas Jaeger proposed a change to openstack/python-ceilometerclient: Improve help strings  https://review.openstack.org/7037016:28
*** SergeyLukjanov is now known as SergeyLukjanov_a16:39
*** SergeyLukjanov_a is now known as SergeyLukjanov_16:40
*** SergeyLukjanov_ is now known as SergeyLukjanov16:41
*** kwhitney has left #openstack-ceilometer16:41
*** viktors has left #openstack-ceilometer16:45
*** SergeyLukjanov is now known as SergeyLukjanov_16:51
*** julienvey_ has quit IRC16:53
*** SergeyLukjanov_ is now known as SergeyLukjanov16:54
*** doug-fish has joined #openstack-ceilometer16:54
*** SergeyLukjanov is now known as SergeyLukjanov_16:55
*** xmltok has joined #openstack-ceilometer16:56
*** vrovachev has left #openstack-ceilometer17:04
*** ildikov_ has joined #openstack-ceilometer17:05
*** yassine has quit IRC17:13
*** yassine has joined #openstack-ceilometer17:14
*** yassine has quit IRC17:16
*** yassine has joined #openstack-ceilometer17:16
*** yassine has quit IRC17:16
*** yassine has joined #openstack-ceilometer17:17
*** yassine has quit IRC17:18
*** yassine has joined #openstack-ceilometer17:19
*** yassine has quit IRC17:21
*** yassine has joined #openstack-ceilometer17:22
*** sayali__ has joined #openstack-ceilometer17:25
*** yassine has quit IRC17:27
*** yassine has joined #openstack-ceilometer18:27
*** KresiusMengg has joined #openstack-ceilometer18:55
*** KresiusMengg has left #openstack-ceilometer18:55
*** jaypipes has quit IRC18:59
*** eglynn has quit IRC19:04
*** jaypipes has joined #openstack-ceilometer19:09
*** eglynn has joined #openstack-ceilometer19:23
*** julienvey_ has joined #openstack-ceilometer19:53
*** julienvey_ has quit IRC19:53
*** yassine has quit IRC20:12
*** yassine has joined #openstack-ceilometer20:12
dhellmannjd__: ping?20:32
jd__dhellmann: ?20:32
dhellmannjd__: do you still need that pbr release? I'm trying to catch mordred to see if he has any objections20:33
jd__yes20:33
dhellmannjd__: the gate runs from source, so where do you need the release? it's late enough in the afternoon on a friday that I'd like to hold off until monday if that's not a problem20:37
jd__I don't understand the question but I'm ok to wait monday20:38
dhellmannjd__: why do you need a release?20:38
jd__dhellmann: to depends on it and add the support for python 3 tests in oslo-incubator20:38
dhellmannok, right, the tests not in the gate don't run from source20:39
dhellmannI was thinking about the fact that devstack gate uses the git version of pbr, not a pypi package20:39
dhellmannignore me20:39
jd__got it :)20:39
jd__yeah I do need a release, but no hurry, I can prepare a patch without that for monday anyway20:39
dhellmannjd__: will email monty about doing a release, and go ahead and tag it monday morning20:39
*** sayali__ has quit IRC20:40
jd__thumbs up20:40
*** thomasem has joined #openstack-ceilometer20:50
*** thomasem has quit IRC21:39
*** jdob has quit IRC21:40
*** openstackgerrit has quit IRC21:53
*** openstackgerrit has joined #openstack-ceilometer21:53
openstackgerritEoghan Glynn proposed a change to openstack/ceilometer: Selectable aggregation functions for statistics [WIP]  https://review.openstack.org/7045422:06
*** doug-fish has quit IRC22:17
*** yassine has quit IRC22:26
*** jmckind has quit IRC23:04
*** julienvey_ has joined #openstack-ceilometer23:21
*** julienvey_ has quit IRC23:25
*** julienvey_ has joined #openstack-ceilometer23:26
*** prad_ has quit IRC23:31
*** julienvey_ has quit IRC23:33
*** eglynn has quit IRC23:37
*** Alexei_987 has quit IRC23:41

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!