Friday, 2015-12-18

*** Qiming has quit IRC00:20
*** pratikmallya has joined #senlin00:37
*** gongysh has quit IRC00:48
*** Qiming has joined #senlin01:04
*** gongysh has joined #senlin01:05
*** gongysh has quit IRC01:05
openstackgerritjunxu proposed openstack/senlin: This patch adds ScalingPolicy's event property checking in 'policy_check' function.
*** Yanyanhu has joined #senlin01:37
*** pratikmallya has quit IRC02:04
*** elynn has joined #senlin02:07
*** pratikmallya has joined #senlin02:30
openstackgerritOpenStack Proposal Bot proposed openstack/python-senlinclient: Updated from global requirements
*** pratikmallya has quit IRC02:58
*** gongysh has joined #senlin03:22
openstackgerritMerged openstack/senlin: Add blue-green deployment as a feature request
*** gongysh has quit IRC03:45
*** gongysh has joined #senlin04:04
*** xujun has quit IRC05:24
*** gongysh has quit IRC05:46
*** yuanying has quit IRC05:52
*** xujun has joined #senlin06:26
Qiminghi xujun06:33
Qimingabout the patch you proposed, please sync with Yanyanhu06:33
QimingI hope we can come up with an elegant solution for that but I myself don't have bandwidth to review it at the moment06:34
openstackgerritQiming Teng proposed openstack/python-senlinclient: Remove
xujunok, thanks06:50
*** gongysh has joined #senlin07:12
*** gongysh has quit IRC07:17
*** gongysh has joined #senlin07:30
openstackgerritQiming Teng proposed openstack/senlin: Add 'dependency' table
QimingHi, guys, I'm reworking the DB dependency support by adding new DB tables07:57
Qimingthere will be patches coming in, please help review07:57
Yanyanhudefinitely a way worth to try07:58
Yanyanhuwill help to review07:58
Qimingwe have to ...07:58
Qimingwe cannot imagine how to let sqlalchemy to do a SQL like: UPDATE action set depends_on=<existing list minus an action> where action_id=<ID>08:00
Qimingmore over, we cannot set the action status to READY if 'depends_on' becomes empty08:00
Qimingcombined with ORM, this is not a solvable problem using a list column08:01
openstackgerritQiming Teng proposed openstack/senlin: Temporarily disable lock stealer
elynnHi Qiming, What's the reason that we should disable lock stealer? ^08:23
Qimingit is in the locking critical path08:23
Qimingit is calling action.set_status instead of doing db api call directly08:24
Qimingit is doing unnecessary RPC checks of the owning engine's liveness08:24
Qimingeven I have only one engine, it is still invoking an RPC which takes a long time to respond08:25
Qimingthe execution path is interwined with the more complex issue of DB action dependency checking08:25
Qimingso ... currently I'm disabling it for debugging08:25
Qimingwe can revisit this afterwards08:26
elynnSo you mean we should store the engine status in db and directly check with db api?08:26
Qimingno further thought on that yet08:27
elynnYou are right, it's better to check if the owner is current engine, so that we can save a RPC call.08:30
QimingORM is disgusting08:31
Qimingwhen you are checking the owner, you can never be sure that the owner field has been changed from DB08:31
elynnSo you suspect lock stealer might the reason fail the functional tests?08:33
Qiminglock stealer is one of the reasons that makes the debugging a pain08:34
elynnI saw a spec in nova to migrate their db to use new enginefacade from oslo_db. I'm not sure that new enginefacade will help us or not. I will help to debug functional tests failure.08:35
elynnCan you point me where to start?08:35
Qimingenginefacade might be good08:35
Qimingplease check the oslo.db/docs/usage document08:36
QimingI'm not seeing the migration to oslo.db is giving us any advantage08:36
Qimingfor simple setter(writer) and getter(reader) functions, it looks helpful, but our current implementation is not that ugly08:37
Qimingfor more complicated scenarios, oslo_db is not helping us a lot08:37
Qimingbesides that, oslo.db is introducing READER and ASYNC_READER kind of options, which is further complicating the whole path of DB interaction08:38
Qimingit would be good to understand how enginefacade is constructed, how would we use it in Senlin db to simplify the code08:38
*** Liuqing has joined #senlin08:40
elynnif we only use 1 senlin-engine, will functional tests fail or not?08:43
Qimingstill fails08:44
Qimingyou can check the recent log of
elynnMy local env it passed, bug jenkins failed, so weird...08:45
Qimingit is not reproducible08:45
Qimingthus a problem we need to solve seriously08:46
Qimingthe root cause of this problem is about transaction isolation08:46
Qimingsql_alchemy and oslo_db is having a lot of (invalid) assumptions and doing a lot of hacks on our behalf08:47
Qimingfor example, the auto_commit of a sqlalchemy connection is set to True, the expires_on_commit is set to False by oslo_db, and set to True in sqlalchemy be default08:48
Qimingyou will never know whether your change to a python object is persisted into database in a predictable way08:48
*** xujun has quit IRC08:48
Qimingusing current database design, it would be impossible to have sqlalchemy generate a SQL to remove an action_id from depends_on field, for example08:49
Qimingif we are using ORM, the whole transaction design is becoming very very fragile08:49
QimingI have spent the whole week on this, only to learn such a hard lesson08:50
elynnsqlalchemy is not easy to understand... I spent a lot of time on that but still not understand that well...08:52
elynnI will have a look to see if i can help.08:52
*** xujun has joined #senlin08:53
Qimingthe problem is that people are doing too much assumptions in each layer08:56
QimingI'm not thinking that adding oslo_db is making our lives any easier08:56
elynnYou mean in senlin codes or in sqlalchemy?08:57
Qimingin senlin code08:59
Qimingthe annotation of enginefacade.reader looks charming08:59
Qimingbut most of the time, we don't have problem with such a simple function08:59
Qimingmigrating to that may help solve some simply transaction problems09:00
Qimingwe yet to evaluate that09:00
Qimingit would be good to start with some simple functions such as cluster_get, cluster_get_all etc09:00
elynnThanks, hope we can solve this soon...09:02
elynnNew gerrit sucks, can't find the patches that depends on current one.09:07
*** Yanyan has joined #senlin09:08
*** Yanyanhu has quit IRC09:08
Qiminga lot of people will complain09:11
*** Yanyan has quit IRC09:14
*** xujun has quit IRC09:20
openstackgerritMerged openstack/senlin: Fix action dependency problem
openstackgerritQiming Teng proposed openstack/senlin: Add 'dependency' table
*** gongysh has quit IRC09:25
*** openstackgerrit has quit IRC09:32
*** openstackgerrit has joined #senlin09:33
*** Qiming has quit IRC09:38
*** Liuqing has quit IRC09:46
*** zhenguo has quit IRC10:02
*** Qiming has joined #senlin10:32
openstackgerritMerged openstack/senlin: Add 'dependency' table
*** elynn has quit IRC11:01
openstackgerritQiming Teng proposed openstack/senlin: More test cases for purge_deleted
*** lixinhui__ has joined #senlin12:28
openstackgerritMerged openstack/senlin: Temporarily disable lock stealer
openstackgerritQiming Teng proposed openstack/senlin: More test cases for purge_deleted
*** lixinhui__ has quit IRC13:24
*** Qiming has quit IRC16:52
*** bdrich has joined #senlin17:24
*** bdrich_ has joined #senlin18:04
*** bdrich has quit IRC18:08
*** bdrich_ has quit IRC18:11
*** bdrich_ has joined #senlin18:40
*** openstackgerrit has quit IRC19:17
*** openstackgerrit has joined #senlin19:18
*** pratikmallya has joined #senlin19:47
*** pratikma_ has joined #senlin19:49
*** pratikmallya has quit IRC19:53
*** pratikma_ has quit IRC21:01
*** rebase has joined #senlin21:02
*** rebase has quit IRC21:06
*** pratikmallya has joined #senlin21:09
*** pratikma_ has joined #senlin21:10
*** pratikmallya has quit IRC21:14
*** bdrich_ has quit IRC22:09
*** pratikma_ has quit IRC22:56
*** pratikmallya has joined #senlin23:02
*** pratikmallya has quit IRC23:03
*** pratikmallya has joined #senlin23:03
*** pratikmallya has quit IRC23:05
*** pratikmallya has joined #senlin23:39

Generated by 2.14.0 by Marius Gedminas - find it at!