Friday, 2015-08-14

openstackgerritxu-haiwei proposed stackforge/senlin: Handle exceptions which happen in heat driver
openstackgerritMerged stackforge/senlin: Profile types API test case fix
*** xuhaiwei has quit IRC01:05
*** xuhaiwei has joined #senlin01:08
*** Yanyanhu has joined #senlin01:34
*** jdandrea has quit IRC01:40
*** mathspanda has joined #senlin01:46
openstackgerritxu-haiwei proposed stackforge/senlin: Handle exceptions which happen in heat driver
openstackgerritYanyan Hu proposed stackforge/senlin: Fix gate job of Senlin functional test
openstackgerritYanyan Hu proposed stackforge/senlin: Fix gate job of Senlin functional test
*** Qiming_ has joined #senlin02:12
*** Qiming_ has quit IRC02:34
openstackgerritYanyan Hu proposed stackforge/senlin: Fix gate job of Senlin functional test
openstackgerritMerged stackforge/senlin: Updated from global requirements
*** Qiming has joined #senlin02:48
Yanyanhuhi, Qiming, made a test and found 'DEVSTACK_GATE_TIMEOUT' is not an env variable visible for devstack...02:48
YanyanhuI'm now using ERROR_ON_CLONE to have a try02:49
Qimingcheck the last few lines, it says cloning not allowed02:50
Yanyanhuyes, this is because ERROR_ON_CLONE is set to True02:50
Qimingwhy the clone was failing02:51
Yanyanhubecause there is a judgement in git_clone function which fails the installtion when it found ERROR_ON_CLONE=True02:52
Yanyanhuactually the git clone operation is not exectued02:52
Qimingfrom the name, ERROR_ON_CLONE should be true02:52
Qimingor else, senlin itself may fail the install, right?02:53
Qiming2015-08-14 02:27:44.774 | +++ trueorfalse False RECLONE02:53
Qiming2015-08-14 02:27:44.779 | ++ RECLONE=False02:53
Qimingthis looks like a problem02:53
Yanyanhunope, actually this flag is always False when devstack is running locally02:53
YanyanhuFalse is the default value02:54
Yanyanhulet me find the code02:54
Qimingso, ... pls spend some time looking into it02:54
Yanyanhusince gate will use this flag to avoid erroneous branch cloning to ensure the test target is correct02:55
Yanyanhuso this flag will be set to True when devstack is running in gate02:56
Yanyanhubut usually when we use devstack to install openstack env locally, this flag is set to False by default02:56
Yanyanhuso I think maybe we can use it to check whether the installatoin is happening in gate side02:57
Yanyanhuwill make test using this way03:00
openstackgerritMerged stackforge/senlin: Fix gate job of Senlin functional test
Qiming\o/ gate job is in!03:26
Yanyanhuyes :)03:26
Qiminghi, guys04:07
QimingThis is a .... weird ... proposal to Heat04:07
QimingI was trying to understand what is actually being proposed04:07
Qimingit is quite relevant a feature request (usage scenario) for cluster management04:07
QimingI'm suspecting that the proposal should appear here04:07
xuhaiweiseems like what senlin is doing04:19
Yanyanhuhmm, have the same feeling... will go through it04:19
Qimingthe positive side is: there are real requirements to manage resource groups in a flexible or even customizable way, Senlin is doing the right thing04:20
Qimingthe negative side: Senlin is not a famous project, people still struggle to inject this kind of features to projects like Heat04:20
Qimingto be objective, it is beyond Heat's scope04:21
Yanyanhuthat's true :(04:21
Yanyanhuso maybe we can try to lead this kind of requirement04:21
xuhaiweiwhat about inviting him to senlin04:21
QimingI told him this:
Yanyanhuhi, Qiming, seems line54-65 is duplicated in this file?04:34
Qimingah, yes04:35
Yanyanhuok, will add comment in the patch04:35
Qimingactually, the code comes from SDK:
Qimingplease propose a patch to the file with a resync04:37
Qiminga resync is necessary because there are patches like this:
Qimingdoubting why sdk leave this in its examples folder ...04:40
openstackgerritYanyan Hu proposed stackforge/senlin: Fix a typo in lbaas driver plugin
xuhaiweiI am doing the exception handling of keystone_v3 module04:58
xuhaiweithere are ResourceNotFound exception raised from endpoint_get and service_get04:59
xuhaiweiwhich means endpoint and service are not found04:59
Qimingthose are not user-visible "resources"04:59
xuhaiweithis exception should be caught in upper layer and then raise HttpInternalServerError?04:59
xuhaiweisince user doesn't know anything about these resources05:00
Qimingyes I think so05:00
Qimingit is kind of an error out of our control05:01
Qimingif those things do happen, it IS an 500 error05:01
openstackgerritMerged stackforge/python-senlinclient: Add unit test cases for sdk module
openstackgerritMerged stackforge/senlin: Fix a typo in lbaas driver plugin
openstackgerritQiming Teng proposed stackforge/senlin: Fix action base module test case
openstackgerritQiming Teng proposed stackforge/senlin: Test case for lb_delete with no physical objects
*** branw has joined #senlin06:00
openstackgerritYanyan Hu proposed stackforge/senlin: Fix a bug in lbaas driver
Yanyanhuhi, Qiming, found an issue about using oslo_context.get_current()06:28
Yanyanhuin lb_policy we use this way to get context for accessing db_api06:29
Yanyanhubut seems this context is different from action.context and thus cause the Node.load_all get [] return_value06:29
Yanyanhualthrough there is a node in cluster06:29
YanyanhuI check the user attr of oslo_context.get_current(), and it's None...06:30
openstackgerritMerged stackforge/senlin: Fix a bug in lbaas driver
Qimingthat is annoying06:31
YanyanhuI guess there might be some place to let's update the value of oslo_context.get_current()06:31
Yanyanhunot sure what I'm thinking is right06:31
Qimingis this happening in test cases or in real run?06:32
Yanyanhuin real run06:32
xuhaiweiI also found this problem in the test case06:32
xuhaiweiit cost me a lot time06:32
Yanyanhudid you find the resolution?06:33
xuhaiweiI added the project parameter, seems like that06:33
Yanyanhuyou mean manually update the context returned by oslo_context.get_current()?06:34
xuhaiweino, just give me a minute06:35
Qimingdon't do that dirty hack06:35
Qimingyou will get yourself to endless trouble06:35
YanyanhuI think maybe we can use this method, RequestContext.update_store()06:35
Yanyanhuit will update the global current context to a specified context06:36
Qimingeach action when started execution, should assume the context from the action.context06:36
xuhaiweiI added the 'project': self.context.project to the values{} to create node, because it calles for 'project'06:37
Qimingthat one was missing?06:37
xuhaiweiseems so06:37
Qimingthis above line didn't work?06:38
xuhaiweidoes the 'context' really contain 'project'?06:39
Qimingthere are cases where we were missing the initialization, e.g.
Qimingwe need to track the node creation proceduer then06:40
Qimingwhere we create a new node object, we need to pass in the context06:40
Qimingor else the 'project' won't get assigned06:41
QimingI only find two places where Node is created06:45
Qimingin engine/service for node_create and in cluster_actions06:45
Qimingin both places, we are passing in the context06:45
Qimingin node_create, we even pass kwargs for user, project ...06:46
Yanyanhuso this error shouldn't happen?06:47
Qimingit should not06:48
Yanyanhubut for the question about oslo_context.get_current(), it is different06:49
Qimingit is getting context from current thread's local storage06:49
YanyanhuI made a simple test using update_store but didn't work06:49
Qimingdon't do that, PLEASE06:50
Qimingwe are not there to do a hack yet06:50
YanyanhuI guess the global variable _request_store = threading.local() is always None in our cases?06:51
Qiminghere, when we load an action from db06:52
Qimingwe are not initializing the action.context with that from the database?06:53
Yanyanhuhmm, not sure, but I tried to use action.context to fetch node from DB and found it works well06:54
Yanyanhulet me make a check06:54
Qimingwe did that:
Qimingsomewhere somehow we lost the project info06:55
Yanyanhuseems so06:57
Qimingplease try dump the context content06:58
Qimingmaybe the class inheritance is hurting us06:58
Qimingwe have already overridden the to_dict() and from_dict() method from oslo_context.context.Context06:58
Yanyanhuyou mean the content of current context got from oslo_context?06:58
Yanyanhulet me make a test06:58
Qimingyes, maybe it contains 'tenant' instead of 'project'06:59
Qimingreally hate the 'context' thing07:00
Qimingshould kill it thoughly one day07:00
Yanyanhudump result07:00
Qimingwe are using it only for db transactions once the request comes to engine07:01
Yanyanhuseems the project info is lost in the current context07:01
Yanyanhuother info is correct07:02
Qimingokay, find out where we initialized such kind of a "partial" context then07:02
Qimingnote that it contains trusts07:03
Qimingwe did this somewhere07:03
Yanyanhuso it is not a 'bare' one07:04
Yanyanhuthis place?07:05
Yanyanhumaybe we should use 'project' rather than 'project_id' here07:05
Qiminghow could that happen07:06
Yanyanhunot sure...07:06
Yanyanhubut this didn't solve haiwei's problem?07:06
Qimingokay, try it07:06
Qimingwill diagnose that one later07:06
xuhaiweiI am in a meeting now, will help investigate it07:08
Yanyanhuseems we can't specify project and trust at the same time in ctx07:09
Yanyanhumultiple scope exception was raised07:10
Yanyanhuah, I remeber we fix a similar problem before07:10
Yanyanhuso we actually didn't need project in the context when we want to talk with other openstack services07:11
Yanyanhutrust + username/pass of senlin admin user is enough07:12
Yanyanhubut project is needed when we access DB...07:12
Qimingokay, that seems a big problem07:14
YanyanhuI remebered this line is before the line207 in the old version07:14
Yanyanhuso this error didn't happen...07:15
Qimingseems that root cause of project missing from context07:15
Yanyanhusince we haven't built the context for talking with neutron lbaas07:15
Yanyanhuan accidential correctness07:16
Yanyanhumaybe we should provide an individual param for DB access07:16
Qimingeach time you build a new context, you that context is saved in your TLS07:17
Yanyanhuwe don't use it for any other cases except DB operation07:17
Qimingthat is why get_current() seems a good thing to use07:17
Yanyanhuso maybe we use a variable for service access, e.g. openstack services and another variable for DB access07:17
Yanyanhubut this could make thing complicated?07:18
Qimingwe don't need a 'context' instance when talking to openstack services07:19
Qimingwe need a dict07:19
Yanyanhuthat is something like credential07:19
Yanyanhuthen we need something, maybe 'context' for DB07:19
Qimingyes, maybe we should start a compaign to cleanse interactions with openstack services07:20
Qimingwe avoid using the term context07:20
Qimingwe use context for DB operations as usual07:20
Yanyanhuagree with this plan07:21
Qimingthat will (hopefully) get eliminated eventually07:21
Qimingit is just a connection params, we should not construct a context there07:22
xuhaiweiwhat is the 'context' actually for?07:22
Qimingit contains authentication information when you talk to an openstack service07:24
xuhaiweiwhy is it needed by DB ?07:24
Yanyanhuxuhaiwei, actually it is weird and I'm also not clear about the exact reason...07:24
Yanyanhuseems DB need some user information to do some filtering job07:24
Yanyanhuor some access limitation07:24
Yanyanhubut in my thought, just user info is enough, we actually didn't need those content about credential07:25
Qimingit is used as a vehicle for db sessions07:26
Qimingit has been a traditional issue07:26
Qimingwe may need to revise that as just 'session', which seems more appropriate07:27
Qimingneed a better understanding of DB Facade before doing that07:29
YanyanhuI think this can help to solve the problem completely07:30
Qimingeven with that, we may still need a db_context kind of thing, to verify if the requester can read a specific object07:31
Yanyanhuyes, this is something will be kept07:31
Yanyanhuactually I think other service should also meet this kind of problem when they try to mix trust and project together in a context07:32
xuhaiweiQiming, I remembered 214line is why I added the project parameter to values07:33
YanyanhuI think maybe in first step, we keep current context for DB07:33
Yanyanhuand add another entity for talking with other services07:34
Yanyanhuor embeded the credential into current context like this:07:36
YanyanhuRequestContext: {07:36
Yanyanhu  'user': xxx,07:36
Yanyanhu  'password': xxx,07:36
Yanyanhu  'project': xxx,07:36
*** Tennyson has joined #senlin07:37
Yanyanhu  'credential': {'user':, 'password', 'trust'},07:37
Yanyanhu  ...07:37
Yanyanhuwe just use crendetial part when we tried to create connection07:37
Yanyanhuhmm, seems a little messy...07:38
YanyanhuQiming, before this job, I think maybe we can use a workaround to make lb policy work so peiyu and other guys can continue their testing? e.g. mova line 218,219 to line 20607:41
*** jroyal has joined #senlin07:43
Qimingat the same time, please help remove the initialization of "another" requestcontext when talking to sdk07:44
*** jroyal has quit IRC07:47
openstackgerritYanyan Hu proposed stackforge/senlin: Use a workaround to make lb_policy work
Yanyanhuhi, Qiming, xuhaiwei, the temporary solution for lb_policy has been proposed here, thanks
*** Tennyson has quit IRC08:03
*** Tennyson has joined #senlin08:04
openstackgerritYanyan Hu proposed stackforge/python-senlinclient: Sync ProfileAction implementation with lastest SDK code
openstackgerritYanyan Hu proposed stackforge/python-senlinclient: Sync ProfileAction implementation with latest SDK code
*** wizardYVE has joined #senlin08:12
openstackgerritMerged stackforge/senlin: Use a workaround to make lb_policy work
openstackgerritMerged stackforge/python-senlinclient: Updated from global requirements
*** Tennyson has quit IRC08:30
*** Tennyson has joined #senlin08:31
openstackgerritMerged stackforge/python-senlinclient: Sync ProfileAction implementation with latest SDK code
openstackgerritMerged stackforge/senlin: Handle exceptions which happen in heat driver
openstackgerritYanyan Hu proposed stackforge/senlin: Verify gate can work correctly with new functional test cases
*** Tennyson has quit IRC09:01
*** Tennyson has joined #senlin09:03
Qimingbye guys, :)09:14
Yanyanhuhave a good trip :)09:14
*** Qiming has left #senlin09:14
*** Qiming has quit IRC09:14
*** Tennyson has quit IRC09:26
*** mathspanda has quit IRC09:27
*** Tennyson has joined #senlin09:28
*** Tennyson has quit IRC09:30
*** lixinhui_ has joined #senlin09:31
*** wizardYVE has quit IRC09:31
lixinhui_how to define node group in senlin09:31
Yanyanhunode group? you mean create several nodes at the same time without cluster?09:32
Yanyanhuhi, lixinhui_, you mean nova's server-group?09:34
lixinhui_From model perspective09:39
lixinhui_one cluster consists of one or more group of nodes09:40
lixinhui_one group has one profile09:40
lixinhui_I think09:40
Yanyanhuhmm, I think senlin doesn't have this conception yet09:40
lixinhui_just like heat09:40
Yanyanhunow each node actually can have its own profile09:40
lixinhui_it can define resource group and instance number09:40
Yanyanhusounds like nested cluster :)09:41
lixinhui_little like that09:41
*** Liu has quit IRC09:41
lixinhui_that is okay09:42
Yanyanhunot sure whether we should this since in current design, we assume all nodes in cluster are a individual member09:42
Yanyanhuthis is a simple structure I think :)09:43
lixinhui_usual case is dependency may exist between different09:43
Yanyanhuok, I know what you mean09:44
Yanyanhuyou mean maybe the deployment sequence of 'nodes' can be controlled?09:44
Yanyanhubased on their dependency relationship09:44
lixinhui_yes, that is one kind of dependency09:44
lixinhui_does senlin has such attribute to describe that?09:45
Yanyanhuactually in my opinion this is in Heat's scope honestly :)09:45
lixinhui_I see09:46
lixinhui_I am learning heat these days09:46
Yanyanhuthe deployment sequence of resources that has dependency relatoinship is one of the control flow of orchestration serivce I think09:46
lixinhui_may pose more questions in next few days09:46
Yanyanhuyes, you can find related support in heat09:47
Yanyanhuno problem, just leave message in irc ;)09:47
Yanyanhumaybe we can also open a section in etherpad to hold question and answer09:48
Yanyanhuwill ask qiming for this09:48
Yanyanhuwill leave for home, see U guys next week. Have a good weekend :)09:49
*** Yanyanhu has quit IRC09:50
branwfrom nova.openstack.common.gettextutils import _09:59
branw2015-08-14 05:31:35.144 TRACE heat-engine ImportError: No module named gettextutils09:59
lixinhui_Bye, Yanyan10:11
lixinhui_Qiming_ has started his vacation10:12
lixinhui_You can join #heat10:12
lixinhui_to pose your question10:12
lixinhui_and catch Yanyanhu instead when working time10:13
*** LiuWei has joined #senlin10:28
*** branw has quit IRC10:39
*** mathspanda has joined #senlin11:36
*** mathspanda has quit IRC11:36
lixinhui_Qiming is on vacation since this weekend till next thursday12:03
lixinhui_you can post your question to #heat12:04
LiuWeien,I have known it from branw12:04
lixinhui_or catch Yanyanhu when dailight12:04
*** lixinhui_ has quit IRC12:05
LiuWeiokay,have a nice weekend,heihei12:05
*** lkarm has joined #senlin12:49
*** jroyal has joined #senlin13:03
*** lixinhui_ has joined #senlin13:07
*** LiuWei has quit IRC13:15
*** jdandrea has joined #senlin13:34
*** lixinhui_ has quit IRC13:45
*** jroyal has quit IRC13:51
*** mathspanda has joined #senlin13:53
*** jroyal has joined #senlin13:59
*** Qiming has joined #senlin14:29
openstackgerritMerged stackforge/senlin: Updated from global requirements
openstackgerritMerged stackforge/senlin: Test case for lb_delete with no physical objects
openstackgerritMerged stackforge/senlin: Fix action base module test case
openstackgerritMerged stackforge/senlin: Unit test for actions in engine service
openstackgerritMerged stackforge/senlin: Unit test for custom action module
openstackgerritQiming Teng proposed stackforge/senlin: Unit test for events in service engine
*** lkarm has quit IRC14:54
openstackgerritMerged stackforge/senlin: Webhook API optimization with test cases
openstackgerritMerged stackforge/senlin: Replace flavor_get_by_name with flavor_find
*** lkarm has joined #senlin14:59
*** Qiming has quit IRC15:13
*** mathspanda has quit IRC15:15
*** openstackgerrit has quit IRC15:31
*** openstackgerrit has joined #senlin15:32
*** jroyal has quit IRC16:05
*** jroyal has joined #senlin16:30
*** xuhaiwei has quit IRC16:35
*** jroyal has quit IRC16:39
*** jroyal has joined #senlin17:31
*** lkarm has quit IRC19:47
*** jroyal has quit IRC19:47
*** lkarm has joined #senlin19:51
*** jroyal has joined #senlin19:51
*** lkarm has quit IRC20:25
*** jroyal has quit IRC20:28
*** jroyal has joined #senlin21:16
*** jroyal has quit IRC21:21
*** jroyal has joined #senlin22:57
*** jroyal_ has joined #senlin22:58
*** jroyal_ has quit IRC22:59
*** jroyal_ has joined #senlin23:00
*** jroyal has quit IRC23:01
*** jroyal_ has quit IRC23:04

Generated by 2.14.0 by Marius Gedminas - find it at!