Monday, 2017-03-13

*** karthikpr has quit IRC00:41
*** karthikpr has joined #openstack-watcher01:01
*** karthikpr has quit IRC01:02
*** zhurong has joined #openstack-watcher01:11
*** sanfern has quit IRC01:49
*** karthikpr has joined #openstack-watcher01:53
*** karthikpr has quit IRC02:42
*** karthikpr has joined #openstack-watcher02:42
*** sanfern has joined #openstack-watcher03:03
*** sanfern has quit IRC03:11
*** sanfern has joined #openstack-watcher03:14
*** ianychoi has joined #openstack-watcher03:46
*** karthikpr has quit IRC04:23
*** karthikpr has joined #openstack-watcher04:27
*** karthikpr has quit IRC04:45
openstackgerritSanthosh Fernandes proposed openstack/watcher master: Add gnocchi support in basic_consolidation strategy
*** karthikpr has joined #openstack-watcher05:03
*** thorst_afk has joined #openstack-watcher05:19
*** thorst_afk has quit IRC05:19
*** thorst_afk has joined #openstack-watcher05:28
*** sanfern has quit IRC05:32
*** thorst_afk has quit IRC05:32
*** sanfern has joined #openstack-watcher05:32
*** thorst_afk has joined #openstack-watcher05:43
*** thorst_afk has quit IRC05:47
*** sanfern has quit IRC06:36
*** sanfern has joined #openstack-watcher06:37
*** dtardivel has joined #openstack-watcher06:53
*** aspiers has quit IRC08:08
*** aspiers has joined #openstack-watcher08:11
*** openstackgerrit has quit IRC08:18
dtardivelYumeng: Hi Yumeng08:26
Yumengdtardivel:hi David08:26
*** vincentfrancoise has joined #openstack-watcher08:26
dtardivelYumeng: I reviewed your last version of spec last friday.08:27
Yumengdtardivel: Yes, I have noticed that! Many thanks08:28
dtardivelPart of comments was just about trying to understand your use case from a technical point of view.08:29
Yumengand I was just discussing with licanwei08:29
dtardivelIf  you want to use Ironic, it's ok. But we  have to do it properly, by building a baremetal data model08:30
dtardiveland parameters for Action 'power off/on' should be extract from this model.08:32
*** licanwei has joined #openstack-watcher08:37
Yumengdtardivel:  ironic is not used for baremetal management in this spec.  if you mind discussing later? I will add a diagram into the spec to show why we want to use ironic which might be easier to be understood.08:53
*** alexchadin has joined #openstack-watcher10:08
*** licanwei has quit IRC10:13
*** openstackgerrit has joined #openstack-watcher10:21
openstackgerritMerged openstack/watcher-specs master: Add team priorites for Pike
*** zhurong has quit IRC10:21
*** zhurong has joined #openstack-watcher10:21
*** sanfern has quit IRC10:34
*** dtardivel has quit IRC11:30
*** alexchadin has quit IRC11:46
*** alexchadin has joined #openstack-watcher11:47
*** figleaf is now known as edleafe12:21
openstackgerritzhongshengping proposed openstack/puppet-watcher master: Fix typo
*** zhurong has quit IRC13:01
*** ragahv has joined #openstack-watcher13:25
ragahvi am facing problem while running audit13:26
ragahvwhen i run audit i get error  The cluster state is not defined13:26
ragahvcan anyone help to resolve this issue..or tell me whether i missed some configuration13:26
vincentfrancoiseragahv: hi13:27
ragahvHi vincent13:28
ragahvreq-34955670-b106-4615-ae69-7211b5dedd6d None None] The cluster state is not defined 2017-03-13 14:52:49.160 TRACE watcher.decision_engine.audit.base Traceback (most recent call last): 2017-03-13 14:52:49.160 TRACE watcher.decision_engine.audit.base   File "/opt/stack/watcher/watcher/decision_engine/audit/", line 130, in execute 2017-03-13 14:52:49.160 TRACE watcher.decision_engine.audit.base     solution = self.do_e13:28
vincentfrancoiseragahv: can you give some detail/logs regarding your issue13:28
ragahv2017-03-13 14:52:49.160 ERROR watcher.decision_engine.audit.base [req-34955670-b106-4615-ae69-7211b5dedd6d None None] The cluster state is not defined 2017-03-13 14:52:49.160 TRACE watcher.decision_engine.audit.base Traceback (most recent call last): 2017-03-13 14:52:49.160 TRACE watcher.decision_engine.audit.base   File "/opt/stack/watcher/watcher/decision_engine/audit/", line 130, in execute 2017-03-13 14:52:49.16013:28
ragahvwhen i run audit13:29
ragahvi get this13:29
vincentfrancoisecan you also give me the version/commit hash of your Watcher?13:29
vincentfrancoiseragahv: ^13:29
ragahvi am using ocata release13:30
ragahvis that you are asking13:30
vincentfrancoiseragahv: ok thanks13:32
vincentfrancoiseragahv: I already have my guess, just verifying ;)13:32
ragahvvincent: ok13:33
vincentfrancoiseragahv: ok so my guess is that there is a bug that was found during the last few days before the final release which was backported to the ocata branch that you probably do not have13:36
vincentfrancoiseragahv: watcher 1.0.0 has this bug whereas v1.0.1 has it fixed13:37
openstackgerritzhongshengping proposed openstack/puppet-watcher master: Fix ci
openstackLaunchpad bug 1662810 in watcher "model_root.get_node_instances(node) returns none instances" [Critical,Fix released] - Assigned to licanwei (li-canwei2)13:38
vincentfrancoiseragahv: can you check if you have this fix on your side?13:39
openstackgerritzhongshengping proposed openstack/puppet-watcher master: Fix ci
ragahvVincent: Fix is in / watcher/decision_engine/model/collector/ file13:44
ragahvvincent:i have watcher version 1.0.113:46
ragahvvincent:and i could see fix present13:46
vincentfrancoiseragahv: ok good13:46
ragahvvincent: but still i am getting error13:46
vincentfrancoiseragahv: do you get this error everytime? or is it pseudo-random?13:47
ragahvvincent: everytime13:48
vincentfrancoiseragahv: can you send me the Watcher logs for the decision-engine process you have when the error occurs?13:49
ragahvi have shared it above13:49
openstackgerritMerged openstack/watcher-specs master:     Define when an action plan is stale/invalid
vincentfrancoiseragahv: the error you get is a consequence of another error that should have occured in the background13:51
vincentfrancoiseragahv: so the error you get should have logged another error before crashing during the execution of your audit13:51
vincentfrancoiseragahv: and this is this initial error I am asking you to find if you can ;)13:51
ragahvvincent:  ok..i will rerun it and check13:52
vincentfrancoiseragahv: thanks13:52
ragahvvincent:please find below log13:55
vincentfrancoisecan you paste them on please?13:55
vincentfrancoisevfrancoise: will be easier to read for me ;)13:55
ragahvvincent:pasted..i could one notification error13:57
vincentfrancoiseragahv: can't see anything13:57
ragahvvincent: in logs before the error i reported13:57
ragahvraghav: i have pasted log13:58
ragahvvincent: i have pasted the log13:58
vincentfrancoiseragahv: need to send the resulting link ;)13:58
vincentfrancoiseragahv: thx13:58
vincentfrancoiseragahv: did you configure the nova notifications as described in ?14:00
ragahvvincent:yes its same14:02
*** karthikpr has joined #openstack-watcher14:07
vincentfrancoisealexchadin: ^ any guess?14:09
alexchadinhi ragahv14:10
ragahvhi alex14:10
*** karthikpr has quit IRC14:10
alexchadinragahv: did you take these logs from decision engine logs?14:14
ragahvalex: yes14:14
vincentfrancoiseragahv: BTW, when I re-read the logs you sent, the error I was referring to should be another one14:15
alexchadinragahv: isn't there any other logs from API?14:15
vincentfrancoiseragahv: notifications.error is just Watcher notifying whoever is interested that things didn't go well14:15
ragahvvincent: yes14:16
vincentfrancoiseragahv: in case you want to plugin it to some supervision tool or 3rd party software14:16
alexchadinvincentfrancoise: maybe background update of CDM isn't good?14:16
vincentfrancoisealexchadin: yes that's what I think14:16
vincentfrancoisealexchadin: but doesn't tell me what is going wrong14:17
alexchadinwe need more infos14:17
vincentfrancoisealexchadin: 1st option I see, the CDM is stale14:17
vincentfrancoisealexchadin: 2nd option, some option: a notification-related CDM update crashes14:18
vincentfrancoisealexchadin: 3rd option: The background job that rebuilds the CDM from scratch every hour or so is crashing14:19
vincentfrancoisealexchadin: I would personally think it's option 2 or 3 but we need more as you said14:19
*** karthikpr has joined #openstack-watcher14:26
ragahvVincent: hi14:33
ragahvvincent: any clue14:33
vincentfrancoiseragahv: can you give us the full trace of what you get from the moment you restart your decision-engine process until you get the error given that you enabled the debug mode in your config file?14:38
ragahvvincent: ok sure14:38
vincentfrancoiseragahv: thx14:39
alexchadinragahv: it would be great if you provide us logs from the very beginning of service working14:43
*** karthikpr has quit IRC14:50
ragahvalex: log is very big14:51
vincentfrancoiseragahv: if it fits, send it all please14:51
ragahvvincent: is there any way i can share log file14:52
vincentfrancoisewhat's the overal size of the log?14:52
ragahvvincent: till the ppoint we reach error message it nearly  650 lines14:54
ragahvvincent: i have cut some code and pasted some portion of entire code14:56
alexchadinvincentfrancoise: lol15:01
alexchadinvincentfrancoise: have you found it too?15:01
alexchadinvincentfrancoise: 105 line15:01
*** karthikpr has joined #openstack-watcher15:10
*** karthikpr has quit IRC15:11
vincentfrancoisealexchadin: yes 00mValueError: invalid literal for int() with base 10: 'region!child@1'15:21
ragahvvincent: is it issue related to some package15:23
*** ianychoi has quit IRC15:29
vincentfrancoiseragahv: don't know why yet, but it seems like you have a field in nova that Watcher couldn't parse "region!child@1"15:31
vincentfrancoiseragahv: problem is that I can't tell which one it is that contains this15:32
ragahvvincent: ok ..i wil try to ..If you find anything please let me know15:36
*** alexchadin has quit IRC15:36
vincentfrancoiseragahv: can you run a CLI command for me please15:37
ragahvvincent: yeh tell15:37
vincentfrancoiseragahv: and tell me where in nova you see this "region!child@1" value15:37
*** karthikpr has joined #openstack-watcher15:39
*** ianychoi has joined #openstack-watcher15:40
vincentfrancoiseragahv: should be displayed in "openstack hypervisor list --long" or in "openstack hypervisor show <UUID>"15:41
ragahvyes its the ID in hypervisor list15:42
vincentfrancoiseragahv: ok so I get it now...15:42
vincentfrancoiseragahv: that's a bug on our side15:43
vincentfrancoiseragahv: Thanks for telling us15:43
ragahvvincent: ohh15:43
vincentfrancoiseragahv: can you check if it works if I tell you the fix?15:43
ragahvvincent : sure15:44
vincentfrancoiseragahv: this line
vincentfrancoiseragahv: should actually be '"id": wfields.StringField(),'15:45
vincentfrancoiseragahv: this way, it should be able to parse the hypervisor ID as a string and not an int15:46
ragahvvincent:ok i have updated in my code15:47
ragahvvincent: just restarting watcher decision engine is fine right?15:47
vincentfrancoiseragahv: yes15:47
vincentfrancoiseragahv: I am rechecking the nova API doc and it still seems odd to me though...15:48
vincentfrancoiseragahv: it worked?15:49
ragahvvincent: yes15:49
vincentfrancoiseragahv: ok now what I don't understand is why you have a string as hypervisor ID15:50
vincentfrancoiseragahv: referring to,list-hosts-detail,list-hypervisors-detail#list-hypervisors15:51
vincentfrancoiseragahv: or even,list-hosts-detail,list-hypervisors-detail,list-hypervisors-details-detail,show-hypervisor-details-detail#show-hypervisor-details15:51
vincentfrancoiseragahv: the ID field should be an integer15:51
vincentfrancoiseragahv: "id body integer The id of the hypervisor."15:52
ragahvvincent: yes...i too was going to ask you its type is integer how come it came as string15:52
vincentfrancoiseragahv: so now I am not so sure the problem comes from Watcher anymore15:52
vincentfrancoiseragahv: what's the version of your Nova setup BTW?15:53
vincentfrancoiseragahv: isn't that that novaclient's version?15:56
ragahvvincent: yes15:56
ragahvvincent: its novaclient version15:57
*** karthikpr has quit IRC15:58
*** karthikpr has joined #openstack-watcher16:00
ragahvvincent:its 15.0.116:00
vincentfrancoiseragahv: ok so ocata version too then16:01
ragahvvincent: yes16:01
vincentfrancoiseragahv: well, this is odd...16:02
vincentfrancoiseragahv: could you ask the nova folks if they know why it's not an integer?16:02
ragahvvincent: whats problem16:02
vincentfrancoiseragahv: because the docs says integer so I cannot raise a bug if what you say goes against the Nova doc and I can't tell why...16:03
vincentfrancoiseragahv: also DB-wise, id is marked as an Integer in Nova codebase (see
vincentfrancoiseragahv: ocata version
ragahvvincent: ok16:06
*** karthikpr has quit IRC16:28
openstackgerritEmilien Macchi proposed openstack/puppet-watcher master: Add bindep support
*** karthikpr has joined #openstack-watcher16:50
*** vincentfrancoise has quit IRC17:14
*** openstackgerrit has quit IRC17:18
*** karthikpr has quit IRC17:37
*** thorst_afk has joined #openstack-watcher18:04
*** thorst_afk has quit IRC18:04
*** karthikpr has joined #openstack-watcher18:06
*** sanfern has joined #openstack-watcher18:09
*** karthikpr has quit IRC18:11
*** karthikpr has joined #openstack-watcher18:28
*** wootehfoot has joined #openstack-watcher19:43
*** openstackgerrit has joined #openstack-watcher19:44
*** karthikpr has quit IRC19:47
*** karthikpr has joined #openstack-watcher19:54
*** karthikpr has quit IRC20:16
*** wootehfoot has quit IRC20:24
*** wootehfoot has joined #openstack-watcher20:41
*** karthikpr has joined #openstack-watcher20:46
*** wootehfoot has quit IRC21:15
*** aspiers has quit IRC22:03
*** karthikpr has quit IRC22:12
*** karthikpr has joined #openstack-watcher22:17
*** sanfern has quit IRC22:30
*** sanfern has joined #openstack-watcher22:34
*** karthikpr has quit IRC22:36
*** karthikpr has joined #openstack-watcher22:37
*** wootehfoot has joined #openstack-watcher22:58
*** karthikpr has quit IRC22:59
*** karthikpr has joined #openstack-watcher22:59
*** karthikpr has quit IRC23:05
*** aspiers has joined #openstack-watcher23:37
*** wootehfoot has quit IRC23:37
*** karthikpr has joined #openstack-watcher23:54
*** karthikpr has quit IRC23:58

Generated by 2.14.0 by Marius Gedminas - find it at!