Wednesday, 2017-09-27

*** liyi has joined #senlin01:41
*** liyi has quit IRC03:30
*** maestropandy has joined #senlin04:54
maestropandyHi all.. after creating cluster using senlin. status is "WARNING" why it so ?04:59
maestropandyhere the cirros file and cluster and nova output
maestropandyruijie: I checked nova logs, no isses05:00
chenyb4maestropandy, you can execute "senlin cluster-show xxx" find status_reason05:02
maestropandychenyb4: status is
maestropandyalso the nova list VM state is ERROR, where if i create stack using heat, spinning normal VM no issues..05:04
chenyb4maestropandy, Hi, you must find the vm error reson, if the nova vm appear error, senlin node create don't  succee05:06
chenyb4maestropandy, and cluster status maybe unable to change to active05:07
Qimingyes, better check the reason why the VM created is in ERROR status05:07
Qimingif you do senlin node-show <node> you may find some hints there05:08
maestropandysure i will check and get .. but meanwhile understand me how to create heat template and do scaling using senlin.. whether aodh/ceilometer need to be installed here, because ceilometer consumes lot of memory and jammed our rabbitmq pipeline.. we feel senlin is good05:09
Qimingsenlin supports manual scaling and auto-scaling05:09
Qimingby "manual scaling", you can do "cluster-scale-out", "cluster-scale-in", "cluster-resize", "cluster-nodes-add", "cluster-nodes-del"05:10
Qimingby "auto-scaling", you will have to provision your own monitoring software/solution that triggers a scale change05:11
QimingSenlin provides you the receiving end for such events, which is called a "receiver"05:11
maestropandywhile exapnading created cluster getting error as "transistion failure"
Qimingmaestropandy, time for you to dig into nova side05:13
maestropandyQiming: I am thinking to use collectd to trigger alert, apart from ceilomter do you suggest better way to trigger alert which should consume less memory05:13
Qimingyou may get more info from 'nova show', to diagnose the vm failure05:13
Qimingas for monitoring solution, we don't have any bias/preferences05:14
Qimingonce you create a receiver (by specifying the cluster, the action to trigger), you will get an unique webhook URL05:14
Qimingyou, as the solution designer/implementer, decide from where you will invoke that webhook05:15
Qimingit can be anything that can invoke a simple HTTP call to the webhook05:15
maestropandywe are holding big private cloud setup. around 500+ nodes.. and faced challenges with ceilometer..05:15
maestropandyin recent summit we attended your session and holded for some time.. now resumed work and am incharge for POC senlin05:16
Qimingno comment to ceilomter, aodh, panko, monsca, nagios, ...05:16
Qimingat the very beginning, we tried to model the "alarm" component into senlin, but we soon abandoned that idea05:17
Qimingit is very difficult to generalize05:17
Qimingcustomer use cases are so different05:17
maestropandyunderstood, so in senlin,as now PoC done with ceilometer or other plugins.. i saw senlin presentations, its not mentioned so.. if am missed please correct e05:19
Qimingintegration with aodh is documented as an example05:19
maestropandyQiming: nova has sql connectivity issue.. but wondering while creating via heat working fine,, any way i have restarted nova service will see05:20
maestropandycan i get some handson documents to integrate aodh with senlin, also i dont want to use aodh. can i use aodh itself05:20
maestropandyi dont want to use mongodb in case of ceilometer05:21
maestropandycan i use aodh to store DB05:22
QimingI think ceilometer has long abandoned mongo05:22
Qimingthey tried mysql again05:23
Qimingand then they decided to build their own TSDB (timeseries database), which is called Gnocchi05:24
maestropandyas per your ceilometer link, if i want heat.stack profile can i change "type: os.nova.server" to "type: os.heat.server" then will work ? bcz for me need heat stack.. so05:24
Qimingit should work, though you have to make sure the stack (template) contains VM servers that are properly tagged05:25
maestropandyQiming: yes you are right, I tried to point gnocchi as DB to point ceilomter, but didnt worked.. do you know how to point gnoccchi to use as DB for ceilometer ? also senlin scaleout default is 3600 .. may i know wher it config file sits and to change05:25
Qimingwhatever monitoring solution you deploy, you will have to filter and group VMs from the same cluster and then decide whether/when to scale05:26
Qiming3600 default is an option that can be tuned05:26
Qimingit used to be a global option, but recent patch has made it configurable on a per-cluster basis05:27
Qimingfor the global option, search 'default_action_timeout' on this page:
maestropandyas per, under receiver mentioned that "Each receiver is usually created for a specific purpose, so for different purposes you may need to create more than receivers." so its like different alarms/config value as like ceilometer ? can't i can keep two "ACTIONS" at a time in receiver like SCALE_OUT && SCALE_IN05:31
Qimingyou will need two receivers for two actions05:32
Qimingmost likely you will need two alarms as well05:32
maestropandybut i can map to single cluster with two actions & receivers ?05:33
maestropandyQiming: why "most likely you will need two alarms as well" I mean what are they.. ?05:33
Qimingthere is no limitation on the number of receivers you associate with a single cluster05:34
Qimingfor example, you will do this:05:34
Qiming- when average cpu utilization is above 80% for 3 minutes, scale out by 1 node05:34
Qiming- when average cpu utilization is below 40% for 10 minutes, scale in by 1 node05:35
maestropandyunderstand, but need seperate receiver for each actions to.. bcz i tried like this "openstack cluster receiver create --action CLUSTER_SCALE_OUT --params count=2 --cluster mycluster r_01  --action CLUSTER_SCALE_IN --params count=2 --cluster mycluster r_0105:35
Qimingoh, no ...05:35
Qimingyou cannot do that in a single command05:36
maestropandytried seperately.. as well. where i can see two alarm are mapped to cluster05:36
Qimingnot two alarms yet, you only created two receivers for your cluster05:37
Qimingthe next step is to trigger those receivers from your favorite monitoring software05:37
*** liyi has joined #senlin05:38
Qimingif you want, you can trigger them using cURL05:38
Qimingyou can trigger a scaling operation from crond05:38
maestropandynow i have created two actions with seperate receivers to single cluster, can i see what are the alarms mapped to that cluster. bcz cluster show not showing details.05:39
Qimingyou do receiver-list, receiver-show05:39
maestropandyalso please let me know how to integrate gnocchi with senlin,.. didn't find answer yet for it so please05:40
Qiminginstalling and configuring gnocchi is out of senlin's scope05:40
maestropandyu mean "openstack cluster receiver-list" ?05:40
Qiminggnocchi is about meter sample storage05:41
Qimingwe don't touch meter samples directly05:41
Qimingwe only provide the integration with aodh (the alarming service under the telemetry project) as an example05:41
maestropandyunderstand..FYI>. if install aodh and gnocchi its has dependencies with master branch of aodh..05:44
maestropandyQiming: I am looking for senlin administraton docs / CLI for senlin.. as of now familiar with most commands, but looking cheatsheet :) or senlin administration05:46
Qimingno cheat :)05:52
Qimingmost docs can be found here:
*** liyi has quit IRC05:56
maestropandyQiming: I think after creatinng cluster, it state_reason shows "number of active node is below desired capacity (2) " thiugh i create two nodes and add this cluster .. do you think it status will change..05:57
maestropandysenlin node-list .. totally three nodes all went to error state05:58
Qimingas I have mentioned05:58
Qimingyou may first want to dig into nova to find out why you cannot create a VM successfully05:59
maestropandysorry, i forgot. I will check ..06:00
*** liyi has joined #senlin06:07
*** maestropandy has quit IRC06:11
*** maestropandy has joined #senlin07:00
*** maestropandy1 has joined #senlin07:04
*** maestropandy has quit IRC07:07
*** ruijie has quit IRC07:19
*** ruijie has joined #senlin07:19
*** ruijie has quit IRC07:20
*** maestropandy has joined #senlin07:35
*** maestropandy1 has quit IRC07:35
*** ruijie has joined #senlin08:08
*** liyi has quit IRC08:18
*** XueFeng has quit IRC08:31
*** XueFeng has joined #senlin08:32
*** ruijie_ has joined #senlin08:56
*** ruijie has quit IRC08:59
*** maestropandy has quit IRC09:10
*** maestropandy has joined #senlin09:30
*** maestropandy has quit IRC09:51
*** maestropandy has joined #senlin09:54
*** maestropandy1 has joined #senlin09:55
*** maestropandy has quit IRC09:58
*** maestropandy1 has quit IRC10:18
*** ruijie_ has quit IRC11:11
*** maestropandy has joined #senlin12:41
*** jmlowe has quit IRC12:48
*** jmlowe has joined #senlin13:34
*** jmlowe has quit IRC14:31
*** maestropandy1 has joined #senlin14:34
*** maestropandy has quit IRC14:36
*** jmlowe has joined #senlin14:37
*** jmlowe has quit IRC14:48
*** jmlowe has joined #senlin14:48
*** jmlowe has quit IRC15:03
*** jmlowe has joined #senlin15:04
*** Tut__ has joined #senlin15:11
Tut__Hi. As soon as I enable senlin-dashboard (stable/ocata devstack on Ubuntu) my Horizon stops to work with JS error "Uncaught Error: [$injector:unpr] Unknown provider: <- <- horizon.cluster.profiles.service". Same behavior on pike and master. Any ideas?15:16
*** jmlowe has quit IRC15:32
*** Tut__ has quit IRC16:02
*** Tut__ has joined #senlin16:02
*** maestropandy1 has quit IRC16:18
*** jmlowe has joined #senlin16:35
*** eandersson has joined #senlin17:54
*** XueFeng has quit IRC18:12
*** XueFeng has joined #senlin18:12
*** jmlowe has quit IRC19:25
*** jmlowe has joined #senlin23:32
*** jmlowe has quit IRC23:52
*** jmlowe has joined #senlin23:53

Generated by 2.15.3 by Marius Gedminas - find it at!