13:01:05 <yanyanhu> #startmeeting senlin
13:01:06 <openstack> Meeting started Tue Jun 28 13:01:05 2016 UTC and is due to finish in 60 minutes.  The chair is yanyanhu. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:01:07 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:01:09 <openstack> The meeting name has been set to 'senlin'
13:01:20 <yanyanhu> hi
13:01:28 <elynn> o/
13:01:57 <yanyanhu> hi, elynn
13:02:01 <yanyanhu> hi, lixinhui_
13:02:05 <yanyanhu> long time no see :P
13:02:33 <yanyanhu> ok, here is the agenda, plz feel free to add any item you want to discuss
13:02:38 <yanyanhu> https://wiki.openstack.org/wiki/Meetings/SenlinAgenda#Weekly_Senlin_.28Clustering.29_meeting
13:03:06 <yanyanhu> #topic newton workitem
13:03:17 <yanyanhu> https://etherpad.openstack.org/p/senlin-newton-workitems
13:03:29 <yanyanhu> here is the etherpad track our newton workitem
13:03:39 <yanyanhu> first item, testing
13:03:47 <yanyanhu> tempest API test has been done
13:03:56 <yanyanhu> and tempest functional test is in good progress
13:04:03 <elynn> Dose release notes in?
13:04:14 <yanyanhu> I think all existing functional test cases have been migrated(some in progress)
13:04:39 <yanyanhu> elynn, not yet, several patches are still under review :)
13:04:53 <yanyanhu> but I think we can finish it in this week
13:05:15 <yanyanhu> the new gate job for tempest functional test is also available now
13:05:35 <yanyanhu> although it is experimental and doesn't vote
13:05:58 <yanyanhu> after the functional test migration is done, I will propose a patch to remove the old functional test gate job
13:06:06 <yanyanhu> it will be replaced by the new one :)
13:06:27 <yanyanhu> then we will have two jobs enabled in both check and gate pipeline and they will vote
13:06:45 <yanyanhu> ok, this is the tempest test part
13:06:58 <yanyanhu> about rally plugin, didn't get time to work on it this week
13:07:13 <yanyanhu> but our patch to add cluster/profile plugin for rally got first +2
13:07:19 <yanyanhu> need another +2 and workflow
13:07:28 <yanyanhu> basically, it looks good now
13:07:40 <yanyanhu> ok
13:07:55 <yanyanhu> hi, lixinhui_ , around?
13:08:07 <yanyanhu> any update on HA related work :)
13:08:28 <yanyanhu> I think Qiming didn't get time to work on it in last week
13:08:35 <lixinhui_> yes
13:08:48 <lixinhui_> I finished the fencing tests
13:08:59 <yanyanhu> great!
13:09:01 <lixinhui_> need to discuss how to bring into
13:09:03 <lixinhui_> Senlin
13:09:06 <yanyanhu> that is important for our HA solution
13:09:08 <yanyanhu> ok
13:09:26 <lixinhui_> And I tried the resilient
13:09:34 <yanyanhu> I think we can make further discussion in irc channel to decide how to merge into current HA framework
13:09:42 <lixinhui_> elastic cluster template with ceilometer/aodh/gnocchi
13:09:52 <lixinhui_> yes, yanyanhu
13:10:17 <yanyanhu> so monitoring has been included?
13:10:31 <yanyanhu> for failure detection?
13:11:47 <yanyanhu> actually I'm thinking how to build the basic workflow of our HA solution, including leverage other monitor service to detect failure in different layers
13:11:48 <lixinhui_> what the failure detection?
13:12:02 <yanyanhu> like host/VM crash or app failure
13:12:04 <lixinhui_> the failure detection is based on node status
13:12:18 <lixinhui_> yes
13:12:20 <lixinhui_> yanyanhu
13:12:23 <yanyanhu> yes, from nova notification, e.g.
13:12:26 <yanyanhu> for VM failure
13:12:43 <yanyanhu> I think Qiming was working on this?
13:12:44 <lixinhui_> one problem I met is about heat
13:12:49 <yanyanhu> yes?
13:13:06 <lixinhui_> where do you know to set the timeout or retry time
13:13:18 <yanyanhu> retry for?
13:13:20 <lixinhui_> creation of loadbalancer is very slow
13:13:31 <lixinhui_> then heat stack-create will keep retrying
13:13:32 <yanyanhu> yes, it is sometimes
13:13:48 <yanyanhu> I'm not sure there is such an option to customized
13:13:54 <lixinhui_> then there will be several loadbalancers under creation
13:13:57 <yanyanhu> hi, elynn, do you have any idea?
13:14:35 <lixinhui_> I think there should be somewhere can be customized
13:14:49 <lixinhui_> and I know Qiming is adding listener
13:14:56 <lixinhui_> for detection functions
13:15:09 <yanyanhu> I guess that is a fixed value defined in heat engine?
13:15:09 <yanyanhu> so you put all those resources in a heat template?
13:15:31 <lixinhui_> yes
13:15:35 <elynn> retry time for creating resource?
13:15:40 <lixinhui_> yes
13:15:43 <lixinhui_> elynn
13:16:03 <yanyanhu> lixinhui_, maybe you can try to split lb resource from other ones if the stack creation always failed for the timeout of lb creation
13:16:13 <lixinhui_> to avoid creations duplicated creation of loadbalancer
13:16:44 <lixinhui_> yanyanhu, I am using the same template as we presented in Austin
13:16:58 <lixinhui_> which is committed into as a tutorial
13:17:03 <yanyanhu> lixinhui_, that should happened I think if you mean duplicated creation of lb for timeout retry
13:17:03 <elynn> I can't remember any property or configurations about that.
13:17:20 <yanyanhu> lixinhui_, I see
13:17:30 <lixinhui_> that is okay elynn
13:17:38 <lixinhui_> octavia is so slow
13:17:38 <yanyanhu> sorry, that shouldn't happen
13:17:49 <yanyanhu> so haproxy works well
13:17:51 <elynn> I can check later
13:18:01 <yanyanhu> but octavia doesn't?
13:18:05 <lixinhui_> that will be very nice, elynn
13:18:12 <elynn> That would be a problem if it happens on presentation...
13:18:18 <lixinhui_> yes, yanyanhu
13:18:30 <yanyanhu> I see
13:18:39 <lixinhui_> elynn, nsx and haporxy work well
13:18:43 <yanyanhu> elynn, hope we can find some solution for it
13:19:00 <yanyanhu> or in worst case, use some workaround
13:19:00 <lixinhui_> why so many people use octavia
13:19:12 <yanyanhu> lixinhui_, we don't :)
13:19:31 <lixinhui_> yanyanhu, what kinds backend for IBM
13:19:32 <lixinhui_> ?
13:19:35 <elynn> We can at least change the hardcode retry times in our env ;)
13:20:10 <yanyanhu> I'm not sure. But I think octavia is better than haproxy hosted in network controller
13:20:24 <elynn> Saw an option client_retry_limit
13:20:32 <elynn> we can try to increase it in heat.conf
13:20:43 <yanyanhu> not only reduce the risk of single failure, but also much better scalability I think
13:20:50 <lixinhui_> oh, elynn?
13:21:11 <yanyanhu> sorry， single point failure
13:21:11 <lixinhui_> okay yanyanhu
13:21:18 <lixinhui_> I see
13:21:36 <yanyanhu> BTW, I have draft a topic proposal about HA for Barcelona summit
13:21:43 <yanyanhu> https://www.openstack.org/summit/barcelona-2016/call-for-presentations/manage/15037/summary
13:21:56 <yanyanhu> hope we can finish our HA design and make a presentation for it
13:22:07 <yanyanhu> I added you two and Qiming's name as speaker
13:22:17 <elynn> And also we can specify an timeout parameter when creating new stack.
13:22:31 <yanyanhu> I think we can make further discussion to see how can we build this demo
13:22:40 <lixinhui_> okay, yanyanhu. will read it and raise discussion
13:22:58 <lixinhui_> sounds like a better solution, elynn
13:23:02 <yanyanhu> lixinhui_, thanks :) I think HA is an important feature we need to finish in this cycle
13:23:56 <yanyanhu> although maybe the basic one, we hope it is a complete loop, from failure detection to recovery
13:24:00 <lixinhui_> is that a cli parameter? elynn
13:24:23 <elynn> lixinhui_: yes
13:24:38 <lixinhui_> okay, I will try it
13:24:47 <lixinhui_> yes, yanynahu
13:24:53 <yanyanhu> lixinhui_, timeout works well I think
13:25:01 <yanyanhu> for heat stack creation
13:25:16 <lixinhui_> :)
13:25:37 <yanyanhu> you can have a try. I tried to increase it from 1 hour to 10 hours when deployed a very large and complicated stack in softlayer
13:25:39 <yanyanhu> :)
13:26:09 <elynn> That is so long.
13:26:11 <lixinhui_> happy to know
13:26:34 <yanyanhu> ok, I think we can collect our idea about HA topic using  this existing etherpad: https://etherpad.openstack.org/p/senlin-ha-recover
13:26:53 <yanyanhu> elynn, yes, since the deployment of some services inside VM is very slow :)
13:26:57 <yanyanhu> like DB2
13:27:14 <yanyanhu> and we used software deployment for it
13:27:21 <elynn> that's true...
13:27:33 <yanyanhu> anyway, it works good
13:27:54 <yanyanhu> ok, so this is about HA?
13:27:59 <lixinhui_> yes
13:28:01 <yanyanhu> we can have more discussion offline
13:28:06 <yanyanhu> thanks, xinhui
13:28:06 <lixinhui_> sure
13:28:22 <lixinhui_> pleasure
13:28:22 <yanyanhu> next one
13:28:27 <yanyanhu> :)
13:28:31 <yanyanhu> lets skip document
13:28:43 <yanyanhu> I guess haiwei is not here?
13:29:00 <yanyanhu> I noticed he proposed patch to add docker driver
13:29:16 <yanyanhu> although just for most basic operations like creating, deleting
13:29:22 <yanyanhu> it's a startpoint I think
13:29:45 <yanyanhu> will talk with him to add more driver interfaces
13:30:10 <yanyanhu> umm, for other left items, we have no progress I think
13:30:18 <yanyanhu> so topic 2
13:30:27 <yanyanhu> #topic proposal for summit
13:30:47 <yanyanhu> hi, elynn, lixinhui_ , any idea :)
13:30:53 <yanyanhu> besides the HA one
13:31:14 <elynn> Not sure what else we can propose
13:31:25 <lixinhui_> cluster.do
13:31:42 <lixinhui_> I think that is worthy a talk
13:31:47 <yanyanhu> you mean deployment? lixinhui_
13:31:52 <yanyanhu> of app
13:31:57 <lixinhui_> yes
13:32:14 <lixinhui_> and any cluster management
13:32:15 <yanyanhu> I see. It's very useful and powerful I believe
13:32:33 <yanyanhu> just I feel we may need a use case as reference
13:32:55 <lixinhui_> integration will some given agent
13:32:59 <yanyanhu> otherwise, pure technical discussion could be difficult to understand for audience who are not familiar with Senlin code
13:33:23 <lixinhui_> yes
13:33:24 <yanyanhu> lixinhui_, for agent, you mean?
13:33:37 <elynn> agree
13:33:39 <lixinhui_> agent of some given monitor
13:34:05 <yanyanhu> so this for HA solution?
13:34:15 <yanyanhu> as monitoring part?
13:34:20 <lixinhui_> I think it should be a separate topic
13:34:26 <yanyanhu> ok
13:34:27 <elynn> We can think of some use cases, like scaling from a standby pool, green-blue deployment
13:34:39 <lixinhui_> cool, ekynn
13:34:41 <yanyanhu> elynn, that's interesting as well
13:34:44 <lixinhui_> elynn
13:34:54 <yanyanhu> ok, I will create an etherpad to collect these ideas
13:34:55 <elynn> How to use senlin for better management.
13:35:07 <yanyanhu> and then we can discuss and refine them to see what we can propose
13:35:47 <elynn> yes
13:35:48 <yanyanhu> #action yanyanhu to create an etherpad to collect proposal ideas
13:35:48 <yanyanhu> oh, BTW, I also talked with eldon to see whether we can propose one to share their experience on managing cluster in large scale
13:35:48 <yanyanhu> eldon from cmcc
13:36:16 <lixinhui_> cool yanyanhu
13:36:18 <yanyanhu> they have tried to use senlin to manage  cluster consists of a thousand of VMs?
13:36:19 <yanyanhu> I think
13:36:37 <lixinhui_> you know I always wanna to learn more details
13:36:39 <yanyanhu> so that can be a very good demonstration and sharing
13:36:50 <yanyanhu> they also met some problems I believe
13:36:58 <yanyanhu> lixinhui_, sure, me too :)
13:37:07 <lixinhui_> :)
13:37:14 <yanyanhu> so lesson and learn
13:37:18 <yanyanhu> ok
13:37:28 <yanyanhu> will add it as well
13:37:47 <yanyanhu> I will create the etherpad and post the link into irc channel
13:38:10 <yanyanhu> the deadline is 13th ?
13:38:38 <elynn> yes
13:38:39 <yanyanhu> so we need to propose before that date :)
13:38:51 <elynn> about 2 weeks
13:38:55 <yanyanhu> yea
13:39:08 <yanyanhu> lots of thinking needed :P
13:39:26 <yanyanhu> ok, that's for proposal for summit
13:39:43 <yanyanhu> #topic open discussion
13:39:57 <yanyanhu> any other items you guys want to discuss?
13:40:43 <elynn> no from me
13:40:55 <yanyanhu> if not, we can finish 20 minutes earlier :)
13:41:11 <lixinhui_> Qiming should be a good dad
13:41:28 <elynn> Congratuate to him :)
13:41:36 <yanyanhu> lixinhui_, I believe so :)
13:41:39 <yanyanhu> yea
13:41:53 <lixinhui_> :)
13:42:11 <yanyanhu> so maybe a month later, we can go to his home to see his baby:)
13:42:18 <lixinhui_> haha
13:42:22 <yanyanhu> heihei
13:42:29 <yanyanhu> lets go together
13:42:37 <lixinhui_> he will be managed by two women
13:42:41 <elynn> haha
13:42:42 <lixinhui_> since then
13:42:43 <yanyanhu> haha
13:42:49 <yanyanhu> lixinhui_, LOL
13:42:55 <yanyanhu> yep
13:43:03 <yanyanhu> ok, thank you so much for joining
13:43:12 <yanyanhu> I think we can finish the meeting now
13:43:14 <lixinhui_> see you next time
13:43:20 <yanyanhu> lets make further discussion later
13:43:20 <elynn> cu
13:43:22 <yanyanhu> see U
13:43:27 <yanyanhu> #endmeeting