Thursday, 2017-08-24

*** hidekazu has joined #openstack-watcher00:32
*** thorst_afk has joined #openstack-watcher00:45
*** thorst_afk has quit IRC00:49
*** yuanying_ has joined #openstack-watcher00:58
*** yuanying has quit IRC01:02
*** yuanying has joined #openstack-watcher01:02
*** yuanying_ has quit IRC01:02
*** thorst_afk has joined #openstack-watcher01:23
*** thorst_afk has quit IRC01:46
*** yuanying has quit IRC01:50
*** zhurong has joined #openstack-watcher01:54
*** yuanying has joined #openstack-watcher02:01
*** thorst_afk has joined #openstack-watcher02:19
*** thorst_afk has quit IRC02:19
*** thorst_afk has joined #openstack-watcher03:20
*** thorst_afk has quit IRC03:25
*** nicolasbock has quit IRC03:35
*** thorst_afk has joined #openstack-watcher04:21
*** thorst_afk has quit IRC04:26
*** zhurong has quit IRC04:58
*** zhurong has joined #openstack-watcher05:06
*** thorst_afk has joined #openstack-watcher05:22
*** thorst_afk has quit IRC05:26
*** thorst_afk has joined #openstack-watcher06:23
*** thorst_afk has quit IRC06:27
*** thorst_afk has joined #openstack-watcher07:24
*** thorst_afk has quit IRC07:28
*** alexchadin has joined #openstack-watcher07:31
*** vincentfrancoise has joined #openstack-watcher07:35
*** thorst_afk has joined #openstack-watcher08:25
*** thorst_afk has quit IRC08:29
hidekazumany fixed bugs were cherry-picked to stable/pike..08:33
*** alexchadin has quit IRC08:53
*** alexchadin has joined #openstack-watcher08:54
alexchadinaspiers: hi08:55
aspiershi alexchadin08:58
aspiersI'll be free to talk in about 30 mins, is that OK?08:58
alexchadinaspiers: yeap08:58
openstackgerritHidekazu Nakamura proposed openstack/watcher-specs master: Add cdm-scoping spec
*** hidekazu has left #openstack-watcher09:21
*** thorst_afk has joined #openstack-watcher09:25
*** suzhengwei has joined #openstack-watcher09:28
*** thorst_afk has quit IRC09:30
*** vincentfrancoise has quit IRC09:43
*** vincentfrancoise has joined #openstack-watcher09:45
aspiersalexchadin: ok09:45
*** alexchadin has quit IRC09:49
*** alexchadin has joined #openstack-watcher09:54
alexchadinaspiers: ping09:54
alexchadinaspiers: back yo your proposal, what strategy do you want to suggest?09:55
aspiersso I figured out how to do VM consolidation via linear programming09:56
aspiersit's actually pretty easy09:56
aspiersthe only problem is the algorithmic complexity, which would probably require partitioning the cloud up into chunks (e.g. max 500 servers per chunk)09:56
aspiersbut I don't think this would be an issue09:56
alexchadinyou speak about vms or nodes (500 servers)?09:57
aspierscompute hosts09:57
alexchadinoh, ok09:57
aspiersand then once the optimal placement is calculated, we can use the software I already wrote for ordering the migration rearrangements09:57
aspiersand IIUC we also now have a power-off strategy, right?09:58
aspiersso they could all be combined to minimise the amount of energy spent on running compute hosts09:58
aspiersis there already a VM consolidation strategy?09:58
alexchadinaspiers: some of them are presented already09:59
alexchadinlet me see09:59
alexchadinaspiers: there is one proof of concept:
alexchadinoh, it's about balancing, not consolidation10:01
aspiersyeah, that's the opposite10:01
alexchadinaspiers: consolidation one:
aspiersOK, looking10:01
alexchadinaspiers: we also have basic consolidation strategy, but it's more about proofing that watcher works10:02
aspierswhich one is that?10:02
alexchadinaspiers: I'd like to ask you to create appropriate blueprint with pointing out methods of linear programming you plan to use in your strategy10:05
aspiersok cool10:05
aspiersis it already possible to combine the existing consolidation strategies with the power off strategy?10:06
alexchadinto be honest, we haven't planned to cross strategies as they are independent by its nature10:07
alexchadinaspiers: instead of it, you are free to use any actions watcher provides10:07
aspiersoh I see, so the solution would include both migrations and power off?10:08
openstackgerritMerged openstack/watcher master: Remove watcher_tempest_plugin
alexchadinaspiers: yes10:09
openstackgerritMerged openstack/watcher master: Updated from global requirements
alexchadinaspiers: you may use any of these methods:
alexchadinactons, excuse me10:10
aspierscool, that's really helpful thanks!10:10
alexchadinlist of these actions is result of your perfect super best consolidation strategy ;D10:10
aspiersso the solution will provide an ordering of the actions, but then the planner can reorder them?10:11
aspiersI think any solutions generated in this way would have a very specific order which could not be easily changed10:11
alexchadinaspiers: yes, we have two planners to use:
alexchadinaspiers: unfortunately I should go for doings, we have pluggable planner mechanism, you may provide yours if you want10:13
aspiersalexchadin: I have other things to discuss too10:15
alexchadinaspiers: I'll ping you once get back10:15
aspiersok great!10:15
*** alexchadin has quit IRC10:16
*** vincentfrancoise has quit IRC10:22
*** thorst_afk has joined #openstack-watcher10:26
*** thorst_afk has quit IRC10:31
*** nicolasbock has joined #openstack-watcher10:33
*** dpawlik_ is now known as dpawlik10:51
*** nicolasbock has quit IRC10:54
*** nicolasbock has joined #openstack-watcher11:08
*** thorst_afk has joined #openstack-watcher11:27
*** alexchadin has joined #openstack-watcher11:31
*** thorst_afk has quit IRC11:32
*** alexchadin has quit IRC11:36
*** alexchadin has joined #openstack-watcher11:37
*** vincentfrancoise has joined #openstack-watcher12:15
*** thorst_afk has joined #openstack-watcher12:16
*** Yumeng has quit IRC12:21
openstackgerritMerged openstack/watcher master: Fix KeyError exception
*** vincentfrancoise has quit IRC12:43
*** vincentfrancoise has joined #openstack-watcher12:44
*** alexchadin has quit IRC12:52
*** alexchadin has joined #openstack-watcher12:53
alexchadinaspiers: I'm back13:01
alexchadinsballe_: hi13:11
*** vincentfrancoise has quit IRC13:13
*** vincentfrancoise has joined #openstack-watcher13:14
aspiersalexchadin:  hi13:15
alexchadinaspiers: pong13:15
aspiersalexchadin: so one question I had is around terminology13:16
aspiersI don't understand this definition13:16
aspiersit doesn't seem to relate to any other concept for grouping machines in OpenStack13:16
aspiersand it also conflicts with other definitions of "cluster" in OpenStack13:16
aspierswhat does "managed by the same controller node" mean?13:17
alexchadintypical grouping in OpenStack is being managed by Availability Zones, Host Aggregates and Cells, right?13:18
aspiersand regions13:18
alexchadinyes, I've forgot about it13:18
alexchadinwhat other definitions of cluster have you found?13:19
aspiersthe control plane typically contains HA clusters13:20
aspiersand also Senlin manages clusters13:20
aspiersalso the control plane does not typically run on a single controller13:20
aspiersit gets split across multiple nodes, sometimes even multiple clusters13:20
aspiersso it does not really make sense to say that a machine is managed by one controller node13:21
aspiersso I'm trying to understand what that definition really means13:21
aspiersis Watcher tracking its own grouping of compute nodes?13:21
aspierswhere is a Watcher "cluster" defined?13:22
alexchadinbasically, we tracked all VMs and Compute Node we could find13:22
aspiersso that is the whole compute plane of the cloud?13:23
alexchadinthen, we have implemented Audit Scope that allows us to restrict scope of resources which are using during working of strategy13:23
aspiersOK but where is a Watcher "cluster" defined? in the Audit Scope?13:24
alexchadinaspiers: I haven't met it in the code :)13:24
alexchadinWe have Cluster Data Model13:24
aspiersyes, I saw that13:25
aspiersI am trying to understand what a Watcher cluster really is13:25
alexchadinIt's a graph with nodes and related VMs13:25
aspiersis it a strict subset of the compute plane, or just the whole compute plane?13:25
alexchadinaspiers: to give you the answer I need to know what compute plane is13:26
aspiersthe compute plane is all the compute hosts in the cloud13:26
alexchadinso I'll read this article now13:26
aspierssome people *might* also include nova-{scheduler,conductor} etc. in the compute plane, but IMHO they are part of the control plane instead13:27
alexchadinaspiers: Nova Cluster Data Model collects all compute nodes and their VMs in background13:29
alexchadinCinder Cluster Data Model does the same thing for Volume Nodes13:30
aspiersalexchadin: I guess you mean
aspiersalexchadin: yeah so it seems that it really refers to *all* compute hosts and instances, i.e. the whole compute plane13:36
aspiersin which case I think "cluster" is really not a good term to use13:37
alexchadinaspiers: if you want to rephrase the definition, feel free to submit the patch13:37
aspiersalexchadin: OK, I'll have a think about the best way to deal with it. Thanks!13:37
alexchadinaspiers: thanks for your help!13:38
aspiersalexchadin: welcome :) I had one more question13:38
aspiersalexchadin: are there plans to extend the scope of Watcher beyond optimization?13:38
alexchadinaspiers: We didn't plan to do it. We have some thoughts about including some things to optimize (i.e. containers), but there weren't discussions about extending the scope13:40
alexchadinaspiers: We have pretty straightforward goal, why should we extend the scope?13:41
aspiersalexchadin: I don't think you should :)13:41
aspiersand even if you optimize containers, that is still optimization, so it is still inside the current scope of Watcher's mission statement13:42
aspiersI am asking because there have been one or two blueprints submitted recently which extend the scope *outside* optimization13:42
*** vincentfrancoise has quit IRC13:42
alexchadinaspiers: which ones?13:43
aspiersinto also handling failures13:43
*** vincentfrancoise has joined #openstack-watcher13:44
aspiersif Watcher starts to tackle auto-healing of failures then it is no longer just optimization, but also high availability13:44
aspiersand then it starts to overlap with other OpenStack projects which already exist13:44
aspiersthis particular spec is targetting the exact same failure scenario which there is already a big project addressing13:45
aspiersso I don't understand why it is being proposed13:46
aspiersI asked on for an explanation but I didn't get one yet13:47
alexchadinI haven't approved it yet, it sounds fair enough13:47
alexchadinWatcher is created to reduce total cost of ownership by using optimization models with different goals13:49
aspiersyes, that makes sense to me13:51
aspiersand by definition optimization makes a cloud which is already working, better.13:51
aspiersfixing a broken cloud is not optimization13:51
aspiersI suppose some people might want to argue with the last point13:52
alexchadinit's more about Python script bound by something like cron than optimization algorithm13:52
aspiersthe best definition I could find was
aspiersand that article does not mention anything about optimization fixing failures13:53
aspiersit also does not mention HA13:53
aspiersbut even if hypothetically optimization *did* include fixing failures, it would still not make sense for Watcher to duplicate the work of existing OpenStack projects dedicated to HA13:54
alexchadinaspiers: I think it's time to call suzhengwei :)13:54
aspiersthe question is simple: if you want a compute HA solution, why not just use Masakari? it is a well-established OpenStack project, which the OpenStack HA community has agreed is the best solution13:55
aspiersinstead of reinventing the wheel, it would be more effective to collaborate on the existing solution13:55
alexchadinI agree13:56
alexchadinaspiers: well, it's more about fixing post-failure state of cloud than optimising the living, right?13:57
aspiersalexchadin: masakari is about fixing post-failure state, yes. It does not do any optimisation13:58
aspiersalexchadin: so currently there is a nice clean divide between the responsibilities of masakari and watcher13:58
alexchadinaspiers: I was speaking about proposed strategy13:58
aspiersoh yes, the proposed strategy changes that divide so that it is no longer clear13:58
aspierswith that strategy, Watcher would be not just handling optimisation, but also post-failure states, so it would overlap with masakari13:59
openstackgerritOpenStack Release Bot proposed openstack/puppet-watcher master: Update reno for stable/pike
aspiersalexchadin: another problem with this proposal is that it is impossible to do safe compute HA without fencing, and Watcher has no fencing mechanism14:04
aspiersalexchadin: Masakari uses Pacemaker for fencing in order to ensure that VM resurrection via nova-evacuate is safe14:05
aspiersalexchadin: so in order to implement this strategy in Watcher, you would need to add a dependency from Watcher on Pacemaker or some other cluster manager which implements quorum/voting/consensus/fencing14:06
alexchadinaspiers: I've left my questions here:
aspiersalexchadin: OK thanks14:10
aspiersI explained the need for fencing in Austin
alexchadinaspiers: thanks for link14:12
*** alexchadin has quit IRC14:46
*** ianychoi has joined #openstack-watcher16:00
*** openstackgerrit has quit IRC16:04
*** vincentfrancoise has quit IRC16:13
*** vincentfrancoise has joined #openstack-watcher16:14
*** efoley has joined #openstack-watcher16:44
*** efoley has quit IRC16:54
*** vincentfrancoise has quit IRC16:57
*** openstackgerrit has joined #openstack-watcher19:08
openstackgerritAlex Schultz proposed openstack/puppet-watcher master: Update versions for Queens cycle
*** thorst_afk has quit IRC20:16
*** thorst_afk has joined #openstack-watcher20:19
*** thorst_afk has quit IRC20:24
*** thorst_afk has joined #openstack-watcher20:36
*** yuanying_ has joined #openstack-watcher23:59

Generated by 2.15.3 by Marius Gedminas - find it at!