14:00:32 #startmeeting watcher 14:00:33 Meeting started Wed Jun 1 14:00:32 2016 UTC and is due to finish in 60 minutes. The chair is acabot. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:34 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:36 The meeting name has been set to 'watcher' 14:00:41 hi 14:00:47 hi 14:01:02 agenda for today #link https://wiki.openstack.org/wiki/Watcher_Meeting_Agenda#06.2F01.2F2016 14:01:54 hello 14:01:57 hi 14:02:02 hi 14:02:04 hi 14:02:10 hi 14:02:23 hi 14:02:25 o/ 14:02:31 hi 14:02:41 hi tpeoples ; good to see you ;-) 14:02:53 #topic Announcements 14:03:05 big tent :) 14:03:07 #info Watcher has been accepted for the OpenStack big tent 14:03:15 last night 14:03:26 so congrats to everyone 14:03:30 great news! 14:03:37 +1 14:03:43 #link https://review.openstack.org/#/c/320941/ 14:03:43 o/ 14:04:09 #link http://eavesdrop.openstack.org/meetings/tc/2016/tc.2016-05-31-20.02.log.html if you want to look at the TC discussion 14:04:15 o/ 14:04:26 just some quotes I'd like to share with you 14:04:28 beautiful! 14:04:36 lots of operators seemed interested in the project at the ops meetup in manchester 14:04:47 I like to think of it as more generally the background process that will apply some long-term optimization policy as you have churn on your resources 14:04:56 "cloud defrag" 14:05:05 another example use case besides power saving: shuffle things around to get a bunch of empty hypervisors, update kernels on the empties, move things back around to get other empties 14:05:13 "policy based live migration" is what it's called in some other things ... 14:05:33 * johnthetubaguy nods at good highlights picked out there 14:05:37 all very great comments 14:06:05 also pretty great diveristy achieved pre-officialness 14:06:10 diversity* 14:06:16 the only "issue" we had was the tag "diverse-affiliation" 14:06:36 ttx : thx ;-) 14:07:15 so now we can check metrics on stackalytics #link http://stackalytics.com/?project_type=openstack&metric=commits&module=watcher-group 14:07:29 +1 14:08:18 we need more contribs to lower bcom contributions under 50% 14:08:32 if we want to get the "diverse-affiliation" tag ;-) 14:08:35 :) 14:09:16 #info This achievement has been announced on the ML 14:09:24 #link http://lists.openstack.org/pipermail/openstack-dev/2016-June/096340.html 14:09:39 I'm trying to invite more colleagues to watcher's contributing 14:10:12 I hope it will all help you to get more traction inside companies 14:10:44 lets move to watcher activity now 14:10:44 Rally peoples are interested in Watcher 14:10:59 #info Newton-1 release has been packaged on May 30th watcher v0.27, python-watcher client v0.23 & watcher-dashboard v0.3 14:11:18 all packages are available on pypi & launchpad 14:11:25 becoming big tent project definitely helps with traction I think 14:11:50 +1 14:11:55 +1 14:12:12 tkaczynski : I hope so and we will see that in the coming weeks, I would like to have bcom contribs under 50% by our next mid-cycle ! 14:12:39 #info seanmurphy published a blog post about Watcher 14:12:45 #link https://blog.zhaw.ch/icclab/employing-openstack-watcher-in-geyser-to-make-openstack-more-energy-efficient/ 14:12:58 thx seanmurphy 14:13:17 np - it’s not super high visibility stuff tbh 14:13:21 that is super. thx 14:13:39 but the bit of work that we’re doing here is within the context of that proj, so we have to promote it a bit 14:13:43 nice ! 14:13:45 do we have release notes somewhere with features released in newton-1 ? 14:14:03 I forgot to mention it but I wrote down an article on bcom website #link https://b-com.com/en/news/back-austin-openstack-summit-antoine-cabot-gives-us-news-about-watcher-project 14:14:44 tkaczynski : actually no, we rely on launchpad to identify blueprints that have been implemented, we should probably improve this 14:15:00 acabot: thanks 14:15:18 tkaczynski : I will look at it as soon as we have the Watcher doc hosted on openstack.org 14:15:31 +1 14:15:36 tkaczynski acabot: we should maybe use reno (http://docs.openstack.org/developer/reno/) to generate the release notes 14:16:00 #action acabot look at improving release notes for Watcher / use reno 14:16:20 #info mid-cycle details will be announced on the ML by EOW 14:16:36 sballe_ : how many attendees do you have ? 14:16:45 vincentfrancoise: good idea, but I guess this will require more discipline in putting the right comments in the commits or whatever is used to generate these release notes 14:16:49 for now 6 14:16:55 so we can use more 14:17:11 jwcroppe: tpeoples jed56 acabot hvprash and sballe 14:17:38 I hope to get more people from intel. I just sent them out the notification 14:17:40 ok can you send an email on the ML to promote it ? 14:18:01 sure 14:18:09 will do later today 14:18:12 tkaczynski: Probably yes, but we already write openstack compliant commit messages so this should help 14:18:18 ML activity will help a lot I think - we should also send an 'intro' note maybe now that we're Big Tent? 14:18:28 #action sballe_ send an email on the ML to promote our mid-cycle meetup 14:18:48 jwcroppe : I did http://lists.openstack.org/pipermail/openstack-dev/2016-June/096340.html 14:19:10 #info Watcher puppet repository is now available 14:19:38 thanks to the puppet team 14:19:49 #link https://github.com/openstack/puppet-watcher 14:19:50 acabot: for the internal intel clluster should edwin and I ask for rally to be installed? 14:20:04 +1 puppet team 14:20:23 we are looking for feedback from Rally team 14:20:30 alexchadin : any update on this ? 14:20:34 thanks to danpawlik for initiating it as well ;) 14:20:37 acabot: yes 14:20:37 IMHO, rally is design to benchmark Openstack 14:20:50 ok let us know since we are being ask for setup requirements 14:20:56 yes thx to danpawlik (I was looking for his IRC nick...) 14:21:03 acabot: I got instructions from Rally team of modifying Rally for Watcher Support 14:21:11 I agree but I am not a rally expert so not sure what to do if things go wrong 14:21:14 I can handle this if you want 14:21:27 but I can learn :) 14:21:36 sballe_ what about telemetry ? 14:21:39 alexchadin: what are the modifications they want to perform ? 14:21:53 edwin asked for ceilomter/monasca 14:21:56 we are community, each brings a lot for this project :) 14:22:00 hvprash: ^^ 14:22:20 alexchadin: Do you have plan to enhance rally for watcher? 14:22:28 thx sballe_, if any help needed on ceilometer will be happy to 14:22:41 jed56: we need WatcherOSClient in rally.osclients.OSClient 14:22:53 jed56: gate jobs 14:23:09 jed56: contexts 14:23:12 jed56: http://rally.readthedocs.io/en/latest/plugins/context_plugin.html 14:23:19 I'm not sure that we need to stress the watcher api 14:23:26 jed56: and scenarios http://rally.readthedocs.io/en/latest/plugins/scenario_plugin.html 14:23:50 acabot: jed56 do we think ceilomter will scal eok for the 30 node test? 14:23:53 gzhai2: I do 14:23:54 the watcher-api didn't need to scale a lot 14:24:33 I would prefer to ask for Ceilomter as the back end then Monasca. I have no real experience with Monasca 14:24:45 sballe_ : dtardivel is looking at testing the latest version and ceilometer and probably bench Monasca later on 14:24:59 sballe : I don't think so 14:25:00 ok perfect 14:25:20 does anybody remember waht the etherpad was called for this activity? 14:25:42 https://etherpad.openstack.org/p/watcher-test-environment-specifications 14:25:58 alexchadin: we need more a tool the generate specifics workloads in the VMs 14:26:05 sballe_ : bookmark this one https://etherpad.openstack.org/p/watcher-etherpads ;-) 14:26:18 perfect 14:26:21 jed56: what kind of loads? 14:26:29 jed56: CPU, RAM, Disk? 14:27:02 alexchadin : yes if we want to trigger migrations based on CPU load we need to deploy loaded VMs 14:27:03 We can have several scenario depending of the goals 14:27:26 I was using the stress tool for that 14:27:42 there is a newer version stress-ng 14:27:48 vtech : that's what we did also but its far from reality ;-) 14:27:57 there are some tools which can generate some specific workloads, like CPU, redis, etc. we used that for demo 14:28:13 tkaczynski: yes filebench for example 14:28:36 I can try to find our what we used if that helps 14:28:40 we found ceilometer to slow down the watcher api quite a bit - don’t have specifics to hand but can provide some details on the openstack-watcher chat in next day or two - even for a small scenario 14:28:45 I would need to ask my colleagues 14:28:50 this is what we need to do now, define what will run in the infra to demonstrate watcher capabilities 14:28:51 stress-ng provides much more control than the original stress - http://kernel.ubuntu.com/~cking/stress-ng/ 14:29:09 vtech : thx, I will look at it 14:29:18 The perfect tool can generate noisy neighboor , unbalanceload, etc 14:29:34 jed56 doo we have one of those 14:29:37 there is the google perfkit benchmark. haven't tried it yet though - https://github.com/GoogleCloudPlatform/PerfKitBenchmarker 14:29:41 any other announcement ? 14:29:51 vtech: I agree but they are micro-benchmark 14:29:58 acabot: could we create a etherpad about stress and workload tools ? 14:30:20 hvprash: I tried perfkitbenmark 14:30:35 any better ? 14:30:37 dtardivel: I am adding it to the etherpad for the test env 14:30:40 dtardivel : I would prefer to create a sub-section in https://etherpad.openstack.org/p/watcher-test-environment-specifications 14:30:51 acabot: +1 14:30:54 acabot: I am doing that 14:30:58 jed56, sure, I just thought stress-ng worth mentioning... 14:31:09 IMHO, we need some to tool to stress a real infrastructure but also use simulators 14:31:27 sballe_ ok 14:31:27 jed56, dtardivel feel free to add to the etehrpad 14:31:43 I have a POC which connect the watcher strategies with CloudSIm 14:31:57 What's CloudSim 14:32:19 #link www.cloudbus.org/cloudsim 14:32:35 this an academic cloud simulator 14:32:40 *is 14:33:03 #link http://beyondtheclouds.github.io/VMPlaceS/ 14:33:47 sorry but I'd like to come back to the agenda :-) 14:33:51 lol 14:33:54 #topic Review Action Items 14:33:56 :-) 14:34:19 #info Add Watcher Continiously Optimization spec has been merged #link Add Watcher Continiously Optimization spec has been merged 14:34:35 alexchadin is working on the implem 14:34:40 +1 14:34:43 +1 14:34:51 +1 14:34:56 Audit parameters should be persistent in Db needs review from core #link https://review.openstack.org/#/c/317130/ 14:35:01 it is ready to review 14:35:12 #action acabot sballe_ jwcroppe review https://review.openstack.org/#/c/317130/ 14:35:13 yeah, need more comments 14:35:36 hvprash: I will have a look on it 14:35:38 alexchadin : sorry I missed it 14:35:42 will look 14:35:49 thx 14:36:09 alexchadin needs more reviewers on https://review.openstack.org/#/c/321411/ 14:36:45 I will look at it too 14:36:46 i'll take a look at that 14:36:56 #info Cluster model objects wrapper has been unassigned from tpeoples #link https://review.openstack.org/#/c/287019/ 14:36:57 I can review it 14:37:25 acabot: does vincentfrancoise plan to pick that up? i can help with the implmentation, but don't think i'll be able to drive the whole thing 14:37:28 #action sballe_ tpeoples vincentfrancoise review https://review.openstack.org/#/c/321411/ 14:37:49 vincentfrancoise: I appreciate this:) 14:38:18 #action dtardivel review https://review.openstack.org/#/c/317130/ :) 14:39:00 tpeoples : vincentfrancoise plan to work on it as soon as efficacy indicator will be merged but he will need help on implementation 14:39:19 acabot: +1 14:39:53 ok acabot.. let me get back into the swing of things and i'll start hacking at that like i planned to originally... :( :). when vincentfrancoise frees up he can help out 14:40:19 Graph model describing virtual and physical elements in a data center & Consolidation of multiple Action-plans into a single one need reviews #link https://review.openstack.org/#/c/298891/ https://review.openstack.org/#/c/298871/ 14:40:38 acabot you can pick me 14:41:02 #action jed56 review https://review.openstack.org/#/c/298891/ & https://review.openstack.org/#/c/298871/ 14:41:13 lets move to Watcher 14:41:30 #info plugins-parameters has been merged #link https://review.openstack.org/#/q/status:merged+project:openstack/watcher+branch:master+topic:bp/plugins-parameters 14:41:44 2 new strategies have been merged Workload balance migration strategy implementation & Add Overload standard deviation strategy 14:41:52 #link https://review.openstack.org/#/c/292188/ 14:42:00 #link https://review.openstack.org/#/c/292188/ 14:42:17 #link https://review.openstack.org/#/c/298891/ 14:42:36 Watcher CLI 14:43:06 #info openstackclient-plugin has been merged #link https://review.openstack.org/#/q/topic:bp/openstackclient-plugin 14:43:27 so now you can use watcher with "openstack optimize" CLI ;-) 14:43:38 +1 14:43:46 About the watcher cli, please have a look on latest documentation. Command syntax has changed !!! 14:44:38 you can still use both "watcher" and "openstack optimize" command 14:44:56 #link http://factory.b-com.com/www/watcher/doc/python-watcherclient/openstack_cli.html 14:45:17 Watcher puppet 14:45:48 danpawlik has submited the initial commit https://review.openstack.org/#/c/323864/ 14:46:08 and need reviewers (if you are familiar with puppet :-)) 14:46:23 danpawlik: +1 14:46:46 acabot: people from puppet are very helpful and I guess there will be no problem in review :) 14:46:57 danpawlik : ok 14:47:04 #topic Blueprint/Bug Review and Discussion 14:47:32 acabot: who can review optimization-threshold? 14:47:32 #info Newton-2 is our next milestone due July 11th (just before the mid-cycle) 14:47:51 gzhai2 : right I missed this one, sorry for that 14:48:25 gzhai2 : still a jenkins issue ? 14:48:38 gzhai2 needs reviewers on https://review.openstack.org/#/c/297556/ 14:48:39 acabot: I think so. 14:49:15 jenkins has py34 and multinode error. 14:49:36 who wants to review https://review.openstack.org/#/c/297556/ ? 14:49:51 gzhai2: I have made the review but still in draft for now 14:50:08 vincentfrancoise: thanks! 14:50:21 Newton-2 #link https://launchpad.net/watcher/+milestone/newton-2 14:51:02 tkaczynski : deadline is July 11th, are you back on the scoring module ? 14:51:04 gzhai2: There is the question of how to validate the parameters that still bugs me but I will be discussing this with jed56 before posting the review ;) 14:51:32 +1 14:51:41 vincentfrancoise: we can discuss it in watcher chanel after meeting. 14:51:53 acabot: TAP sprint for the release is finishing next Tuesday (7th June), from Wednesday I'm back 100% to Watcher 14:52:04 +1 14:52:15 tkaczynski: great :) 14:52:19 tkaczynski : ok I will keep it for Newton-2 then 14:52:33 gzhai2: sure 14:52:43 we still have an unassigned BP regarding policies #link https://blueprints.launchpad.net/watcher/+spec/watcher-policies 14:52:56 does anyone wants to take it ? 14:53:00 I should be able make some good progress, not sure if finish 100%, documentation etc. 14:53:37 tkaczynski: break down your changesets so we can validate the code even if you didn't finish the doc 14:54:08 vincentfrancoise: will do my best :) 14:54:22 tkaczynski : do you plan to come at the mid-cycle ? 14:54:59 5 minutes left for open discussions 14:55:00 I would like to, but I need to talk to management about funding 14:55:08 #topic Open Discussion 14:55:22 tkaczynski : still the same issue at Intel ;-) 14:55:35 should we add puppet repository and watcher-dashboard to project-config to get IRC notifications from CI ? 14:55:39 acabot: regarding the policies BP, shouldn't nearly all of the APIs be admin only? trying to think why an end user should have access, unless we would be using policies for certain types of admins 14:56:01 tpeoples: +1 14:56:06 Why not 14:56:12 acabot: always :) but I think it makes a sense to look at every dollar spent :) it would definitely help to have meetup somewhere closer 14:56:13 I will adapt Rally for Watcher and see what we can got from this 14:56:15 tpeoples : dtardivel had a look at it a couple of months ago 14:57:37 tkaczynski: it is at the Intel location. aybe you can visit wih other peopel 14:57:49 coming back to my original discussion, should we add IRC notifications on our 2 new repos ? 14:58:18 acabot yes 14:58:20 might as well as long as it isn't spammy 14:58:30 What the Openstak way? we are now an official project 14:58:31 vote ? :) 14:58:34 I really like to have on IRC but I dont want to spam everyone 14:58:44 tpeoples: yes, we could have different types of admin user. We want just to allow an admin to configure rules from a policy file. 14:58:47 sballe_: maybe. I should know more in few days I guess 14:59:17 dtardivel: sure, that sounds reasonable... just don't think many deployers will utilize that sort of feature 14:59:24 #startvote should we add IRC notifs ? yes, no 14:59:24 Begin voting on: should we add IRC notifs ? Valid vote options are yes, no. 14:59:25 Vote using '#vote OPTION'. Only your last vote counts. 14:59:36 #vote yes 14:59:38 #vote yes 14:59:39 #vote yes 14:59:39 #vote yes 14:59:47 #vote yes 14:59:47 #vote yes 14:59:49 #vote yes 14:59:51 #vote yes 14:59:55 #vote yes 15:00:01 #endvote 15:00:02 Voted on "should we add IRC notifs ?" Results are 15:00:03 yes (9): vincentfrancoise, tpeoples, alexchadin, hvprash, jed56, dtardivel, acabot, gzhai2, sballe_ 15:00:07 ok done 15:00:18 I am glad we voted ;-) 15:00:20 FYI, I'm starting to develop a POC to avoid to bypass the nova scheduler rules 15:00:26 acabot: by the way, I was thinking in the meantime about scoring module in Watcher and it seems that I will need to make some changes comparing to what is in the spec 15:00:28 #action dtardivel add watcher-puppet & watcher-dashboard notifs on IRC 15:00:46 acabot: do I need to update the spec and go through the review again? 15:00:50 jed56: 1 15:00:51 we have to end the meeting 15:00:55 thx 15:00:56 bye 15:00:59 thx 15:01:00 bye 15:01:01 #endmeeting