Tuesday, 2016-07-12

*** esberglu has joined #openstack-watcher00:09
*** thorst has joined #openstack-watcher01:21
*** thorst has quit IRC01:47
*** thorst has joined #openstack-watcher01:47
*** thorst has quit IRC01:51
*** thorst has joined #openstack-watcher01:55
*** thorst has quit IRC01:56
*** thorst has joined #openstack-watcher01:56
*** thorst has quit IRC02:02
*** diga_ has quit IRC02:23
*** jinquan has quit IRC03:20
*** jinquan has joined #openstack-watcher03:21
*** diga has joined #openstack-watcher03:50
*** Kevin_Zheng has quit IRC05:23
*** esberglu has quit IRC05:41
openstackgerrithongzhezheng proposed openstack/python-watcherclient: Remove the blank space between the function name and the parenthesis.  https://review.openstack.org/34069005:54
openstackgerritlicanwei proposed openstack/watcher: Fix unittest in test_api.py  https://review.openstack.org/34069406:06
*** vincentfrancoise has joined #openstack-watcher06:52
openstackgerritMerged openstack/python-watcherclient: Remove the blank space between the function name and the parenthesis.  https://review.openstack.org/34069007:02
tkaczynskihi guys07:14
vincentfrancoisetkaczynski: hi07:15
tkaczynskiI have a quick question about openstack/watcher plugin architecture07:15
tkaczynskiso if I want to deliver a new strategy (and in the future a new scoring engine as well) as a plugin, I need to prepare a stevedore plugin which essentially is similar to the watcher plugin itself and configured in local.conf file, correct?07:16
*** dtardivel has joined #openstack-watcher07:16
vincentfrancoiseyes but then it depends on whether you want to enable many scoring engine at once or if you just want some flag option to decide which one to enable07:19
openstackgerritDavid TARDIVEL proposed openstack/python-watcherclient: Add support continuously-optimization  https://review.openstack.org/33902607:20
vincentfrancoisealso, local.conf would be in the case of a deployment, but ultimately, that's the watcher.conf file that needs to be changed07:20
openstackgerritDavid TARDIVEL proposed openstack/watcher-dashboard: Add support continuously-optimization  https://review.openstack.org/32709507:21
tkaczynskiideally what I want is to have "dynamic" behavior. that means that I would like to have a single entry point for the plugin, but then have the ability to add/remove scoring engines on the fly07:21
tkaczynskithe plugin would take care of internal dispatching, but still somehow I need to sync the db, so the new plugins will be visible in the watcher API and watcher CLI07:22
vincentfrancoiseoh yeah I remember07:22
tkaczynskiso what I really need is to somehow expose "db.syncScoringEngines()" method from watcher plugin to my plugin07:24
tkaczynskiso I kind of have a dependency between plugins as well (scoring engine plugin depends on watcher plugin)07:24
tkaczynskiany suggestions for an elegant solution?07:25
vincentfrancoisewhat is the difference you make when you say scoring engine plugin vs watcher plugin (too much plugins I'm getting confused :p)07:25
tkaczynskiI'm talking now about general plugins which you declare in the local.conf file for your openstack / devstack. watcher is simply a plugin there, right?07:26
tkaczynskiI plan to implement also an external plugin for Intel's TAP scoring engines, which will allow to integrate TAP (Intel product) with Watcher07:27
tkaczynskiso I don't want it to integrate directly with watcher (so the watcher has no dependency on TAP), but deliver as a separate, completely optional plugin07:28
tkaczynskithis is why I'm working on scoring engine abstraction in Watcher. the ultimate goal is to enable such scenarios, so everybody can implement their own scoring engines and configure as openstack plugins and watcher will recognize them07:29
vincentfrancoiseok I see07:32
tkaczynskiso I believe it should just work for simple scenarios. if my additional plugin will define an entry point /class for a scoring engine, watcher during startup will recognize that and load in the same way as it does with built in strategies, goals etc. correct?07:34
openstackgerritMerged openstack/watcher: Make default Planner generic to handle new action  https://review.openstack.org/33892107:35
tkaczynskiI should be able now to define such plugin for a strategy, configure it in local.conf file (point to some Git repo) and it should work. the thing is I've never tried that, so maybe my understanding is wrong :)07:36
vincentfrancoiseyes correct ^^07:36
openstackgerritMerged openstack/python-watcherclient: Add support continuously-optimization  https://review.openstack.org/33902607:36
tkaczynskithis is essentially talking about that: http://docs.openstack.org/developer/watcher/dev/plugin/base-setup.html07:37
vincentfrancoiseso I recap what I understood jsut to make sure07:37
tkaczynskiso I want to do the same thing for scoring engines! with one exception: normally watcher loads and syncs with db all the plugins during start-up07:38
tkaczynskiI would like to be able to force db-resync07:38
tkaczynskilet's say from a command line07:38
vincentfrancoiseyou will write a stevode scoring engine plugin which is a an client that can call any scoring engine given that the associated (devstack-like) watcher plugin has been enabled behind the hood07:38
vincentfrancoisetkaczynski: ^^ I believe we can make that happen fairly easily07:39
vincentfrancoiseAnd because it's sync, you should be able to run as many time as you want07:40
tkaczynskiyes, but this is run as part of the watcher decision engine service07:40
tkaczynskifrom a command line I need a way to trigger it again07:40
vincentfrancoiseyes but it shouldn't break anything if you re-run it afterwards07:41
tkaczynskicorrect. but how to re-run it from command line?07:41
vincentfrancoisethe only issue would be be a race condition07:41
vincentfrancoiseyou will have to create a new command line then07:42
vincentfrancoiseor add a subcommand to an existing one07:42
vincentfrancoisesay, watcher-db-manage07:42
vincentfrancoisebut the functionality is already there so it won't be complicated07:43
tkaczynskiI was thinking about that, but watcher-db-manage seems quite specific to administrative tasks, like create db, purge db, etc.07:44
vincentfrancoisein https://github.com/openstack/watcher/blob/master/watcher/cmd/dbmanage.py#L55-L58 you can add a sync() command based on how the other commands are implemented07:44
vincentfrancoiseor if you prefer an other command07:44
vincentfrancoisethat's easy as well07:44
tkaczynskisure, but will that be easy if that other command will come in a third party plugin?07:45
vincentfrancoiseCreate a new module like watcher/cmd/dbsync.py07:45
vincentfrancoiseand add the main() function into the setup.cfg like https://github.com/openstack/watcher/blob/master/setup.cfg#L36-L4007:45
tkaczynskibecause it should know the watcher DB credentials etc07:45
vincentfrancoisethe command cannot comment from your plugin07:46
tkaczynskiyes, I know how to do that! but this additional tool should be in the third party plugin07:46
vincentfrancoisewatcher should go and load the extra from your 3rd party plugin instead07:46
tkaczynskibecause it will be specific to TAP07:46
tkaczynskicorrect! ^^07:47
tkaczynskibut I still need to trigger watcher to load that and this trigger has to be from third party plugin07:47
tkaczynskithe use case is: third party plugin will provide a scoring engine plugin (one entry point for watcher-defined entry points) and a command line tool, which will have options like add/update/remove scoring TAP engine. each call should then trigger watcher to reload scoring engines and sync DB07:49
tkaczynskithis is what I want :)07:49
tkaczynskiquestion is how to expose this db sync from watcher to third party plugin07:50
tkaczynskiand I think it's ok to limit this sync to scoring engines for now, if you don't want that for other plugins (strategies or goals)07:51
vincentfrancoise(sorry daily)07:53
tkaczynskino worries, please take your time :)07:54
vincentfrancoiseok so let's say you have a new watcher command I will call watcher-sync07:55
vincentfrancoisethis command has an initial command that does the syncing07:56
vincentfrancoiseand you can register subcommands via 3rd party plugins07:56
vincentfrancoisethen when the TAP subcommand is called, it can ask the parent to call the sync with the extra07:57
vincentfrancoisewould it be enough or am I still missing something?08:00
openstackgerritMerged openstack/watcher: Add new documentation section for Watcher policies rules  https://review.openstack.org/33830508:14
tkaczynskithinking :)08:18
tkaczynskiit seems that this could be enough. my CLI tool (from third party plugin) would then call watcher-sync and that's it.08:20
vincentfrancoiseyes because it will have access to the parent08:20
tkaczynskiI don't think that the subcommands are needed08:21
vincentfrancoiseif not then that's fine by me both ways anyway ")08:21
tkaczynskiwhat about authentication? everybody could call watcher-sync?08:21
tkaczynskior this ". openrc admin" would be needed?08:21
vincentfrancoiseIMHO no need for protecting the script because running it means you already have access to the machine and to other commands like create_schema08:22
tkaczynskisounds good. so I will prepare this watcher-sync script as part of my next changes with scoring engines. for the moment I will only do the scoring engine sync. is that ok? or better to include everything (strategies and goals)?08:24
vincentfrancoisethe smaller for reviews the better08:24
tkaczynskithanks for help!08:25
vincentfrancoiseyou're welcome :)08:25
openstackgerritlicanwei proposed openstack/watcher: Fix unittest in test_api.py  https://review.openstack.org/34069408:54
*** alexchadin has joined #openstack-watcher09:07
hanrong1I meet a problem. I upload a patchset for wachter-dashboard policies:  https://review.openstack.org/#/c/339082/11/watcher_dashboard/api/watcher.py.  In this file I wrote some code to modify horizon's settings' POLICY_FILES attribute which is defined in /horizon/openstack_dashboard/settings.py file. Although the effect is achieved, but I think there may be a better way to achieve it for these horizon's plugin projects.09:30
*** brunograz has joined #openstack-watcher09:55
alexchadinhi vincentfrancoise10:49
alexchadinLooks like I've found bug10:49
alexchadinin CLI10:49
alexchadinexception module doesn't really has ConnectionRefused error10:50
alexchadinvincentfrancoise: but it has CommunicationError https://github.com/openstack/python-watcherclient/blob/master/watcherclient/exceptions.py#L2110:51
*** alexchadin has quit IRC10:55
*** alexchadin has joined #openstack-watcher11:02
*** diga has quit IRC11:04
*** alexchadin has quit IRC11:26
*** alexchadin has joined #openstack-watcher11:26
*** alexchadin has quit IRC11:28
*** alexchadin has joined #openstack-watcher11:31
*** thorst has joined #openstack-watcher11:41
*** hanrong1 has left #openstack-watcher11:49
*** alexchadin has quit IRC12:15
*** alexchadin has joined #openstack-watcher12:16
openstackgerritMerged openstack/watcher-dashboard: Add support continuously-optimization  https://review.openstack.org/32709512:32
*** hvprash has joined #openstack-watcher12:36
*** hvprash has quit IRC12:40
*** hvprash has joined #openstack-watcher12:41
*** hvprash has quit IRC12:45
*** jed56 has joined #openstack-watcher12:47
openstackgerritAlexandr Stavitskiy proposed openstack/python-watcherclient: Fix for error importing of exception class  https://review.openstack.org/34093513:12
*** esberglu has joined #openstack-watcher13:18
openstackgerritDavid TARDIVEL proposed openstack/watcher-dashboard: [WIP]Add policies for API access control to watcher-dashboard  https://review.openstack.org/33908213:19
openstackgerritDavid TARDIVEL proposed openstack/watcher-dashboard: Add policies for API access control to watcher-dashboard  https://review.openstack.org/33908213:20
tkaczynskihi guys13:20
tkaczynskiI tried to use the latest watcher repo and it looks like decision engine is not starting (using latest master)13:21
tkaczynskilooks like this goal is indeed missing, because when I run "watcher goal list" I get:13:22
vincentfrancoiseyou have to re-run a "sudo pip install -e ." inside you /opt/stack/watcher to install the airflow entrypoint13:23
tkaczynski?? ^^13:24
tkaczynskito me the problem is with "_" missing in the goal name. the name contains space13:24
vincentfrancoisetrue !!! I didn't see that one...13:25
tkaczynskihow is that possible?13:25
vincentfrancoiseI guess he never tested it properly...13:26
tkaczynskithe goal implementation returns a name value with space...13:26
dtardiveltkaczynski: I will push a fix ASAP13:27
tkaczynskijust wondering - no test failed because of that?13:27
vincentfrancoisewe do a lookup on the goal name so if he doesn't write it correctly it won't work13:27
vincentfrancoiseyeah that's clearly a missing test here13:27
tkaczynskiok, thanks guys :) please let me know when this is fixed, I'll rebase13:28
vincentfrancoiseif I remember it correctly we test on a dummy case but I didn't write a test that would do a test matrix with all existing strategies and try to find their related goal13:28
tkaczynskisure, at least this time I've found a real problem, no something with my env :)13:29
vincentfrancoiseyes, thanks for spotting it :)13:29
tkaczynskiwell, my environment is not working now. thanks for fixing :)13:29
tkaczynskiregarding tests: I believe there is a integration test, which is starting watcher in real environment, right? no unit test is even needed, because this problem fails the whole decision engine service (and devstack)13:32
vincentfrancoiseyes but it takes a lot of time to set up integration tests for strategies so if we sufficently can raise our confidence with a unit test then I would go for this option13:33
vincentfrancoisealthough I agree all strategies should definitely have some integration tests13:34
tkaczynskiI would say just  smoke test would be the minimum. devstack should at least start with watcher from master13:34
openstackgerritDavid TARDIVEL proposed openstack/watcher: Bad goal name for airflow_optimization  https://review.openstack.org/34094613:34
vincentfrancoisecan you create a bug about it then? maybe someone other than me will accept doing this :p13:36
tkaczynskiI don't know how to create bug :P13:37
*** hvprash has joined #openstack-watcher13:42
*** hvprash has quit IRC13:44
openstackgerritDavid TARDIVEL proposed openstack/watcher: Bad goal and strategy name for Airflow Optimization  https://review.openstack.org/34094613:47
openstackLaunchpad bug 1602273 in watcher "Latest Watcher taken from master should never crash" [Undecided,New]13:47
tkaczynskidtardivel: I see that I've opened a pandora box ;)13:48
dtardiveltkaczynski: yes :)13:49
*** brunograz has quit IRC13:49
*** brunograz has joined #openstack-watcher13:50
dtardivelWe have also a pb with our unitary tests which should have detected this pb13:50
tkaczynskigood that this wasn't released and tagged :)13:51
vincentfrancoisefor sure ^13:52
*** michaelgugino has joined #openstack-watcher13:53
*** hvprash has joined #openstack-watcher13:55
*** hvprash_ has joined #openstack-watcher13:56
*** hvprash_ has quit IRC14:00
*** hvprash__ has joined #openstack-watcher14:00
*** hvprash has quit IRC14:00
*** hvprash__ has quit IRC14:02
*** hvprash has joined #openstack-watcher14:02
*** jinquan has quit IRC14:03
*** jinquan has joined #openstack-watcher14:05
*** hvprash_ has joined #openstack-watcher14:13
*** hvprash has quit IRC14:16
*** jinquan1 has joined #openstack-watcher14:25
*** jinquan1 has quit IRC14:41
*** jinquan1 has joined #openstack-watcher14:44
*** jed56 has quit IRC14:55
openstackgerritDavid TARDIVEL proposed openstack/watcher: Update unitaty tests to verify strategy and goal name  https://review.openstack.org/34098514:56
*** alexchadin has quit IRC15:13
dtardivelvi edleafe x15:13
*** dronshaw has joined #openstack-watcher15:13
edleafedtardivel: ?15:18
*** bauzas is now known as bauzas_notabot15:40
*** mestery has quit IRC15:49
*** mestery has joined #openstack-watcher15:50
*** brunograz has quit IRC15:59
*** jinquan1 has quit IRC16:03
*** bauzas_notabot is now known as bauzas16:15
openstackgerritYatin Kumbhare proposed openstack/watcher: Add Python 3.5 classifier and venv  https://review.openstack.org/34107916:35
*** diga has joined #openstack-watcher16:38
*** thorst is now known as thorst_afk16:42
openstackgerritOpenStack Proposal Bot proposed openstack/python-watcherclient: Updated from global requirements  https://review.openstack.org/34006416:42
openstackgerritOpenStack Proposal Bot proposed openstack/watcher: Updated from global requirements  https://review.openstack.org/33995216:44
openstackgerritOpenStack Proposal Bot proposed openstack/watcher-dashboard: Updated from global requirements  https://review.openstack.org/34017616:44
*** michaelgugino has quit IRC16:49
*** hvprash has joined #openstack-watcher16:58
*** hvprash_ has quit IRC16:58
*** hvprash_ has joined #openstack-watcher17:05
*** hvprash has quit IRC17:09
openstackgerritMichael Gugino proposed openstack/watcher: WIP: Implement goal_id, strategy_id and host_aggregate into Audit api  https://review.openstack.org/33016517:36
*** thorst_afk is now known as thorst17:45
openstackgerritMichael Gugino proposed openstack/watcher: WIP: Implement goal_id, strategy_id and host_aggregate into Audit api  https://review.openstack.org/33016518:00
*** hvprash_ has quit IRC18:05
*** hvprash has joined #openstack-watcher18:06
*** wootehfoot has joined #openstack-watcher18:10
*** hvprash has quit IRC18:12
*** hvprash has joined #openstack-watcher18:12
*** wootehfoot has quit IRC18:12
*** hvprash_ has joined #openstack-watcher18:13
*** hvprash__ has joined #openstack-watcher18:15
*** hvprash has quit IRC18:17
*** hvprash_ has quit IRC18:18
*** hvprash has joined #openstack-watcher18:19
*** hvprash__ has quit IRC18:22
*** hvprash has quit IRC18:28
*** hvprash has joined #openstack-watcher18:28
*** wootehfoot has joined #openstack-watcher18:31
*** hvprash has quit IRC19:08
*** hvprash has joined #openstack-watcher19:10
*** hvprash_ has joined #openstack-watcher19:11
*** hvprash has quit IRC19:12
*** hvprash has joined #openstack-watcher19:12
*** hvprash_ has quit IRC19:16
*** hvprash has quit IRC19:24
*** hvprash has joined #openstack-watcher19:25
*** hvprash_ has joined #openstack-watcher19:31
*** hvprash has quit IRC19:34
*** hvprash_ has quit IRC19:39
*** hvprash_ has joined #openstack-watcher19:39
*** hvprash_ has quit IRC19:40
*** hvprash_ has joined #openstack-watcher19:44
sballe_edleafe: Did acabot mentioned that we might like to go out for dinner or have lunch with the nova team next week. Is that a possibility?19:47
edleafesballe_: No, he didn't19:48
edleafesballe_: You buying? :)19:48
sballe_Intel is paying for lunch :) as far as I know19:49
sballe_so maybe lunch is better :)19:49
openstackgerritSusanne Balle proposed openstack/watcher-specs: Added blueprint priorities for Newton  https://review.openstack.org/34116320:05
openstackgerritMichael Gugino proposed openstack/watcher: WIP: Implement goal_id, strategy_id and host_aggregate into Audit api  https://review.openstack.org/33016520:08
openstackgerritSusanne Balle proposed openstack/watcher-specs: Blueprint priorities for Newton  https://review.openstack.org/34116320:23
*** wootehfoot has quit IRC20:53
*** thorst has quit IRC21:16
*** thorst has joined #openstack-watcher21:17
*** thorst_ has joined #openstack-watcher21:18
*** thorst__ has joined #openstack-watcher21:20
*** thorst has quit IRC21:21
*** thorst_ has quit IRC21:23
*** thorst__ has quit IRC21:24
*** dtardivel has quit IRC21:57
*** esberglu has quit IRC22:06
*** thorst has joined #openstack-watcher22:25
*** thorst_ has joined #openstack-watcher22:29
*** thorst has quit IRC22:30
*** thorst_ has quit IRC22:34
*** hvprash_ has quit IRC22:44
*** hvprash has joined #openstack-watcher22:44
*** hvprash_ has joined #openstack-watcher22:45
*** hvprash_ has quit IRC22:48
*** hvprash has quit IRC22:49
*** thorst has joined #openstack-watcher23:51

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!