17:06:56 #startmeeting Rally 17:06:57 Meeting started Tue Dec 9 17:06:56 2014 UTC and is due to finish in 60 minutes. The chair is msdubov_. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:06:58 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:07:01 The meeting name has been set to 'rally' 17:07:13 rvasilets Hi :) 17:07:20 Hi 17:07:30 sup 17:07:39 amaretskiy1, redixin, hi! 17:07:44 hi 17:07:46 andreykurilin, hi! 17:07:48 o/ 17:08:23 olkonami hi :) 17:08:26 hi 17:08:49 Okay let's start! 17:08:54 #topic NetworkContext 17:09:09 amaretskiy What was your progress with this during the past week? 17:09:19 amaretskiy1, Is it ready to use? 17:09:39 https://review.openstack.org/#/c/103306/ is usable 17:09:48 but tests are incomplete 17:10:04 so I'm going to submit final patch set in next 2 hours 17:10:16 and the patch will be ready to final review 17:10:17 great! 17:10:24 I believe we will merge it soon 17:11:23 eom 17:11:24 amaretskiy1 Great! Have you done some testing on it? Because there are guys out there interested in this, so it is important to be sure it works when we are giving them a link to this patchset 17:11:49 I have local fuel on VirtualBox 17:11:52 amaretskiy1, what about add more scenarios to gate-rally-dsvm-neutron-rally job? This can help be sure that network context works correctly 17:11:56 with two small clusters 17:12:07 it would be nice to get feedback from that guys if something is broken 17:12:16 one with nova-network and another is with neutron 17:12:27 so everything is working for me 17:12:30 eom 17:13:02 amaretskiy1, Okay, thanks! 17:13:02 there is a scenario added within the patch 17:13:07 single scenario 17:13:11 eom 17:13:29 amaretskiy1:Well tell a bit about this scenario, please :) 17:13:50 https://review.openstack.org/#/c/103306/87/rally-jobs/rally.yaml,cm 17:14:08 take a look at context: network: { ... } 17:14:24 here we have a scenario with network context enabled 17:14:43 amaretskiy1:I see 17:14:55 andreykurilin, Do you think this is enough? 17:15:19 msdubov_: hm 17:15:51 msdubov_: this is new scenario for gate-rally-dsvm-rally, but i want new scenarios for gate-rally-dsvm-neutron-rally :) 17:16:16 amaretskiy1 ^ 17:16:33 yep, I will add one 17:16:59 Okay 17:17:32 #topic gate-rally-dsvm-verify gate job 17:17:44 So we have a new job for everything tempest-related 17:17:55 \o/ 17:18:02 like launching verification tests via Tempest 17:18:14 or functional testing for the verification part 17:18:33 So andreykurilin, could you please tell us a bit about the progress here? 17:18:43 msdubov_: sure 17:19:06 the patch, whcih configure this job, is ready for review 17:19:13 #link https://review.openstack.org/#/c/139262/ 17:19:41 the results page of this job looks like other rally jobs 17:19:45 #link http://logs.openstack.org/62/139262/30/check/gate-rally-dsvm-verify/d21e464/ 17:20:33 this job runs two verifications, display the results in different formats and finally compare them 17:21:17 recently, "compute" set is used for testng verification, but in future I wants to add random selecter for set name 17:21:40 this will have to increse coverage 17:22:10 andreykurilin, Nice! One more thing to clafiry: are you going to add some functional tests like those you removed from tests/functional/test_cli_verify.py, or will you leave it as is? 17:22:19 also, "rally task " command will be added for tempest tasks 17:23:02 msdubov_: I don't want to add any functional tests in tests/functional/test_cli_verify.py 17:23:16 msdubov_: I prefer to delete this module in near future 17:23:30 andreykurili Okay, and what about that "rally task" command? What will it look like? 17:24:05 why not just remove test_cli_verfy.py just now? 17:24:26 msdubov_: "rally task" contains scenarios related to tempest. https://github.com/stackforge/rally/tree/master/doc/samples/tasks/scenarios/tempest 17:24:43 msdubov_: I want to test it in this job too 17:25:03 msdubov_: for example, launch singel tempest test https://github.com/stackforge/rally/blob/master/doc/samples/tasks/scenarios/tempest/single_test.json 17:26:11 redixin: I leave this module, because I want to share imformation about new job 17:27:11 redixin: when all become accustomed to the new job, this module should be removed 17:27:15 eom 17:27:19 andreykurilin, Okay, let it be so 17:27:35 #topic General Rally code improvement 17:28:11 Okay this is quite a broad topic. Since we are moving towards our first release, we are doing our best to make the Rally code more neat 17:28:40 E.g. there is a patch that covers all benchmark scenarios with docstrings and unifies their style: https://review.openstack.org/127192 17:28:55 This is very important for the "rally info" command as well to be more informative 17:29:13 And rvasilets is currently working on 100% functional test coverage for CLI 17:29:22 rvasilets, Seems like you are almost done with that? 17:30:09 Yes 17:30:27 During the last week I have wrote the code for the 100% coverage functional tests. There are a couple of issues with this patch but I hope they would be solved soon. You can look at patch here: https://review.openstack.org/#/c/138134/ and I need you help with this 17:31:05 andreykurilin, amaretskiy1, redixin Any new ideas why those tests fail in such a weird way? :) 17:31:28 I have only one suspicion that fails all tests in patch where is in string like 17:31:38 looking 17:31:48 self.rally = utils.Rally() in setUp 17:31:54 function 17:32:50 It's only onlу dependence that I found 17:32:57 rvasilets, Well, as far as I know, setUp() is launched prior to EACH test method 17:33:04 But I have no clue why it so 17:33:12 rvasilets, So this shouldn't be the problem... 17:34:10 IF we look at test that pass jenkins, for example test_cli_task 17:34:23 there is now such line in this module 17:35:06 something wrong with config file. seems like file /tmp/.rd.json is rewritten somehow during tests 17:36:23 anyway it is not best time to dive into code and logs =) 17:36:26 Mow? I found that my theory wrong? never mind. It's wrong for test_cli_show 17:36:58 redixin, Agree, let's move further 17:36:59 redixin: maybe we should use different names for separate tests? 17:37:17 Also I have made bug ticket on launchpad and fix it https://bugs.launchpad.net/rally/+bug/1399675 . It was about incorrect output of rally info find for SLA. Thx, EOM. 17:37:19 Launchpad bug 1399675 in rally "Incorrect work rally info find for SLA" [Low,Fix released] 17:37:24 andreykurilin, maybe 17:38:06 So one more thing to make our code more neat is to use the decorator syntax to mark deprecated things. 17:38:22 olkonami, That's what you are currently working on, how are the things going? 17:38:59 deprecated decorator takes more time than I planned :( 17:39:44 the first idea was to check scenarios, runners, contexts and slas for deprecation in it's base class, but than it becomes clear that this way we should have multi checks for one type if it presents in config several times 17:40:09 so now I collect all checks in one method into the benchmark engine class 17:40:34 I think it's better to check all benchmark components for deprecation in one place 17:40:48 but this method looks ugly :( 17:40:52 https://review.openstack.org/#/c/138489/4/rally/benchmark/engine.py 17:41:11 mayby someone will have any ideas how to improve it 17:41:19 olkonami, We'll review it, thanks! 17:41:22 and I am still in process of writing unit tests for it 17:42:51 eom 17:44:40 olkonami, thanks 17:44:46 #topic Rally roadmap 17:45:18 boris-42 was very kind to prepare a roadmap doc during his vacation https://docs.google.com/a/mirantis.com/spreadsheets/d/16DXpfbqvlzMFaqaXAcJsBzzpowb_XpymaK2aFY2gA2g/edit#gid=0 17:45:37 this will hopefully make our work more transparent for the community 17:47:35 It can be seen from this doc for example that a couple important tasks like multi-scenario load support are currently blocked by the ongoing refactoring 17:47:39 let's keep that in mind 17:47:51 #topic Free discussion 17:48:16 amaretskiy1, andreykurilin, redixin, olkonami, rvasilets Any other things we haven't covered yet? 17:48:30 none from me 17:48:35 hm... 17:48:35 no 17:48:43 -_- 17:48:45 no 17:48:56 I have no ideas about what we missed:) 17:49:20 Okay, thanks for participation then! 17:49:26 #endmeeting