17:27:50 #startmeeting Rally 17:27:51 Meeting started Tue Jan 14 17:27:50 2014 UTC and is due to finish in 60 minutes. The chair is boris-42. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:27:52 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:27:54 The meeting name has been set to 'rally' 17:28:23 hughsaunders hi 17:28:26 hi 17:28:34 sorry I late a bit=) 17:29:14 os let's start 17:29:21 seems like nobody is here=) lol 17:30:07 okay I have a couple things that I would like to disucss 17:30:30 #topic Rally & Benchamrk engine better result collecting 17:30:58 So the main idea that now we are collection only times of whole scenario loop 17:31:35 but it could be improved to collect results of all atomic actions 17:31:45 that would be good 17:31:59 so we already have some kind of base classes 17:32:10 https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/utils.py 17:32:17 so with such atomic actions 17:32:23 we should make some decorator 17:32:35 that will be add on them and that will contain name of "parameter" 17:33:29 then we will store all data in class object (like now https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/servers.py#L38 this work) 17:33:59 and then get it https://github.com/stackforge/rally/blob/master/rally/benchmark/runner.py#L55 17:34:05 here from class instance 17:34:21 and push in key like full_times 17:34:37 hughsaunders thoughts ^ 17:35:46 boris-42: I think more granular results would be useful 17:36:03 like I was attempting with scenario_specific_results 17:36:14 yep but that is a bit different 17:36:23 and could be implemented together=) 17:36:35 I will try to find time to retest your patch soon 17:36:43 sounds like you have a plan that could work across all scenarios? 17:36:52 hughsaunders yep 17:37:22 hughsaunders just add decorators to all "atomic" actions (like _boot_server) 17:37:37 that will store in class object 17:37:41 all timings 17:37:51 and then after execution of method get them 17:37:59 that makes sense 17:38:24 so this will be feature of scenario runner not every benchmark 17:39:06 okay so it seems like good idea 17:39:09 boris-42: may be able to auto-decorate with a metaclass? 17:39:22 not sure 17:39:31 I'm not sure either, would have to do some reading 17:39:48 I mean I would like to decorate only "atomic" actions 17:40:31 if we continue storing all such methods in couple classes 17:40:41 like now https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/keystone/utils.py#L35 17:40:47 then it is possible 17:41:28 ok, probably simpler to use a decorator as you suggested 17:41:44 yep to specify some beauty names 17:41:55 like here https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/keystone/utils.py#L41 17:41:59 anyone else here BTW? 17:42:08 miarmak should be=) 17:42:11 + 17:42:14 hi 2 all) 17:42:36 hi miarmak :) 17:43:27 so .. I think that we could continue discussing this 17:43:32 in other place 17:43:43 like ether pad but the major idea is ok 17:43:49 #topic Rally & Tempest 17:44:02 miarmak could you share your results ? 17:45:01 boris-42: : over what period? 17:45:12 miarmak about Rally & Tempest 17:45:29 * IgorYozhikov is now away: went away... 17:46:26 tomorrow I'll start investigate this issue. Today and yesterday i tryed to finish all my activities in other issues 17:46:49 so you didn't start work around this part? 17:46:55 Okay I will just share for others 17:47:07 We are going to add some kind of cloud verification 17:47:18 I was wondering if tempest has/needs a plugin system, so stackforge projects can use that to provide their own tests 17:47:37 hughsaunders hmm yep I think it need 17:47:39 Yesterday I installed rally, but had some problems with it usage (Hugh alredy have reported this bug) 17:47:50 miarmak and fixed=) 17:48:07 oh, it's great) hughsaunders: Thanks! 17:48:25 so the tempest & rally integration means next one 17:48:39 we have at this moment part that creates "deployments" 17:48:45 part that runs "benchmarks" 17:49:07 and there is no way before running benchmark to ensure that whole cloud works properly 17:49:15 so we would like to add 1 more command 17:49:27 something like openstack-rally verify --deploy-id 17:49:46 that will make proper tempest config 17:49:51 and run tempest against cloud 17:50:16 miarmak hughsaunders I think it will be nice functionality ^ 17:50:34 ahh, so we don't actually need rally specific tests, we run standard tempest tests before benchmarking 17:50:46 not only before 17:50:49 when you would like 17:51:13 yeah 17:51:14 just "button" to run tempest against coud 17:51:17 cloud* 17:52:34 so I think it's ok? 17:52:36 hughsaunders ^ 17:52:39 miarmak ^ 17:52:42 I think that would be useful 17:52:59 I think so 17:53:07 ok nice 17:53:11 last topic for today 17:53:20 #topic Rally & Benchmarks Inside VMs 17:54:12 so the ideas is to make next steps 17:54:51 boris-42: next step: get my review to patchset 20 ;-) 17:55:02 lol=) 17:55:23 using heat install inside VMs different benchmark suits 17:55:26 then run them 17:55:29 collect info 17:55:34 present to user 17:55:48 so we can use current benchmark engine 17:56:06 and I don't know probably N different benchmark scenario for N different suits 17:56:42 each of benchmark scenario will have some (probably hardcoded heat template) that makes required installation 17:57:06 so we will be able to run a lot of simultaneously benchmarks in cloud 17:57:30 and check that there is no difference if we run with 1 active user and with N active user 17:57:47 hughsaunders miarmak thoughts ^ 17:58:07 that would be good, I prob won't have time to work on it soon though :( 17:58:24 also have to go at in two mins 17:58:30 =) 17:58:40 why hardcoded? We can make several heat templates and use necessary щту 17:58:46 one* 17:59:06 because every heat template is related with some benchmark suit 17:59:16 different benchmark suit will be in different scnearios 17:59:32 because processing of output data will be done in different ways 17:59:40 but this is all under discussion 17:59:46 so i will make some ether pad 17:59:50 oh, ok 18:00:01 about this and probably send to mailing list 18:00:08 Okay we have to end meeting 18:00:11 see you later 18:00:17 #endmeeting