17:06:23 #startmeeting Rally 17:06:24 Meeting started Tue Dec 10 17:06:23 2013 UTC and is due to finish in 60 minutes. The chair is boris-42. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:06:25 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:06:27 The meeting name has been set to 'rally' 17:07:00 msdubov harlowja jaypipes hi 17:07:12 boris-42 hi 17:07:22 jd__ hi 17:09:00 Alexei_987 hi 17:09:09 Hi :) 17:09:15 jog0 ping 17:09:43 Ok let's start 17:10:13 #topic Updates in benchmark engine 17:10:22 msdubov can you share with us 17:10:30 msdubov your recent efforts 17:11:15 boris-42 The main update is no doubt the patch implementing the generic cloud cleanup after launching benchmarks implemented by boden 17:11:40 It has been merged today and required me to put a lot of efforts on rebasing my patches 17:12:25 As for my patches, they are implementing the init() procedure, which is a kind of the opposite thing to cleanup. In init(), one can initialize some resources required for a benchmark scenario, e.g. servers 17:12:57 msdubov I saw some concerns from Alexei_987 17:13:01 I also exemplify it with the patch that implements the benchmark scenario for setting and deleting metadata to a predefined (in init()) server 17:13:21 boris-42 Yep they are addressing the way the benchmark engine works currently 17:13:30 boris-42: msdubov yes 17:13:37 Alexei_987 do you have some ideas how to improve it? 17:13:47 The restrictions posed by multiprocessing tool used in the benchmark engine lead to a somewhat ugly way of implementing the engine. 17:13:51 *pool 17:13:51 boris-42: msdubov our benchmarking system is a lot alike unittest framework 17:14:06 Alexei_987 not sure 17:14:07 and I think we should reuse their code and experience 17:14:12 why not? 17:14:24 they have setUp/tearDown + run 17:14:41 why not using threads? 17:14:42 Alexei_987 yep but they don't have billions of temp users 17:14:50 otherwise we'll have to reinvent the wheel all over again 17:14:51 Alexei_987 projects 17:14:58 Alexei_987 resources that should be cleaned 17:15:10 why do you think they don't? 17:15:11 Alexei_987 and unit test are run only 1 time 17:15:23 unittest is used in many various scenarios 17:15:43 Alexei_987 we already use pytest as a base of our benchmark system 17:15:51 Alexei_987 it was made by Eugeniy 17:15:51 anyway I'm not talking about using unittest 17:15:53 eyerediskin I suppose threads will pose the same restrictions on the things we can pass to the called methods? The primary problem actually is we can't pass openstack clients to benchmark scenarios 17:16:15 I'm talking about that current design that we have has many issues 17:16:23 Alexei_987 like? 17:16:25 related to code organization 17:16:30 object creation and so on 17:16:52 Alexei_987 and how are you going to avoid object creation using pytest unit test or something other? 17:16:57 I've mentioned one of them to msdubov 17:17:11 for example classmethod can only be called under certain conditions 17:17:15 Alexei_987 I mean let's start from task 17:17:30 Alexei_987 we have 1 method that is benchmark scenario 17:17:41 Alexei_987 Yep the possible bug you mentioned indeed can take place, but in fact the benchmark engine manages classes in such a way that this cannot happen 17:17:49 ok.. I mean that unittest and other big frameworks have certain way and procedure of when and how to create objects 17:18:05 Alexei_987 could you explain 17:18:12 Alexei_987 on current task 17:18:23 for example our problem is that we have to pass objects to other process 17:18:30 which is bad 17:18:40 Alexei_987 yep but how are you going to fix it? 17:18:53 they should be created where they are actually used 17:19:01 Alexei_987 no 17:19:01 create object in process instead of passing it? 17:19:02 without passing them all over the system 17:19:08 yes 17:19:08 Alexei_987 no 17:19:12 why not? 17:19:13 Alexei_987 because the case is another 17:19:23 just like setUp method in tests 17:19:26 Alexei_987 But we may launch a benchmark scenario millions of times, and we want to create OpenStack clients only once 17:19:39 why? 17:19:48 Alexei_987 because we are testing not abstract shit=) 17:19:52 objects are created once thread/process is started 17:19:52 creation of a python object takes almost 0 ms 17:20:04 Alexei_987 creation of billions users take a lot of time 17:20:09 agreed with Alexei_987 17:20:16 users yes but not clients 17:20:25 you can pass user credential 17:20:28 plain dict 17:20:35 and create a complex object later 17:20:36 Alexei_987 and Floting IPs FixedIps 17:20:39 Alexei_987 images 17:20:45 Alexei_987 other stuff that you need in init 17:20:54 Alexei_987 also if you put this to thread 17:21:07 Alexei_987 then you will need to create them in each method 17:21:34 this stuff created once per process/thread. created only objects needed by process/thread 17:21:36 what do you mean by each method? 17:21:37 Alexei_987 eyerediskin https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/servers.py#L29-L37 17:22:00 Alexei_987 eyerediskin this method get's random clients in cls 17:22:21 clients? 17:22:26 I don't see any clients in there 17:22:26 Alexei_987 yep clients 17:22:31 Alexei_987 (chosen randomly of many client users) 17:22:44 Alexei_987 they are used in utils 17:22:50 so it chooses a random user credential 17:22:57 please be precise 17:23:00 Alexei_987 https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/utils.py#L28-L49 17:23:04 client is not tied to user 17:23:16 the only things that connects it to the user is auth token 17:24:04 again let's separate clients and users 17:24:16 user -> auth_token is just a string 17:24:16 Alexei_987 again I don't care about users 17:24:20 Alexei_987 in becnhmarks 17:24:22 why not? 17:24:26 Alexei_987 I care about clients 17:24:32 clients? 17:24:47 then I must have missed something 17:24:48 Alexei_987 glacne, nova, keystone, …. clients for this user 17:24:53 so... 17:24:58 there is a word user 17:25:13 Alexei_987l so I would like to be able cls.clients("nova").servers.create(server_name, image_id, 17:25:14 flavor_id, **kwargs) 17:25:20 cls.clients("nova") 17:25:27 use from benchmark dirrectly 17:25:30 yes but user is hidden inside 17:25:37 Alexei_987 yep 17:25:50 so we can change code in such a way that we only pass users 17:25:53 and not clients 17:26:00 Alexei_987 why 17:26:11 Alexei_987 to create in each method client every time by hand? 17:26:17 Alexei_987 it is called "copy paste" 17:26:22 (facepalm) 17:26:33 client manager can handle this 17:26:34 Alexei_987 in each benchmark I need clients 17:26:46 and reuse 1 client object for 1000 benchmarks 17:26:55 just bind it to certain user before usage 17:27:20 the main idea is that we have to define data flow 17:27:30 and data dependencies 17:27:42 Alexei_987 hm I don't understand 17:27:42 so we won't have to apply dirty hacks to make it work 17:28:16 Alexei_987 we need to be able to pass the same data in every thread 17:28:25 Alexei_987 or even more different data 17:28:29 why we should do it? 17:28:32 Alexei_987 in every call of method 17:28:39 we should only pass data that is used in that thread 17:28:47 Alexei_987 ? 17:28:54 Alexei_987 and it could be all data from INIT 17:29:05 Alexei_987 that is called only 1 time 17:29:15 Alexei_987 like create 20k fixed IPs 17:29:35 and 1 tests uses all 20k fixed ips? 17:29:49 Alexei_987 I don't know what will test use 17:30:17 ok.. such discussion is not productive at all 17:30:23 need additional digging in here 17:30:29 Alexei_987 lol 17:30:39 Alexei_987 we should talk about user case 17:30:51 Alexei_987 my user case was next 17:30:57 Alexei_987 use case* 17:31:37 Alexei_987 then it will be more clean 17:32:03 msdubov could you write etherpad with full description about our use cases and how they are implemented 17:32:16 msdubov Alexei_987 ^ so we will be able to continue this discussion next week 17:32:37 okay let's close this discussion a got to another topic 17:32:41 ok :) 17:32:48 boris-42 ok 17:32:55 #topic Profiling system 17:33:15 Alexei_987 could you say for jd__ and jog0 current status of your work 17:33:16 cool :) 17:33:29 I have 1 patch in ceilometer on review 17:33:40 that adds data collector functionality 17:33:57 + I have 1 patch in process of polishing in nova 17:34:13 that adds library that will collect and send data 17:34:21 that's the current status 17:34:34 I also have some problems that we can discuss 17:34:50 Alexei_987 could you share with links to the patches? 17:35:06 https://review.openstack.org/#/c/60262/ 17:35:13 https://review.openstack.org/#/c/60797/ 17:35:17 Alexei_987 and what problem do you have? 17:35:38 so the problem is: how and where do we store a profiler object on single service 17:36:14 Alexei_987 could you explain? 17:36:16 this is kind of cross functionality that should be accessible from anywhere 17:36:29 but we should not bind it to singleton of any kind 17:36:48 in openstack I didn't find a proper way of doing such a thing 17:37:03 it's somewhat common with logging system 17:37:16 but logger is created for each file separately 17:37:24 Alexei_987 but why we are not able just to put Profiler class to oslo-incubator 17:37:31 Alexei_987 and copy paste it to all projects? 17:37:42 nah.. the code will lie in oslo 17:37:55 Alexei_987 and what is the problem? 17:37:59 I mean when we create a profiler object: Profiler(service='nova') 17:38:05 where do we put it? 17:38:29 You should call it Profiler(service="oslo") and it will be replaced during sync to ("proect_name") 17:38:32 Alexei_987 ^ 17:38:47 Alexei_987 and this could be done in oslo code actually 17:38:58 hm the name? 17:39:02 or the object? 17:39:10 name ok.. 17:39:12 Alexei_987 all can be stored there 17:39:15 ha ha 17:39:24 you should ask Roman Po.. 17:39:34 how much pain they have now cause of global config object 17:39:53 Alexei_987 first of all they have another problmes 17:40:00 yes... 17:40:03 Alexei_987 they would like to move the code in separated lib 17:40:09 but library should not create any active objects 17:40:10 Alexei_987 that use another libs 17:40:14 that's why it's libraryt 17:40:27 library should only provide functionality 17:40:45 Alexei_987 but actually we are going to patch with our lib another libs 17:41:00 Alexei_987 like oslo.log, oslo.messegin? 17:41:09 oslo.log - no 17:41:14 messaging - maybe 17:41:24 but mostly it will be used on higher level code 17:41:30 such as controllers, etc. 17:42:12 Alexei_987 so I don't see any problem to just take Profiler() code put it into oslo 17:42:19 and I do :) 17:42:20 Alexei_987 then sync a create locally in every code 17:42:28 Alexei_987 project* 17:42:34 Alexei_987 and then use it for tracing 17:42:46 code yes 17:42:48 objects no 17:42:50 Alexei_987 and have 2 new config options (in oslo or projects) 17:42:59 Alexei_987 you don't need to store any objects 17:43:30 Alexei_987 what is the problem? 17:43:40 Alexei_987 I didn't have any problems with using DB code in all projects 17:43:52 Alexei_987 and DB code is much more complex that this 50lines 17:43:59 Alexei_987 and have much more dependencies 17:44:03 there are some problems related to ENGINE being a singleton 17:44:13 Alexei_987 Engine? 17:44:22 yes.. connection object 17:44:35 Alexei_987 you are speaking now about DB? 17:44:43 (facepalm) 17:44:46 1 sec 17:44:51 Alexei_987 omg 17:45:24 https://github.com/openstack/oslo-incubator/blob/master/openstack/common/db/sqlalchemy/session.py#L356 17:45:27 ENGINE 17:45:31 global object 17:45:41 cause of it we cannot implement slave reads properly 17:46:17 anyway... 17:46:33 so there is a problem in this object placement.. 17:46:44 and I suggest to move on for now 17:46:50 Alexei_987 okay 17:46:52 so we don't waste all the time 17:47:09 #topic Deploy engines 17:47:21 eyerediskin hey can you share with your updates 17:47:36 Fuel engine is coming soon. One or wo days 17:47:45 eyerediskin oh nice 17:48:23 eyerediskin I saw some changes are checking configs 17:48:31 eyerediskin around checking configs* 17:48:41 yep, this is almost done 17:49:06 eyerediskin what is left? 17:49:40 config validation in providers. Maxim is working on it 17:50:05 https://review.openstack.org/#/c/60275/ 17:50:09 https://review.openstack.org/#/c/60030/ 17:50:22 some patches waiting for +2 =) 17:51:01 eyerediskin https://review.openstack.org/#/c/60030/ <- +2ed 17:51:22 eyerediskin and how about LXCEngine? 17:51:49 eyerediskin are you working around it, to make it work inside OS cloud? 17:52:02 I gonna continue with it after fuel engine is done 17:52:14 eyerediskin okay nice 17:52:41 okay let's move to the next topic 17:52:42 boris-42: I'got solution in my mind. Just need to code it 17:52:51 eyerediskin nice 17:53:01 #topic Rally as a Service 17:53:32 We would like to add Web UI, and make it possible to work together with one Rally instances 17:53:44 So the right way is to make from Rally service 17:53:53 and make something like in nova and other projects 17:54:01 we will build REST with pecan 17:54:30 And make API + Managers (or master + workers) using oslo.messeging 17:54:58 I don't think that these decisions requires discussion 17:55:10 we'll use the same async approach with eventlet and hookers? 17:55:11 But there is a discussion how to make it better 17:55:19 Alexei_987 ? 17:55:38 nonblocking code and all the stuff like in nova 17:55:50 in our service 17:56:10 Alexei_987 There will be API that just make plain operation (get data from DB) or stat task and make record in DB 17:56:25 And call managers that will actually run all other stuff 17:56:30 ok.. so no async code and no eventlet :) 17:56:31 cool 17:56:32 Alexei_987 so we will use rpc.cast calls 17:56:46 Alexei_987 for long operations 17:57:07 Alexei_987 so yep no eventlet shit 17:57:10 ok I got it 17:57:23 And we actually started discussion 17:57:31 About how to make this possible inside Rally 17:57:36 and what changes we should make 17:57:55 https://etherpad.openstack.org/p/rally-service-arch 17:57:58 here is it ^ 17:58:18 it is just draft not ready so if everyone is interested it can help with advices 17:58:29 So okay 17:58:30 that is all 17:58:35 #topic free discussions 17:58:56 So if somebody have any questions or something else just ask 17:59:30 there is blueprint in devstack 17:59:58 https://blueprints.launchpad.net/devstack/+spec/devstack-rally 18:00:08 meybe someone wants to implement it 18:00:26 eyerediskin could you specify drafter, assignee and so on? 18:00:32 eyerediskin and series 18:00:42 eyerediskin I think that it will be nice to implement it 18:01:04 eyerediskin especially if we are going to make from Rally service 18:01:49 okay let's finish this meeting 18:01:53 #endmeeting