14:00:08 #startmeeting Rally 14:00:09 Meeting started Mon Aug 15 14:00:08 2016 UTC and is due to finish in 60 minutes. The chair is andreykurilin. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:10 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:14 The meeting name has been set to 'rally' 14:00:21 o/ 14:00:34 o/ 14:00:39 o/ 14:01:58 ok, I had at least 3 topics to discuss 14:02:35 #topic optional-requirements. Do we need them at all? 14:02:46 We created this file long time ago 14:03:20 when we had no ability to add custom requirements(those one which are missed from global-requirements) to requirements.txt 14:03:38 Now, we have ability to add whatever we want to requirements.txt 14:03:49 As I remember there should be packages that are not in global requirements 14:04:09 I think that is reasonable to catch ImportError on specific clients initialization and print help message for installing pythonclient- 14:05:03 rvasilets_: yes, but we decided to do it since we had no ability to add requriements to requirements.txt 14:05:11 and now we had ability to do it 14:05:21 amaretskiy: yes, we had such validator 14:05:28 but why we need it now? 14:05:40 we can move all requirements to our main requirements.txt 14:05:50 I don't know why we have it for now 14:06:07 and why we can't to have only requirements file 14:06:26 so I propose to do 2 things 14:07:11 First of all, decide what kind of packages can be added to our requirements 14:07:38 for example package can be added to requriements.txt if it supports py2 and py3 14:08:07 and such requirements for requirements should be listed somewhere. the best place - header of requriements.txt 14:08:18 Ideas? 14:09:09 :( 14:09:19 #agree with note in requirements header 14:09:29 Okey our constrains could be printed in any way 14:09:29 amaretskiy: thanks :) 14:09:35 for example like header 14:10:15 after we agree with "requirements for requirements", we can move to reviewing optional-requirements and move them to requirements.txt 14:10:48 I will propose a patch soon 14:10:59 what constrain do we have? - I don't simple answer. Why we can't add there all packages and as a comment write what python it maintain? 14:11:21 it's not clear for me what we will do with some "exotic" optional requirements 14:11:59 rvasilets_: if some specific package doesn't support for example python 3, rally will failed on py3 env(while installing rally requirements) 14:12:03 like senlin client, etc... will we make this package mandatory by moving to requiremensts.txt ? 14:12:37 amaretskiy: I don't think that they are exotic. if we have plugins for them in our tree - they are normal requirements 14:12:45 This question need some research) 14:12:51 What is the best practice here 14:13:30 just idea to discuss: how about runtime validation of required package for plugin 14:13:57 plugins can run validation if some package (+version) is available 14:14:14 rvasilets_: so pip support python classifiers in requirements.txt . so it is possible to mark which packages should be installed for specific python version, but plugin discovery will fail if importerror occure 14:14:26 amaretskiy: so it is good idea 14:14:37 the implementation is simple 14:16:09 amaretskiy: from another point of view, if I install some package(in our case it is rally), I do not want to install it's requirements manually, I expect that `pip install somepackage` installs everything required 14:17:44 I don't want to decide during the meeting such important thing because I don't want to create my own bycicle. I'd prefer to think more or to google best practicies 14:19:11 rvasilets_: ok, I give you a week and we will return to this topic at the next meeting:) 14:19:32 let's move to the next topic? 14:19:34 okay, lets put runtime idea on a shelf 14:20:03 but not to trashcan :) 14:20:27 heh 14:21:03 andreykurilin, thx) 14:21:05 move on? 14:21:08 yes 14:21:21 #topic Move platform requirements from install_rally.sh to separate file 14:21:30 ok, I have one more topic related to requirements 14:22:13 You mean platform relateed requirements 14:22:14 Currently our install_rally.sh script has several lists of packages need for rally 14:22:23 like Solaris 14:22:25 etc 14:22:26 example https://github.com/openstack/rally/blob/master/install_rally.sh#L282 14:23:11 why is install_rally.sh a bad place? 14:23:23 amaretskiy: because there is better way 14:23:36 OpenStack-infra team provided new tool for package requriements 14:23:45 and it is possible to list all requirements in separate file 14:23:47 andreykurilin, Yes this would be nice to add platform related files if we want to support different platforms 14:23:57 like https://github.com/openstack/python-novaclient/blob/fe1fd3321bce29f283e010db90423ddcd7d4f7fd/other-requirements.txt 14:24:07 this more OOP=) style 14:24:46 I'm definetely for approach used in novaclient 14:25:04 rvasilets_: it is used in most of openstack projects now:) 14:25:16 okey) 14:25:43 amaretskiy: your ideas ? 14:25:56 #agree to move platform requirements to other-requirements.txt !!! 14:26:01 nice:) 14:26:06 i like this 14:26:10 :) 14:26:27 so we can move to next topic 14:26:42 #topic [andreykurilin] get rid of jsonschema where it is possible 14:26:50 - it is easy to write wrong jsonschema which will work for wrong json (we had broken schema for ExistingCloud for long time) 14:26:54 #superagree :) 14:26:56 - there is no way to display jsonschema in user-friendly way 14:27:01 - there is no good way to leave comments/descriptions in jsonschema 14:27:30 I decided to start this topic after discussion about parameters of our contexts 14:27:37 however sometimes we have to do complicated validation 14:28:05 i think we could write own simple validator, not so powerful as jsonschema, but faster 14:28:10 +1 14:28:29 also, there are a lot of places when we have simple validation of parameters 14:28:30 andreykurilin, I'm working on it under validator topic 14:29:11 I have one example where jsonschema validator is redundant 14:29:17 contexts 14:29:43 what do you suggest to use instead? 14:29:48 jsonschema is used there for validation of input arguments 14:30:06 rvasilets_: we can validate input arguments as it done for scenarios 14:30:38 not strictly for input arguments but also for checking the format of proper context dection 14:30:44 section* 14:31:13 rvasilets_: so I did not say that we should not use jsonchema at all :) 14:31:41 I said that we should not use it whatever it possible like it done in current code 14:31:43 this will be possible after validator plugin base will be introduced 14:33:00 rvasilers_: I doesn't depend on validation plugin base. we can start refactoring some places even now. we don't have complex jsonschemas for contexts 14:33:36 rvasilets_: but I agree that new validators will allow to build better validation 14:33:47 *better validation for contexts too 14:33:56 We can't add new validators for context because it make more work for me then 14:34:27 rvasilets_: we do not need to add new validators for context:) 14:34:43 we can use things which are already in our code 14:34:47 ...interesting))) 14:35:06 https://github.com/openstack/rally/blob/master/rally/task/scenario.py#L138-L166 14:35:09 this one 14:35:39 this code checks that all scenario arguments specified in correct way 14:35:58 it equals to what we do with jsonchema in contexts 14:36:12 I guessalmost everything that have substring validation or validator in our code should be moved under plugin base 14:36:53 substring in the name* 14:37:00 rvasilets_: yes. but we can reuse now what we already have) 14:37:19 until we started refactoring such peaces 14:37:21 this will add extra work 14:37:28 Why we can't add it later 14:38:25 also this is work item that was told by Boris to me. To avoid usage of jsonschema 14:38:53 rvasilets_: for example because we need to list contexts parameters at http://rally.readthedocs.io/en/latest/plugin/plugin_reference.html . Users are need to read the code to know what parameters should be specified for specific context 14:39:01 it is bad user-experience 14:39:09 Also this is logical to refactor validation after big validator change would be merged 14:40:00 rvasilets_: based on my previous experience, the process of merge some big refactoring is too long and can take years... 14:40:01 Such bad-experience we could avoid by simply add schema field to the docs 14:40:25 rvasilets_: it is impossible to add comments to schema 14:40:39 and schema can not be displayed in user-friednly way 14:41:00 rvasilets_: btw, I had such patch https://review.openstack.org/#/c/327592/ 14:41:15 why we need to add comment to schema? 14:41:30 rvasilets_: because we need to discribe what parameter foo means 14:42:03 for example - https://github.com/openstack/rally/blob/master/rally/plugins/openstack/context/manila/manila_share_networks.py#L40-L68 14:43:47 rvasilets_: ok, could you give me any ETA for finishing(patches on review) ability to use validators for contexts ? 14:43:51 Okey we could add additional fields into schema or to wait validators. Also I guess that error msg after wrong format of context schema should help user to understand what format they can write 14:45:05 rvasilets_: additional fields will make schema more huge and unreadable 14:45:18 for developers 14:45:58 no, developers know jsonchema and can read it. jsonchema is not end-user thing :) 14:46:23 rvasilets_: so what about eta for validator refactoring? 14:47:31 ETA - this is one of the most important feature that is in progress for now. And I can tell that ready for review patches will be in the nearest month then it depends from reviewers how quick it would be 14:48:39 rvasilets_: ok, lets wait one month. 14:49:28 anything else to discuss? 14:49:50 Nothing from me 14:50:36 amaretskiy? 14:51:26 nothing from me but REVIEW THE RAAS, finally!!! 14:51:56 scary) 14:51:59 heh 14:52:00 ok 14:52:02 let's finish 14:52:39 agree 14:52:47 #endmeeting