17:04:27 #startmeeting qa 17:04:28 Meeting started Thu May 2 17:04:27 2013 UTC. The chair is jaypipes. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:04:30 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:04:32 The meeting name has been set to 'qa' 17:04:33 davidkranz: Error: Can't start another meeting, one is in progress. 17:04:44 jinx 17:05:02 davidkranz: you want to lead? go for it :) 17:05:18 Agenda at https://wiki.openstack.org/wiki/Meetings/QATeamMeeting 17:05:46 Before we get to that, I noticed in my recent stress submission that pep8 failed. 17:06:10 There are a bunch of stacktraces in pep8 runs for tempest even when jenkins says it is ok. ANy one know about that? 17:06:19 no, not I 17:06:38 http://logs.openstack.org/27950/1/check/gate-tempest-pep8/3105/console.html.gz 17:07:13 Also, what do you run in a tempest dir to get all the stuff that jenkins is diong for the pep8 test? 17:08:07 I did './run_tests.sh -N -p' but that passes and jenkins is still failing pep8. 17:08:29 davidkranz: looks like it failed, not succeeded... are you sure gerrit reported it a success? 17:08:49 jaypipes: Gerrit reported failure but I got success when I ran it before submitting. 17:08:52 davidkranz: does -p in run_tests.sh actually look in the stress/ directory though? 17:09:19 jaypipes: Ah, perhaps that is the problem. Maybe I have to add it to some config. 17:10:06 Let's move on. 17:10:15 Any issues with reviews? 17:10:32 davidkranz: I see this: 17:10:32 [flake8] 17:10:32 ignore = E125,H302,H404 17:10:32 show-source = True 17:10:32 exclude = .git,.venv,.tox,dist,doc,openstack,*egg 17:10:42 in tox.ini 17:11:03 davidkranz: so, AFAICT, tox -> flake8 should be picking up the stress/ directory... :( 17:11:10 jaypipes: Then I think "new_stress" would have been included. I will have to investigate more. 17:11:22 ya 17:11:45 Does any one have any comments about stress tests? 17:12:26 davidkranz: Is it configurable parameters 17:12:40 to run stress test? 17:12:57 like time , # of VMs etc 17:13:26 ravikumar_hp: It could be. I haven't gone there yet. Just the basic idea was for reactions to the approach. 17:13:54 ravikumar_hp: Right now you provide those options in each test script. 17:14:05 ravikumar_hp: There could/should be a more general configuration mechanism. 17:14:42 #topic Blueprints 17:15:04 I will create a blueprint for stress tests. 17:15:40 Has every one looked at the blueprints and claimed those to be worked on for Havana? 17:15:58 davidkranz: The stack traces probably caused by inspection or __import__ used by the "flake8". 17:16:18 afazekas: Should I take this up with infra? 17:16:18 davidkranz: I marked milestones and claimed some for our team 17:16:24 ravikumar_hp: Thanks. 17:16:44 davidkranz: yes 17:17:55 Sean was planning to soon obsolete the blueprints that are not claimed. They can always be claimed later. 17:18:13 But we wanted a manageable number to keep track of. 17:19:19 do we really need multiple blueprints for example for keystone v3 api ? 17:19:26 no 17:19:36 afazekas: They should be cleaned up. 17:19:47 Do we really want blueprints just with the list of test cases ? 17:20:18 afazekas: It it will take more than a very small effort, then yes I think we do. 17:20:44 afazekas: Part of the purpose of blueprints is to make sure there is not duplication of effort. 17:20:48 Even smaller effort if you just send the patch :) 17:21:04 afazekas: Unless some one else sends it first. 17:21:25 It only takes a minute to create a blueprint. 17:21:36 There is effort in tracking blueprints though. 17:21:37 The test cases in a blueprint can be done by multiple person 17:22:11 afazekas: True, but that should be coordinated by the blueprint assignee. 17:22:41 It is work to keep track of blueprnits which brings up the next subject which was the idea to have a "PTL" for the QA project. 17:22:45 How the tempest bugs fits into the picture ? 17:23:24 afazekas: I think we are talking about a fairly traditional model for tracking bugs and features. 17:23:52 afazekas: There is a gray area of course for a few missing test cases or something like that. 17:24:30 Does any one have any thoughts about a "PTL"? 17:24:51 The idea is to have some one take on ownership of tracking bugs and blueprints. 17:25:18 And coordinating suggestions for improvements to how we do things. 17:27:49 OK, anything else to discuss today? 17:29:27 any word on CloudCafe? 17:30:30 jaypipes: Does it have more published test cases ? 17:30:58 afazekas: not sure. hadn't last time I checked last Friday... 17:31:10 was hoping to hear from dwalleck or SamDanes 17:32:40 We should have some procedure for coopering on non-gating test tools anyway. 17:33:50 For example on test cases which expects multiple node 17:34:01 afazekas: That is really a matter of multiple groups of people agreeing to use a common infrastructure. 17:34:22 afazekas: The ci group is working on multiple node for jenkins. 17:35:17 do you think it would be possible to add multinode to devstack? 17:35:34 mordred: ping 17:36:33 I mean, would it be difficult to make devstack able to spawn vms for testing? 17:37:14 clarkb, fungi : Could one of you comment on the status of multi-node devstack in jenkins ^^^^^ 17:38:18 Guess they are all busy now. 17:38:52 I'll ping them later and see if I can find something to send out to the qa list. 17:39:04 one 'not in the agenda' question 17:39:08 sup davidkranz 17:39:09 davidkranz: thank you 17:39:30 are there plans for inclusion/creation of tests revolving around heat/ceilometer ? 17:39:37 mordred: ^^^^ discussnio about multi-node in jenkins 17:39:52 great. multi-node jenkins is something I'm quite keen on having 17:39:53 davidkranz: i don't think there's any major status updates on multi-node devstack, but i've not been directly involved 17:40:07 oh, mordred's here 17:40:11 mordred: Is the status or plan for this being tracked some where? 17:40:13 pleia2 is the main one working on that, but she's getting married right now 17:40:29 I believe so - lemme find the bug 17:41:28 giulivo: This was discussed at the summit with the heat and ceilo folks. I had a short IRC chat with heat the other day. 17:41:53 mordred: Is the currently planned the multi-node solution for spiting the test to multiple node, or to have multitude openstack cluster ? 17:41:58 giulivo: I think they will ping us when they are ready to engage. 17:42:08 afazekas: whose are two different things 17:42:35 afazekas: splitting the test to run portions of the tempest tests on different nodes is one thing that testr will be the first step in accomplishing 17:42:56 afazekas: having the cloud against which tempest is running be installed on multi-nodes is a different thing 17:43:09 both, I believe, are important - but have a different set of challenges. :) 17:44:05 * mordred cant' find the CI bug for multi-node right now - it's in there somewhere, I'll ping you with it davidkranz when I find it 17:44:24 mordred: NP. Thanks. 17:45:02 * afazekas is considering creating a test runner which can work faster on a single node 17:45:04 I've also been having good chats with rainya about getting her infra guys hooked in with our infra guys, so hopefully that'll have some good results on our ability to more forward on some of these thigns 17:45:12 afazekas: that should be testr 17:45:16 I would imagine 17:46:01 nova tests increased by 5x speed on my laptop when we moved to testr ... but it's not a straightforward move 17:46:17 as I believe jaypipes will be more than happy to back me up on 17:46:27 mordred: yes, but they CPU sensitive unittests 17:46:39 right. 17:47:01 afazekas: I'm just saying- before we go off making new test runners, let's finish the current test runner migratoin that we're working on 17:47:24 since it's a lot of work, and there are more than one reason we want to accomplish the testr one 17:49:23 mordred: first we should place script in tempest which start the actually used runner with the actual directory structure 17:49:38 Both will be changed soon 17:50:00 I'm confused by what you mean 17:50:44 https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L223 , https://blueprints.launchpad.net/tempest/+spec/tempest-repo-restructure 17:52:02 Now we cannot do restructure and changing the gate script at the same time, with a single commit 17:52:25 The logic around L223 should be in the tempest repo 17:52:57 Implemented as a ./run_tests.sh --gate 17:56:20 afazekas: the gate does not use run_tests.sh 17:56:26 afazekas: it uses tox.ini 17:56:36 it should use in the tempest case 17:56:37 afazekas: or calls nosetests directly 17:56:56 if we creating a venv we are not really using the recent python libraries 17:57:07 afazekas: no venv for tempest. 17:58:40 so, I agree that the extra config of how nose is invoked there is a problem 17:59:04 is the restructure goig to put cli into tempest? 17:59:25 mordred: it already is, AFAIK 17:59:50 k. weird. yeah - in _this_ case - I thnk a run_tests.sh should probably be used 18:00:07 but it should have big bold letters around it saying that it's a hack to solve historical issues 18:00:18 and will go away in the future once we've solvle them 18:00:24 and that we all feel bad about it 18:00:58 bdpayne: Error: Can't start another meeting, one is in progress. 18:01:06 #endmeeting