17:01:28 #startmeeting qa 17:01:28 Meeting started Thu Mar 27 17:01:28 2014 UTC and is due to finish in 60 minutes. The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:01:29 I'll fix it 17:01:30 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:01:32 The meeting name has been set to 'qa' 17:01:39 hi who do we have here today? 17:01:54 hi 17:01:56 o/ 17:02:11 #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_March_27_2014_.281700_UTC 17:02:13 hi 17:02:17 ^^^ Today's agenda 17:02:36 o/ 17:03:05 ok well let's get started 17:03:14 #topic QA Specs Repo Proposals (sdague) 17:03:25 ok, so we have the qa-specs repo 17:03:41 https://github.com/openstack/qa-specs 17:03:49 #link https://github.com/openstack/qa-specs 17:03:51 and we even have an agreed to template now 17:04:06 https://github.com/openstack/qa-specs/blob/master/template.rst 17:04:21 #link https://github.com/openstack/qa-specs/blob/master/template.rst 17:04:37 o/ 17:04:44 so I think the point for this agenda item was to go through a spec or two and see what our feedback should be on it 17:04:56 mtreinish: you have a couple up, right? 17:05:06 do you want to volunteer one of those? 17:05:13 sdague: yeah I pushed up the non high prio bps as specs reviews: 17:05:14 https://review.openstack.org/#/c/82933/ 17:05:19 and https://review.openstack.org/#/c/83264/ 17:06:30 ok, lets take 82933 17:06:47 #link https://review.openstack.org/#/c/82933/ 17:07:19 feedback that I'd like to see there is the that it's a new tool, so it should include where that tool will go in the source tree 17:07:59 and we should probably have the -h output for it mocked out, so what's optional or not can be seen up front 17:08:04 in the spec 17:08:17 then I'd consider it good 17:08:24 anyone else have feedback? 17:08:34 or feel that feedback is too specific? 17:08:35 heh, well it's already in the tools dir :) 17:08:44 sdague: no those are good points 17:08:56 The alternative is to generate the config file and supply the admin creds as an argument 17:09:14 That is basically what I did in the sample generator I submitted as WIP 17:09:40 But seeing the -h output would make it clearer 17:09:48 dkranz: I had an alternative listed in the spec for something like that 17:10:11 mtreinish: Yes, I saw that. My comment was about that part. 17:10:12 so the alternatives section is one I'm sort of meh about 17:10:21 yeah I'll add the -h option, although honestly it'll have only one bool option 17:10:34 sdague: it was in the template so I used it 17:10:44 sdague: meh because you don't like it or because there should not be such a section? 17:10:48 it's important in nova because they get a lot of competing duplicative approaches 17:11:05 dkranz: right, I wonder if people feel it's important for qa-specs in general 17:11:09 I could go either way 17:11:27 sdague: I don't see anything wrong with it but perhaps not required 17:11:36 I think it's listed as optional in the template 17:11:46 mtreinish: ok, that's fine 17:12:06 sdague: that was one of my first review comments when cyeoh posted the first template draft 17:12:17 ok, great 17:12:32 ok, any other feedback on this one? 17:13:00 sdague: that might have already been said, 17:13:54 https://review.openstack.org/#/c/83264/2/specs/add-service-tags.rst is the 2nd one 17:13:56 sdague: but I think once the config-verification is done, it should point to this spec somehow for futur docs 17:14:15 yfried: what exactly do you mean? 17:14:16 *future documentation 17:15:10 sdague: a new guy looking at that module will find all this discussion very helpful 17:15:22 yfried: so the specs repo is largely to define the blueprint 17:15:32 then the implementation will happen there 17:15:46 in tree based on the approved blueprint 17:16:06 I'm not sure it's going to be better than the actual implementation for documentationm 17:16:24 https://review.openstack.org/#/c/83264/2/specs/add-service-tags.rst - mtreinish I couple of comments there 17:16:46 first, like with the last one, it's probably worth making it explicit where the code is getting added in tempest 17:17:03 which is sort of there in prose, but could probably be broken out 17:17:17 sdague: I want remove all unused variables in tempest. What you think about it. 17:17:33 also, I think a code sample of what the code would look like afterwards would be useful 17:17:54 sdague: ok, so basically we should include more examples in specs then 17:18:04 we probably should put that in the template or readme 17:18:05 yeh, I think examples will make it more reviable 17:18:19 at least for me, that's how I wrap my head around things 17:18:38 no that's a good point 17:19:02 vrovachev: ok, is that part of the specs discussion? or is that for open discussion? 17:19:19 i.e. should it wait until the end of the meeting 17:19:33 anyone else have comments on - https://review.openstack.org/#/c/83264/2/specs/add-service-tags.rst ? 17:19:45 vrovachev: yeah that's off topic, if we have an open discussion section you can bring it up then 17:20:38 other than that, I think this is looking pretty solid 17:20:48 sdague: ok cool 17:20:58 so would you say we're open for real on this then? 17:21:01 mtreinish: you want to make another iteration on those? We can handle any remaining feedback in review 17:21:11 sdague: yeah I'll do it after the meeting 17:21:16 mtreinish: lets plan for end of next week to make that open beyond this group 17:21:25 I'd like landed content that are good examples 17:21:42 mtreinish: Good, sorry. 17:22:03 sdague: ok yeah that'll probably be helpful 17:22:35 andreaf: can you convert yours to this template - https://review.openstack.org/#/c/81294/2 ? 17:22:45 I think that would be another good one to review during the week 17:23:07 sdague: sure I was hoping to do that before the meeting but didn't manage to 17:23:14 andreaf: yep, no problem 17:23:41 ok then I guess we can move on if there isn't anything else 17:23:54 sdague: one question re the template, do we include copyright notice in those like the code? 17:24:15 andreaf: yeah the cc license like in the template 17:24:33 andreaf: if your employer really wants a called out copyright, you can 17:24:48 as long as it has the CC license in it 17:25:08 sdague: ok thanks 17:25:29 ok, I think we're making good progress here 17:25:36 anything else on specs? 17:26:01 back to you mtreinish 17:26:08 #topic Blueprints 17:26:22 sdague: did you want to do a high prio bp review 17:26:37 or skip it this week because of the specs discussion 17:27:12 I think we can skip 17:27:32 especially as we seem to have a small audience 17:27:37 ok then did anyone have any bps they'd like to bring up 17:27:41 otherwise we'll move on 17:28:01 mtreinish: ok one 17:28:07 andreaf: sure go ahead 17:28:25 #link https://blueprints.launchpad.net/tempest/+spec/keystone-v3-jobs 17:28:36 it's related to the multi-auth bp 17:28:51 but on the infra side of things 17:29:29 I have to align the spec to the template for this one too, but it would be good to get some review on it 17:29:55 andreaf: yeah we can do the review in gerrit now :) 17:30:00 at the moment is not prioritized, but it shall probably be same prio as the multi auth one 17:30:16 andreaf: yeah that'll come after the spec review (see the readme) 17:30:20 #link https://review.openstack.org/#/c/81307/ 17:30:25 but it seems pretty straightforward 17:31:11 mtreinish: thanks that's all 17:31:22 andreaf: ok thanks 17:31:30 ok are there any other bps to discuss? 17:32:06 ok then let's move on 17:32:10 #topic Neutron testing 17:32:21 mlavalle: are there any updates on neutron testing? 17:32:26 Yes 17:32:35 We have continued merging api tests 17:32:54 I am tracking 28 patchsets. As of today, we have merged 16 17:33:41 There are 5 abandoned. I have sent email to the authors. 3 responded and will be restarting their patchsets over the next couple of days 17:33:49 The other 2 I assigned to myself 17:34:17 I have 2 patchsets that only need one more +2 to merge: 17:34:39 https://review.openstack.org/#/c/63723/ 17:34:39 https://review.openstack.org/#/c/68597 17:34:46 #link https://review.openstack.org/#/c/68597 17:34:54 #link https://review.openstack.org/#/c/63723/ 17:35:01 mlavalle: ok cool 17:35:06 mlavalle: this might be off-topic, but - is there any chance to apply this productive approach to network-scenarios (as well as api) 17:35:10 Please take a look at them and see if we can merge them 17:35:12 hopefully we'll get this all merged before the release 17:35:48 yfried: this is just for getting api coverage on neutron 17:35:55 yfried: of course. I just don't want to spread myself to thin in this cycle, but scenario testing is my next goal 17:36:10 scenarios are a bit more involved so I'm not sure a similar structure would be as effective 17:36:21 unless there was a predefined list of scenarios to implement 17:36:23 we can give it a try :-) 17:36:42 and start with a predefined list 17:36:56 mlavalle: I understand. I think the first step would be to draft a list of scenarios to implement 17:37:03 yes 17:37:19 let's partner on that 17:37:36 that's all I have 17:37:44 ok is there anything else on neutron testing from anyone else? 17:38:11 do we have an idea on how close the full job is? 17:38:17 what is required to make the full neutron job voting ? 17:38:24 sdague: there has been a ml thread on that 17:38:38 salv-orlando and rossella_s have been looking at it 17:39:05 I think there are still a few bugs but it's looking like we'll greenlight the week after release 17:39:20 We've been looking at failure rate, and rossella_s has a script which keeps spinning a patch in the check queue 17:39:38 So far it does not seem much worse than the smoke job 17:39:51 so I would say after release, we switch it to voting 17:39:58 salv-orlando: I wonder if we could get rossella_s talking with the infra team about background queue jobs 17:40:04 instead of doing that in check like that 17:40:17 she 17:40:18 sdague: I'd be glad to talk to them 17:40:19 because that's something we actually very much wanted to get baselines on all the test suites 17:40:21 she's online... 17:40:28 rossella_s: awesome 17:40:29 sdague: do those still get picked up by e-s? 17:40:37 e-s? 17:40:41 mtreinish: the point would be to make them 17:40:44 elastic search 17:40:48 and elastic recheck 17:40:57 so basically we could have control data from master 17:40:59 sorry got my acronyms mixed up 17:41:04 yep, no worries 17:41:09 salv-orlando: it's ok, I'm just lazy 17:41:51 BTW: is the neutron more stable with pgsql based on statistics ? 17:41:55 rossella_s: ok, so how about post meeting we jump to -infra and talk about this 17:41:57 if you were really lazy you would have avoided using the dash as well; you probably had to stretch your right middle finger for that! 17:42:04 heh 17:42:09 sdague: ok, awesome! 17:42:19 afazekas: I would say mysql failure rate is actually higher than postgres 17:42:34 salv-orlando: there is another difference in those jobs 17:42:42 I believe mysql is using the metadata server 17:42:45 and pg is not 17:43:01 does pg uses config drive? 17:43:02 salv-orlando: so the eventlet vs mysql native driver issue is visible.. 17:43:10 salv-orlando: I think so 17:43:24 the layout.yaml definition should say 17:43:32 right. so keep going with the meeting I'll be back in 5 with numbers from the past week 17:43:46 salv-orlando: ok cool 17:43:57 then I guess we should move onto the next topic 17:44:06 #topic Heat testing 17:44:22 sdague, stevebaker: I've seen stuff from both of you on this 17:44:27 are there any updates? 17:45:12 mtreinish: I did the refactor to get the templates out of the code 17:45:17 that's landed now 17:45:27 which I think makes things a lot easier to understand 17:45:33 and review 17:45:40 there are some additional patches inbound 17:45:57 I went through a stack by shardy this morning and gave some feedback 17:46:18 also SergeyLukjanov was talking about adding sahara scenario tests that would use the heat provisioning 17:46:24 which would help all around 17:46:27 ok cool, I'll take a look at them when they come back through 17:46:31 I'm here 17:46:36 reading scrollback 17:46:37 yeah that would be cool to see come through 17:46:45 we'd run it on the heat slow job I'm assuming 17:46:50 the sahara part will have to wait until we cut stable/icehouse 17:47:16 yeah that's fine, it's only a couple weeks away anyway 17:47:17 sdague, yup, I hope to have some PoC heat-based tests for sahara in summit time 17:47:30 which we should probably decide what else has to land before we do that, as we are going to start getting open masters on projects again soon 17:47:40 keystone has a juno master now IIRC 17:48:00 sdague: I think I saw an email from ttx abount cinder too 17:48:10 I think all projects will cut their first i-rc till the end of next week 17:48:15 SergeyLukjanov: that's the hope 17:48:39 sdague, I've prepared change for d-g to cut conditions 17:48:50 https://review.openstack.org/#/c/83075/ 17:48:55 and enable sahara in gate 17:48:59 https://review.openstack.org/#/c/83076/ 17:49:35 of course, the first one depends one rc1 cuts and the second one is on your wish :) 17:50:06 SergeyLukjanov: well I wasn't planning on cutting tempest until release day, which is what've done for the past few cycles 17:50:21 SergeyLukjanov: so I think the issue is we don't get stable/icehouse branches till much later, right? 17:50:37 now we're sitting on milestone proposed? 17:50:40 sdague: yeah I think that's right it's milestone proposed until release day 17:50:54 we should maybe revisit that 17:51:11 anyway, we should move on 10 minutes 17:51:17 ok yeah 17:51:28 oh, yup, sure, so, April 18 will be ready :) 17:51:34 #topic Bugs 17:51:34 we'll 17:51:53 so does anyone have any bugs that they would like to bring up? 17:51:55 * SergeyLukjanov disappearing 17:52:02 or are there any critical bugs that need attention? 17:53:01 ok I guess not 17:53:04 so let's move on 17:53:12 #topic Critical Reviews 17:53:24 does anyone have any reviews they'd like to get eyes on? 17:53:38 I do 17:53:41 as usual: https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/multi-keystone-api-version-tests,n,z 17:53:51 #link https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/multi-keystone-api-version-tests,n,z 17:54:09 andreaf: yeah I'll try to give it a thorough review this week 17:54:15 it does take some time though 17:54:19 It's a chain of 7 patch-sets for the multi-auth bp 17:54:31 mtreinish: thanks - I tried to reduce the size as much as possible 17:54:59 ok are there any other reviews? 17:55:05 mtreinish: in the meantime I started working on auth provided via keystone official client for the scenario tests - bit still WIP 17:55:14 The ssh instance validation things has enough problem without discussing the uec or not uec default image https://review.openstack.org/#/c/81486/ , I would recommend to stay with uec anyway 17:55:47 #link https://review.openstack.org/#/c/81486/ 17:56:12 andreaf: I thought we used the keystoneclient for auth in scenario already 17:56:23 or at least with tenant isolation we do 17:56:28 or did you mean tokens? 17:56:47 mtreinish: yes but it's a fixed version of keystone client 17:56:58 I'm moving it to auth so it uses the right version 17:57:01 ahh ok 17:57:03 * salv-orlando has the failure rate numbers and is waiting for open discussion 17:57:16 salv-orlando: go for it, we only have 3 minutes 17:57:23 or we can take it to -qa after 17:57:30 #topic Open Discussion 17:57:33 salv-orlando: go ahead 17:57:38 past 7 days: mysql failure rate 2.79%, pg: 2.04% 17:57:41 we can move to -qa if it takes longer 17:58:03 there is a tail in mysql of the effects of bug 1283522 17:58:04 Launchpad bug 1283522 in neutron "DB lock timeout errors with parallel operations" [Critical,Fix committed] https://launchpad.net/bugs/1283522 17:58:11 considering past 5 days only 17:58:21 mysql failure rate: 2.26%, pg:3.34% 17:58:41 but the amount of pg jobs is rather small so I would not regard the last info as statistically significant 17:58:45 that's all 17:58:52 salv-orlando: is that the full parallel jobs or the smoke jobs? 17:58:59 oh I guess it's smoke if it's pg nm 17:59:21 ok well we're out of time for today 17:59:23 smoke. Full parallel job we have only 21 baseline runs so far. too little to say anything. But we found 3 failures in full job (1/7) 17:59:34 we can pick anything else up on -qa 17:59:40 thanks everyone 17:59:48 #endmeeting