22:02:09 #startmeeting qa 22:02:10 Meeting started Thu May 8 22:02:09 2014 UTC and is due to finish in 60 minutes. The chair is sdague. Information about MeetBot at http://wiki.debian.org/MeetBot. 22:02:11 hi 22:02:12 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 22:02:14 The meeting name has been set to 'qa' 22:02:34 there's dkranz 22:02:48 Who is here today? 22:02:54 o/ 22:02:55 o/ 22:02:57 David Paterson 22:03:00 o/ 22:03:02 #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_May_8_2014_.282200_UTC.29 22:03:22 dkranz: I just kicked it, but feel free to run it now 22:03:33 dkranz: I am supporting a desployment to one of the data centes at work. Can I give the Neutron update at the start of the meeting, so I can concentrate at work after that? 22:03:40 sdague: There was something wrong with my connection 22:03:51 mlavalle: Yes, go ahead 22:03:58 #topic Neutron Testing 22:04:03 go mlavalle 22:04:22 Of 28 api tests that we are tracking, we h only have 5 more to merge 22:04:36 the rest have merged, so very good progress on that front 22:05:01 nice 22:05:10 I would like the core team to help us reviewing the following 3, to see if we can merge them soon: 22:05:36 https://review.openstack.org/#/c/83627/ 22:05:52 https://review.openstack.org/#/c/67312 22:06:11 https://review.openstack.org/#/c/63723 22:06:35 sounds great 22:06:57 we also have 20 minutes of tempest on the Neutron agenda next Thursday at 9 am in the design summit 22:06:59 #action core review eyes needed on 3 neutron reviews 22:07:09 this is the etherpad that I put togeteher 22:07:15 https://etherpad.openstack.org/p/TempestAndNeutronJuno 22:07:43 In principle, during Juno we will be pursuing four lines of actions 22:07:53 1) increase the number of scenario tests 22:08:07 2) fill any gaps that might have been left in api tests 22:08:18 3) support the nova parity subproject 22:08:26 4) support other suprojects 22:08:38 please feel free to review the etherpad and add to it 22:08:52 sounds good 22:08:56 mlavalle: anything else? 22:09:00 that's all I have 22:09:02 thanks 22:09:19 great 22:09:21 I'll be watching the rest of the meeting, but I might have to drop off 22:09:29 ok, back to agenda as it's ordered 22:09:34 #topic Reminder about summit etherpads 22:09:49 The list of etherpads is here - https://wiki.openstack.org/wiki/Meetings/QATeamMeeting 22:09:59 not all have been created yet 22:10:36 chmouel's UX one is missing 22:10:43 boris-42's rally one is missing 22:10:44 maybe https://wiki.openstack.org/wiki/Summit/Juno/Etherpads#QA ? 22:10:46 #link https://etherpad.openstack.org/p/Juno-QA-design-summit-topics 22:10:53 sdague ? 22:11:27 and maru_afk's functional test one is missing 22:11:42 sdague: Please run the meeting. I keep flaking in and out. 22:11:48 boris-42: etherpad for rally / tempest summit session hasn't been stubbed yet 22:11:55 masayukig: yes 22:11:58 sdague when is deadline? 22:12:06 sooner the better 22:12:20 sdague ok will do (just finished slides) 22:12:23 so people can look and provide feedback pre summit 22:12:41 boris-42: there shouldn't be slides for a design summit session 22:12:53 sdague just small intro 22:13:11 sdague i think it's simpler to start from intro no? 22:13:16 if you want to link some slides in the etherpad for people to read in advance, that's cool, but there shouldn't be slides in the session 22:13:22 that's not what the session is there for 22:13:33 sdague okay I'll just left slides 22:13:40 sdague cause I think intro is required 22:14:24 it's probably worth putting those out on the list in advance as well, just to further highlight them 22:14:43 boris-42: Yes, please send intro to the list 22:15:03 I'm going to spend lots of time tomorrow filling out my etherpad 22:15:14 hehe me too 22:15:43 sdague i know it's not super related to the rally & tempest integration 22:15:55 ok. Reminder to everyone else to handle etherpads for the summit 22:15:55 sdague but I would like to speak about osprofiler & tempest integration 22:16:06 sdague is it ok? 22:16:29 boris-42: I'd say try to keep this narrow to begin with, and if there is more time get there 22:16:33 but 40 minutes goes fast 22:16:42 sdague okay it will be last topic 22:16:46 latest* 22:16:57 and I think we've got some issues on time accounting that we need to make sure we sort out 22:17:16 ok, next topic 22:17:18 sdague: We should move on 22:17:21 #topic Proposal to move success response checking to clients (dkranz) 22:17:33 dkranz_: you have the floor 22:17:42 sdague: I just wanted to see if any one objected to this proposal that was discussed on the ml 22:17:56 or if there were any other comments 22:18:23 I think it was generally agreed. I think it's worth writing up as a qa-spec, and we can approve it through that mechanism 22:18:23 sdague: I don't really see any downside 22:18:32 sdague: ok, I will do that. 22:18:34 seems big enough to be a spec/blueprint 22:18:39 vs. just a bug 22:18:53 #action dkranz to create spec for moving response checking to clients 22:19:09 I think the only details are around multiple allowed success codes 22:19:12 so make sure to call that out 22:19:17 just so we get that right 22:19:26 sdague: Right. I wonder how many there actually are. 22:19:47 sdague: Not counting those that say any 2xx is ok 22:19:53 That's it 22:20:13 cool 22:20:15 next topic 22:20:29 #topic Can we turn on voting of ironic jobs (recent creds change broke it)? (adam_g) 22:20:42 adam_g: That's you 22:21:01 context: some refactoring merged recently that broke some of the non-voting jobs 22:21:10 ironic, and i believe solum 22:21:28 i dont think we can really make these voting until the projects have graduated 22:21:55 adam_g: there's no solum job on tempest I think 22:22:31 andreaf_, oh, maybe not a job in the gate but some of the solum tests (at least reported by devkulkarni earlier) 22:22:58 solum is running a ton of out of tree stuff though, so I consider that a different issue 22:23:06 we'll be making these ironic jobs voting in the ironic gate soon, not sure how to prevent this from happening other than urging people to pay attention to non-voting jobs 22:23:35 and /me being more proactive about catching failures during review :) 22:23:43 adam_g: I think with the # of jobs on a tempest run now, seeing the non voting votes is going to get harder over time 22:24:14 adam_g: I suggest sending a ml message saying that ironic is not gating only because of incubation but is solid. 22:24:17 any idea if there is a gerrit query that would return those changes? 22:24:21 adam_g, sdague: yes it's getting harder, and gate is not very stable (voting and not voting) so it's even harder 22:24:39 dkranz_: +1 sounds good 22:24:53 i just threw up http://no-carrier.net/~adam/openstack/ironic_gate_status.html to help me monitor failures 22:25:02 s/threw up/put up :) 22:25:05 heh 22:25:09 it's funnier the first way 22:25:20 sdague: Because we have non-voting jobs that have been downgraded due to failures and others that are solid but waiting to get in for other reasons 22:25:29 Reviewers just need to know which is which 22:26:24 dkranz_, sdague, adam_g: sortng the jobs and splitting them in sections would help already 22:26:55 dkranz_, sdague, adam_g: but even nicer would be to get some stability stat next to the failing job 22:27:14 i'd be cool if there were a way to categorize both using notes in jenkins comments, based on pass/failure ratio over the last N days/weeks 22:27:19 andreaf_: sure, that's in my elastic recheck set of futurues to give us that 22:27:23 or some check that identified new test failures in an unstable job 22:27:48 ok, this is turning more into brainstorm though, so I think we should table to beers somewhere at summit 22:28:03 is there something actionable beyond sending a heads up to the list? 22:28:04 +1, tho i wont be there so someone will need to drink mine 22:28:09 Really we just need a third tag which says "non-voting but look at a failure before approving" 22:28:16 adam_g: bummer 22:28:35 dkranz_: so realistically that feels to me like a "jenkins 2nd vote" 22:28:51 which we'd need a bunch of infra buy in and refactor on 22:28:57 but is kind of interesting 22:29:05 sdague: is there a place where we can track such "additional topics to chat about at summit"? 22:29:16 sdague: I don't know how much real work should be done here vs just living with it 22:29:31 andreaf_: not atm, you have a suggestion? 22:29:51 I just assume it will come up over coffee / food / beer all week, and my brain will end up full at the end of it 22:29:57 sdague: perhaps another etherpad 22:30:00 dkranz_: agreed, lets move on 22:30:01 sdague: At past summits we have had "qa meetings" 22:30:16 dkranz_: well, usually a lunch somewhere 22:30:34 sdague: that too 22:30:45 #topic Specs Review 22:31:01 ok, time for specs that people want to talk about, and get eyes on 22:31:13 ok 22:31:13 dpaterson: I believe that includes you, right? 22:31:25 dpaterson: go first 22:31:45 or I'll start 22:31:48 https://review.openstack.org/81294 22:32:03 multiauth bp, I think it's ready all comments addressed 22:32:22 and https://review.openstack.org/#/c/81307/ keystone v3 jobs also all comments addressed 22:32:55 andreaf_: this looks pretty good 22:32:56 and I filed a new one today about client manager refactor https://review.openstack.org/92804 for which I'd love some feedback 22:33:22 I'm good on 81294 22:33:31 I'll look at 81307 in the morning 22:33:57 andreaf_: any specific items you want to bring up about them? 22:34:19 or just getting people to look? 22:34:46 the latter 22:34:49 #action qa-specs that are probably ready for final approval https://review.openstack.org/81294, https://review.openstack.org/#/c/81307/ 22:35:01 andreaf_: I already gave my +2 to 81294 and mtreinish just -1 for syntax issue 22:35:21 #action qa-spec on client manager refactor needs review https://review.openstack.org/92804 22:35:31 dpaterson: You there? 22:35:34 mtreinish should be working tomorrow 22:35:34 dkranz_: yes I fixed the issue 22:35:34 yup 22:35:50 Sorry stepped away for a sec 22:35:52 dpaterson: You can discuss your spec 22:35:53 so dkranz_ +2 if you think it still holds and we can nudge him tomorrow for landing 22:36:00 Sure 22:36:02 https://blueprints.launchpad.net/tempest/+spec/post-run-cleanup 22:36:02 https://review.openstack.org/#/c/91777/ 22:36:32 dpaterson: so there is a clerical issue here where the patches need to be merged 22:36:36 dpaterson: You should abandon the old patch and resubmit the new one with no dependency 22:36:39 Basically I have been working with some QA folks on testing a HA configuration and running into issues with Temepst cleanup 22:37:15 dkranz: will look into the problem 22:37:53 dpaterson: one of the concerns I have about this approach is it papers over state corruption issues in the services by just having Tempest clean things up 22:38:10 I'd much rather get the base services fixed to not be in an inconsistent state 22:38:22 I agree 22:38:31 also, we really *can't* access the db directly from tempest 22:38:39 for both design reasons 22:38:43 and practical reasons 22:38:46 but before that happens I would like to have a tool to unblock my guys 22:39:12 sdague: I think he is proposing a script, not having tempest do it automatically 22:39:29 oh, ok, yeh I see that now 22:39:35 sdague: There is already a script in the stress dir but it does not do what is needed. 22:39:38 My proposal is just python, not dep on tempest. 22:39:47 dpaterson: ok sure 22:39:51 dpaterson: My concern is with the database cleanup part 22:40:00 Yes 22:40:16 on the db cleanup part, the issue we had before when we had whitebox testing was the schemas change a lot in a release 22:40:23 so it's basically always breaking 22:40:27 dpaterson: Any such database cleanup will be very fragile 22:40:28 like *always* 22:40:40 I am open to alternatives but currently they are getting into a state where API calls cannot remove objects. 22:40:56 So somekind of surgery is going to be required 22:41:14 dpaterson: I don't doubt that but I'm not sure tempest is the right place for that 22:41:17 dpaterson: sure, so lets get the blueprint cleaned up so it's passing the docs job just to look at it. 22:41:28 dpaterson: Stuff in tempest has to be kept working 22:41:49 dpaterson: And that is hard with code that only runs when things get seriously messed up 22:42:07 I'd be worth trying it, if we also required that any cleanup function needed an upstream bug before it could come in so that we'd actually work towards addressing root issues 22:42:24 sdague: works for me 22:42:25 It would only execute if cleanup is flagged to do so 22:42:31 but we should also be very aware this is probably going to be very fragile 22:42:43 agreed 22:43:07 dpaterson: I also don't think it should be triggered by tempest. It would just be a helper tool that we keep around that people could run manually if they like 22:43:14 it is 22:43:18 +1 22:43:41 dpaterson: ok, so if you can respin the spec, will take a look 22:43:47 are you going to be in atlanta? 22:43:54 dpaterson: It is useful because not every one has to figure out the weird db calls 22:44:05 Sorry not this time, 22:44:06 I expect the review queues are going to go really quiet next week regardless 22:44:18 so it might not be till the week after that people get real eyes on this 22:44:34 dpaterson: interesting. In the HA testing, tempest will stop openstack for checking right switching? 22:45:01 oomichi: Not sure what you mean 22:45:12 I don't quite understand oomichi, 22:46:12 dpaterson: HA switchs active / standby controller-node as I understand. 22:46:50 Tempest doesn't do any switches. The idea is tempest is run and we get a report 22:46:54 oomichi: I don't think tempest would know anything about ha, right? 22:46:56 Then clean the system up 22:47:14 Take down a controller or introduce some other failure 22:47:17 and rerun tempest 22:47:26 dpaterson: oh, I see. thanks 22:47:27 Should get same test report 22:47:41 ok, great, dpaterson you need anything else on this? 22:47:50 nope, thanks 22:48:00 ok, great 22:48:05 #topic Blueprints 22:48:12 tx 22:48:17 any in process blueprints people want to bring up? 22:49:06 I'm assuming most people are prepping for summit 22:49:13 sdague: Just mentioning that we made some good progress on the multi-auth bp with lots of reviews 22:49:27 given that we only have 10 minutes left, lets jump to critical reviews 22:49:31 sdague: Just that https://review.openstack.org/#/c/91899/ is waiting for final approval 22:49:32 #topic Critical Reviews 22:49:37 https://review.openstack.org/#/c/92573/ - connection verification as part of RemoteClient init 22:49:46 that was put in the agenda 22:50:07 dkranz_: +A 22:50:12 sdague: I think yfried is trying to debug some ssh issues we have seen 22:50:15 sdague: thanks 22:50:58 I'll plug the test and worker summary patch - https://review.openstack.org/#/c/92362/ 22:51:09 as we lost test summary with the new subunit trace 22:51:23 that will also let us see worker balance in jobs 22:51:28 which is kind of useful 22:51:52 andreaf_: what's the next patch needed for review in the multi-auth stack? 22:52:07 any other critical reviews people need eyes on? 22:52:14 https://review.openstack.org/#/c/80246/ 22:52:16 #topic Open Discussion 22:52:25 https://etherpad.openstack.org/p/juno-summit-open-topics 22:52:25 ok, let's do open discussion 22:52:50 andreaf_: I like that 80246 is negative LOC :) 22:52:52 sdague: just an etherpad for people to put additional ideas we can discuss at lunch or beer 22:53:08 sdague: :D 22:53:10 andreaf_: great, want to also send that to the mailing list 22:53:17 sdague: ok will do 22:53:45 anything else from folks? 22:54:00 when are folks getting to atlanta? 22:54:10 7pm on Sunday 22:54:15 sdague: Sorry I will miss the dinner. Don't arrive until 8pm 22:54:22 15:30 Sunday 22:54:31 I will also miss the dinner, as I've got the TC / board thing that night 22:54:45 but my evening schedule is always nuts 22:54:59 sdague: Maybe we should have a lunch table on Monday 22:55:27 Anyway, should be a busy week :) 22:55:36 yes, it should be. 22:55:58 I'm getting in Sat because of board things on Sun, so if anyone happens to be around Sat let me know. 22:56:28 ok, I think that's a wrap folks. See you in ATL 22:56:41 and there won't be a meeting next week, because of the summit, so see folks back here in 2 22:56:45 #endmeeting