17:01:43 #startmeeting qa 17:01:44 Meeting started Thu Dec 5 17:01:43 2013 UTC and is due to finish in 60 minutes. The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:01:45 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:01:47 The meeting name has been set to 'qa' 17:02:00 hi who do we have here today? 17:02:03 o/ 17:02:07 hi 17:02:07 o/ 17:02:20 hi 17:02:28 hi 17:02:32 hi 17:02:37 #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_December_5_2013 17:02:38 hi 17:02:43 ^^ Today's agenda 17:02:57 let's get started 17:03:05 #topic Meeting time change proposal 17:03:10 afazekas: meeting 17:03:23 so I sent out an email to the ml about this 17:03:30 mtreinish: yep, thanks for that 17:03:41 we need to change the meeting time, I suggested 2200 utc 17:03:48 which is a bit difficult for people in Europe 17:04:04 I just wanted to make sure to highlight it to people so they got on that thread if they weren't already 17:04:10 hi 17:04:17 so I was thinking we could alternate between this time and then (on Thursday) 17:04:18 mtreinish, 22utc is good for me though (europe) 17:04:26 giulivo: ok cool 17:04:32 is other "europe" against it? 17:04:42 I'd prefer to keep one time 17:04:47 giulivo: mkoderer said it was going to be pretty late for him 17:04:48 mkoderer was hesitant 17:04:51 mkoderer was not thrilled but said he could do it 17:04:57 oh okay sorry 17:05:03 afazekas ? 17:05:10 What about 1 hour earlier? 17:05:18 it's too early for Japan then 17:05:28 6am 17:05:31 I thought they said they could do 6am 17:05:45 dkranz: it also ends up on top of the nova meeting then, right? 17:05:57 sdague: Don't know 17:05:58 yet, isn't it three of us in europe? afazaks will make the difference 17:06:12 But 22:00 is midnight in europe most of the year 17:06:49 well anyway we can take this back out to the list, and do a final vote next week 17:07:02 dkranz, indeed it is late but altenating meetings means we would loose 1 in 2 anyway ... will midnight be better or worst than that? 17:07:12 yep, I'm honestly going to be more +1 on oscillating just because it won't discourage new .eu people 17:07:30 sdague: I was just worried about the disconnect then 17:07:31 I agree. And if giulivo wants to go to the other at midnight that is ok 17:07:46 But it is a tough call 17:07:57 Making it 1 doesn't help if people don't show up 17:07:59 giulivo: I will try to solve to attending the meeting at whatever time 17:08:14 how about next week we do the 22 UTC and start oscillating? 17:08:25 mtreinish: that works for me 17:08:33 want to propose that back to the list? 17:08:38 Sounds reasonable 17:08:47 sdague: yeah will do 17:08:51 and I'll update the wikis 17:08:54 it can start oscillating and then check if stick to a single time 17:09:00 if it doesn't work well we can always revisit it again 17:09:05 +1 17:09:11 +1 17:09:19 22:00 is the best fallback we have so +1 17:09:19 +1 17:09:28 #action mtreinish to update meeting times to oscilate between current time and 2200 UTC 17:09:37 ok then lets move on 17:09:46 #topic Getting additional folks signed up as point people 17:09:51 sdague: this is your's 17:09:54 yep 17:10:24 so basically I'm realizing that there are too many discreet things to keep an eye on in the QA program for me to handle it all well 17:10:28 or any future PTL 17:10:34 Agreed 17:10:44 so I'd like to carve off point people for specific things 17:11:11 which isn't that they'd own all the work, they would just be the point person to make sure we stay on top of it, and harass folks when we don't 17:11:34 I figured 3 things that could be carved off this way are: meeting management, blueprint triage, and bug triage 17:11:55 mtreinish volunteered for meeting management (hence why he's running the meeting and handling the time change) 17:12:06 but I'd still like to get volunteers for the others 17:12:10 I think the problem is the things you are carving off are things that in most engineering orgs are done by managers. We should recruit managers to do them. 17:12:29 dkranz: well our community doesn't work that way 17:12:40 people that do the work, get the trust 17:12:50 sdague: Why not? If a company wants to donate manager time for this... 17:13:20 dkranz: ok. But we don't see those folks as contributors today 17:13:27 sdague: I know 17:13:38 sdague: But we have not tried to get them either 17:13:39 so I don't think that's a productive direction for the conversation 17:13:58 because honestly, point people need to be folks the group trusts 17:14:32 sdague: ok, i'll stop here. But I don't know why being a good hacker makes you more trustworthy at bug triage 17:14:32 because if they don't trust them, then it's sort of useless 17:15:06 I'm not talking some random manager but some one who is engaged. 17:15:07 looks like we still have bugs in Fix Committed status, does anyone has objection against moving them to the Fix released status ? 17:15:25 afazekas, I wanted to ask that too, that's a +1 from me 17:15:27 dkranz: I'm not saying it does. I'm saying there are no non developer candidates in our community that are doing work 17:15:29 That was supposed to happen automiatically but there was a window where it didn't 17:15:37 afazekas: that's kinda off topic, but they should be moved (we do it by default now) 17:15:52 sdague: RIght. I just had some idea to try changing that 17:15:54 so... this is exactly why we need a point person :) 17:16:09 sdague: Agreed. And it should be one of us. 17:16:17 because now the conversation went off into the weeds on bp details, which have gotten lost, because there wasn't one 17:16:38 or more specifically, the person was me, but it's about #12 down my priority list 17:16:44 so effectively will never get done 17:17:20 so I won't bother, but I don't like this "single point of failure" thing 17:17:31 sdague, do you expect this point person to be amongst the core people, or not necessarily? 17:17:57 adalbas: not necessarily, especially for bugs 17:18:00 let's separate bug triage from blueprints because they are a different magnitude of task 17:18:04 yep 17:18:14 and different levels of trust needed I think 17:18:35 but my long term idea is to do some hard work now and later block new blueprints to not find ourselves in the same situation again 17:18:37 so our bug group is open, anyone can triage, so I really think any volunteer on bug triage point person would be welcomed 17:18:50 The problem with random failure bugs, in most cases we cannot do anything with them in tempest, how to handle them ? 17:19:04 afazekas, that's why I'd drop non triaged 17:19:07 afazekas: ok, again, moving into the details 17:19:19 because the point person would get to make some of those calls 17:19:43 giulivo: so it sounds like you are actually taking a run at the blueprint cleanup anyway 17:19:47 sdague, could you point what the bug triage person would be doing (responsibilities)? 17:20:02 sdague, hehehe yes I'd like to indeed! 17:20:04 you want to be the point person for that for now 17:20:14 giulivo: I think yuo'd be great at it :) 17:20:40 adalbas: QA bug czar (Spearheading Tempest Bug triage) 17:20:40 * focus on driving down Tempest bug queue to a useful thing 17:20:40 * nag about top QA bugs in tempest meeting 17:20:40 * organize bug triage days regularly to help with this 17:21:27 but really, why do we need new blueprints if not discussed here or by a core? does "closing" blueprints look so terrible? 17:22:05 giulivo: I don't think so. Again we just need someone to stay on top of it and make sure it doesn't get out of hand 17:22:16 That is the direction nova is going in 17:22:16 hint hint, you sound like a great volunteer for this :) 17:23:17 dkehn, you mean closing "new blueprints to trusted" ? 17:23:21 dkranz, ^^ 17:23:37 ok, well lets move to the next topic now 17:23:44 ?? 17:23:48 sdague can pester people to volunteer after the meeting 17:23:49 giulivo: No, just the idea of exerting more control over accepted blueprints sooner 17:23:49 yeh, I guess we still aren't getting volunteers on it 17:23:51 sdague, if anyone else from the core team are not able to be the point for that and if you are all ok with that, I would be ok to help with the bug triage thing 17:23:52 yep 17:24:10 #topic Blueprints 17:24:12 adalbas: yes, +1. I'm totally happy if yuo wanted to lead the bug triage effort 17:24:42 oops maybe jumped the gun a bit there 17:24:53 mtreinish: nope, it's all good 17:25:02 let's move on 17:25:03 are there any outstanding blueprints that need attention? 17:25:09 I have https://blueprints.launchpad.net/tempest/+spec/input-scenarios-for-scenario 17:25:44 #link https://blueprints.launchpad.net/tempest/+spec/input-scenarios-for-scenario 17:26:11 it's something I suggested we could do at the summit, sdague mentioned it would be a good idea, so I started working on it 17:26:27 andreaf: ok, cool 17:26:50 so I'm moving that into accepted 17:26:50 andreaf: so just need the bp approved? 17:26:59 do you have a WIP up? 17:27:17 giulivo: +1 17:27:36 giulivo: you just volunteered... 17:27:37 mtreinish: I only have it locally, I can upload a WIP 17:28:16 andreaf: ok cool, that would help my understanding of what you're doing there 17:28:17 andreaf: cool, would be great 17:28:23 but it sounds like everyone is on board 17:29:10 sdague, mtreinish: ok cool 17:29:22 ok are there any other bp status updates that people have 17:29:49 mtreinish: Just that the negative test stuff is not really moving quickly 17:30:04 mtreinish: There will probably not be any progress until after the new year 17:30:15 dkranz: is there a bp registered for that one? 17:30:28 sdague: I thought there was but I'll check 17:30:50 #link https://blueprints.launchpad.net/tempest/+spec/negative-tests 17:30:50 dkranz, , https://blueprints.launchpad.net/tempest/+spec/negative-tests ? 17:31:03 Yes 17:31:27 dkranz, i m working with kein omich and others to get the separate files for negative tests 17:31:35 I've been swamped and Marc is out until new year 17:31:40 the patches are added on that blueprint 17:31:41 so we should probably make 2 blueprints 17:31:44 adalbas: Yes, but that is not what this is about 17:31:53 adalbas: can you make another blueprint on the separation? 17:31:54 There should be 2 17:32:04 sdague, dkranz , yes 17:32:08 we'll put them both into an approv state 17:32:11 i can separate them 17:32:24 i ll line that up with the others working on that 17:32:44 sdague, adalbas I can draft and approve it 17:32:58 you meant to have two one for "autogeneration" and one for "separating negative files" right 17:33:01 ? 17:33:06 yes giulivo 17:33:11 right 17:33:57 ok, are there any other blueprints we need to discuss, otherwise let's move on to the next topic 17:34:37 #topic Neutron testing 17:34:46 ok mlavalle your name is on this one 17:34:58 yeap 17:35:16 Over the past few days I've been conducting a gap analysis for API tests 17:35:37 I put the results in the API section of https://etherpad.openstack.org/p/icehouse-summit-qa-neutron 17:36:11 I've finished with core and l3 api's and plan to continue with L4 - L7 over the next few days 17:36:18 great 17:36:47 so are we at the point of turning on the full runs non voting on normal (non experimental) neutron runs? 17:36:59 I will also create a wiki page for new contributors, walking them through the process of writing api tests for Neutron 17:37:01 i.e. how many fails are we down to now? 17:37:10 sdague: I thought we weren't going to do that until parallel worked 17:37:34 ah, right, it will get too long 17:37:47 That wiki will be done early next week and when it';s done i'll advertise it in the ML 17:37:49 mlavalle: yeah, that would be helpful. You've got ~52 tests listed there right? 17:37:57 mlavalle: great 17:38:31 Last nigh I made a little change to the Neutron wiki https://wiki.openstack.org/wiki/Neutron 17:38:51 Look at the Related Projects section close to the top 17:40:03 I am also attending every week the Neutron IRC meeting, so I keep them abreast of what is going on with QA 17:40:38 so do we know where we stand on parallel 17:40:49 salv-orlando: ^^^ 17:40:51 Do we able to create a shared network in parallel safe way? Most of tempest test assumes there is only one network visible in the tenant 17:40:56 i.e. agent rewrite 17:41:03 hi 17:41:05 salv-orlando is working on several patches to Neutron to enable that 17:41:34 he sent an email to the ML a couple of day ago with an update on the status 17:41:44 I know where we stand. To cut a long story short we have fixed the OVS agent, which does not have anymore tremendous lag for wiring ports, but.. 17:42:35 … there is also an issue with the DHCP agent, which takes about 40 seconds in updating the hosts file under load and this causes a failure since the cirros vms send 1 dhcp discover per minute 17:43:05 I am working on that too. So far I am pushing all the stuff I have in a single patch, where I run the parallel job 17:43:21 as soon as I have 10 green runs in a row I will push the patches for review with unit tests and stuff 17:43:24 end of the updated. 17:43:53 salv-orlando: ok, very cool 17:43:56 salv-orlando: based on my work with bug 1251448, we also need to worry when we devete ports 17:43:58 Launchpad bug 1251448 in neutron "BadRequest: Multiple possible networks found, use a Network ID to be more specific. " [High,New] https://launchpad.net/bugs/1251448 17:44:00 delete 17:44:22 and btw, I reckon that with armax's fix for tempest not relinquishing public IPs + another proposed fix for making the public network a /24 we shall be enable to enable full isolation 17:44:35 mlavalle: update my outside of the meeting with this issue, please 17:44:48 salv-orlando: cool 17:44:54 salv-orlando: ok, when that goes through let me know and we can push through turning that on again 17:45:41 is there anything else on the neutron testing front? 17:45:51 not on my side 17:45:59 ok then let's move on 17:46:07 #topic Critical Reviews 17:46:19 does anyone have any reviews they'd like to bring up? 17:46:35 I actually have one today: https://review.openstack.org/#/c/58604/ 17:46:45 mtreinish: I don't think we should discuss it now but please look at https://review.openstack.org/#/c/54948/ 17:47:02 #link https://review.openstack.org/#/c/54948/ 17:47:07 mtreinish: which is a disupte about parallelism and tenant isolation requirements 17:47:09 #link https://review.openstack.org/#/c/58604/ 17:47:30 dkranz: ok, I'll look at it after the meeting 17:48:12 on the review I posted it has 2 -1's but I responded to them with no response so if someone else could take a look at them I'd appreciate it 17:48:22 (the -1s probably mean that no one looks at it) 17:48:30 are there any other reviews to bring up? 17:48:34 https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/nova-v3-api-tests,n,z 17:48:49 it would actually be nice to move those forward, they have been there for a while 17:49:00 and remain about 1/3 of our queue 17:49:10 mtreinish: Over he weekend I will submit a patch to add tenant isolation to the Neutron base test. We will need to approve that quickly to enable new contributors to work on Neutron API tests. I'll ping you when it's in Gerrit 17:49:12 oh, only 20% now 17:49:27 sdague: I have been reviewing these 17:49:28 mlavalle: that is already up on gerrit 17:49:43 sdague: And will continue and hopefully be done by tomorrow 17:49:44 mlavalle: https://review.openstack.org/#/c/53459/ 17:49:57 oh, so maybe someone picked it up from the ether pad…. great! 17:49:58 I guess the only other issue is we really need to not be +2ing heat patches that we can't run 17:50:10 I've been -1ing the stuff that modifies heat slow jobs 17:50:19 because that doesn't run at all yet 17:50:39 sdague: The problem is that they don't run due to an infrastructure issue 17:50:45 hopefully the f20 release next week will let stevebaker get the infrastructure working 17:50:48 sdague: Tempest tests are a high priority for that team 17:50:57 sdague: I hope so 17:51:10 dkranz: then getting the infrastructure working to run them, or running 3rdparty testing needs to be a priority 17:51:16 sdague: If not then we either have to accept them or force people out of upstream 17:51:30 sdague: I'm not really sure what the blocker is 17:51:38 sdague: I'll inquire about that and get a status 17:51:44 dkranz: we can't really add tests until we can confirm they work. We've had issues with that in the paste 17:51:51 s/paste/past/ 17:51:52 so we actually went through it on IRC 17:52:08 disk-image-builder needs pip 1.5 to install 17:52:12 which is not yet released 17:52:13 mtreinish: We have, but this is a little different because they only run in the expewrimental queue 17:52:20 because they need real guests 17:52:21 mtreinish: ANd won't be lit up until they all run 17:52:34 dkranz: no it's not different 17:52:47 the point is we'd have a ton of tests that we have no proof in the community they ever ran 17:53:13 it's the same reason we don't accept tests that start skipped 17:53:31 dkranz, sdague: lets save this for after the meeting. We're down to 7 min. and dkranz still has a topic on the list 17:53:36 sure 17:53:39 sdague: I don't completely agree but let's see if it can be resolved soon and made moot 17:53:51 #topic How can we make sure reviews don't slide >1 week 17:54:00 mtreinish: I think I can withdraw my topic for now since we are doing ok 17:54:00 dkranz: you're up 17:54:14 dkranz: heh, ok cool 17:54:21 then let's open the floor 17:54:22 We should just all make sure to focus on old reviews first 17:54:29 #topic open discussion 17:54:30 even thought it is a pain 17:54:42 dkranz: yeah I always start at the bottom 17:54:42 mtreinish: Question about neutron 17:54:57 dkranz: ok, shoot 17:55:06 yeh, I do as well. though I'm trying to actually start with my review list, so I'm not missing coming back to things 17:55:07 How many of us are comfrotable reviewing neutron complex scenarios for functional correctness? 17:55:22 I'm not 17:55:33 dkranz: I typically try to ask someone from neutron core to comment 17:55:54 dkranz: yeah the same with me, I'm not super familiar with it at this point 17:56:08 dkranz: I can help with it as well 17:56:08 sdague: That is the right answer but it might help to have a slightly more formal way to spread the load on the neutron team for this 17:56:10 that's actually on my list after sorting out volunteers on the tempest items 17:56:24 sdague: ok, great 17:56:42 sdague: I don't want reviews to slide because we are not comfortable pushing them forward 17:56:43 dkranz: so I think the actual solution there is getting one or more point people in each core project that are there to help 17:56:50 sdague: Right 17:57:02 sdague dkranz +1 from on that 17:57:07 sdague: That's the only way that scales 17:57:08 *from me 17:57:22 which I'm going to run on, once I have volunteers signed up for the tempest tasks. We need our house in order first before we ask others for things 17:57:24 sdague: Never mind our three new projects :) 17:58:18 dkranz, sdague: o/ potential marconi-tempest contributor snooping 17:58:25 malini: great 17:58:46 ok, we're about to loose the room 17:58:58 yeah, lets end it here for today 17:59:01 thanks everyone 17:59:07 #endmeeting