17:00:47 #startmeeting qa 17:00:52 Meeting started Thu Oct 23 17:00:47 2014 UTC and is due to finish in 60 minutes. The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:53 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:00:55 The meeting name has been set to 'qa' 17:01:01 hi, who's here today? 17:01:11 o/ 17:01:15 o/ 17:01:27 o/ 17:01:30 #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_October_23rd_2014_.281700_UTC.29 17:01:35 ^^^ Today's agenda 17:02:34 ok let's get started, maybe more people will trickle in later 17:02:41 #topic Blueprints 17:03:11 So I still need to go through the list of open juno bps and move them to kilo 17:03:22 but that's just administrative nothing will change with those 17:03:43 mtreinish: I closed the clients-check-success because it is superseded by clients-return-one-value 17:04:09 dkranz: ok, I was going to ask you for the relationship between the specs as a review comment 17:04:27 dkranz: do you have a change up for dropping the success spec? 17:04:42 mtreinish: I just closed it as Implemented 17:05:15 dkranz: so we're going to call it implemented, then I guess we should move it the implemented dir 17:05:27 mtreinish: It was not completely finished but there is no point in doing that for the remaining few clients separately from the return-one-value 17:05:51 mtreinish: Yes, we should move it or discard it. 17:06:20 dkranz: well that's what I was asking about :) If the status in lp is implemented I say let's just move it 17:06:30 mtreinish: works for me 17:06:45 mtreinish: It was a weird case because it was superseded after being mostly implemented :) 17:06:57 dkranz: ok I'll add it to my pending change doing that 17:07:21 dkranz: and I'll review the new spec this afternoon (sorry I haven't gotten to it yet) 17:07:37 mtreinish: ok, great 17:08:00 ok does anyone have any open blueprints that they would like to update status on or discuss? 17:08:28 mtreinish: resource-cleanup 17:08:45 andreaf: oh, yeah I need to review the framework patch 17:08:48 andreaf: go ahead 17:08:49 mtreinish: I'm waiting for feedback on the proposed framework before I can continue to part 2 17:09:04 mtreinish: yes that's it 17:09:16 andreaf: do you have a link for the minutes 17:09:26 mtreinish: coming... 17:09:39 #link https://review.openstack.org/#/c/115353/ 17:09:49 cool, thanks 17:10:21 Also the ssh-auth one, which is actually a spec, not yet a bp, but I'd love to get input on that - perhaps it's something we could discuss as well on Friday at the summit 17:10:53 andreaf: oh, yeah we punted on that until kilo (which I guess it is now :) ) 17:11:08 andreaf: if you've got another link... 17:11:25 #link https://review.openstack.org/94741 17:11:28 andreaf: About the test-accounts, now that it is finished, is the plan to make using them the default in the gate? 17:11:37 dkranz: it's not ready for that yet 17:11:47 there still some missing functionality before we could gate on it 17:12:03 dkranz, mtreinish: indeed - but I plan to prepare a spec for what's missing 17:12:11 after that's done it's definitely worth discussing 17:12:14 mtreinish: ok, is it ready to use in real deployments? 17:12:30 so we can keep the code exercised 17:12:30 mtreinish: I would like to try but not if it is known to have major issues 17:13:03 dkranz: it has the same networking limitations that non-isolated did. Which can get confusing for multitenant users 17:13:28 but for the non-admin auth side it should work fine as long as you're not relying on networking 17:13:39 dkranz: and also if you run in parallel with pre-provisioned accounts you'll skip admin tests 17:13:40 (admin being the other reason we can't use it in the gate yet) 17:14:01 mtreinish: ok, I thought that if each user had their own network it would be ok 17:14:22 mtreinish: anyway, if there are issues I will raise them outside this meeting 17:14:29 dkranz: yeah if the default is configured sanely it should work, it's just for tests that use fip it might not 17:14:38 there's a assumption that all user will belong to the same network 17:15:02 andreaf: ok. That is the same issue as with nova-network vlan which also does not work 17:15:20 We should really fix that 17:15:27 that'll be part of the second half 17:15:35 mtreinish: ok, fair enough 17:15:51 allowing the network to be specified in the yaml for each cred set 17:15:52 dkranz: yes it shouldn't be to difficult in fact, only we wanted to start with something not too complex at once 17:16:51 ok, moving on I have an update on the tempest-lib bp 17:17:19 finally after about 2 weeks of fighting pypi, requirements, and other issues tempest-lib usage is ready to land in tempest 17:17:29 #link https://review.openstack.org/117649 17:17:38 #link https://review.openstack.org/122469 17:18:21 which switch the cli tests to use tempest-lib and switches subunit-trace to be the tempest-lib copy 17:19:46 on that note I also pushed out the tempest-2 tag to signal the opening of kilo dev 17:20:26 although I guess that's really unrelated :) 17:20:49 ok if there aren't any other bp updates I guess we can move on 17:21:43 #topic Devstack 17:21:49 dtroyer: are you around? 17:22:07 the only recent devstack thin I'm aware of was that stable/juno was cut 17:23:32 ok well if dtroyer shows up later we can come back to devstack 17:23:43 #topic Grenade 17:24:18 so the big update here is thanks to clarkb we've migrated grenade to do juno -> master and icehouse->juno 17:24:53 although during the period we were gating on icehouse->master nova managed to land an upgrade breaking change 17:25:19 * dtroyer runs in at the sound of his name 17:25:42 dtroyer: heh, are there any update on devstack or grenade? 17:27:13 devstack also has a stable./juno cut. otherwise its mostly shaping stuff up for summit. I have a couple of topic we can talk about if nothing more immediate comes up in other sessions 17:28:05 I'm also going to get the bug/bp launchpad stuff under control Real Soon Now… 17:28:29 dtroyer: oh yeah I still need to draft the ML post for devstack and grenade qa-specs usage 17:29:38 ok if there's nothing else on devstack and grenade I guess we'll move on 17:29:49 thats it, thanks 17:30:11 #topic Bugs 17:30:21 #link https://etherpad.openstack.org/p/Tempest-bug-report 17:30:43 I think those numbers are a bit out of date 17:30:57 but it looks like we're finally close to 0 new bugs 17:31:39 we still have ~190 open bugs but getting the incoming triage under control is a good first step 17:32:14 next week's triage volunteer will be gmann: https://etherpad.openstack.org/p/qa-bug-triage-rotation 17:32:35 does anyone have anything else to discuss about bugs? 17:33:59 ok then I guess we'll move on 17:34:08 #topic Critical Reviews 17:34:30 does anyone have any reviews they'd like to bring up? 17:36:16 wow, must be a first no reviews that need extra attention :) 17:37:06 ok, then I guess I'll open the floor 17:37:14 #topic Open Discussion 17:37:29 does anyone have a topic they'd like to discuss that wasn't on the agenda? 17:38:18 mtreinish: My cross-project proposal was discussed positively in the tc meeting. 17:38:20 mtreinish: do you still have open slots at the summit? 17:38:28 andreaf: 1 17:38:30 mtreinish: The one about "impact of moving func tests to projects" 17:38:46 assuming dkranz's topic gets picked up in the cross-project track 17:39:01 mtreinish: Actually it was mentioned that that was the only test-related proposal. 17:39:35 if it does and there isn't another proposal for the slot, I'll give the slot to infra or someone else 17:39:41 dkranz: hmm, that's surprising 17:40:09 mtreinish: I was wondering if discussing what level of validation we want in API tests is worth a session - i.e. do we want all tests to check the created server have proper networking, that volumes attached can actually be seen attached to a VM etc 17:40:43 andreaf: basically a session on your ssh auth spec? :) 17:40:44 andreaf: I think that is part of the "what is tempest after moving fun tests to projects discussion" 17:41:02 mtreinish: I think it is more general than that. 17:41:43 mtreinish, dkranz: well yes the spec is about how to do it, but I wanted to discuss if / where we want to do it 17:41:47 mtreinish: I think we should double up the "future of tempest" slot rather than giving it away 17:42:13 andreaf: right, that was my point I think 17:42:14 80min is really a lot of time for one topic 17:42:28 I'd rather just use part of friday if we can't fit it all into 40 17:42:34 there is this constant problem that tests doing ssh to a VM is less reliable - they are more complex granted but maybe it's also because we do not test it enough in the gate 17:43:01 mtreinish: I think "future of tempest" with folks from outside our group could easily take more than 40 min :) 17:44:08 andreaf: The problem is that it is flaky 17:44:31 andreaf: If ssh was expected to be reliable I would be all for doing it always 17:45:11 I had one more topic 17:45:31 dkranz: there may be other ways to do deeper validation which do not rely on ssh, perhaps custom image logging things on console would be another option 17:45:39 dkranz, go ahead 17:45:43 We should really have many more periodic jobs running various "stress" tests 17:46:18 dkranz: as a summit topic or for discussion now? 17:46:35 I meant discussion now 17:46:47 It is just a matter of who will watch them. 17:46:51 ok, because andreaf has a summit session that too :) 17:46:59 oh, I didn't see that 17:47:08 mtreinish: Is it on the etherpad? 17:47:32 dkranz: QA and CI after merge 17:47:42 dkranz, I have a session about post merge QA 17:47:56 it's not exactly about stress tests, but the same basic issues will be discussed 17:48:01 mtreinish: ok, we can discuss it then. 17:48:16 dkranz: what we really need to have is a dashboard view for the results 17:48:34 mtreinish: yes a dashboard would help 17:48:36 which shows the job history and trends over time 17:48:44 mtreinish: Yes, and a rotation of people to watch 17:48:57 the problem is someone needs to step up to make it 17:49:13 mtreinish: but it could even something like if a job displays signs of definite bit-rot, start voting -1 or patches or so 17:49:58 andreaf: I think that would be much harder to do 17:50:58 mtreinish: it would be harder to implement? Yes but it's something that we need I think, else the gate won't scale in future 17:51:48 mtreinish: ok we discuss this at the summit 17:52:07 andreaf: both to implement and it also very different from what we say gating is now 17:52:10 andreaf: sure 17:52:46 dkranz: anyway I think it would be fine to add more varied stress jobs now we can figure out the reporting/watching side at summit 17:54:19 ok if there's nothing else I guess we'll end here 17:54:22 thanks everyone 17:54:35 #endmeeting