17:00:55 <jaypipes> #startmeeting qa
17:00:56 <openstack> Meeting started Thu Oct 11 17:00:55 2012 UTC.  The chair is jaypipes. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:57 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:58 <openstack> The meeting name has been set to 'qa'
17:01:40 <jaypipes> torandu, davidkranz, donaldngo: hello!
17:01:45 <jaypipes> dwalleck: hello!
17:02:00 <torandu> hello!
17:02:41 <jaypipes> so... our gate has been working well it seems...
17:02:51 <davidkranz> hi
17:03:05 <jaypipes> jeblair set up a nightly job that runs the full suite, IIRC
17:03:19 <dwalleck> howdy!
17:03:22 <davidkranz> jaypipes: Yes. I was going to push it to all projects but won't get to that until after the summit.
17:03:36 <dwalleck> Sorry, was away
17:03:55 <jaypipes> no worries
17:04:06 <jaypipes> dwalleck: so, see my latest email on the swift stuff?
17:04:06 <davidkranz> One concern about running the full gate on all projects is that it will get longer every time a new test is added.
17:04:07 <jeblair> #link https://jenkins.openstack.org/job/periodic-tempest-devstack-vm-cinder/
17:04:15 <jaypipes> jeblair: rock on brother. :) thx!
17:04:43 <dwalleck> jaypipes: How latest? If it was less than an hour ago, no
17:05:19 <davidkranz> jeblair: What do you think a time limit for the full gate should be?
17:05:34 <dwalleck> If it was yesterday, I pushed my changes yesterday afternoon
17:05:46 <jaypipes> dwalleck: ah, k.
17:05:55 <dwalleck> right after you guys pinged me
17:06:01 <jaypipes> dwalleck: yeah, I was out sick yesterday afternoon. so didn't notice it. will get right on it.
17:06:56 <jaypipes> jeblair: looks like that periodic job is only running smoke tests in tempest... I think we wanted to run the whole suite?
17:07:20 <davidkranz> jaypipes: Yes, and I had suggested running it multiple times in a row.
17:08:02 <jeblair> davidkranz: i don't have good data to say based on patch-throughput or another systemic basis; i'd say the main concern is developer tolerance for feedback delay.  for that, maybe an hour?
17:08:31 <davidkranz> jeblair: OK, so we are safe for a while.
17:08:37 <jeblair> jaypipes, davidkranz: can you guys hack up that change then?
17:08:41 <jaypipes> jeblair: we are well below that... I think it's about 700 seconds now
17:08:49 <jaypipes> jeblair: yes
17:09:02 <jaypipes> jeblair: I'll submit a patch to the openstack-ci-puppet stuff
17:09:11 <jeblair> jaypipes: great
17:09:12 <davidkranz> jeblair: At some point we might want to not run swift tests as a gate on nova checkins for example.
17:09:25 <jaypipes> davidkranz: ++
17:10:14 <davidkranz> jaypipes: With the nightly run in place we can adjust the tradeoff between gating and the nightly build as approproate.
17:10:24 <jaypipes> yep
17:10:43 <davidkranz> For now, our problem is still not enough tempest tests :)
17:10:44 <jeblair> davidkranz: i see the desire, as long as we keep in mind it's an integration test designed to catch errors where projects interact.
17:11:06 <dwalleck> jeblair: ++
17:11:08 <davidkranz> jeblair: Right.
17:11:39 <jaypipes> other than the swift stuff and a couple smaller reviews to do for dwalleck (compute cleanup) and torandu (floating IP assignment bug), the other review is from Rohit.
17:11:51 <jaypipes> we should probably give that a full review ASAP today or tomorrow
17:12:04 <jaypipes> 'since it's been around for a while and I don't want rohit to get discouraged.
17:12:14 <jaypipes> it's tests for the live-migration functionality
17:12:24 <jaypipes> which is a big deal and is a good addition IMHO
17:12:26 <dwalleck> I thought Rohit's patch was pending his changes still
17:12:39 <dwalleck> Same with torandu
17:12:42 <davidkranz> jaypipes: I reviewed the live-migration
17:12:55 <jaypipes> doink, sorry, it's not rohit.
17:13:00 <jaypipes> it's lakat
17:13:09 <davidkranz> jeblair: Can you say why https://review.openstack.org/#/c/13101/ is stuck in its current state?
17:13:18 <davidkranz> jeblair: That's the live migration.
17:13:40 <davidkranz> jeblair: There were several reverify but nothing happend. No dependencies are listed.
17:14:20 <jeblair> davidkranz: it's not in zuul's queue.  it may have slipped through a restart
17:14:37 <davidkranz> jeblair: Hmm. How do we kick it?
17:14:49 <jeblair> davidkranz: i'm reverifying now, i'll see if it shows up or hits an error
17:14:57 <jaypipes> jeblair: thx
17:14:58 <davidkranz> jeblair: Great. Thanks.
17:15:31 <jaypipes> OK, davidkranz, anything you want to bring up regarding the QA track next week?
17:15:59 <davidkranz> jaypipes: Not really. I will be happy if we make progress on configuration testing and bringing qa into the open.
17:16:08 <jeblair> oh, you want "recheck" not "reverify"
17:16:10 <davidkranz> jaypipes: THey are related I think.
17:16:14 <jaypipes> k
17:16:16 <jeblair> it's running now.
17:16:22 <jaypipes> jeblair: ah, thanks.
17:16:32 <davidkranz> jeblair: What is the rule for  which to use>?
17:16:40 <davidkranz> jeblair: reverify vs recheck
17:16:57 <jaypipes> davidkranz: re: configuration testing, you are referring to testing different configurations of OpenStack clusters with Tempest? Or something else?
17:16:59 <jeblair> recheck runs the patch-upload tests, reverify runs the merge gate (and tries to merge)
17:17:27 <davidkranz> jaypipes: Yes, what you said. THere are a few more details in the session description.
17:17:56 <jeblair> (so reverify won't do anything if the change doesn't have +2 code review and +1 approved votes)
17:18:17 <davidkranz> jaypipes: The other thing is that there is no scheduled time for a QA group meeting. We should probably set some unconference time for that.
17:18:21 <davidkranz> jeblair: Got it.
17:18:30 <jaypipes> davidkranz: ++
17:19:15 <jaypipes> davidkranz: I'd really love to see a dicsussion on using DTest and/or PyVows as well.
17:19:34 <dwalleck> davidkranz: ++. I think all the sessions are really going to lead into a much larger discussion "where do we go from here" discussion
17:19:37 <davidkranz> jaypipes: I think there is a session that is supposed to include that.
17:20:06 <dwalleck> jaypipes: I wouldn't mind talking about PyVows. I should push what I've been tinkering with into my repo so it's actually visible
17:20:24 <davidkranz> jaypipes, dwalleck : How about Wednesday after lunch?
17:20:41 <dwalleck> sure
17:20:42 <davidkranz> It's probably better to meet after the QA sessoins.
17:21:26 <davidkranz> We can carve out more time on Thursday dynamically if we need it.
17:21:31 <jaypipes> davidkranz: works for me.
17:21:35 <jaypipes> either..
17:22:31 <davidkranz> I will post something on the unconference board for Wed after lunch.
17:23:07 <jaypipes> davidkranz, dwalleck: Also, one thing that we (AT&T) might be able to show is how we have set up our deployment testing in CI. i.e. Testing Chef cookbooks and automated deployment code in Jenkins.
17:23:23 <jaypipes> might be useful to see what we're doing to include that stuff in our testing plans
17:23:34 <davidkranz> jaypipes: That would be great.
17:23:58 <davidkranz> jaypipes: Is that related or comparable to smokestack in any way?
17:24:02 <jaypipes> I know not everyone uses Chef, of course, but the way we're doing stuff is pretty interchangeable with Puppet or Juje
17:24:03 <dwalleck> That'd be great to see
17:24:10 <jaypipes> davidkranz: not comparable...
17:24:41 <jaypipes> davidkranz: it tests/validates the actual Chef cookbooks (ChefSpec) as well as our Cobbler/IPMI autoconfiguration etc
17:24:54 <davidkranz> jaypipes: If we got more volunteers for configuration testing is this something they might use?
17:25:01 <jaypipes> davidkranz: absolutly.
17:25:06 <davidkranz> jaypipes: Cool.
17:25:27 <jaypipes> absolutly == Absolutely, Vodka-flavored.
17:25:28 <davidkranz> jaypipes: I suggest you say a few words about it at the config testing session as well.
17:25:41 <jaypipes> davidkranz: will do! both me and Toanster
17:25:45 <jaypipes> oops, torandu
17:26:20 <jaypipes> OK, the only other topic I had was how we are currently not handling testing of stable/XXX branches well at all.
17:26:47 <jaypipes> and I'm not sure about the priority of maintaining those stable/XXX branches at this point.
17:27:04 <davidkranz> jaypipes: Well, we certainly need to maintain stable/folsom.
17:27:23 <dwalleck> davidkranz: ++. That would be a good start
17:27:52 <davidkranz> I think there is already a daily build for the stable branches whose results are not looked at by any one.
17:28:27 <jaypipes> davidkranz, dwalleck: By "we", who do you refer to? :)
17:29:02 <jaypipes> davidkranz, dwalleck: My personal focus is on trunk -- I don't imagine I will have much time to focus on differences between stable releases (especially X - 2 and older)
17:29:45 <davidkranz> jaypipes: By maintain I meant dealing with any fallout if a checkin to a stable/folsom project branch causes tempest to fail.
17:30:23 <davidkranz> jaypipes: I don't think we should be backporting new tests as a main line activity.
17:30:28 <jaypipes> davidkranz: k
17:30:31 <dwalleck> jaypipes: Whomever thinks its important. I think folks who are trying to stay with stable branches might find it difficult to do so without some type of testing. I also don't know how large of an audience that is
17:30:37 <davidkranz> jaypipes: Though others of course are free to do so.
17:30:39 <jaypipes> davidkranz: much agreed about not backporting.
17:31:36 <davidkranz> jaypipes: So what is the issue, then?
17:32:40 <davidkranz> jaypipes: We are gating stable branches on stable tempest, right?
17:32:44 <jaypipes> davidkranz: oh, I was just thinking about Armando's recent merge request that we basically said "sorry, we're not really adding anything to stable tempest releases, only fixing things"
17:33:12 <davidkranz> jaypipes: OK.
17:33:28 <jaypipes> davidkranz: wanted to make sure I wasn't smoking too much Thai stick when I responded that we weren't looking to accept *new* stuff to stable Tempest releases.
17:33:41 <davidkranz> jaypipes: In that case I think we need a "stable tempest" "team" like there is for other projects.
17:34:04 <jaypipes> davidkranz: agreed, but I don't think I can take part in that team -- if I'm being honest... I just don't have the time.
17:34:18 <davidkranz> jaypipes: I think we should allow people to backport tests to stable branches
17:34:33 <jaypipes> davidkranz: so perhaps it is worth making that team official at next week's summit and broadcasting the intent to maintain a team for stable releases to the broader community
17:34:35 <davidkranz> jaypipes: As long as there is a way we don't have to review all of them.
17:34:50 <davidkranz> jaypipes: ++
17:35:18 <jaypipes> k. well, that is all the things I wanted to bring up today... anybody got anything else?
17:35:31 <dwalleck> nothing here, just heads down
17:35:40 <davidkranz> jaypipes: Not for me.
17:35:44 <jaypipes> torandu: ?
17:35:46 <jaypipes> anything?
17:36:08 <torandu> no. i've been following along. all sounds good
17:36:32 <davidkranz> To all going to the summit, see you there!
17:36:46 <davidkranz> davidkranz: Got to run now...
17:36:54 <jaypipes> see ya
17:36:56 <torandu> later!
17:36:59 <jaypipes> #endmeeting