17:00:27 <jaypipes> #startmeeting qa
17:00:28 <openstack> Meeting started Thu Jan  3 17:00:27 2013 UTC.  The chair is jaypipes. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:30 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:32 <openstack> The meeting name has been set to 'qa'
17:01:02 <zyluo> hi
17:01:20 <jaypipes> Hi all, welcome to 2013's first QA meeting :)
17:01:23 <jaypipes> #link https://etherpad.openstack.org/QAMeeting
17:01:32 <jaypipes> I put a quick agenda in link ^^
17:01:38 <jaypipes> davidkranz_: around?
17:01:40 <mlavalle> hi
17:01:47 <jaypipes> mlavalle: welcome :)
17:01:55 <jaypipes> Ravikumar_hp: morning.
17:01:56 <davidkranz_> jaypipes: Here. Been kind of busy this past week+.
17:02:07 <Ravikumar_hp> hi
17:02:11 <jaypipes> davidkranz_: no worries, we all have I think, with holidays and all..
17:02:11 <sdague> <- here
17:02:25 <sdague> glorious holidays :)
17:02:30 <mtreinish> I'm here too
17:02:34 <jaypipes> mtreinish: hi! :)
17:02:48 <jaypipes> and afazekas is here as well.. let's get started.
17:03:01 <afazekas> hi!
17:03:02 <jaypipes> #topic Reviews: https://review.openstack.org/#/q/status:open+project:openstack/tempest,n,z
17:03:39 <jaypipes> Why don't we tackel the quantum ones first.
17:03:53 <jaypipes> is mnewby around?
17:03:57 <jaypipes> re: https://review.openstack.org/#/c/18507/
17:04:20 <jaypipes> mnewby: the above needs a quick rebase... other than that, ready to go.
17:04:21 <afazekas> Just needs a rebase
17:04:24 <jaypipes> right.
17:04:52 <jaypipes> #action send mnewby a quick email about his rebase
17:05:01 <jaypipes> Next Quantum one is here: https://review.openstack.org/#/c/9507/
17:05:08 <jaypipes> zyluo: that's yours, right?
17:05:25 <zyluo> jaypipes, yeah I'm working on it
17:05:41 <jaypipes> zyluo: http://logs.openstack.org/9507/6/check/gate-tempest-devstack-vm/22729/console.html.gz
17:05:46 <zyluo> could you -2 on this?
17:05:47 <jaypipes> zyluo: that's a simple one to fix...
17:05:58 <jaypipes> zyluo: s/nova/compute
17:06:08 <zyluo> I'll start my own patch..yeah
17:06:18 <jaypipes> zyluo: we underwent a naming change a little while back and I think you just need to update nova to compute...
17:06:29 <jaypipes> zyluo: easiest way is to do this:
17:06:40 <zyluo> jaypipes, will do
17:06:44 <jaypipes> git review -d 9507
17:07:13 <jaypipes> zyluo: ^^ will check out the original branch in gerrit locally. make changes, then git commit -a --amend && git review
17:07:30 <zyluo> jaypipes, thanks! will try
17:07:34 <jaypipes> np :)
17:07:51 <jaypipes> zyluo: feel free to ping me later if you run into questions/issues
17:07:58 <zyluo> got it
17:08:18 <jaypipes> kk, next review.
17:08:20 <jaypipes> #link https://review.openstack.org/#/c/18324/
17:08:23 <jaypipes> Ravikumar_hp: ping
17:08:46 <Ravikumar_hp> jaypipes: new patch is submitted . will get it reviewed
17:08:50 <jaypipes> Ravikumar_hp: you want to check in with Rajalakshmi and see where she is on this one?
17:08:58 <jaypipes> Ravikumar_hp: gotcha, cool. thx!
17:09:35 <jaypipes> sdague: next patch is https://review.openstack.org/#/c/18085/
17:10:00 <sdague> it's simple, people should just +2 it :)
17:10:02 <jaypipes> sdague: I have some reservations about whether this will work properly but I suppose the patch doesn't hurt anything...
17:10:07 <davidkranz> jaypipes: You had commented that this patch was not sufficient to shut the stuff up?
17:10:16 <sdague> yeh, it's helpful for local debug at least
17:10:22 <jaypipes> sdague: I've found that setting -s on nosetests really doesn't do anything to clean up the log mess...
17:10:30 <jaypipes> davidkranz: right, exactly.
17:10:49 <sdague> this was actually about letting things not shut up for fixing
17:11:00 <jaypipes> davidkranz, sdague: what I *think* needs to happen is that in setup_package() in tempest/tests/__init__.py, an initial call to logging.basicConfig() needs to occur.
17:11:03 <afazekas> I have no idea why is -s good, but it will not harm anybody in this form.
17:11:05 <sdague> we needed it to get to the bottom of the flakey behavior
17:11:40 <jaypipes> afazekas: no, I'm saying that -s doesn't actually capture  the log output properly...
17:11:53 <sdague> jaypipes: right, I wasn't actually trying to fix logging
17:12:09 <jaypipes> sdague: ok, then I suppose I'm not quite understanding thsi patch! :)
17:12:12 <sdague> I was trying to let me use prints to debug.
17:12:23 <davidkranz> jaypipes: I think -s just forces the output when it happens.
17:12:33 <davidkranz> Instead of capturing it and printing it at the end.
17:12:34 <afazekas> -s is for printing to the stdout, I assume it is for "print" debugger technics
17:12:42 <sdague> yes, and lets it out for tests that pass
17:12:51 <sdague> because otherwise we only get output for tests that fail
17:12:52 <jaypipes> davidkranz: I haven't seen any difference between running with and without -s... but if you say so, oK :)
17:13:11 <davidkranz> jaypipes: I have seen differences and I think I suggested this to sdague
17:13:21 <jaypipes> sdague: ah, ok. I still see output for non-failures (output from glanceclient, for instance) even when not using -s
17:13:39 <jaypipes> sdague: so I think there's two separate issues...
17:13:58 <davidkranz> jaypipes: You see it without -s, but not until the dashed line stuff at the end.
17:14:04 <jaypipes> sdague: and this perhaps fixes the first -- which is to always output all stdout messages to the screen even if tests succeed.
17:14:15 <jaypipes> davidkranz: gotcha.
17:14:30 <jaypipes> sdague: ok, davidkranz, want to +1 Approve it?
17:14:41 <jaypipes> sdague: sorry I delayed this one so long...
17:14:49 <davidkranz> Yes. I have been waiting for your comments.
17:14:50 <sdague> no worries
17:15:35 <davidkranz> jaypipes: Done.
17:15:47 <jaypipes> ok, off to the tarlac pit it goes then.
17:16:03 <sdague> heh
17:16:15 <jaypipes> alright, I'm going to skip the three oslo ones and do those last...
17:16:23 <jaypipes> next review: https://review.openstack.org/#/c/18775/
17:16:36 <jaypipes> Ravikumar_hp: this is in your neck of the woods I think.
17:16:46 <Ravikumar_hp> jaypipes: new patch is submitted. under review
17:17:16 <jaypipes> Ravikumar_hp: ah, so this one makes the other review irrelevant?
17:17:16 <Ravikumar_hp> will take care of
17:17:43 <jaypipes> Ravikumar_hp: does https://review.openstack.org/#/c/18775/ make https://review.openstack.org/#/c/18324/ abandoned?
17:18:23 <Ravikumar_hp> flavor spec . new patch will be submitted tomorrow
17:18:50 <Ravikumar_hp> 18324 is not abandoned.
17:19:04 <Ravikumar_hp> will run the test as admin user
17:19:16 <jaypipes> Ravikumar_hp: OK, understood.
17:19:17 <Ravikumar_hp> negative test as normal user
17:19:36 <jaypipes> Ravikumar_hp: please mark the ones you are waiting on as Work in Progress.
17:19:44 <jaypipes> just to update their status..
17:19:47 <Ravikumar_hp> sure
17:20:03 <jaypipes> ty
17:20:27 <jaypipes> alrighty, next one is Swift: https://review.openstack.org/#/c/18794/
17:20:51 <Ravikumar_hp> jaypipes: Just got review feedback from afazekas
17:21:03 <Ravikumar_hp> need to check and incorporate
17:21:06 <jaypipes> Ravikumar_hp: right, looks like good feedback too...
17:21:18 <jaypipes> Ravikumar_hp: shall we try to get a swift-core member to review?
17:21:24 <jaypipes> Ravikumar_hp: prolly a good idea :)
17:21:28 <sdague> jaypipes: with mtreinish's changes going in, everything is going to need a rebase I think
17:21:41 <sdague> because tempest.openstack had to change
17:21:43 <jaypipes> sdague: understood.
17:21:57 <jaypipes> I will delay reaching out to swift-core on that one until the rebase is done, then.
17:22:17 <jaypipes> #action jaypipes to reach out to swift-core on https://review.openstack.org/#/c/18794/ after rebase is done
17:22:41 <jaypipes> Next review is Jaroslav's: https://review.openstack.org/#/c/18030/
17:22:45 <sdague> I'll go and run recheck on everything that will need to rebase after so that it will fail early
17:22:52 <jaypipes> jhenner: ping
17:22:59 <jaypipes> sdague: thx
17:23:15 <afazekas> Can you split the https://review.openstack.org/#/c/18794 to two patches ?
17:23:39 <afazekas> It has test cases without questions
17:24:15 <jhenner> jaypipes: pong
17:24:20 <jaypipes> I just did a recheck on jhenner's review... looks like he fixed the pep8 violation
17:24:23 <jaypipes> jhenner: ohai :)
17:24:36 <jaypipes> jhenner: no worries, all good to go on your review. sorry to bother!
17:25:00 <jhenner> jaypipes: ok
17:25:22 <jaypipes> alright, mtreinish and sdague, the remaining 3 reviews are all related to oslo (openstack-common)
17:25:30 <jaypipes> shall we discuss them as a group?
17:25:38 <mtreinish> jaypipes: sure
17:25:40 <sdague> jaypipes: I think mtreinish's are in the gate right now
17:25:48 <sdague> and monty's is obsolete
17:25:50 <jaypipes> mordred: you too, since https://review.openstack.org/#/c/17063/ is your baby.
17:25:57 <jaypipes> ahhh....
17:25:58 <mordred> uhoh
17:26:07 <mtreinish> mordred's patch has stuff with version which I didn't add in mine
17:26:20 <jaypipes> ok, so who needs to rebase? ;)
17:26:26 <mtreinish> mordred does
17:26:27 <jaypipes> first come first serve?
17:26:33 <jaypipes> hehe, ok, mordred..
17:26:35 <mordred> I'm fine deferring
17:26:40 <sdague> mordred: the others should land in an hour or so
17:26:46 <mordred> great
17:26:48 <jaypipes> mordred: looks like you'll need to rebase to add in the versions stuff
17:26:54 <sdague> whenever full gate gets through
17:26:55 <jaypipes> mordred: danke
17:27:14 * mordred looks around and says "testr?", then ducks back into his hole
17:27:15 <mtreinish> also mordred you copied version.py manually I used oslo's scripts to sync
17:27:31 <jaypipes> sdague: ok, so the other two are heading through the gate... so I suppose it's best to just wait the hour or so until that finishes and catch you online after?
17:27:58 <sdague> jaypipes: yeh, any time after they land modred can rebase
17:28:07 <jaypipes> cool.
17:28:27 <jaypipes> ok then, anybody have any review-related stuff before we move on to bugs and blueprints?
17:28:29 <sdague> mordred: cyeoh is looking at testr, he's on holiday now though.
17:29:03 <jaypipes> ok, moving on...
17:29:12 <jaypipes> #topic Bugs -- and cleaning up our mess
17:29:19 <jaypipes> #link http://bit.ly/w4i5td
17:29:29 <afazekas> I think we need test runner which is able to consider server and volume resurce usage when scheduling  a new test item
17:29:56 <jaypipes> afazekas: yes, that has already been agreed upon ;)
17:30:02 <jaypipes> afazekas: we hate nosetests, frankly...
17:30:11 <mordred> nosetests die die die
17:30:19 <jaypipes> afazekas: and a few people are actively looking at testr as a replacement
17:30:19 <mordred> testr for life!
17:30:30 <jaypipes> afazekas: but of course, easier said than done ;)
17:30:35 <afazekas> jaypipes: I have some ideas for test runner :)
17:30:59 <jaypipes> afazekas: please do talk with mordred, sdague and others about testr and your ideas!
17:31:12 <jaypipes> Alright, back to the #topic....
17:31:20 <afazekas> tempest needs some refactoring in order to make the resource uage followable
17:31:22 <mordred> afazekas: lifeless is actually the person you want - but loop sdague and I in
17:31:35 <jaypipes> We have a very large number of open bugs: http://bit.ly/w4i5td
17:31:48 <mtreinish> jaypipes: only 70...
17:31:54 <jaypipes> mtreinish: :P
17:32:08 <jaypipes> mtreinish: 10 for every active QA person! ;)
17:32:13 <sdague> :)
17:32:29 <jaypipes> all jokes aside, we're heading towards the G2 milestone release tomorrow
17:32:33 <afazekas> https://bugs.launchpad.net/tempest/+bug/1080971 this looks easy
17:32:35 <uvirtbot`> Launchpad bug 1080971 in tempest "Tenants for test are not deleted in teardown function." [High,Confirmed]
17:32:41 <jaypipes> and I think today and tomorrow would be a good time to do some bug cleanup.
17:32:59 <Ravikumar_hp> out of 70 some are task items (test development)
17:32:59 <sdague> jaypipes: that sounds reasonable, I'll try to help out there.
17:33:00 <jaypipes> afazekas: hold up one second.
17:33:23 <jaypipes> Ravikumar_hp: yes, indeed.
17:34:05 <sdague> I guess that raises another question, would it make sense to have a dedicated -qa irc channel to help coordinate. It's definitely nice to have that for nova.
17:34:09 <jaypipes> if y'all don't mind, I'm going to send out an email to davidkranz, sdague, and Ravikumar_hp with lists of bugs to investigate, get the current status on, and close out if appropriate.
17:34:28 <jaypipes> sdague: sure, no problem with that... set it up and ML it.
17:34:37 <davidkranz> jaypipes: OK
17:34:41 <sdague> jaypipes: ok, will do
17:35:07 <jaypipes> the four of us can either do the investigation ourselves or further dole out bugs to other QAers who woudl like to help.
17:35:12 <jaypipes> alright with you all?
17:35:24 <sdague> sounds fine
17:35:31 <davidkranz> k
17:35:33 <mtreinish> jaypipes: sure, sounds good
17:35:34 <Ravikumar_hp> ok
17:35:39 <jaypipes> I'll put em all on an etherpad so we can collaborate and cross off.
17:36:09 <jaypipes> #action jaypipes to list open bugs on etherpad and divide between Ravikumar_hp, davidkranz and sdague
17:36:15 <jaypipes> alrighty, onto blueprints
17:36:20 <jaypipes> #topic Blueprints
17:36:21 * afazekas I can't kill bugs before sunday :(, I have other things to do..
17:36:39 <jaypipes> #link http://blueprints.launchpad.net/tempest
17:36:45 <jaypipes> afazekas: no problem at all.
17:37:42 <jaypipes> I'd like to clean up the blueprints as much as possible, but I will take responsibillity for that, since there's only 26 of them.
17:38:00 <jaypipes> I will coordinate with the blueprint proposers and see what's still relevant.
17:38:13 <jaypipes> Ravikumar_hp, I know the two blueprints you asked me to look at. Will do those shortly troday.
17:38:14 <afazekas> https://blueprints.launchpad.net/tempest/+spec/boto-tests-attach-detach-volume the big test case does it
17:38:25 <Ravikumar_hp> jaypipes: ok
17:39:04 <jaypipes> afazekas: I will follow up with you about the boto ones, even though you didn't add those blueprints ;)
17:39:33 <afazekas> jaypipes: ok
17:39:40 <jaypipes> Ravikumar_hp: there are a number of blueprints (ex: https://blueprints.launchpad.net/tempest/+spec/add-xml-support) that I think can be marked Implemented.... will work with you on those.
17:40:02 <Ravikumar_hp> jaypipes: ok
17:40:07 <jaypipes> OK, I want to keep us within our hour, so I'm gonna move on now to sdague's topics...
17:40:17 <jaypipes> #topic Tempest Runtime Growth.
17:40:24 <jaypipes> sdague: dance, partner.
17:40:29 <afazekas> jaypipes: Hopefully next week, I can send patch to devstack for image (s3-materails) prepration
17:40:35 <sdague> yeh, so I had to bump the full gate to a 120 minute timeout
17:40:35 <jaypipes> excellent.
17:40:47 <sdague> because we were regularly getting timeouts a 90 minutes
17:40:52 <sdague> which is long
17:41:07 <jaypipes> crazy long.
17:41:18 <sdague> I'm going to start looking at long tests in there to just figure out if we can do better prior to a testr change
17:41:23 <sdague> by being a little smart
17:41:31 <mordred> testr for life!
17:41:49 <sdague> mordred: yeh, you're going to need some much bigger guests in ci when we get it though :)
17:41:59 <sdague> those puny 4Gb memory ones aren't going to cut it
17:42:05 <mordred> sdague: you are not the only one who wants those...
17:42:05 <jaypipes> sdague: for example... for the XML vs. JSON list XXX tests, we should be able to do a single setup, creating the resources, then test the list operations with XML and JSON, then teardwon.
17:42:18 <sdague> jaypipes: probably
17:42:22 <mordred> sdague: tripleo/baremetal testing wants larger ones too
17:42:46 <sdague> honestly, it's going to just take some time to look and be smart about it
17:42:51 <jaypipes> sdague: well, right now, it's taking 2X as long to do the list tests, because we setup resources twice, once for XML, once for JSON, when we only need to set them up once.
17:43:22 <sdague> ok, fair, though in some cases the test is the resource setup
17:43:22 <jaypipes> sdague: since the list tests aren't testing creation... they test list operations, so there's no need to double the setup.
17:43:34 <jaypipes> sdague: sure, but no the list tests :)
17:43:37 <sdague> right, those didn't seem to be the big hitters on time
17:43:40 <jaypipes> of which there are plenty ;P
17:43:51 <afazekas> We need to use resource pools
17:43:56 <jaypipes> sdague: agreed, but everything counts :)
17:44:04 <sdague> anyway, I think first step is to make it a little simpler to get timing out
17:44:19 <jaypipes> sdague: could you elaborate on that please?
17:44:36 <sdague> jaypipes: longest test runs
17:45:06 <jaypipes> sdague: are you talking about the time each test takes, or the timing out of a build/launch?
17:45:26 <sdague> honestly, I wouldn't mind annotating the tests per the project that they are testing, and have some idea about how much time we're spending on nova vs. cinder vs. swift
17:45:42 <jaypipes> k
17:46:15 <jaypipes> sdague: one of the issues with nosetests (among many others) is that timings don't count setUpClass or tearDownClass, and many times, that's where all the cost is!
17:46:37 <mtreinish> sdague: There's a blueprint on this: https://blueprints.launchpad.net/tempest/+spec/tempest-audit
17:46:48 <sdague> jaypipes: right, but if we instrumented tempest ourselves, we could get those numbers
17:46:57 <jaypipes> sdague: completely agreed.
17:47:00 <sdague> mtreinish: yes, right
17:47:01 <davidkranz> In the medium term I think we need to separate integration tests from api tests so we can only run the api rests on every build if the relevant project.
17:47:14 <davidkranz> ^^ tests
17:47:14 <uvirtbot`> davidkranz: Error: "^" is not a valid command.
17:47:33 <sdague> anyway, I think the point I wanted to bring up is: when reviewing tests, be careful on how much runtime they are adding right now
17:47:55 <sdague> for instance, the first round of swift tests had an object expires test with a sleep 30 in it
17:47:56 <jaypipes> no disagreement from me on that one!
17:48:28 <sdague> I think we can move on topic wise, hopefully something more interesting to say by next week
17:49:14 <Ravikumar_hp> sdague: object expire is not gated test
17:49:29 <sdague> Ravikumar_hp: it's a tempest full test
17:49:35 <sdague> so it's part of that 90 minutes
17:50:06 <jaypipes> #topic PostgreSQL in gate.
17:50:47 <sdague> ok, so I've got a review in to turn on non voting postgresql runs on tempest and devstack gates
17:50:59 <sdague> because I had to fix issues for each of them yesterday :)
17:51:14 <afazekas> Is PostgreSQL supported by heat or quantum now?
17:51:34 <sdague> my intent, unless there are objections, is to eventually have them be voting
17:51:50 <sdague> afazekas: no idea on heat, but it's not core, so I'm not sure that matters
17:51:52 <zyluo> not used in Quantum.
17:52:03 <sdague> zyluo: does quantum have a database?
17:52:11 <zyluo> yes it does
17:52:39 <jaypipes> zyluo: does Quantum use SQLAlchemy?
17:52:43 <sdague> my understand was core projects in openstack needed to be db agnostic
17:52:48 <jaypipes> zyluo: if it does, it supports PostgreSQL :)
17:52:51 <zyluo> jaypipes, yes SQLA
17:52:58 <sdague> jaypipes: well, possibly :)
17:53:08 <sdague> my experience is there are mysqlisms that need to be watched for
17:53:09 <afazekas> jaypipes: yes. IMHO installer related qustion
17:53:11 <zyluo> oh, yeah it should
17:53:15 <jaypipes> zyluo: I think you mean that Quantum's gate hasn't supported PG so far :)
17:53:37 <zyluo> yes, that's what I meant
17:53:39 <sdague> so what's in tempest full today passes on pg
17:53:39 <jaypipes> :)
17:54:09 <sdague> had we had this turned on previously, we would have caught a cinder mysqlism that was turned up in a tempest test
17:54:40 <jaypipes> sdague: so are you trying to get a quorum on turning on PG in the main project gate?
17:54:43 <sdague> so, anyway, expect to see the pg runs showing up (non voting), and please flag things that fail on it
17:54:49 <jaypipes> kk
17:55:07 <sdague> well, I'm actually hoping their is no objection to the non-voting
17:55:29 <jaypipes> no objection from me.
17:55:36 <sdague> and want to prep people for the idea of turning it voting some time in the future
17:55:41 <jaypipes> anbody have objections?
17:55:48 <sdague> that would be some sort of general agreement needed before we do that
17:55:49 <mtreinish> jaypipes: none from me
17:56:30 <jaypipes> alrighty..
17:56:48 <jaypipes> unless anybody has anything else to bring up, we're close to our hour...
17:56:55 <jaypipes> #topic Open discussion
17:58:03 <afazekas> Do you mind if I change the resources (server,volume) to "objects" and give them methods including  delete ?
17:58:28 <afazekas> I want to count and trace the usage
17:59:09 <sdague> I created #openstack-qa, people should feel free to join
17:59:23 <sdague> afazekas: non from me, having accounting in there would be great
18:00:03 <afazekas> ok
18:02:03 <jaypipes> afazekas: fine by me.
18:02:07 <jaypipes> #endmeeting