17:00:33 <sdague> #startmeeting qa
17:00:34 <openstack> Meeting started Thu Jul 18 17:00:33 2013 UTC.  The chair is sdague. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:35 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:37 <openstack> The meeting name has been set to 'qa'
17:00:46 <sdague> hey folks, who's around for the meeting?
17:00:49 <mtreinish> hi
17:00:50 <afazekas> hi
17:00:51 <mkoderer> Hi!
17:01:23 <sdague> also, if anyone cares, and has google hangouts working, I'm going to run my video camera open for the meeting - https://plus.google.com/hangouts/_/4770be3e1806b15ec7aaf802806072fbfc5c4a64?hl=en
17:01:24 <dkranz> hi
17:01:30 <krtaylor> o/
17:01:47 <mkoderer> sdague: cool I will try it
17:01:51 <sdague> ok, lets get agenda up
17:02:29 <sdague> agenda - https://wiki.openstack.org/wiki/Meetings/QATeamMeeting
17:03:02 <sdague> ok, topic 1 Havana-2 status check in - https://launchpad.net/tempest/+milestone/havana-2 (sdague .. et al)
17:03:39 <sdague> so I'm going to have to push a bunch of stuff to H3
17:03:40 <mtreinish> so we need to either close those out or move them to H3?
17:03:56 <sdague> yeh, I wanted to see if anything else made H2 before I do the mass change
17:04:10 <sdague> updates from folks?
17:04:29 <sdague> I guess while we are waiting for those, we'll do testr check in
17:04:34 <sdague> mtreinish: you're up
17:04:46 <mtreinish> ok sure
17:04:58 <afazekas> https://review.openstack.org/#/c/35516/ I would like too have more ayes on this change,  before adding  everything else
17:05:19 <mtreinish> so we've got testr running tempest now as a non voting job on every check job in zuul
17:05:21 <sdague> afazekas: ok, cool, will look after meeting
17:05:37 <sdague> #note testr non voting runs now in the check queue
17:05:43 <mtreinish> results are looking promising tempest takes about ~15.5min in the gate
17:05:54 <sdague> #info testr non voting runs now in the check queue
17:06:01 <sdague> nice
17:06:04 <sdague> very cool
17:06:06 <afazekas> mtreinish: cool
17:06:14 <mtreinish> right now we're hitting 4 failures with the jobs, I've outline them here: https://etherpad.openstack.org/debugging-testr-tempest
17:06:45 <mtreinish> once we sort out these issues and the nonvoting jobs are stable enough we'll be able to migrate everything to using testr
17:06:51 <sdague> that will be awesome
17:07:16 <sdague> we still have the smoke test issue right, where smoke tests will run more than they should?
17:07:16 <sdague> which is important for the neutron smoke job
17:07:17 <mtreinish> so I'll appreciate any help with the debugging of those issues
17:07:37 <sdague> any additional volunteers to help track down testr races?
17:07:43 <mtreinish> sdague: so I think that can be sorted using subunit-filter
17:07:55 <afazekas> I would like to help in tracing the issues
17:08:02 <mtreinish> afazekas: great
17:08:13 <sdague> nice, mtreinish and mkoderer on the hangout now with me :)
17:08:23 <sdague> afazekas: cool, that woudl be great
17:08:39 <sdague> #action afazekas and mtreinish to work through testr issues to get that to be our primary runner in the gate
17:08:55 <sdague> mtreinish: you should mute :)
17:08:57 <mtreinish> also, I should note that there is an issue with the test count being reported with testr
17:09:02 <mtreinish> sorry
17:09:24 <sdague> mtreinish: cool, is there a bug up with testr on that?
17:09:39 <afazekas> IMHO we should have more stress task and job combination  in order to simplify , the everything racing with everything issue
17:09:54 <mtreinish> not yet, but soon I'm collecting some logs to give to lifeless. Then I'll report it.
17:10:02 <sdague> afazekas: I suspect the testr races are actually tempest's fault
17:10:05 <afazekas> and configure jobs for the suspected combinations
17:10:06 <sdague> not the system
17:10:18 <sdague> but I agree that the stress tests will shake out good things
17:10:30 <sdague> ok, how about we talk about stress tests
17:10:38 <sdague> good segway... mkoderer
17:10:39 <afazekas> sdague: I hope it will be tempest failure, it is more easy to debug :)
17:10:39 <mkoderer> ok great
17:10:44 <sdague> afazekas: :)
17:10:59 <mkoderer> so let's have a short look to the open items
17:11:16 <mkoderer> I think we can finish nearly everything next week...
17:11:28 <sdague> awesome
17:11:28 <mkoderer> open for me is the jenkins stuff
17:11:38 <mkoderer> what is the current state here?
17:11:57 <mkoderer> I think there is already something running right?
17:12:01 <mtreinish> mkoderer: what jenkins stuff? we've got the periodic job working again after the json fix I pushed yesterday
17:12:03 <sdague> mkoderer: which jenkins issue?
17:12:16 <mkoderer> so we have those items open:
17:12:19 <mtreinish> mkoderer: https://jenkins.openstack.org/job/periodic-tempest-devstack-vm-stress/
17:12:32 <mkoderer> Create a json test description for a standard test to be run in a periodic jenkins job: INPROGRESS
17:12:43 <mtreinish> mkoderer: that's done
17:12:57 <sdague> mtreinish: can you update the blueprint?
17:13:04 <mkoderer> ok cool... so I hope we can finish it next week
17:13:10 <sdague> they've got a good whiteboard list of tasks in it
17:13:16 <mkoderer> afazekas and me will add some new tests
17:13:18 <mtreinish> sdague: sure, I'm really bad about dealing with blueprints...
17:13:18 <sdague> #link https://blueprints.launchpad.net/tempest/+spec/stress-tests
17:13:32 <afazekas> https://jenkins.openstack.org/job/periodic-tempest-devstack-vm-stress/lastBuild/console
17:13:32 <sdague> mtreinish: well I guess a scolding is in order :)
17:13:55 <sdague> cool, dkranz on as well
17:14:07 <mtreinish> sdague: I often forget to link my commits to my own blueprints....
17:14:10 <sdague> ok, any other blueprints?
17:14:31 <mtreinish> sdague: well, I added one for creating service tags...
17:14:37 <mtreinish> but that is pretty self explanatory
17:14:48 <sdague> I'll move stuff to H3 tomorrow. I'm actually curious what happened to the HP folks that were adding tests, I haven't seen updates on their BPs for a while
17:15:00 <sdague> mtreinish: great
17:15:24 <sdague> ok, moving on from blueprints
17:15:34 <Bageshree> sdague: we got approval from our internal process so HP will be doing checking in H3 mostly
17:15:34 <sdague> #topic Mailing list move (voting on moving to -dev) (sdague)
17:15:50 <sdague> ok, so we discussed this on the mailing list some already
17:16:02 <sdague> but I wanted to open up for discussion, vote here.
17:16:26 <sdague> my concern is that by having our own mailing list we end up in a funny 'qa corner', which doesn't include the main dev teams
17:16:39 <mtreinish> sdague: I'm fine with moving as long as there is a tag we agree on using for qa related topics
17:16:41 <sdague> which I'd like to avoid, and be more integrated with the dev conversations
17:16:43 <dkranz> sdague: I think anything that separates qa from dev should be eliminated
17:16:55 <sdague> sure, my suggestion is going to be a [qa] tag
17:17:04 <sdague> which would apply to tempest, grenade, or other related things
17:17:18 <sdague> given qa is the program name
17:17:23 <mtreinish> sdague: that's fine, just wanted there to be convention (makes mail filtering easier :)
17:17:34 <krtaylor> +1
17:17:36 <sdague> are there any descenting opinions?
17:17:39 <Bageshree> +1
17:17:43 <mkoderer> +1
17:17:50 <dkranz> +1
17:17:51 <sdague> ok, wait, let me use the vote thing :)
17:18:04 <dkranz> sdague: smiles all around
17:18:13 <mlavalle> +1
17:18:13 <sdague> #vote move mailing list traffic to openstack-dev list
17:18:36 <sdague> hmm... maybe I don't know how to make that work :)
17:18:43 <sdague> well we're +1s all around so far
17:18:45 <sdague> any -1s?
17:19:09 <sdague> ok.... we'll call that done deal then
17:19:18 <sdague> I'll work with infra to get us sorted tomorrow.
17:19:28 <mtreinish> sdague: #startvote
17:19:38 <mtreinish> http://ci.openstack.org/meetbot.html
17:20:01 <mtreinish> sdague: ^^^ maybe you should read that as a meeting chair
17:20:11 <sdague> #startvote move mailing list traffic to openstack-dev? Yes, No
17:20:12 <openstack> Begin voting on: move mailing list traffic to openstack-dev? Valid vote options are Yes, No.
17:20:13 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
17:20:19 <sdague> #vote Yes
17:20:26 <mtreinish> #vote Yes
17:20:33 <dkranz> #vote Yes
17:20:35 <Bageshree> #vote Yes
17:20:36 <mlavalle> #vote Yes
17:20:39 <afazekas> #vote Yes
17:20:46 <sdague> mtreinish: yes, probably... :)
17:20:51 <mkoderer> #vote Yes
17:21:11 <krtaylor> #vote Yes
17:21:16 <sdague> ok, last chance to voice...
17:21:17 <dansmith> #vote you guys are just lonely
17:21:17 <openstack> dansmith: you guys are just lonely is not a valid option. Valid options are Yes, No.
17:21:21 <dansmith> oh, damn.
17:21:27 <sdague> heh :)
17:21:37 <sdague> #endvote
17:21:38 <openstack> Voted on "move mailing list traffic to openstack-dev?" Results are
17:21:39 <openstack> Yes (8): krtaylor, mlavalle, mtreinish, dkranz, afazekas, sdague, Bageshree, mkoderer
17:21:46 <sdague> ok
17:22:00 <sdague> #agreed moving mailing list traffic to openstack-dev with [qa] tag
17:22:08 <sdague> #action sdague to work with infra to implement
17:22:18 <sdague> ok, thanks mtreinish for the assist
17:22:27 <sdague> next topc
17:22:35 <sdague> #topic White Box Tests
17:22:48 <sdague> I don't know who listed this on the agenda, can they step forward?
17:23:28 <sdague> also, reminder, if you have working google hangout, dkranz, mkoderer, mtreinish, and I are at - https://plus.google.com/hangouts/_/4770be3e1806b15ec7aaf802806072fbfc5c4a64?hl=en all muted, but you can see video
17:23:29 <dkranz> sdague: I put it there but I think it was on behalf of afazekas
17:23:38 <sdague> ok, afazekas you want to speak to it?
17:23:42 <afazekas> So my question about the whitebox test, do we really want to direct DB updates by tempest ?
17:24:03 <mtreinish> afazekas: I'd say no, but I thought that there was more in whitebox than just that
17:24:09 <sdague> afazekas: that's a good question
17:24:36 <afazekas> mtreinish: mostly state transition matrix
17:24:39 <sdague> afazekas: if we are sure that we can cover that content back in unit tests, I'd be happy to get it out of there
17:24:58 <sdague> I think jaypipes brought that in to handle some bugs we weren't having any luck in in unit test tracking down
17:25:06 <sdague> especially in nova state transitions
17:25:17 <sdague> but we aren't actually running them right now
17:25:18 <afazekas> sdague: IMHO some parts are even better can be covered by stress tests
17:25:19 <sdague> right?
17:25:26 <sdague> afazekas: that's true
17:25:36 <sdague> if we can cover them other places, I'm totally good with removing them
17:25:53 <dkranz> sdague: I think there is a place for whitebox tests but the ones that cannot be covered otherwise are pretty low priority now.
17:26:26 <sdague> yeh, what I'd say right now is they aren't a focus, if someone wants to clean them up, or to drop them because we've got it covered else where, I'd +2 it
17:26:43 <dkranz> sdague: We could just scrutinize new submissions more closely to make sure they really need to be whitebox
17:26:51 <afazekas> Another related question, do we want to test state transition impossibilities (negative) by tempest (stable states, so expected to be gate friendly)
17:26:55 <dkranz> sdague: If there are any
17:26:56 <sdague> yeh, I don't think we've had that many new submissions there
17:27:20 <sdague> afazekas: if we think it will expose a bug, sure
17:27:28 <afazekas> ok
17:27:50 <sdague> anything else on white box tests?
17:28:05 <afazekas> I saw skip removal review
17:28:19 <sdague> well even so, whitebox aren't in full
17:28:30 <sdague> so a skip removal doesn't do anything
17:28:41 <sdague> except for mtreinish's all periodic run
17:29:02 <mtreinish> sdague: I didn't set that up...
17:29:28 <sdague> ok, so for the all periodic (not mtreinish's) :)
17:29:53 <sdague> ok, anything else on white box?
17:30:05 <sdague> #topic Critical Reviews
17:30:09 <afazekas> Ok so someone secretly not restored the state matrix to the old one
17:30:32 <sdague> afazekas: ok, is that something we need to fix?
17:30:46 <afazekas> I do not think so
17:30:47 <sdague> also, critical reviews time, pimp your reviews that you need landed
17:30:56 <sdague> or that someone should look at
17:31:01 <afazekas> But someone at the nova side should confirm it
17:31:06 <dkranz> sdague: We need to deal with the slow heat issue.
17:31:21 <dkranz> sdague: It is not really waiting for code review
17:31:28 <sdague> dkranz: well, my current strategy is testr
17:31:32 <dkranz> sdague: But we have no way to run it now.
17:31:48 <sdague> because the tempest runs dropped a lot with that
17:31:55 <dkranz> sdague: OK, we can push the limits of that
17:32:09 <sdague> dkranz: agreed, and if we do, we split the heat case off
17:32:12 <dkranz> sdague: sbaker said there were more slow tests coming
17:32:25 <dkranz> sdague: I think he is just waiting for a way to run them.
17:32:32 <sdague> ok, I guess those guys are all asleep right now, right?
17:32:39 <dkranz> sdague: Yeah.
17:32:57 <sdague> dkranz: ok, how about we come back to this after running on reviews
17:33:01 <dkranz> sdague: Sure
17:33:14 <sdague> just so that we don't prevent people from posting reviews they need eyes on, then I'll #topic it
17:33:27 <sdague> ok, any more reviews?
17:33:33 <sdague> or any reviews (I
17:33:34 <afazekas> https://review.openstack.org/#/c/33211/
17:33:37 <dkranz> sdague: A lot just need +A
17:34:04 <sdague> afazekas: did we figure out why that went non votable?
17:34:40 <mtreinish> dkranz: I'll make an effort to do some real reviews today. I've been bad about that this week.
17:34:47 <sdague> dkranz: sure, but if there were specific reviews that are hot that we need eyes on, this is a good time to bring them up
17:34:52 <sdague> basically a hot reviews time
17:35:15 <afazekas> sdague: I think every review is important for someone
17:35:16 <sdague> also, remember, even if you aren't qa-core we could use review comments
17:35:16 <dkranz> sdague: I think just afazekas leak stuff which was already mentioned
17:35:34 <sdague> afazekas: true, but not all of them have the same impact on the project
17:35:51 <dkranz> Of course we should try to review as rapidly as we can.
17:35:57 <sdague> yep
17:36:04 <afazekas> sdague: the core reviewers main duty to review , AFAIK
17:36:16 <mkoderer> I will need this one https://review.openstack.org/#/c/36820/ for the blueprint
17:36:27 <dkranz> There must be some gerrit bug with https://review.openstack.org/#/c/33211/
17:36:32 <mkoderer> but dkranz found something so... I need to fix it first... ;)
17:36:41 <sdague> ok, cool, got it up in my window
17:37:03 <sdague> dkranz: could andreaf have made it a draft?
17:37:46 <dkranz> sdague: I don't know. It doesn't look any different except for can't review
17:37:51 <sdague> #action sdague to take https://review.openstack.org/#/c/33211/ to -infra to figure out why we can't vote on it
17:37:57 <sdague> because I think it was +2 all around
17:38:07 <dkranz> sdague: Right.
17:38:24 <sdague> ok, any other reviews?
17:38:26 <sdague> going once....
17:38:33 <sdague> going twice....
17:38:39 <sdague> ok, moving on
17:38:44 <sdague> #topic Heat tests in gate
17:39:07 <dkranz> sdague: Even if we make a separate job, it could still take too long.
17:39:11 <mtreinish> dkranz: so how slow is slow for these tests?
17:39:12 <sdague> dkranz: any idea from sbaker how long running they think the heat tests are going to be?
17:39:26 <sdague> mtreinish: well they are going to use real OS images, not cirros
17:39:28 <dkranz> The one there is supposed to be 10min
17:39:47 <dkranz> I think we need to make a separate job and give it a budget.
17:40:01 <sdague> dkranz: ok, I'm fine with that
17:40:21 <sdague> you want to take making that job?
17:40:23 <mtreinish> dkranz: yeah if 1 test is 10min that's probably all we can do.
17:40:50 <dkranz> sdague: I can do that but won't get to it until Monday because I am out tomorrow.
17:40:55 <sdague> sure, that's fine
17:41:04 <sdague> I think, given their length, I'd like to start them on testr
17:41:16 <sdague> even if that means they start non voting
17:41:25 <sdague> assuming that's going to help with their overall run
17:41:26 <dkranz> sdague: We can not mark it as gate and then run the heat tests in the other job.
17:41:34 <mtreinish> sdague: well it's going to be just the heat jobs right?
17:41:44 <mtreinish> sorry heat tests in the heat job.
17:41:50 <mtreinish> not everything + heat
17:41:54 <dkranz> mtreinish: Right
17:42:06 <sdague> mtreinish: right
17:42:18 <sdague> we're going to need to separate heat from the rest of api
17:42:32 <dkranz> sdague: Why?
17:42:42 <sdague> well... conceptually
17:42:49 <dkranz> sdague: Sure.
17:42:52 <sdague> because the main runs are going to be api - heat
17:43:00 <sdague> can we do that sanely in testr?
17:43:02 <mtreinish> dkranz: otherwise they'll get picked up when we tell testr/nose to run tests in tempest.api
17:43:09 <sdague> without having to list a million subdirs?
17:43:13 <dkranz> sdague: But if heat is not marked gate it won't run in the main ijob.
17:43:25 <dkranz> and the heat job can just select heat
17:43:33 <mtreinish> sdague: I'll prioritize figuring out how to run subunit-filter with our env
17:43:39 <sdague> I'm not convinced we actually got that tag working in the gate
17:43:46 <afazekas> sdague: the autoscaling (scenario?) is the only slow heat test
17:43:47 <sdague> actually the service tag bits would solve this
17:43:51 <dkranz> mtreinish: But if it is not gte it won't run
17:44:01 <sdague> afazekas: ok, so there will be heat tests that don't do that?
17:44:13 <mtreinish> dkranz: we run tempest.api tempest.scenario tempest.cli and tempest.thirdparty in the gate
17:44:16 <mtreinish> we don't use the tag
17:44:17 <sdague> so maybe the right thing to do is put the autoscaling tests outside of api
17:44:25 <afazekas> sdague: the existing ones are ok IMHO
17:44:31 <sdague> afazekas: ok, cool
17:44:34 <dkranz> mtreinish: Oh. So why do we have it if we are not using it?
17:44:49 <sdague> dkranz: because it got only partially implemented
17:44:50 <mtreinish> dkranz: no idea I just know what's being run now
17:45:11 <dkranz> We really need to decide if we are using that or not.
17:45:21 <dkranz> It's silly to have it and not use it.
17:45:24 <sdague> dkranz: agreed, lets solve the heat thing first
17:45:29 <mtreinish> well, it is a pain to individually tag 1063 tests...
17:45:34 <sdague> well, a lot of it was about dealing with testr
17:45:40 <dkranz> sdague: OK
17:45:43 <mtreinish> but who was doing that work
17:45:46 <sdague> because we had no idea if that was going to work in time
17:46:05 <sdague> but now that it's close we can probably do a purge of the gate tag once it lands
17:46:11 <dkranz> mtreinish: A bunch of stuff with 'smoke' was submitted. I thought it was all but I guess not.
17:46:13 <sdague> ok, anyway, the heat autoscale tests
17:46:32 <mtreinish> sdague: I'll clear out gate when I start working on the service tags
17:46:44 <sdague> what if we put that in as a scenario test, use the heatclient for those
17:46:47 <dkranz> Can't we exclude the heat directory with testr?
17:47:04 <sdague> dkranz: well afazekas just said that a lot of the heat tests are fine in api
17:47:09 <sdague> and not super long running
17:47:13 <sdague> just the autoscaling bits
17:47:21 <sdague> which feels like a scenario test to me anyway
17:47:34 <mtreinish> if there are heat api tests in there now they're being run
17:47:40 <sdague> yes, there are a couple
17:47:55 <sdague> but I'd like to keep their simple api testing in the main runs
17:48:00 <dkranz> sdague: OK, so let's make it scenario.
17:48:11 <sdague> as that means it gets tested on other dbs, with neutron, etc.
17:48:18 <sdague> and only push autoscale seperate
17:48:25 <sdague> cool
17:48:31 <dkranz> We still need to keep it out of the main run.
17:48:37 <sdague> yep
17:48:57 <dkranz> We could just mark it slow
17:49:05 <sdague> but we'll figure out a tag to exclude on for that
17:49:08 <dkranz> and skip those in the main run
17:49:27 <sdague> sure, or something, lets work that out later.
17:49:32 <sdague> you want to communicate with sbaker about the approach?
17:49:46 <dkranz> sdague: OK
17:50:41 <sdague> #info have the heat team submit the autoscaling tests as scenario tests to make it easy to give them their own time budget
17:50:56 <sdague> #action dkranz to work with sbaker to get any other details sorted out
17:51:06 <sdague> cool
17:51:08 <sdague> ok
17:51:29 <dkranz> I had one more issue I should have put on the agenda
17:51:30 <sdague> #topic Open Discussion
17:51:43 <sdague> ok, any other topics
17:51:43 <sdague> 9 minutes left :)
17:51:53 <dkranz> testing and stability of python clients
17:52:00 <afazekas> Adding test cases with skip attribute
17:52:18 <dkranz> people have started talking about this as if it were true but we don't test it at all
17:52:18 <sdague> go ahead
17:52:30 <sdague> dkranz: who talked about it?
17:52:41 <dkranz> That is, using a new client library will work with older versions
17:52:42 <mtreinish> afazekas: don't we shouldn't add a test case unless there is a way to run it in ci
17:52:42 <sdague> maybe that's just education
17:52:59 <sdague> dkranz: right, I think there is a missing kind of test run
17:53:09 <sdague> which is new clients on stable release
17:53:18 <dkranz> sdague: The context was with uncapping versions of client libs
17:53:31 <sdague> dkranz: right
17:53:36 <dkranz> sdague: It was agreed that uncapping was good because it was "not needed"
17:53:46 <afazekas> mtreinish: I have no idea how many test case can be lost in somewhere in the history and now,  they should be able to run..
17:53:58 <sdague> dkranz well actually it was agreed that it was good, because otherwise we release broken clients
17:54:01 <dkranz> sdague: I sent an email to the list about this a few days ago.
17:54:04 <sdague> which happened with keystone
17:54:16 <Bageshree> a last q: have we started putting together etherpad for tempest/testr or any framework topic to be discussed in summit?
17:54:18 <dkranz> sdague: I agree with the principle, but we don't test it at all
17:54:27 <sdague> dkranz: not directly
17:54:31 <sdague> but we do in the interactions
17:54:38 <sdague> because nova uses keystone client
17:54:46 <dkranz> sdague: Yes, that gives us some coverage
17:54:47 <sdague> and glance client, cinder, neutron client
17:54:52 <sdague> under the covers
17:55:07 <sdague> that was what we got ourselves in trouble with, as we weren't testing their git trees
17:55:09 <mtreinish> afazekas: we have the skip tracker tool to see if bugs are fixed but it's not perfect
17:55:09 <dkranz> But it does not test that a client still runs against a previous release that has the same api version
17:55:19 <sdague> dkranz: correct
17:55:26 <mtreinish> afazekas: I'm sure there are skips we can remove in there now, but we don't want to add things with skips
17:55:27 <sdague> and for that I think we need a new stable bitrot job
17:55:34 <dkranz> sdague: We would need to run old stable branch scenario tests with current clients
17:55:41 <sdague> right, exactly
17:55:50 <sdague> anyone want to volunteer to get that spun up?
17:55:58 <sdague> it should be reasonably simple
17:56:08 <dkranz> sdague: That will be hard going back becaues the scenario tests are  not many
17:56:14 <dkranz> But should get better in time
17:56:24 <sdague> dkranz: actually, api does a lot of testing indirectly
17:56:34 <sdague> because of the integration points
17:56:38 <dkranz> sdague: No, api does not use client libs
17:56:45 <sdague> dkranz: not directly
17:56:53 <mtreinish> dkranz: it doesn between the services
17:57:02 <mtreinish> for example nova uses glanceclient to make images calls
17:57:03 <sdague> but compute/volumes tests us cinderclient for nova => cinder comms
17:57:09 <sdague> right, exactly
17:57:17 <dkranz> mtreinish: Yes, you're right
17:57:25 <sdague> so it won't catch everything, for sure
17:57:27 <dkranz> I will create a blue print for this.
17:57:35 <sdague> but it will expose totally broken things
17:57:38 <sdague> and should help
17:57:48 <sdague> dkranz: great
17:58:09 <sdague> #action dkranz to create blueprint for doing stable tests with upstream git clients
17:58:16 <sdague> ok, anything in the last minute?
17:58:19 <Bageshree> repeating my q again but have we started putting together etherpad for tempest/testr or any framework topic to be discussed in summit?
17:58:21 <sdague> I've got a hard stop
17:58:32 <dkranz> Bageshree: Not yet.
17:58:37 <sdague> Bageshree: not really, it's just what's in the code
17:58:45 <Bageshree> thanks
17:59:11 <sdague> ok, well I've got to run, so I'll close out the meeting, please continue the conversation in #openstack-qa
17:59:20 <sdague> thanks all for comming
17:59:23 <sdague> #endmeeting