17:00:26 <jaypipes> #startmeeting qa
17:00:27 <openstack> Meeting started Thu Jan 31 17:00:26 2013 UTC.  The chair is jaypipes. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:28 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:30 <openstack> The meeting name has been set to 'qa'
17:00:39 <jaypipes> good morning QAers
17:00:49 <afrittoli> good morning
17:01:00 <chunwang> hi
17:01:11 <mtreinish> jaypipes: morning
17:01:32 <jaypipes> lots of open reviews :(
17:01:43 * jaypipes bad for not being able to do reviews :(
17:02:42 <chunwang> why not being able to do reviews?
17:02:54 <jaypipes> chunwang: w$rk getting in the way :)
17:03:04 <ravikumar_hp> hi
17:03:29 <jaypipes> ravikumar_hp: heey
17:03:37 <davidkranz_> Here
17:03:41 <mlavalle> Hi
17:03:54 * jaypipes hoping that afazekas or davidkranz_ might be able to bring me up to speed on what's gone on in tempest over past week.
17:04:15 <davidkranz_> jaypipes: Mostly a lot of new code submitted.
17:04:36 <davidkranz_> jaypipes: And the formal proposal from sdague  to gate all projects (except horizon) on full tempest.
17:04:43 <jaypipes> yes, I saw that
17:05:02 <davidkranz_> jaypipes: I'm not sure why the reviews have piled up but we should discuss that.
17:05:21 <mtreinish> davidkranz_: afazekas has pushed >50% of the current queue
17:05:27 <jaypipes> davidkranz_: lack of reviewers/approvers is my guess ;)
17:05:31 <mtreinish> I'm sure that has something todo with it
17:05:36 <jaypipes> indeed!
17:05:43 <davidkranz_> jaypipes: Yes, but the question is why?
17:06:10 <jaypipes> davidkranz_: people working on other things, I assume
17:06:21 <davidkranz_> Speaking for myself, I don't know anything about the boto stuff so did not feel I could review it.
17:06:33 <jaypipes> understood
17:07:00 <jaypipes> #topic performance of tempest
17:07:29 <jaypipes> I'd like to get an update from sdague and others about the testtools conversion and improvments in performance.
17:07:36 <davidkranz_> jaypipes: OK, but I want to go back to the review topic later.
17:07:53 <mtreinish> jaypipes: sdague can't make it today
17:08:11 <jaypipes> mtreinish: do you have enough info about the topic to speak on it?
17:08:51 <mtreinish> jaypipes: unfortunately, not really. I know that the testtools conversion got merged, but I don't know about the status of testr
17:09:12 <jaypipes> ok
17:09:18 <jaypipes> we will leave that to next week then
17:09:23 <jaypipes> #topic Reviews
17:09:46 <jaypipes> davidkranz_: I also don't know much about the Boto stuff which is why it's been difficult to do for me.
17:10:01 <davidkranz_> jaypipes: Tempest has become very broad.
17:10:20 <jaypipes> davidkranz_: yes indeed.
17:10:28 <ravikumar_hp> jaypipes: our team is writting boto tests for Ec2 .
17:10:41 <ravikumar_hp> Jaypipes: do you think developers can review
17:10:47 <jaypipes> ravikumar_hp: which developers?
17:11:00 <ravikumar_hp> Nova dev team that supports Ec2 ..
17:11:06 <davidkranz_> jaypipes: I think we need to subdivide into groups that can review particular topics.
17:11:38 <davidkranz_> Or in some cases not try to verify every aspect of the tests which involves detailed understanding of the api.
17:11:47 <ravikumar_hp> davidkranz_: +1
17:11:48 <jaypipes> ravikumar_hp: tough to identify those people...
17:11:56 <davidkranz_> This is not ideal but we need to change something.
17:13:41 <davidkranz_> How about this:
17:13:55 <jaypipes> davidkranz_: perhaps we just need to institute tempest review days
17:14:15 <davidkranz_> If a submission goes more than 24 hours with no review, some one with less specific knowlege can review it.
17:14:41 <jaypipes> davidkranz_: anyone should be able to review any patch at any time...
17:15:06 <jaypipes> davidkranz_: if the reviewer is not a subject matter expert, they can still comment on style, robustness, and other things.
17:15:49 <davidkranz_> jaypipes: Right. So we could say that a core review +1 means "style, etc" is OK but not verified all the functions.
17:16:09 <davidkranz_> But what if no one feels comfortable giving a +2?
17:16:21 <jaypipes> sure, that makes total sense, and that's pretty much what I've been doing
17:16:40 <davidkranz_> jaypipes: OK, then the issue may be (lack of) review days.
17:16:48 <jaypipes> davidkranz_: if nobody feels comfortable +2ing then we need to bring in the devs from Nova and other projects to make a decision
17:17:16 <davidkranz_> So how long should a submitter expect to wait for an approval?
17:17:18 <jaypipes> davidkranz_: but I still think review days are upon us
17:17:33 <jaypipes> davidkranz_: right now... upwards of a week, which is unacceptable
17:17:39 <davidkranz_> I would like a 24 hour turnaround ideally.
17:17:45 <jaypipes> davidkranz_: we should endeavour to keep it <48 hours IMHO
17:17:59 <jaypipes> davidkranz_: for at least an initial comment on the review
17:18:21 <davidkranz_> jaypipes: Do you know how review days are set up on other teams?
17:18:32 <jaypipes> davidkranz_: in addition, adding more people to oepnstack-qa-core would be helpful.
17:18:32 <ravikumar_hp> davidkranz_: jaypipes: yes . better less tahn 24 hrs as many reviews go through 4 or 5 patches
17:18:50 <jaypipes> davidkranz_: it's up to the PTL in other dev teams...
17:19:10 <davidkranz_> jaypipes: OK, just looking for an example to follow.
17:19:12 <jaypipes> davidkranz_: I think we need to have weekly review captains to herd everyone and get reviews pushed
17:19:48 <davidkranz_> jaypipes: Yeah. Is there a way to "request review" if you are not the submitter?
17:19:55 <davidkranz_> The button doesn't seem to work.
17:20:07 <jaypipes> davidkranz_: if it doesn't work, it's a bug and notify mordred
17:20:08 * afazekas here
17:20:09 <davidkranz_> That's what the captain needs to do
17:20:30 <davidkranz_> The button works if I am the submitter but I can't add a reviewer to some one else's change.
17:21:13 <davidkranz_> jaypipes: Grr, I think I misunderstood the UI. Never mind.
17:21:19 <jaypipes> davidkranz_: k
17:21:39 <davidkranz_> Should we maintain the captain list in an etherpad or wiki?
17:21:54 <jaypipes> davidkranz_: yes, I think so.
17:21:57 <ravikumar_hp> davidkranz_: I was able to add reviewers ..
17:22:08 <davidkranz_> ravikumar_hp: I misunderstood the UI.
17:22:09 <ravikumar_hp> in UI for others submission
17:22:45 <jaypipes> davidkranz_: so I would propose: 1) increase number of people on openstack-qa-core, 2) have weekly review captain on rotation, and 3) have qa-core members assigned to review days of week.
17:23:04 <chunwang> BTW, what's the qualification to be a patch reviewer of tempest?
17:23:10 <davidkranz_> jaypipes: Sounds good.
17:23:11 <jaypipes> chunwang: nothing.
17:23:21 <jaypipes> chunwang: anyone is encourage to do reviews
17:23:30 <davidkranz_> jaypipes: I would also have an entry for each project where people can put their names as knowing something about it.
17:23:37 <jaypipes> chunwang: however only members of openstack-qa-core can +1 Approve a patch.
17:23:43 <jaypipes> sorry, +2 Approve...
17:23:58 <jaypipes> davidkranz_: excellent. can you handle creating those pages?
17:24:03 <ravikumar_hp> jaypipes: +1 , 1) 2) is fine
17:24:06 <davidkranz_> jaypipes: That list could be use by the captain to add reviewers when necessary.
17:24:18 <jaypipes> davidkranz_: ++
17:24:46 <davidkranz_> Sure. Etherpad is easier to edit but wiki is less subject to screwup.
17:25:03 <jaypipes> ty
17:25:03 <davidkranz_> jaypipes: I'll go with the wiki.
17:25:08 <jaypipes> k
17:25:38 <chunwang> jaypipes: got it
17:26:21 <jaypipes> OK, so there are a number of folks that could be proposed to qa-core... I will be watching over the next couple weeks for folks who are actively doing reviews and following up. and I will propose those folks for qa-core
17:26:29 <jaypipes> I will send out an email shortly saying that.
17:26:30 <davidkranz_> ravikumar_hp: Can you guys review the boto stuff today?
17:26:49 <ravikumar_hp> davidkranz_: will try .
17:27:09 <davidkranz_> Guess I'm captain this week :)
17:27:13 <jaypipes> davidkranz_: indeed
17:28:25 <ravikumar_hp> davidkranz_: do you have the review id/subject handy  ?
17:28:26 <afazekas> davidkranz_: link ?
17:28:57 <davidkranz_> ravikumar_hp: There are several of them in the queue.
17:29:11 <mtreinish> afazekas: I think davidkranz_ is talking about your boto patches
17:29:19 <davidkranz_> mtreinish: That's right.
17:30:05 <afazekas> davidkranz_: ok, I thought I forget to review one change ..
17:30:43 <davidkranz_> afazekas: We need to review your changes.
17:31:12 <davidkranz_> afazekas: You are doing a lot of stuff and it would help is there were blueprints that provided some context for reviewers.
17:31:42 <afazekas> davidkranz_: small changes
17:32:11 <jaypipes> email sent...
17:32:16 <afazekas> davidkranz_: bug fix and reformating mostly
17:33:16 <davidkranz_> afazekas: I understand but we need to be careful about changing code just because we like it better a different way.
17:33:17 <mtreinish> afazekas: Along this note, I meant to ask if any of the reformats are related can you put them in a series with the same branch. Even just that would help with the context.
17:33:49 <jaypipes> mtreinish: ++
17:36:16 <davidkranz_> Any other topics?
17:36:33 <mtreinish> davidkranz_: there is the coverage analysis I've been doing
17:36:35 <jaypipes> davidkranz_: none from me. I'm eager to get on reviews...
17:36:42 <mtreinish> I sent a mail out to the list about it
17:36:44 <jaypipes> mtreinish: oh yeah... how is that going?
17:36:48 <afazekas> mtreinish: now these changes are mostly independent, but  mostly have second step, but these next step codes are even not created
17:37:09 <Shree_Quantum_HP> davidkranz_: yes we will like to discuss more about it
17:37:17 <mtreinish> https://etherpad.openstack.org/coverage-analysis and https://etherpad.openstack.org/MissingTempestTests
17:37:19 <jaypipes> afazekas: I don't understand that last comment... could you explain?
17:37:47 <jaypipes> mtreinish: ah, well done.
17:37:56 <mtreinish> jaypipes: it's going well, I've found some gaps, mostly on negative tests
17:38:20 <mtreinish> I'm still not done, but this what I've found so far
17:38:21 <afazekas> jaypipes: I modified lot of different unrelated part of the tempest , without marking the comment to dependent to each other
17:38:38 <jaypipes> afazekas: that makes reviews quite difficult...
17:38:49 <afazekas> jaypipes: As I said the bug fix commits will be continued some code cleanup
17:39:23 <afazekas> jaypipes: can I fix multiple unrelated bugs in one commit ?
17:39:46 <jaypipes> afazekas: yes
17:39:57 <jaypipes> afazekas: but only if they are small...
17:40:25 <afazekas> jaypipes: can I do code reformatting and reorganization, in bugfix commit ?
17:40:41 <jaypipes> afazekas: what I was saying is that it is difficult to review stuff that is *both* fixing a bug AND changing formatting or style because you prefer something some different way in the code
17:41:31 <jaypipes> afazekas: In addition, commit messages must be more descriptive :)
17:41:38 <davidkranz_> afazekas: I think code reorganizations should be discussed before code is submitted for review if it is significant.
17:41:38 <jaypipes> afazekas: example: https://review.openstack.org/#/c/20683/
17:41:58 <jaypipes> afazekas: Fixes *what* about boto intialization?
17:42:08 <afazekas> now all bot test are skipped
17:42:12 <afazekas> boto
17:42:20 <davidkranz_> Reorganization is expensive both to reviewers and in lost knowlege of where things are and how they work.
17:42:43 <jaypipes> afazekas: what I am saying is that you need to make the commit messages more descriptive, otherwise reviewers don't really understand what the patch is attempting.
17:42:58 <davidkranz_> jaypipes: And why.
17:43:18 <afazekas> OK
17:44:04 <afazekas> May be it was obvious just for me :)
17:44:28 <chunwang> for the ssh related test, I suppose they will be impacted when quantum applied, since the host cannot ping vm directly in quantum. Is there any change on the way to fix this?
17:44:28 <jaypipes> afazekas: yes, always err on the side of being TOO descriptive :)
17:45:53 <jaypipes> chunwang: I'm sorry, I'm not catching you...
17:46:12 <jaypipes> chunwang: what about Quantum prohibits a VM from being reachable via SSH?
17:46:22 <chunwang> yes
17:47:01 <mlavalle> chumwang: no, it doesn't
17:47:11 <chunwang> after quantum applied, the network related execution should add network uuid, since they are in different namespaces
17:49:16 <chunwang> like "ping <vm ip>" should be "ip netnc exec <network uuid> ping <vm ip>"
17:49:32 <chunwang> ssh is similar...
17:50:05 <jaypipes> hmm, I think we will need some documentation about that...
17:51:37 <jaypipes> OK, do we have any further discussion before we wrap up?
17:52:29 <jaypipes> OK, wrapping up...
17:52:32 <jaypipes> #endmeeting