09:00:51 <oomichi> #startmeeting qa
09:00:52 <openstack> Meeting started Thu Aug  6 09:00:51 2015 UTC and is due to finish in 60 minutes.  The chair is oomichi. Information about MeetBot at http://wiki.debian.org/MeetBot.
09:00:53 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
09:00:56 <openstack> The meeting name has been set to 'qa'
09:01:11 <oomichi> hi who's here today?
09:01:18 <jordanP> o/
09:01:19 <gmann> o/
09:01:32 <ylobankov_> o/
09:01:33 <dolpher1> hi oomichi
09:01:48 <oomichi> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_August_6th_2015_.280900_UTC.29
09:01:53 <NithyaG> o/
09:01:54 <oomichi> ^^^ today's agenda
09:02:05 <oomichi> dolpher1: hi :)
09:02:19 <oomichi> masayukig: around?
09:02:21 <dmellado> Hi, is it late for adding a topic about cirros and dhcpv6? ;)
09:02:53 <xwizard> hi :)
09:02:54 <oomichi> dmellado: that is fine, please update the wiki
09:03:02 <dmellado> oomichi: ty!
09:03:13 <oomichi> ok, let's get started
09:03:21 <oomichi> #topic Specs Reviews
09:03:31 <oomichi> Does anyone have any open specs reviews to discuss today?
09:03:44 <oomichi> #link      https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
09:03:51 <andreaf> o/
09:04:18 <masayukig> o/
09:04:29 <andreaf> oomichi: I need to update https://review.openstack.org/173334 - I'll try to do so this week
09:04:47 <gmann> oomichi: I again forgot to review your microversion spec :(
09:04:52 <gmann> ll do tomorrow
09:05:03 <oomichi> anderstj: cool, thanks in advance :)
09:05:06 <dmellado> oomichi: done
09:05:19 <oomichi> gmann: yeah, that is what I want to say, thanks
09:05:24 <oomichi> dmellado: cool
09:05:57 <oomichi> so there are not so many specs now, is it ok to go to the next topic?
09:06:32 <oomichi> ok, go to next topic
09:06:52 <oomichi> #topic Tempest
09:07:16 <oomichi> I have one item on this topic for service clients.
09:07:31 <oomichi> now we are concentrating on compute servie clients as the first step.
09:07:48 <oomichi> there are 3 topics of patches
09:07:55 <jordanP> yes, 3 topics
09:07:57 <oomichi> # https://etherpad.openstack.org/p/tempest-consistent-service-method-names-works
09:08:02 <oomichi> #link https://etherpad.openstack.org/p/tempest-consistent-service-method-names-works
09:08:10 <oomichi> #link https://etherpad.openstack.org/p/tempest-service-client-return-value
09:08:18 <oomichi> #link https://etherpad.openstack.org/p/tempest-service-client-unit-tests
09:08:25 <mkoderer> hello friends, sry I am late
09:08:26 <oomichi> the first one is almost done
09:08:37 <oomichi> mkoderer: hey, good to come :)
09:08:48 <oomichi> the second is managed by gmann
09:09:03 <oomichi> gmann: can you introduce the current status?
09:09:18 <gmann> oomichi: yea, I updated on compute clients first
09:09:32 <gmann> me and jordanP just discussed those in qa channel
09:09:47 <gmann> actually unit tests and response one conflict with each other
09:10:09 <gmann> it will be good if we postponed response one and merge UT first
09:10:28 <oomichi> ok, that is good
09:10:32 <gmann> after UT patches i can just rebase mine
09:10:41 <oomichi> so we need to concentrate UT patches now.
09:10:44 <jordanP> I agree in this order. gmann is more experienced so we can quickly iterate with him after UT patches land
09:10:57 <gmann> jordanP: yea
09:11:15 <gmann> oomichi: we can review UT one fast :)
09:11:24 <oomichi> and the contributer of UT is new now, so I'd like to get more volunters for UT works.
09:11:39 <jordanP> so I am not super happy with the current UT patches, it's good start
09:11:46 <jordanP> but they don't assert* a lot
09:12:05 <jordanP> and how the unicode vs bytes strings are handled is a bit weird
09:12:16 <jordanP> I left a comment 3 days ago on https://review.openstack.org/#/c/204035/
09:12:35 <oomichi> jordanP: so does it say that seems redundant?
09:12:39 <jordanP> my point is, if we want to test proper unicode support, then we should have some "weird" characters in our string
09:13:16 <jordanP> no I am good with "duplicating tests", one with unicode as input, one with bytes string
09:13:20 <oomichi> jordanP: ah, on nova side, validation ways are different between APIs.
09:13:53 <jordanP> but those are unit tests, Nova is "not involved", I mean, http calls are mocked
09:13:58 <oomichi> jordanP: and my concern is that these tests mislead to users when seeing Tempest tests.
09:14:15 <jordanP> oomichi, what do you mean ?
09:14:16 <mkoderer> jordanP: to test unicode fuzzy test would be helpful..
09:14:43 <jordanP> mkoderer, that's an option and would be easy to do
09:14:52 <jordanP> mkoderer, you mean pick "random unicode char" ?
09:14:52 <oomichi> jordanP: some Nova apis denies not-ascii data as requests.
09:15:32 <oomichi> jordanP: but if including byte inputs in all tests, users misunderstand that is valid request
09:15:50 <mkoderer> jordanP: yeah, if you pick random unicode chars the tests can fail unforeseen
09:15:51 <oomichi> jordanP: but this my concern is small
09:16:53 <jordanP> oomichi, you mean't "unicode  inputs"
09:16:54 <jordanP> ?
09:17:20 <jordanP> as far as I know, nova should always accept byte inputs (~ ascii)
09:17:53 <jordanP> no, no ascii, nevermind :)
09:17:54 <oomichi> jordanP: yes, unicode input also is denied on some APIs.
09:18:24 <jordanP> I think we shouldn't concider what nova supports or not, we are testing tempest clients
09:18:50 <jordanP> I don"t expect people to read the unit tests in order to understand nova's api
09:19:01 <oomichi> jordanP: yeah, I feel you are right. and validation ways also will be changed on nova side.
09:19:23 <oomichi> jordanP: so my concern was a little overkill ;)
09:19:52 <jordanP> overall i'd like more eyes on the UT patch series, so far it's been only me :)
09:20:01 <jordanP> the contributor is really reactive
09:20:01 <oomichi> so is it fine to test randome unicode data on UT?
09:20:19 <jordanP> i think it is
09:20:28 <mkoderer> jordanP: this ones: https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:service-client-unit-tests,n,z
09:20:33 <mkoderer> right?
09:20:35 <jordanP> mkoderer, yes
09:20:46 <mkoderer> jordanP: k, I will add them to my review list
09:20:54 <oomichi> ok, that's fine
09:21:02 <oomichi> mkoderer: cool
09:21:09 <jordanP> mkoderer, but be nice with the guy :) he's new and he has already done a good job
09:21:26 <jordanP> (I like your unicode fuzzing idea)
09:21:42 <mkoderer> jordanP: haha I am always nice (hopefully)
09:21:51 <jordanP> i know :)
09:22:06 <oomichi> ok, can we move from this service client topic?
09:22:35 <oomichi> dmellado: the next is your turn :)
09:22:52 <dmellado> Sure, first of all sorry for the late notice and I wasn't really sure where to address the topic
09:23:00 <dmellado> I've been checking this issue in cirros https://bugs.launchpad.net/cirros/+bug/1366326
09:23:00 <openstack> Launchpad bug 1366326 in CirrOS "ipv6 support" [Medium,Confirmed]
09:23:01 <uvirtbot> Launchpad bug 1366326 in cirros "ipv6 support" [Medium,Confirmed]
09:23:02 <uvirtbot> Launchpad bug 1366326 in cirros "ipv6 support" [Medium,Confirmed] https://launchpad.net/bugs/1366326
09:23:21 <dmellado> where I tried to add ipv6 stateful tests to the scenario/test_network_v6
09:23:31 <oomichi> (oops, a little big earchquak here)
09:23:35 <dmellado> And I don't know if there's any current idea or status about it
09:23:57 <dmellado> but the current cirros image doesn't support DHCPv6, is there any known workaround or plan to address this?
09:23:58 <gmann> oomichi: scary  for me :)
09:25:03 <jordanP> dmellado, so no support at all of dhcp v6 in cirros ?
09:25:12 <dmellado> Not as far as I know
09:25:46 <dmellado> I'm not a real ipv6 expert, but I saw only ping6 from ipv6 tools in cirros
09:25:54 <jordanP> so there's little we can do about Cirros..., not that I know at least
09:25:55 <dmellado> and was curious about how this was being done so far, only slaac?
09:26:21 <jordanP> dmellado, I see the bug report, maybe you could propose "a patch"
09:26:32 <oomichi> mtreinish recommended to user the other image on the report
09:26:57 <mkoderer> for Manila scenario test we are using other images than cirros
09:27:19 <dmellado> mkoderer: but could that be used to pass the gate later?
09:27:34 <dmellado> mkoderer: which ones, just out of curiosity?
09:27:59 <mkoderer> dmellado: the are in the gate.. gate-manila-tempest-dsvm-neutron-scenario
09:28:13 <dmellado> mkoderer: I'll take a look at it, ty!
09:28:17 <jordanP> dmellado, maybe https://github.com/openstack/manila/blob/master/devstack/plugin.sh#L97
09:28:30 <mkoderer> but fixing cirros is most propably the best way :)
09:28:33 <jordanP> +1
09:28:42 <dmellado> Yeah I do agree on that
09:28:48 <oomichi> agree with the best way
09:28:51 <jordanP> dmellado, I think your best option is to spend some time to try to submit a patch to cirros
09:28:52 <dmellado> I'll try to ping the cirros devs so they would add ipv6-tools
09:28:59 <dmellado> or submit a patch myself
09:29:06 <jordanP> at least start a thread on their mailing list
09:29:13 <dmellado> thanks jordanP and oomichi ;)
09:29:45 <dmellado> hopefully my next topic for next time wouldn't be an earthquake too xD
09:30:09 <oomichi> dmellado: yeah, I hope so. sorry about that :(
09:30:12 <dmellado> I guess that's was all from my side, so if you want you can move to the next topic, and again thanks!
09:30:19 <jordanP> np !
09:30:30 <oomichi> ok, let's move to the next topic
09:30:32 <oomichi> #topic Devstack
09:30:55 <oomichi> does anyone have anything to discuss on devstack this week?
09:31:46 <oomichi> (sorry, I need to ask another topics of tempest before. I'd like to do it on open discussion)
09:32:20 <oomichi> ok, well if there isn't anything else lets move on
09:32:29 <oomichi> #topic Grenade
09:32:49 <andreaf> oomichi: do we have anyone from devstack/grenade team in a TZ compatible with this meeting?
09:33:08 <oomichi> anderstj: I guess no.
09:33:09 <mkoderer> don't think so
09:33:17 <masayukig> andreaf: heh, nice question :)
09:33:32 <gmann> yea, may be we can skip those topic for this TZ meeting?
09:33:46 <oomichi> ok, I will ask the teams for this time meeting later.
09:33:58 <oomichi> gmann: yeah, I think so.
09:34:14 <oomichi> ok, let's move to the next topic.
09:34:15 <oomichi> #topic Critical Reviews
09:34:34 <oomichi> does anyone have any reviews they'd like to get more reviewed?
09:35:04 <oomichi> I have one:) https://review.openstack.org/#/c/169126/
09:35:08 <andreaf> oomichi: well it's not critical - but it's the starting point for moving cred providers to tempest lib: https://review.openstack.org/#/c/185080/7
09:35:08 <oomichi> #link https://review.openstack.org/#/c/169126/
09:35:26 <jlanoux> yes
09:35:52 <oomichi> anderstj: that seems critical for me.
09:36:00 <oomichi> #link  https://review.openstack.org/#/c/185080
09:36:01 <jlanoux> #link  https://review.openstack.org/#/c/208481/
09:36:12 <jlanoux> #link  https://review.openstack.org/#/c/207913/
09:36:27 <jlanoux> #link  https://review.openstack.org/#/c/209144/
09:36:30 <andreaf> oomichi: I'll review your spec today or tomorrow
09:36:36 <gmann> andreaf: that looks good, just 1 comment about updating hacking.rst
09:36:38 <oomichi> anderstj: thanks :)
09:37:06 <oomichi> jlanoux: this is penstack-infra/project-config patch.
09:37:10 <oomichi> oops, typo.
09:37:14 <oomichi> andreaf: thanks
09:37:30 <oomichi> jlanoux: can we(qa team) help it?
09:37:33 <gmann> andreaf: on https://review.openstack.org/#/c/185080/
09:37:37 <mkoderer> andreaf: and I will review your cred provider patch..
09:38:11 <jlanoux> oomichi: I think some of the QA guys have infra privileges and in anyway if we can push that
09:38:27 <andreaf> gmann: thanks, I'll fix that
09:38:39 <gmann> andreaf: Thanks
09:38:44 <oomichi> jlanoux: ah, the second is tempest patch.
09:38:55 <jlanoux> oomichi: and the third one as well
09:38:58 <andreaf> jlanoux, oomichi: I think only sdague from QA is in project-config-core
09:39:06 <oomichi> jlanoux: yeah, ok. I got it.
09:39:10 <mkoderer> jlanoux: what is the current state of SSH validation in general?
09:39:16 <gmann> jlanoux: i will review tempest ssh one tomorrow
09:39:52 <jlanoux> mkoderer: good - only scenario tests are left for migration - then we can improve
09:39:55 <jlanoux> gmann: thanks
09:40:39 <mkoderer> jlanoux: ok cool
09:40:54 <jordanP> another topic: have we ever concidered third party CI for tempest. We broke a 3rd party ci a few days ago, the CI tests SRIOV feature in nova. If that CI commented we would have known something was wrong. (the revert of the bad patch is here btw: https://review.openstack.org/#/c/209039/ )
09:41:56 <oomichi> jordanP: thanks for pointing it out. will review it alter this
09:42:27 <mkoderer> jordanP: if project mirgate to templest plugin concept we could have even more coupling with 3rd ci's
09:42:35 <andreaf> jordanP: there are a lot of third part CI tests - I think it's up to the maintainers of third party CI system to decide which patches they want to monitor and test
09:42:49 <mkoderer> I am fine with 3rd CI in tempest... if they are maintained in a good manner
09:42:51 <jordanP> yeah but are we open to it ?
09:42:58 <andreaf> I mean a lof of third party CI systems
09:43:23 <jordanP> no 3rd party CI comment on tempest patchs
09:43:44 <dmellado> those 3rd party are now and then offline, would that be ok?
09:44:01 <andreaf> jordanP: we use to have some 3rd party CI systems voting against tempest - but I don't see them anymore
09:44:22 <jordanP> the problem is that we have tempest code that is never run in the Gate
09:44:30 <jordanP> like sr-iov related tests...
09:44:30 <mkoderer> andreaf: there were mostly flaky...
09:44:33 <andreaf> jordanP: I'm not opposed to them voting on tempest - but again it's something that the 3rd party system maintainers should decide I think
09:44:58 <jordanP> not voting, but maybe commenting
09:45:10 <mkoderer> jordanP: for such test we definitly need 3rd party CI in tempest
09:45:31 <jordanP> which we have, but they are silent, unless we broke them and we hear from them then :)
09:46:55 <oomichi> jordanP: how to know this problem?
09:46:59 <gmann> but only issue will be their maintenance if they are voting
09:47:09 <oomichi> jordanP: did you receive any comment from them?
09:47:35 <jordanP> oomichi, yes here: https://review.openstack.org/#/c/200819/ after the patch was merged
09:47:52 <oomichi> jordanP: the reverting patch doesn't contain a bug report. so I'd like to know how to work them together
09:47:54 <jordanP> gmann, they should not vote, but leave a comment
09:48:20 <oomichi> jordanP: ah, I see.
09:48:33 <mkoderer> I can talk to the NFV guys and asking about they opinion on that
09:48:39 <jordanP> oomichi, I can't fill a bug report for a system I don"t own or even run
09:48:43 <mkoderer> I will bring that up in the next telcowg meeting
09:49:45 <mkoderer> since SRV-IO is mostly driven by the telco vendors
09:49:58 <jordanP> I am fine with how things are now, but there"s room for improvement
09:50:07 <jordanP> mkoderer, yep, that would be good
09:50:13 <jordanP> mellanox seems to have a working CI
09:50:25 <oomichi> jordanP: yeah, I agree
09:50:45 <oomichi> we will face this kind of problem easily in the future
09:50:53 <oomichi> again
09:51:39 <jordanP> we really shouldn"t accept any new code that is not exercised in the Gate or in commenting/voting 3rd party ci
09:52:37 <mkoderer> jordanP: +1
09:52:48 <gmann> jordanP: totally agree
09:53:11 <oomichi> jordanP: yeah, non-opearated tests are trash
09:53:18 <mkoderer> can I give a short status to tempest plugins?
09:53:30 <oomichi> mkoderer: please :)
09:54:06 <mkoderer> so I proted all tests from manila to the interface
09:54:09 <mkoderer> https://review.openstack.org/#/c/201955/
09:54:19 <mkoderer> it need some little nits and then it's ready to merge
09:54:42 <mkoderer> I am currently a bit uncertain about the still exiting dependency of the tests with tempest itseld
09:54:53 <mkoderer> s/itseld/itself/
09:55:12 <mkoderer> it's currently using tempest.clients, tempest.config, tempest.test
09:56:00 <mkoderer> so if tempest changes something it could break the tests in manila
09:56:25 <jordanP> ( mkoderer I see you've met Jay.Shen a.k.a the english teacher... man last time I almost lost my temper...)
09:56:29 <gmann> mkoderer: i think we have plan to move base tests class, clients in lib
09:56:41 <mkoderer> jordanP: haha :)
09:57:25 <mkoderer> gmann: yeah.. so if everyting is ready in tempest-lib the dependency should be gone
09:57:31 <jordanP> gmann, but it's now going to happen real soon
09:57:36 <gmann> mkoderer: yea
09:57:50 <gmann> jordanP: not soon :)
09:58:04 <jordanP> *yeah not soon !
09:58:15 <jordanP> what a typo :)
09:58:18 <oomichi> I hope soon
09:58:48 <mkoderer> :)
09:58:53 <oomichi> mkoderer: so tempest-lib work is blocking your manila work now ?
09:59:51 <mkoderer> oomichi: not blocking but we have to be careful about it
10:00:10 <jordanP> it's tricky...
10:00:11 <oomichi> mkoderer: ok, I see the situation
10:00:16 <jordanP> (time is over...)
10:00:24 <oomichi> oops, thanks all
10:00:31 <mkoderer> oomichi: thx for hosting the meeting!
10:00:32 <gmann> Thanks
10:00:37 <oomichi> #endmeeting