09:00:05 #startmeeting qa 09:00:06 Meeting started Thu Nov 12 09:00:05 2015 UTC and is due to finish in 60 minutes. The chair is oomichi. Information about MeetBot at http://wiki.debian.org/MeetBot. 09:00:07 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 09:00:09 The meeting name has been set to 'qa' 09:00:17 who's here for today's meeting 09:00:22 o/ 09:00:30 o/ 09:00:31 o/ 09:00:56 #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_November_12th_2015_.280900_UTC.29 09:01:02 ^^^ today agenda 09:01:12 Ok, let's get started 09:01:34 #topic Mitaka Sumit: Summarize 09:01:53 we had Tokyo summit and we need to summarize it as conclusion 09:02:08 hey oomichi it was me who proposed that as I couldn't attend the mitaka summit and wanted to get to know the main action points ;) 09:02:14 but to be honest, I saw this agenda item at this time :( 09:02:28 and I forgot the link of mitaka prio etherpad 09:02:56 dmellado: hey, I am serching this 09:03:02 #link https://etherpad.openstack.org/p/mitaka-qa-priorities 09:03:04 https://wiki.openstack.org/wiki/Design_Summit/Mitaka/Etherpads#QA 09:03:06 ? 09:03:15 andreaf: thanks ! :-) 09:03:37 yea in that we have conclusion and target dates 09:03:50 As the etherpad, we have conclusion 09:03:54 gmann: yeah 09:03:57 oomichi: awesome 09:04:09 oomichi, dmellado: I guess we could have a message in the ML like other projects did - but I'd say that's something for mtreinish if he wants to :) 09:04:11 the target date is closed on some items 09:04:30 andreaf: sounds fine for me 09:04:37 andreaf: yeah, nice idea 09:04:47 andreaf: mtreinish didn't it yet? 09:05:04 I don't see it now 09:05:32 yea, on ML it is not yet 09:05:39 ok 09:05:43 #action mtreinish will send a mail of tokyo summit conclusion -dev ml 09:05:50 how about ^^^ ? 09:05:56 +1 09:06:03 yea 09:06:37 ok, will ask him. maybe he will know it by this log :) 09:07:18 to be honest, microversion test is a little close 09:07:31 and we need to get +A on the spec 09:07:41 but I will talk about this later 09:07:47 oomichi: I can lend a hand on that if you're tigh on schedule 09:08:18 dmellado: thanks :-) ok, let's talk about this later :) 09:08:22 sure 09:09:02 the mail will be very important for tracking our aciton items 09:09:29 can we move to the next topic? 09:09:39 fine for me 09:09:41 yea 09:09:48 thanks 09:09:50 #topic Specs Reviews 09:10:01 #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z 09:10:35 some available specs are updated 09:11:18 gmann: can we talk about microversion testing? 09:11:23 oomichi: yea 09:11:29 gmann: that is the first spec on the list 09:11:33 oomichi: as you mentioned spec is close to merge 09:11:35 #link https://review.openstack.org/#/c/169126/12 09:11:48 hi. (sorry I'm late...) 09:11:55 hi jordanP ;) 09:12:00 jordanP: hi :-) 09:12:03 i was thinking about updating ref there for base patch 09:12:16 oomichi: what do you think? 09:12:21 or leave as it is 09:12:33 gmann: that means not spec patch? 09:12:47 gmann: base patch means tempest patch instead 09:13:31 oomichi: in spec, there is comment about having base patch as reference to understand the spec clearly 09:13:44 oomichi: yea Tempest patch as POC 09:13:51 gmann: ah, nice to do that 09:14:04 oomichi: ok will update after meeting 09:14:14 gmann: cool, thanks :-) 09:14:28 so after that it should be fine to merge soon :) 09:14:43 gmann: yeah, we can do that later 09:15:08 andreaf: your resource spec is there on the list 09:15:46 andreaf: do you have comments on that? 09:15:49 oomichi: yes I updated the spec after the session in Tokyo - it would be great to get reviews on it 09:16:19 andreaf: thanks for doing that, unnecessary spaces at line 102: NIT ;-) 09:16:41 oomichi: ok I will re-spin it once I get more comments :) 09:16:44 andreaf: yea, ll try to review it tomorrow. 09:16:52 thanks 09:16:57 andreaf: yeah, will review it soon. that is good for production testing 09:17:31 maybe we can move the next topic 09:17:55 #topic Tempest 09:18:14 dmellado: the first is yours. can you ;-) 09:18:33 sure ;) 09:19:05 dmellado: can you introduce it? 09:19:09 There's this script that we've been using in order to get a tempest.conf file 09:19:26 who was initialy written by dkranz 09:19:41 and I think that it'd be a nice tool to have upstream in order to configure tempest not only against devstack 09:19:48 but for any cloud too 09:19:54 https://github.com/redhat-openstack/tempest/blob/master/tools/config_tempest.py 09:20:08 #link https://github.com/redhat-openstack/tempest/blob/master/tools/config_tempest.py 09:20:12 I was speaking with jordanP about this and he suggested creating also some functional tests for it 09:20:42 dmellado: what kind of functional tests? 09:20:55 what do you think about the idea? 09:20:58 (and what would you suggest about the steps to do so) 09:21:24 oomichi: jordanP suggested something in the lines of comparing the tempest.conf file generated by devstack with one created by this tool 09:21:36 but I'm all open to suggestions 09:21:44 dmellado: ah, I see. 09:22:05 dmellado: but what is the main difference between devstack and this scirpt? 09:22:11 dmellado: does this tool create test resources, or it discovers them or both? 09:22:18 or run "verify_tempest_config.py" on that file 09:22:21 oh, sorry 09:22:27 somewhere did we talk about generating the conf using tempest CLI ? m do not remember exactly 09:22:35 well, basically this script can be used to create a tempest.conf file aganinst any cloud 09:22:37 and not only devstack 09:22:49 it can also create the resources, if needed 09:23:02 andreaf: both, like it create flavor etc if there is no 09:23:18 #link https://github.com/redhat-openstack/tempest/blob/rebased-upstream/tools/config_tempest.py#L520 09:23:32 dmellado: that means the script gets extensions' list via REST API and generates the conf? 09:23:46 andreaf: do you remember to have CLI to generate conf like that ? 09:23:46 oomichi: exactly 09:23:49 dmellado: based on the available extensions/features 09:24:04 I'm reviewing it myself but it'll be a nice additions 09:24:09 may be we can use there 09:24:21 dmellado: Rally can generate tempest.conf as well 09:24:27 dmellado: we tried to do that before in tempest, but the idea was denied by sdague and mtreinish 09:24:40 oomichi: that was bad=( 09:24:40 like the cleanup script and so on 09:24:40 maybe we can integrate them all with the cli... 09:24:44 oomichi: we have spec for that 09:24:50 because if the detecting feature contains a bug, most tests will be skipped 09:24:55 as gmann said 09:25:18 oomichi: why? 09:25:24 just read 09:25:35 basically, we need to understand what tests will run against the clouds 09:25:36 dmellado, gmann, oomichi, boris-42: I'm not against having the script, but I would not use in the gate the discovery part 09:25:52 andreaf: my usecase wouldn't be in the gate 09:25:55 discovery is a bad idea, it's better to configure what we expect to have 09:25:57 but as a tool for tempest 09:26:16 it can also accept a set of options to override 09:26:17 dmellado: if the script misunderstands there is not available extensions on the cloud, the corresponding tests are skipped 09:26:21 a first step would be to add this script in Tempest repo, add some tests around it. No use it directly for the gate. 09:26:29 yea on gate discovery much more risky 09:26:41 because if you disable something by mistake, or if discovery is broken, you will stop testing things and not even realise 09:26:55 andreaf: exactly 09:26:57 andreaf: exactly 09:27:06 gmann: conflict :) 09:27:12 then I'll create some tests and add the script as a "feature" for using it 09:27:35 if it sounds ok for all of you guys ;) 09:27:40 if during cloud upgrade some extensions got broken then tempest would be able to catch those 09:27:59 however the part that generates the resources or the accounts file could be used, and perhaps replace devstack implementation of it 09:28:04 would -> would not 09:28:28 andreaf: actually I was thinking about getting rid of that 09:28:48 dmellado: basically, I can understand this feature because OpenStack has a lot of APIs and it is difficult to set up Tempest by hands 09:28:50 and just rely on accounts.yaml script 09:28:51 guys, for now it's just a tool that can be useful to some, we have a folder called tools in tempest 09:29:06 jordanP: that was my intended path 09:29:35 it is better to discuss it on the ml for getting opinions before having it on tempest 09:29:48 jordanP: sure, but it can be useful for people if it works, and to keep it working it's best if we can exercise it 09:29:51 because pre-PTL and PTL are against it 09:30:04 andreaf, exactly. That's why I asked to add tests 09:30:33 maybe a non voting job, or some unit tests (I'd rather have functional tests than units) 09:30:43 or how about proposing the spec? 09:30:45 but let's agree on the idea first 09:30:48 oomichi: andreaf how about having it added to the tools folders with some tests after removing the create part 09:31:04 a spec for something that is already here, I am not sure 09:31:18 dmellado, let's raise the issue on the ML 09:31:33 I'll send an email 09:31:35 jordanP: it was there. but good to first get agreement on ML before spec 09:31:40 dmellado: thanks:) 09:31:50 dmellado: Thanks. 09:32:07 #action dmellado will send a mail about auto-genenating conf script to -dev ml. 09:32:07 np, let's see what the feedback is and let's agree on its usecase 09:32:26 dmellado: thanks again 09:32:30 np oomichi 09:32:31 #link https://review.openstack.org/#/c/94473/ 09:32:36 previous spec for the same 09:33:03 should we revisit the spec? 09:33:12 gmann: nice point 09:33:23 dmellado: it is nice to propose the other spec 09:33:23 dmellado: i would suggest after ML conclusion 09:33:41 +1 on that 09:34:27 I'd like to move the next item 09:34:34 the next item is mine 09:34:42 that is Service clients 09:34:52 #link https://etherpad.openstack.org/p/mitaka-tempest-service-clients 09:35:12 I created an etherpad ^^^ for managing this 09:35:32 and we need to create a lot of patches for that before M-2 09:36:02 please write your names as Assignees if interested in ;) 09:36:30 my mistake was I decided the deadline is M-2 anyways 09:36:48 oomichi, so bold of you ! :) 09:36:54 lol 09:37:06 heh 09:37:10 I can help starting in 10 days, I have a lot to do right know 09:37:25 I still think it's doable 09:37:26 jordanP: mtreinish denied to change the deadline 09:37:35 oomichi, I don't remember that ! 09:37:46 :D 09:37:48 so we need to implemet them before ! 09:37:58 jordanP: thanks ;-) 09:38:08 oomichi: I'm trying to get volunteers to help on that - if we had a stable set of clients by M2 in tempest-lib it would be great 09:38:18 oomichi: would your script still work for those? xD 09:38:32 andreaf: thanks ! :) 09:38:45 dmellado: yes, that should work 09:38:49 here, I committed on tempest/services/volume/json/*.py 09:38:51 oomichi: after microversion i can also help 09:38:59 then I can take some of those too 09:39:34 jordanP: did you post the patches? 09:39:38 oomichi: I'll ping you after the meeting for this 09:39:55 dmellado: yeah, lets talk about it later 09:40:02 gmann: thank you also 09:40:15 oomichi, no. I just added my name in the etherpad 09:40:20 not 'git commit' 09:40:32 jordanP: thanks much! 09:40:58 ok, I can get many people 09:41:15 lets move to the next item 09:41:24 gmann: can you introduce API microversions testing? 09:41:33 oomichi: sure 09:41:45 gmann: thanks 09:41:53 as discussed early, spec is almost in 09:42:03 and in parallel there are base patches out 09:42:06 #link https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/nova-microversions-tests,n,z 09:42:36 currently m implementing microversion for Nova v2.2 microversion to tests the base framework 09:42:47 might be able to do by tomorrow 09:42:54 gmann: cool, does it work now? 09:43:06 early feedback is nice if all can review those base patches 09:43:09 gmann: ah, I see. 09:43:24 oomichi: not yet completed, still doing version of schema for v2.2 09:43:35 gmann: nice progress 09:44:02 oomichi: as mentioned in spec, once framework is tested well then we can think to move that in lib 09:44:12 deadline is M-1 :) 09:44:22 gmann: yeah, too close 09:44:33 gmann: but it is available since current progress 09:44:55 gmann: it is enough to implement v2.2 test only before M1 09:45:02 oomichi: thinks so. lets see 09:45:17 oomichi: yea so that we can tests the framework on gate 09:45:44 gmann: cool, I see 09:45:50 gmann: I will review them later 09:45:56 oomichi: Thanks 09:46:10 * oomichi having AIs after meeting 09:46:27 #action oomichi will review microversion tests patches 09:46:38 thats all on microversion testing 09:46:51 gmann: thanks 09:47:02 lets move to the next topic 09:47:22 we can skip devstack and grenade, maybe as every meeting 09:47:36 #topic OpenStack Health 09:47:47 #link https://etherpad.openstack.org/p/openstack-health-tracking 09:48:17 can someone talk about this? 09:48:37 I don't have a very consistent update to provide here, is anyone around who can talk about this else I'll do my best 09:48:52 is glauco around? I don't think so 09:49:02 dmellado: no probably not 09:49:17 andreaf: could you ? 09:49:25 dmellado: well o-h is not integrated in the OpenStack css 09:49:41 andreaf: wasn't that a WIP? 09:50:21 dmellado: yes probably, but at least now the page headers and footers are maintained when browsing through o-h 09:50:46 work is still in progress to make performance improvements for the test view 09:51:01 #link http://status.openstack.org/openstack-health/#/ 09:51:08 http://health.openstack.org/ doesn't look right :) 09:51:16 that seems nice now 09:51:36 really easy to know current situation :) 09:52:08 and there is one feature that would be a great usability improvement, which is data grouping by arbitrary run metadata key 09:52:22 but that requires some normalisation of the rest API first 09:53:05 the routes of the rest API have been growing rather organically, so we need a bit of rework there 09:53:19 andreaf: what kind of normalisation? 09:53:50 andreaf: do we need to implement microversion on o-h also? ;) 09:54:10 :p 09:54:36 oomichi: I think the main issue is the we now have /projects , where project is a specific metadata key, and we need to support / instead 09:55:18 andreaf: that seems nice direction 09:55:41 oomichi: I know mtreinish does not want the API to be tightly coupled to the UI workflow (page 1 to 4), but tbh I don't have all the right details here to provide a proper update on this point 09:56:24 andreaf: yeah, it is so fast step to do that on o-h 09:57:04 andreaf: thanks for providing 09:57:19 ok, lets move to the next topic 09:57:27 #topic Critical Reviews 09:57:38 do we have any critical patches for reviewing 09:57:41 ? 09:58:29 * oomichi masayukig introduce o-h for japanese users on http://thinkit.co.jp/story/2015/11/10/6594 09:58:51 cool 09:59:34 cool pic there :) 09:59:51 I had https://review.openstack.org/#/c/168762/ but andreaf already -1 :p 10:00:04 oomichi: also periodic-qa and periodic-stable are part of the collected data now 10:00:42 oh, time comes 10:00:44 thanks all 10:00:48 Thanks all 10:00:48 #endmeeting