17:00:55 #startmeeting qa 17:00:56 Meeting started Thu Sep 29 17:00:55 2016 UTC and is due to finish in 60 minutes. The chair is oomichi. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:57 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:00:59 The meeting name has been set to 'qa' 17:01:10 hi, who are joining today? 17:01:20 \o 17:01:22 Hi 17:01:29 o/ 17:02:11 ok, lets start 17:02:21 #topic Next Summit 17:02:38 we have 7 slots for qa project design sessions. 17:02:58 and we are gathering some ideas for these slots on #link https://etherpad.openstack.org/p/ocata-qa-summit-topics 17:03:27 oops, I needed to pick up today agenda before that: #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_September_29th_2016_.281700_UTC.29 17:03:45 (going back to the summit) 17:04:20 now there is not good enough ideas and it is great to put the ideas on that more :) 17:04:45 rough ideas also are enough at this time 17:05:25 somebody has some ideas at this time? 17:06:06 ok, lets move to the next topic. 17:06:18 #topic Specs Reviews 17:06:36 #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z 17:07:00 there are not any changes since the previous meeting 17:07:36 on the spec. so I feel it is fine to skip this today if no topic on us 17:08:25 #topic Tempest 17:08:53 #link https://review.openstack.org/#/q/project:openstack/tempest+status:open 17:09:22 Now I am trying to define some guideline about negative tests on #link https://review.openstack.org/#/c/379013/ 17:09:48 some good feedback is gotten already, so I will update it after this meeting 17:10:30 are there other items about Tempest today? 17:10:46 I have something, if that's okay 17:11:01 DavidPurcell: Please go ahead :) 17:11:05 Thanks 17:11:35 The company I work for has written a framework and suite of tests for testing RBAC policy enforcement within Tempest. 17:11:53 If there is interest, we were hoping to begin work on pushing it out to the community. 17:12:19 DavidPurcell: that sounds similar to: https://review.openstack.org/#/c/314704/ 17:12:40 DavidPurcell: do you have a link to the tool? 17:12:46 DavidPurcell: that is interesting. Now which projects(nova, cinder, etct) are supported? 17:13:06 Currently I think we have tests for: Nova, Glance, Cinder, Murano, Neutron, Heat 17:13:39 I have seen the Cinnamon-Role project (the one in the review), but it seemed to be abandoned in May. 17:13:55 DavidPurcell: cool, yeah I also want to see the implementation way with some link if possible 17:14:32 DavidPurcell: yeah I don't think rlossit has had time to touch it a while 17:14:43 I didn't prepare something for this meeting, but I can have something to show soon 17:15:04 DavidPurcell: thanks :) 17:15:57 mtreinish: Right, that is one of the reasons I was asking, I wanted to know if that project had been dropped, and if an alternative solution was open to consideration. 17:16:17 DavidPurcell: one question about this kind of tests: We can change policy on productions as they want, so how to verify these policies work fine on the tests? 17:16:53 The policy configuration files should be input on the tests? 17:17:00 DavidPurcell: I think there would be, there has been desire for an rbac tool like this in qa. It's mostly been a matter of someone writing one 17:17:50 the blocker on cinnamon-role was it's use of unstable tempest interfaces. We were planning to add it to qa when that was fixed (by stabilizing the interfaces into tempest/lib) 17:17:51 Our current solution involves reading the apis from a yaml file we created based on the policies, and then using that to determine if the api being tested should be accessible by that role 17:18:49 DavidPurcell: but if your implementation works, I definitely think it's worthwhile to look at it as an alternative 17:19:03 it's hard to say anything concrete without having a repo to look at though 17:19:22 humm, interesting. yeah I am looking forward to seeing it 17:19:27 Agreed, I wish I could just share it now, but internal restrictions :( 17:20:09 Would the openstack-qa chat be the best place to discuss it once I have approval to share code? 17:20:10 DavidPurcell: no problem, that is common thing in companies 17:20:38 DavidPurcell: yeah, that is a right channel 17:20:51 okay, thanks a lot! 17:21:23 DavidPurcell: thanks for picking this up. are there another topic about Tempest today? 17:21:37 all: are there another topic about Tempest today? 17:21:38 I wanted to talk about this patch: 17:21:43 #link https://review.openstack.org/#/c/373428/ 17:22:04 mtreinish: ok, please go ahead :) 17:22:16 it's still a wip, but I wanted to know if people had feelings on using ostestr as the underpinnings of tempest run instead of testr directly? 17:22:24 because that's what the patch is proposing 17:23:05 slowrie, dpaterson: ^^^ 17:23:14 mtreinish: maybe do you have some concern about this? 17:23:44 oomichi: well we made a decision to use testr directly when we did the initial implementation (I don't remember the reason why though) 17:23:45 mtreinish: the patch itself seems reducing the code from Tempest by using it 17:24:04 it does but it leverages ostestr to do a lot of work under the covers 17:24:19 and ostestr itself runs testr by subprocessing out 17:24:35 which I think is my bigger concern, is that we're adding another fairly complex layer in the middle 17:24:53 oomichi, mtreinish, could you spare some minutes for review? https://review.openstack.org/#/c/378483/ 17:25:42 so the call path becomes tempest run -> ostestr -> testr -> CONC x subunit.run -> testtools.run -> unittest.run 17:26:16 mtreinish: oops, that seems very complex.. 17:26:37 oomichi: well even if you remove ostestr it's still complex :) 17:26:53 I'm not opposed to doing this, but I wanted other people's thoughts on it 17:27:15 mtreinish: yeah, TBH I was surprised calls of testr at this time :) 17:27:42 mtreinish: ok, could you put some comment on https://review.openstack.org/#/c/373428 ? 17:27:54 mtreinish: the above explanation is good for reviewing 17:28:09 lhx_: sorry for late response, please go ahead 17:28:51 lhx_: ah, that is simple patch. I will review it after this meeting. 17:29:09 oomichi, sorry for interrupt you. thx for review 17:29:31 lhx_: no problem, and thanks for doing that 17:30:04 oomichi, haha, it's my job. 17:30:05 ok, are there any topic about tempest today? 17:30:19 lhx_: :) 17:30:55 ah, I need to bring one thing before moving 17:31:06 #link https://github.com/oomichi/bug-counter#current-graph 17:31:37 the bug number of Tempest is still decreasing by triage. Thanks for helping many people 17:31:52 oomichi: hmm, we need to make the lines bigger in the legend. It's hard to see the colors 17:33:06 oomichi: how many NEW bugs does tempest have? It's good to have that in single digits 17:33:14 mtreinish: hehe, yeah, it is great to do that. but that is not my work :) 17:33:18 it looks like 30-40 by the graph 17:33:48 mtreinish: yeah, maybe around 30 on NEW 17:34:01 mtreinish: and the number seems increasing in this week 17:34:14 so we need to do some triage about that 17:34:56 mtreinish: do you have some idea for puting a digits on the graph? 17:35:47 and alternative way is #link https://review.openstack.org/#/c/372581/ 17:35:57 oomichi: that get's tricky to add, and makes it difficult to read. (that's why interactive graphs are nice) I can add more Y axis tick points to try and make it clearer 17:36:16 the patch adds long-term bug progress on bugstate 17:36:59 mtreinish: can we add Y axis on right side? 17:37:20 oomichi: lets pick this up after the meeting 17:37:31 mtreinish: yeah, I agree 17:37:45 ok, let's move on 17:38:08 #topic DevStack + Grenade 17:38:38 Do we have some items necessary to discuss them today? 17:39:00 the big thing for devstack and grenade this week is that we've branched them for newton 17:39:36 cool, we are moving to Ocata now 17:39:45 looking at the newton branch though we still have some post branch stuff to do 17:40:18 mostly just updating lib/tempest with the max microversions and the hard coded extension list 17:40:34 we also need to do the extension list for liberty, and mitaka (that came up on the ML) 17:40:46 oomichi: do you think you could push patches up for all that? 17:41:27 mtreinish: ah, I missed the mail of this topic maybe, is that for Nova extension or the other projects? 17:41:59 #link http://lists.openstack.org/pipermail/openstack-dev/2016-September/104754.html 17:42:11 it was about adding a neutron test for an extension added in mitaka 17:42:29 we never hard coded the extension list on liberty or mitaka, so tests for new features fail on stable 17:44:21 mtreinish: humm, is it ok to just put some extensions list on stable devstacks? 17:45:10 I will follow that after meeting anyways. 17:45:29 #action oomichi follows http://lists.openstack.org/pipermail/openstack-dev/2016-September/104754.html 17:45:33 it's a documented part of each release... 17:45:38 #link https://wiki.openstack.org/wiki/QA/releases#Devstack 17:45:47 it's part of branchless tempest's implementation 17:46:00 but sure, we can follow up after the meeting 17:46:32 mtreinish: thanks. yeah it is nice to do that after meeting 17:46:56 do we have more topics about devstack + grenade today? 17:47:06 ok, lets move on 17:47:14 #topic openstack-health 17:47:36 #link https://review.openstack.org/#/q/project:openstack/openstack-health+status:open 17:48:24 do we have discussion items about o-h ? 17:48:39 nothing from me this week 17:48:51 there are some patches on the list, and it is nice to just review 17:49:02 mtreinish: thanks, ok lets move on 17:49:17 #topic Critical Reviews 17:50:33 I'd like to pick #link https://review.openstack.org/#/c/379013/ up as a critical review. I hope a concenses is great from many reviewers 17:50:59 do we have more patches as critical or important? 17:51:52 nothing from me this week 17:52:27 ok, let's move on 17:52:41 #topic Open Discussion 17:53:37 somebody have some discussion items today? 17:54:29 ok, lets end this meeting 17:54:33 thanks all 17:54:36 #endmeeting