22:00:25 #startmeeting qa 22:00:26 Meeting started Thu Apr 17 22:00:25 2014 UTC and is due to finish in 60 minutes. The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot. 22:00:27 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 22:00:29 The meeting name has been set to 'qa' 22:00:31 o/ 22:00:35 hi 22:00:36 hi who do we have here today? 22:00:40 hi 22:00:47 #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_April_17_2014_.282200_UTC.29 22:00:53 ^^^ Today's agenda 22:01:41 o/ 22:01:46 all hail the juno PLT! :) 22:01:56 PLT? 22:01:56 hail 22:02:00 PTL 22:02:03 heh 22:02:20 thanks, I guess the torch was passed today wasn't it 22:02:27 yep 22:02:33 well let's get started 22:02:43 #topic Summit sessions (mtreinish) 22:02:53 #link 22:02:53 https://etherpad.openstack.org/p/Juno-QA-design-summit-topics 22:03:21 so I just wanted to remind people that if they have session proposals the deadline for summit.openstack.org is the 20th 22:03:37 so make sure you get your session proposals in before then 22:03:41 I'll do write ups in the morning for my 2 22:03:48 sdague: ok thanks 22:03:54 my english skills are mostly faded for the day 22:04:04 that's fine 22:04:11 masayukig: also you had an entry in the etherpad 22:04:18 Yes, I've added a proposal of Tempest GUI. 22:04:18 can you add it to the tool? 22:04:27 masayukig: yeah the diagram looked cool 22:04:40 mtreinish: Thanks :0 22:05:24 the only other session proposal in the etherpad that didn't have an entry in the tool was mlavalle yfried__ 22:05:32 I'll ping them about it tomorrow 22:05:49 I plan to have a discussion about the session at the next meeting 22:05:57 after the list of proposal is finalized 22:06:20 does anyone have anything else about summit sessions? 22:06:29 where is discussion on qa-specs going to slot in? 22:07:15 I was planning to discuss it in the juno policy session I was going to have 22:07:32 I don't think we'll have enough slots for a dedicated session 22:07:38 sure 22:07:43 that's not in the list yet right? 22:07:50 no although it's in the tool 22:07:54 I'll add it to the etherpad 22:07:55 ok, cool 22:08:03 it doesnt' really need to be in the etherpad 22:08:11 that's what I thought too :) 22:08:29 I was probably going to make that the last session for the qa track 22:08:34 yep 22:08:47 ok lets move on then 22:08:55 how many slots for qa in the summit? 22:09:04 oomichi: it's looking like we'll have 10 slots for qa 22:09:28 which is down 3 from HK 22:09:41 #topic Oslo Liaison (mtreinish) 22:09:43 thanks, I got it. 22:09:56 #link https://wiki.openstack.org/wiki/Oslo/ProjectLiaisons 22:10:14 so the oslo guys are looking for people from the other projects to be focal points as the convert things to libs 22:10:19 and to keep up with oslo stuff 22:10:24 for better cross team communication 22:10:39 I figured we should have someone from the tempest-core group also on that list 22:10:47 does anyone want to volunteer? 22:11:45 well I'll take that as a TODO then 22:12:01 #action mtreinish to find a volunteer to be oslo liason 22:12:34 ok if no one is going to volunteer today 22:12:37 let's move on 22:12:48 get people to sign up at summit with promise of beer :) 22:13:01 sdague: that's not a bad idea 22:13:12 #topic Blueprints 22:13:41 I guess let's start with the specs review first 22:14:02 #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z 22:14:24 it looks like we have 4 specs without a -1 22:14:29 with the release this week, I was lazy on qa-specs 22:14:31 and one of those has a +2 from dkranz 22:14:34 mtreinish: So I started the run-as-non-admin one even though it has not yet been approved 22:14:50 dkranz: yeah we need to get more eyes on these 22:15:01 I'll make another pass through them tonight or tomorrow 22:15:05 I'll take a pass on monday, I expect tomorrow is mostly going to be recovery 22:15:25 I also will see it 22:15:31 dkranz: I'd still like to see a +2 from someone else before I +A something 22:15:35 mtreinish: you should probably approve my blueprint now that you land the spec as well 22:15:37 :) 22:15:54 sdague: oh yeah that's probably a good idea :) 22:16:02 oomichi: ok thanks 22:16:27 mtreinish: you going to send out the email announcing qa-specs and process to people? 22:16:34 yeah I plan to do that tomorrow 22:16:38 I think we've got enough examples now 22:16:40 mtreinish: I think we are dealing a little with the fact that approval used to be yeah, yeah, click the button 22:16:41 awesome 22:16:54 sdague: yeah 3 is probably enough 22:16:56 mtreinish: And now requires much more work from reviewers 22:17:04 dkranz: yeah I think that's part of it 22:17:15 people aren't used to looking at this on a regular basis yet 22:17:28 mtreinish: I made a bookmark :) 22:17:38 right, so probably specs review should be part of every meeting. And everyone needs to try to keep on top of those. 22:18:00 sdague: yeah I'm going to make it a separate agenda item moving forward 22:18:14 +1 22:18:39 also should encourage anyone to help review these 22:18:42 even if not core 22:18:46 all comments are good 22:18:47 yeah definitely 22:18:52 I'll make that part of the email 22:19:02 #action mtreinish to send an email about qa-specs to the ML 22:19:03 great 22:19:32 ok I think we can go onto bp status now, unless someone has something more to discuss about the specs review 22:19:52 so on the agenda there was a bp linked 22:19:55 #link https://blueprints.launchpad.net/tempest/+spec/nova-api-attribute-test 22:20:03 oomichi: ^^^ I think that's yours right 22:20:08 did you put it on the agenda? 22:20:10 mtreinish: I put that there 22:20:13 dkranz: oh ok 22:20:23 dkranz: go ahead then 22:20:29 mtreinish: There was an issue in reviews about the status of extra keys added to a return dict 22:20:43 mtreinish: and whether the schema should allow additoinalProperties or not 22:21:05 THere was a comment that it would be easier to extend if we allowed properties to be added 22:21:15 But the stability guidelines say you need an extension for that 22:21:32 oomichi: Do you have a comment about that? 22:21:43 dkranz: to add something to the body you need an extension 22:21:48 doesn't jsonschema allow for inheritance or something similar though 22:21:55 jaypipes: was talking about that before on the ML 22:21:56 dkranz: do you mean schema should block non dict data? 22:22:29 oomichi: The issue I think was whether our validation should accept extra values or insist on a specific set 22:22:54 If we don't set additionalProperties to False then new keys can be added without failing the test. 22:23:21 do we have a feeling on how the nova team feels about it? 22:23:24 dkranz: yes, right. and the default value of additionalProperties is True. 22:23:32 oomichi: exactly 22:23:49 so now validation accepts extra attributes. 22:23:52 sdague: I'm not sure but we should find out 22:24:01 sdague: Because some schemas have already gone in. 22:24:05 sure 22:24:10 sdague: I only noticed this recently 22:24:23 ok, maybe something to queue up for the nova meeting next week? 22:24:39 Do we have one of us that attends there as well? 22:24:42 sdague: yeah that's probably a good idea 22:24:58 well it's an hour before this one 22:25:05 :) 22:25:06 we should get a volunteer for that 22:25:16 dkranz: do you want to drive that on the nova meeting? 22:25:25 mtreinish: ok 22:25:28 it's a bit early for oomichi right? 22:25:33 dkranz: cool thanks 22:25:36 6am 22:25:42 the nova meeting does do a morning EST one too 22:25:45 yes, 22:25:50 oomichi: are you on that one? 22:25:53 a little for me:-( 22:26:01 #action dkranz to discuss jsonschema additionalProperties on nova meeting 22:26:07 oomichi: what about when they flip to late meeting for you? 22:26:15 mtreinish: I have some sympathy for ssaying that extra values could be added. 22:26:31 but I will join it. 22:26:34 I'm not sure what the difference is to say you check for an extension and then look for the value 22:26:39 6:00am meeting. 22:26:42 vs just seeing if the value is there 22:26:55 sdague: it's 1400 or 2100 UTC 22:27:09 dkranz: the theory is that it's discoverable in advance 22:27:39 sdague: sure, but is the real value of that compensating for extension hell over time? 22:27:43 mtreinish: yes, jsonschema allows to specify another schema doc. 22:27:53 sdague: I'm just saying it is debatable 22:27:59 dkranz: absolutely 22:28:11 our current model just sucks around all this, no argument here :) 22:28:11 sdague: BUt we really have to nail it down soon. 22:28:38 sdague: ok, we'll see what the nova folks say 22:28:46 jaypipes will fix it all, I have faith ;) 22:29:02 heh 22:29:08 dkranz: ok is there anything else on this bp? 22:29:13 I guess so:) 22:29:16 No 22:29:32 ok then does anyone else have a bp to bring up or discuss? 22:29:59 branchless-tempest 22:30:08 this is very close 22:30:14 sdague: ok cool 22:30:19 what's left? 22:30:23 https://review.openstack.org/#/c/88302/ - will make us gate tempest master on stable/icehouse 22:30:44 all the stable/icehouse projects are gating on tempest master because of the way our branch fall through works 22:31:18 ok, yeah that should do it 22:31:20 and the service selection code for devstack-gate was redone so it would be sane to extend it to extensions 22:31:27 though the extensions work isn't done yet 22:31:36 https://review.openstack.org/#/q/status:open+project:openstack-infra/devstack-gate+branch:master+topic:feature_grid,n,z 22:31:42 ok, well we shouldn't have any trouble until a new extension or feature is added 22:31:43 that all went into the gate 22:31:48 correct 22:31:50 sdague: What's the reason for including non-voting icehouse jobs? 22:32:02 they aren't non voting 22:32:12 sdague: the cells job? 22:32:56 good point 22:33:00 I'll dump that later 22:33:14 I thought that had been moved to experimental, apparently missed it 22:33:39 dkranz: -1 it 22:33:48 mtreinish: k 22:34:03 yeh, clarkb wants another change as well 22:34:05 so -1 away 22:34:21 sdague: shouldn't we gate on some of those icehouse jobs too? 22:34:37 mtreinish: with clean check, I'm less concerned about that 22:34:46 ok yeah that's fair 22:34:54 it won't get to gate without it passing 22:34:59 exactly 22:35:05 I figure lets try with check only for now 22:35:19 and if we break ourselves, move more into gate 22:35:20 ok, I was thinking maybe just the mysql full for a sanity check 22:35:23 but that's fine 22:35:51 we are going to apply tempest/master to stable/havana also? 22:35:56 mtreinish: I think relying on check is going to be more common going forward 22:36:00 oomichi: if we can make it work 22:36:10 oomichi: it doesn't work right now 22:36:18 mtreinish: because we can spread different configs across check 22:36:21 it's still a little debatable if it's fully worth the time, given it eol in 5 months 22:36:25 oomichi: dkranz has a spec proposal in progress for that effort 22:36:44 I'm honestly fine with just taking this moving forward 22:36:49 and leaving havana a branch 22:36:54 thanks, I will see it later. 22:37:06 sdague: Yes, one of things we have to discuss is that havana is not going eol soon for a lot of people, presumably including refstack 22:37:15 sdague: But not now 22:37:24 dkranz: sure 22:37:43 now new API test, which is added since Icehouse, is in review. https://review.openstack.org/#/c/84977 22:38:15 oomichi: the API is new in juno? 22:38:23 so I guess we need some option which controlls enable/disable the API test for havana. 22:38:26 oomichi: hmm make sure you put an extension decorator around it then 22:38:52 sdague: no since Icehouse. server-group API of Nova. 22:39:00 oomichi: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/test.py#n182 22:39:03 mtreinish: I am going to look at the havana thing when I am done with the non-admin which will hopefully be soon. 22:39:23 oomichi: if it's in icehouse, we're good still 22:39:27 oomichi: or is it larger scope than just an api extension? 22:39:38 sdague: yeah it won't fail, but we still want to make sure we properly flag things 22:39:42 dkranz: ok 22:39:45 oomichi: https://github.com/openstack-infra/devstack-gate/blob/master/features.yaml - the theory is add the supported list of compute extensions into this mix as well 22:40:27 then devstack-gate will set the right extension lists per feature and branch 22:40:40 we'll talk about it at summit as well 22:40:55 OK, a lot of info to me. I will check them later carefully. 22:41:28 sdague: ok is there anything else on branchless tempest? 22:41:33 nope 22:41:43 ok then does anyone else have a bp to discuss? 22:41:49 otherwise we'll move onto the next topic 22:42:21 #topic Neutron Testing 22:42:31 so mlavalle said he couldn't make it today 22:42:37 but he put an update on the agenda 22:43:00 the take away is that there still 9 outstanding neutron api patches 22:43:15 he pointed out 3 that have 1 +2: 22:43:22 #link https://review.openstack.org/#/c/67547 22:43:28 #link https://review.openstack.org/#/c/63723 22:43:34 #link https://review.openstack.org/#/c/66541 22:44:00 the only other thing there was he proposed another joint qa neutron session 22:44:06 I guess to plan out things for juno 22:44:27 sounds like a good idea 22:44:32 yeah it does 22:44:44 although I think we're in really good shape on that front compared to at the start of I 22:45:17 I'll talk to mestery about that session and coordinate the scheduling for it 22:45:30 ok does anyone else have anything to say on this topic? 22:45:58 Slightly related ... 22:46:04 HenryG: sure 22:46:40 So marun_afk couldn't attend today but he is planning to submit a session too 22:46:52 also very slightly related (but more to the previous topic) A joint Refstack/QA meeting possible? 22:47:00 HenryG: yeah I talked to him about that today 22:47:05 it's already on the etherpad 22:47:10 HenryG: I spoke to him. He put it on the etherpad 22:47:13 he said he'd put it in the tool 22:47:24 ok, cool 22:47:37 rockyg: are you thinking of that as a design summit session? 22:47:37 rockyg: probably not unfortunately unless it's in other projects 22:47:49 rockyg: the qa track is going to be full 22:48:11 oh I just assumed you meant a summit session 22:48:14 No. Just an irc or phone conference to get some more info on how our stuff dovetails with the new, improved tempest 22:48:17 Does refstack have any slots 22:48:32 it looks like it has one - http://summit.openstack.org/cfp/details/197 22:48:33 rockyg: yeah that's something we can do 22:48:38 We've got one to introduce everyone to RefStack. 22:48:43 rockyg: That would be good since I think refstack is part of what instigated this 22:48:45 Sorry, Refstack. 22:48:50 rockyg: send an email to the ML about it 22:48:57 Will do. 22:49:07 we should make sure to try to get that to not overlap the QA track 22:49:16 so folks here can go to the refstack one 22:49:33 sdague: yeah I'll talk to ttx about that (it looks like he's scheduling that track) 22:49:55 ok well let's move on to the next topic 22:49:57 yeh, I just left a comment 22:50:06 we could also do refstack / tempest as a lunch thing one day 22:50:20 Thanks for the scheduling help 22:50:32 Oooh. I kinda like that. 22:50:32 sdague: yeah that would be good too 22:50:56 #topic Heat Testing 22:51:06 oh, update here 22:51:07 Refstack was supposed to be on Icehouse for the summit, but looks unlikely. 22:51:08 sdague: so I know we have some updates here right 22:51:17 the heat job is currently non voting 22:51:30 http://jogo.github.io/gate/ - explains why 22:51:42 the failure rate was going way out of control 22:51:56 sdague: ooh nice spike 22:51:58 it didn't hit 100%, that's an artifact of rolling averages 22:52:04 but it was above 50% 22:52:12 heh ok 22:52:22 there are a few possible issues in it, heat team is looking at them 22:52:40 but we need to not be bouncing things like the icehouse branch opennings 22:53:04 yeah it's unfortunate though, we just turned it on not too long ago 22:53:04 hopefully the failure rate can be addressed and it can come back to voting some time next week 22:53:19 yeh, but honestly every new major effort has fits and starts like this 22:53:29 yeah that's true 22:53:30 we flipped the neutron job non voting a couple times last year 22:53:54 was the switch to parallel execution part of the increase in instability? 22:53:55 the heat jobs are starting to get enough content that they are flushing out some nice bugs as well 22:54:09 that bug we managed to fix 22:54:15 sdague: ok 22:54:18 it was actually a keystoneclient bug 22:54:25 on cert corruption 22:54:44 this looks more like there are some async behavior where not expected 22:54:52 and possibly the fedora guest is crashing 22:55:11 is there a reason it's a fedora guest? 22:55:18 that seems pretty heavyweight for the gate 22:55:30 a number of the heat tests need real cloud-init or cfn-tools 22:55:36 mtreinish: the heat slow tests need a "real" image 22:55:37 neither of which are provided by cirros 22:55:44 dkranz: well some of them do 22:55:50 hmm ok 22:55:58 I think we could actually do some with just curl in cirrors 22:56:07 sdague: right. I suppose those that don't could be moved to the regular run 22:56:20 dkranz: they will still be slow :) 22:56:35 Ah, yes. 22:56:40 honestly, I think the heat dedicated job makes sense 22:56:45 yup 22:56:47 it provides plenty of time budget 22:56:55 just got to work through some issues here 22:57:05 I expect to be working with them through juno to help 22:57:33 sdague: ok is there anything else on heat? 22:57:37 nope 22:57:37 I have to go but I can finish run-as-non-admin in short order I think once https://review.openstack.org/#/c/88347/ goes in 22:57:46 dkranz: ok 22:57:58 ok with <3 min let's do reviews 22:58:03 #topic Critical Reviews 22:58:15 does anyone have any reviews that they'd like to get eyes on? 22:59:02 not from me, just nap time after the release 22:59:07 heh ok 22:59:17 well I guess if there aren't any reviews we can just end it here today 22:59:30 thanks everyone 22:59:35 Thanks, guys! 22:59:44 thanks:) 22:59:47 #endmeeting