15:01:45 #startmeeting DefCore 15:01:46 Meeting started Wed Jun 17 15:01:45 2015 UTC and is due to finish in 60 minutes. The chair is eglute. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:47 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:50 The meeting name has been set to 'defcore' 15:01:52 o/ 15:01:55 o/ 15:01:56 o/ 15:02:02 0/ 15:02:09 o/ 15:02:10 o/ 15:02:10 0/ 15:02:19 o/ 15:02:53 #link https://etherpad.openstack.org/p/DefCoreFlag.4 Etherpad for today 15:03:01 Hello Everyone! thanks for joining. We are taking attendance by raising hands o/, so please do so if you have not done so yet 15:04:21 thank you markvoelker for sending out the link to the etherpad. 15:04:30 * markvoelker tips hat 15:04:45 Hello all. 15:04:47 #topic mid-cycle meeting 15:05:10 thank you everyone that had voted on the doodle poll. 15:05:13 #link http://doodle.com/karnnaxfrefumneb 15:05:47 i made a mistake in the austin dates, off by one day. if you voted for austin, does that make a difference? 15:05:48 Anyone here who hasn't answered the poll that would like to real quick before we discuss? 15:05:58 eglute: not for me 15:06:27 Just doodled 15:07:06 * markvoelker hums Final Jeopardy music 15:07:21 eglute: I'm going to be in Austin for those days, so I would just need to extend my stay there 15:07:28 eglute: no ## Attendees: in etherpad? 15:07:33 based on the poll, looks like Austin right after the board meeting will be the time/place. we will work on finalize the details 15:07:33 But would have to do that anyway. 15:07:41 kbaikov: we've started doing that via IRC logs 15:07:46 kbaikov meeting bot takes attendance for us 15:07:53 got it 15:08:02 that is why everyone o/ raises hands :) 15:08:05 or speaks. 15:08:19 So, looks like Austin is polling pretty strongly... 15:08:33 yes it is. markvoelker do you think you would be able to make it? 15:08:57 I will need to see if I can shuffle a few things, but I'm optimistic 15:09:39 excellent. anyone else that might have good reasons for not holding midcycle in austin that week? 15:09:49 o/ 15:10:17 eglute: none from me. What was the actual hosting venue for Austin? Foundation HQ again? 15:10:25 if not, i will work on finalizing details/invite so that people can plan travel 15:11:10 markvoelker we are still working on that. Rackspace Austin office is one of the options. i will ask foundation about hosting at their HQ, unless people have other ideas 15:11:18 was thinking IBM may host too 15:11:42 ok, cool. Will await further details and start shuffling. 15:11:52 I'm letting foundation staff know that we're going to be in town, and I'll let you know if we have space. 15:12:12 Depending on how many people are coming it may get cramped. 15:12:17 that would work as well. i am confident we will find the place, and in the mean time, i will send out "hold the date" invite for travel planning purposes 15:12:53 #action eglute send out hold the date invite for DefCore Mid-Cycle in Austin, TX week of July 28th 15:13:28 #topic capabilities subdivision into v1.3 15:14:06 VanL hogepodge have you had a chance to work on capabilities subdivision? 15:14:17 eglute: I have not. 15:16:04 hogepodge: vanl: do you need help from the rest of us, or just been busy? 15:16:25 I've been traveling, sorry 15:16:30 * markvoelker totally understands being busy 15:16:30 markvoelker: mostly busy, but can turn my attentions back to it. 15:16:38 I was on the other side of the clock and massively jetlagged :( 15:17:04 I know Catherine worked on the capabilities as well, has she been attending the IRC meetings? 15:17:05 No worries, we have some breathing room yet according to the timeline (see bottom of etherpad). Just wanted to see if you needed assistance 15:17:23 So we have a proposal (in the googlesheet) that we worked on that has the mapping from tests to new capabilities 15:18:04 That has not been properly entered into a updated json, due to difficulties making sure that all the details are just right. 15:18:13 VanL hogepodge agree with markvoelker, let us know if you need help. also, dwalleck offered to help as well, i can connect you if you would like 15:18:37 VanL: eglute: markvoelker: I'll start on, and try to finish it, today. 15:18:49 thank you hogepodge VanL 15:18:55 Has anyone looked at the reformulated capabilities? 15:19:13 Are there any concerns? 15:19:15 VanL: I've perused them a bit 15:19:18 No major concerns 15:19:29 #action hogepodge VanL will work on subdividing the capabilities 15:19:31 It appears to be a much nicer breakdown than we had in the past. 15:19:38 agreed! 15:19:59 VanL, no, they look good to me 15:20:06 we've got some patches that will need to be rebased 15:20:06 Hmm, we should link to that sheet again....let me dig up the URL 15:20:14 but that's not an issue 15:20:24 Ok, in which case is the next step getting that accurately reflected in a json doc patch? 15:20:26 #link https://docs.google.com/spreadsheets/d/15Fkt2n95WPPe7CYH-0WIKz3dhMeW3FW2gl5aSlamEeY/edit#gid=561264013 capabilities breakout 15:20:33 Not actually subdividing 15:20:50 It would be reflecting that subdivision in our official docs. 15:20:53 Is that right? 15:21:12 VanL: It would be transforming the next document to match the google docs reclassification 15:21:22 Right. 15:21:37 * rockyg is now present 15:21:37 I was referring to #action hogepodge VanL will work on subdividing the capabilities 15:22:05 Should that be: #action hogepodge VanL will work on creating a patch for the json reflecting subdivided capabilities 15:22:47 #info merging that patch will put the flag patches into conflict, but I'll maintain those as the repository is changed 15:23:05 thank you hogepodge 15:23:20 hogepodge: is it reasonable to think the majority of those flag patches could land first? Seems like a lot of them are close. 15:24:01 we could review the patches now, if that would help 15:24:34 markvoelker: I think a bunch can, I'd like to see them reviewed by the community, and some are just waiting on the process updates 15:24:47 markvoelker: esp the tests being removed. 15:25:07 did we reach a good place w/ flag rules? 15:25:27 hogepodge: got it. Either way should be pretty simple to maintain those, so no real need to worry about the order I suppose. 15:25:29 given that we can add new rules when we find reasons 15:26:09 #link https://review.openstack.org/#/c/188661/ Current discussion on flag rules 15:26:35 I am ok with that 15:26:54 #topic review hacking file 15:27:51 I really need to take a pass on 188661. I'll do that now. 15:27:59 So, that makes at least two folks that would like the list to be non-exhaustive. Anyone else have strong opinions? 15:28:18 if the security item the primary question? 15:28:20 anyone else had a chance to take a look at https://review.openstack.org/#/c/188661/ 15:28:50 I'd be happy to update the wording on line 61 per my last comment on the patch if nobody feels differently. 15:28:54 non-exhaustive +1 15:28:58 I'm OK if we hold on it and add it when needed 15:29:14 exhaustive with room for updates so that expectations are clear but not inflexible? 15:29:26 non-exhastive +1 15:29:33 So if the reason is not on the list, update the process then update the flag? 15:29:35 *non-exhaustive 15:30:18 hogepodge that's exactly my concern. 15:30:29 we'd need people to add a new flag if they come up with a new reason 15:30:37 I think we need a "misc" category to avoid burying newly identified distinctions in existing categorie/facets. 15:31:02 * markvoelker pushes updated wording on line 61 to make this non-exhaustive 15:31:17 See patchset 6 please. 15:31:29 I believe we expect that someone who wants a flag to name which valid reason they're citing, yes? 15:31:47 #link https://review.openstack.org/#/c/188661/5..6/HACKING.rst 15:31:56 purp: yes, per rule D307 15:32:11 that does not link up w/ my understanding 15:32:14 Okay, then the behavior would seem to work out like so: 15:32:23 I thought we were trying to keep flags limited to reasons identified 15:32:50 1. Find flaggable thing, 2. Search valid reasons, 3. Pick one that's closest since it's required and I want to get this done. 15:33:02 zehicle_irl: err...wait, I thought you just said you wanted the list to be non-exhaustive and for us to be able to add to it? 15:33:06 purp, what if it does not match? 15:33:15 zehicle_irl i think in this case we risk missing good reasons 15:33:19 sorry, not being clear 15:33:26 If it doesn't, then I have a much harder task to submit a flaggable thing. 15:33:36 I expected that the list of flag reasons would be the exhaustive list 15:33:44 I have to reach out to #defcore, get them/us to agree, make a new category, then submit the flag. 15:33:48 purp: that's part of the point. New reasons needs to be reviewed. 15:33:56 if someone has a reason that's not covered, we'd have to add it 15:34:19 the original text was correct IMHO 15:34:26 hogepodge: totally agree. Would prefer a "misc" category which requires us to validate, categorize into an existing category, or make a new one. 15:34:35 zehicle_irl: ++ 15:34:40 Many will fail validation (as now) 15:34:48 purp +1 15:35:00 I'm worried that a "misc" flag just puts us back where we were 15:35:06 Some will be validly in an existing category, but linguistic barriers made it hard for submitter to understand where. 15:35:28 Very few might coalesce around a real, new category. 15:35:39 but then we'd have to go back and fix the flags 15:35:53 would rather have that discussion up front 15:36:07 zehicle_irl: if they submit to misc, there's extra delay and process. Makes a counter=incentive to the "just put it all there and let #defcore sort it out" 15:36:09 could create some delays, but it makes the decisions pretty clean 15:36:24 Or vice versa: falgs properly categorized get fast attention. 15:36:24 we have an example of a new flag need and then have a concrete discussion about it 15:36:42 in that case, we'd want to reference the D### info in the flag details in JSON 15:36:55 purp: I'm worried misc would be a dumping ground and a way to admit things without proper review 15:36:57 I'm not tremendously worried about someone requesting a flag that only fits dubiously into an existing category actually... 15:37:03 purp: especially at the last minute 15:37:06 I think we as reviewers would catch that during review. 15:37:23 if the flag is logical, adding it would be easy 15:37:30 hogepodge: I'd be comfortable saying that "misc" flags are never issued; they have to move to a category. 15:37:32 if the flag is not then we'd need discussion. 15:37:39 And decide either "yes, this fits an existing reason" or "crap, we never thought of that, let's get a new reason added pronto" 15:37:41 Makes the submission easy, and puts the burden in the review process as should be 15:38:02 I'd be OK if we put "pending" instead of "misc" 15:38:26 so that's like the old D999 pixie dust - you can use that for a patch but we won't merge it 15:38:32 I guess where it lies for me is that I want people to *submit* things they believe need to be flagged for examination; that's data which we learn from. I don't want to make *approval* easier. 15:38:43 zehicle_irl ++ 15:38:44 +1 pending 15:38:49 pending is good. i think we will have "crap we never thought of that category" 15:39:10 (and +1 pixie dust. I used to work for Disney) 15:39:27 purp: So what about this... 15:39:32 i was sad to see pixie dust be removed. 15:39:38 * rockyg sneezes 15:39:47 Bless you, rockyg 15:39:55 * purp listens to markvoelker 15:39:59 * zehicle_irl buys some glitter 15:40:07 purp: Rather than adding a "misc" or "pending" category, what if we ammended the wording on line 61 again such that we basically say 15:40:52 If you think something needs to be flagged and it doesn't meet the criteria below, please submit your flag request anyway and DefCore will consider whether or not the rationale is reasonable so we can add a corresponding rule 15:41:19 (that is terrible wording that I wouldn't actually put in the file....but you get the picture) 15:41:29 markvoelker: I have this urge to have a real reason so a flag request can be validated as requiring one. 15:41:30 That way we don't have "dumping ground" category 15:41:40 I liked purp 's suggestion to have a rule for that 15:41:53 markvoelker: and I respect that we don't want a dumping ground. Did we feel like we had one pre-categorization/ 15:41:54 the placeholder / pending discussion 15:41:55 ? 15:41:58 But we do have a way for people to say "this is a valid reason for not requiring a test, please consider it" 15:42:28 especially because it lets us get the request on record 15:43:00 Perhaps I'm not seeing what the "pending/placeholder" rule you're suggesting would actually look like.... 15:43:05 What would it actually say? 15:43:28 Actually, I think this would be a stronger argument in that someone would need to submit the reason, with the example test. They would either show a strong corelation or not 15:43:43 [D405] ________ [fill in the blank] 15:43:44 ok, so everyone agrees that we dont want a dumping ground. I like markvoelker proposal 15:44:02 [D999] Insufficient Pixie Dust. Flag reason doesn't fit existing categories; suggest new category. Will be subject to lengthy review. 15:44:21 D999 placeholder flag pending discussion. Use this flag if none of the above fit. 15:44:27 so, test sky is green needs to be flagged because new rule: sky where this stuff runs is always blue 15:44:39 I'm going to have to practice typing to keep up with this crowd. 15:44:41 purp that is not bad. I think that any new category proposed will be discussed at this meeting 15:44:49 Usually it's just my stupidity in the way. 15:45:29 OK, I see where you're going. I can live with that. 15:45:33 purp - could you make the hacking file changes to describe this process 15:45:34 ? 15:45:42 * rockyg typing is too slow for group plus too sleepy for group 15:46:02 * eglute gives rockyg huge cup of coffee 15:46:02 #action purp to make hacking file changes to re-add D999 and require D### in all flag submissions 15:46:11 I'd like to also make sure that the D### are referenced in the flag (and maybe into the JSON too?) 15:46:11 That covers it? 15:46:18 +1 15:46:37 Coolio. We'll look at JSON file after we rip my patch to shreds. 15:46:48 zehicle_irl: hrm. You mean reference DXXX in the flag field added to the .json file? 15:46:51 * zehicle_irl happy that we have a way to close flags for now 15:47:06 markvoelker, I think it may be worthwhile 15:47:12 could just be in the comment field 15:47:19 or we could call it out as a new field in the schema 15:47:31 since there are no flags, it would not be a schema impact to add it now 15:47:31 zehicle_irl i think i like comment idea 15:47:54 if we added a field to the JSON then we'd be certain that it was included 15:48:02 I'll have to ponder that a bit. These rules have been pretty fluid so not sure it'll be very exact in practice (at least for now). 15:48:11 * zehicle_irl not overly committed to that course 15:48:18 In JSON schemas, I'm a big fan of having either a parseable comment field for key-value pairs, then moving the good ones into the schema; or having a "blind data" field for the same thing. 15:48:29 E.g. if you add a JSON field, you also have to specify the SHA of the HACKIGN file at the time and OMG red tape 15:48:49 * eglute does not like red tape 15:48:51 markvoelker: exactly. Schema changes are expensive, and should be. 15:48:59 markvoelker, oh, dear. no 15:49:38 Ok, I think we have a reasonable AI here and can continue to discuss in purp's revision of the patch 15:49:44 We have 10m, so let's move on? 15:49:49 agreed 15:49:50 markvoelker: +1 15:49:56 +1 - good discussion 15:50:14 #topic strategic test planning 15:50:17 (other possible misc: change D404 - Reason Not Found.) 15:50:21 (sorry =) 15:50:28 purp i like that 15:50:32 * zehicle_irl bows before purp 's geekness 15:50:38 #link http://lists.openstack.org/pipermail/defcore-committee/2015-June/000818.html 15:50:44 * purp blushes 15:50:51 I sent a mailing out to the list. 15:50:58 I miss the old "Yahoo" add at Pacbell center field. It was 404 feet deep. 15:51:02 I will not accept the patch unless the pending rule is 404 15:51:14 Essentially we have implicit requirements for resources that need to be available for tests. 15:51:20 ^^ link to hogepodge email 15:51:51 I'd like us to start making those requirements explicit so it can give a technical requirement as to what should be considered as required by our criteria. 15:52:07 +1 on explicit requirements 15:52:36 hogepodge, are you seeing issues with this in the vendor space? 15:52:36 Also buried in there is the idea that mapping APIs to capabilities would be useful for developers who want to write portable apps on OpenStack. 15:52:37 hogepodge: So I guess I have two main thoughts on that. First: how well aligned is Tempest to that today and how willing is QA to refactor tests to get them in compliance? 15:52:54 hogepodge, +1 on caps being generally useful 15:53:24 markvoelker: Tempest does a good job of abstracting away resources so you can tell when test needs something 15:53:41 johnthetubaguy what is your opinion on explicit requirements for resources 15:54:06 I think some in QA are on board (with one notable exception, who thinks all tests are interop ready by default) 15:54:14 eglute: so I like the idea, I just not sure what it would look like right now 15:54:15 markvoelker: what Hogpodge said, but beyond that, the tempest folks need educating on separability of functionality in tests 15:54:40 hogepodge: so you're saying there actually wouldn't be many tests that need refactoring in order to meet this bar? 15:54:53 I think the onus to work on identifying or writing new tests would be on "us" ("us" being those interested in interop) 15:55:05 what hogpodge said 15:55:23 markvoelker: We don't have many tests to begin with. 127, and I think many are fine. 15:55:23 hogepodge: so I think identifying what users need but is not tested yet, is a great way to move foward 15:55:34 hogepodge: that's partly my concern. How many people here have contributed to Tempest regularly in the past? 15:55:40 yes, also checking api coverage. 15:55:51 E.g. is it realistic for us to take this on? 15:55:55 markvoelker: me. 15:55:57 We're a pretty small group.... 15:55:59 but, with api tests shifting eventually to projects, these tests could be low hanging fruit for new contributors 15:56:23 I'm sort of thinking we'd need QA's buy-in on the refactoring at the very least, and their help actually writing the code at worst. 15:56:26 markvoelker: It would be a long term project, likely a year. I have resources this summer to devote to it. 15:56:38 hogepodge: cool 15:56:51 And what about in-tree tests in the projects (Swift comes to mind) 15:56:54 ? 15:56:55 One important thing is that we don't break what's already there, and if it's overwhelming we're not stuck. 15:57:13 hogepodge: rather than required resources, do we really want a list of "use cases" that need all the steps testing? 15:57:17 We're going to run out of time for another topic too, which I'll just throw out there. 15:57:29 markvoelker: notmy name is working with QA on the swift stuff 15:57:32 3 minute warning. we can move the discussion to the #openstack-defcore if people have the time 15:58:12 Nova has some plans around "feature classification" that touch on this tempest test coverage topic 15:58:16 I brought up the nova/glance(v1/v2) flags in the cross project meeting yesterday. Hoping a spec will land for L1 to add v2 support to the nova image api 15:58:35 rockyg: yes, but what I'm getting at is: is the swift team onboard with maxmimum resource spec? And for that matter, all the other projects? If we try to refactor their tests to meet this, will they accept the patches? 15:58:38 hogepodge: so I don't think Nova support the v2 image API should block progress here 15:58:47 hogepodge: there are much bigger issues that need addressing 15:58:59 at least I think we need to drill down to whats required here 15:59:20 I guess what I'm getting at here is that I like the idea being proposed here, but I think it needs pretty broad visibility in order to be effective. 15:59:31 please comment on my mailing list post regarding the proposal. It's there to spark discussion and I expect it to be critically reviewed 15:59:39 hogepodge: we need a single API to upload, download and list images in an OpenStack deployment, thats just not a thing right now, and there is no way to discover it right now 15:59:51 hogepodge: I have been unable to find your ML post, what was the subject? 15:59:55 hogepodge: +1, but I'd suggest it also hit the -dev mailer 15:59:58 So, I think the way to get a new test/test fixed is to write the test case as a doc (reason for test, steps), get it approved, then implement in line 15:59:59 It's in the defcore list 16:00:01 we are out of time- if you can continue discussion, lets move it defcore irc. 16:00:04 ...oh 16:00:07 not sure I am on that list 16:00:28 * markvoelker sees johnthetubaguy's last words as good reason for this to go to -dev too 16:00:35 johnthetubaguy: http://lists.openstack.org/pipermail/defcore-committee/2015-June/000818.html 16:00:38 johnthetubaguy,probably not....yet :-) 16:00:41 well, I am on the list, but its at the bottom of my list of things to read 16:00:46 sadly :( 16:00:47 anyone know if there is another meeting in this room 16:01:21 eglute: they usually start shouting at me round about now 16:01:28 ah ok 16:02:00 hogepodge: sorry confused, I was meaning the email on glance v2, I didn't see that one 16:02:09 johnthetubaguy: I haven't sent that one yet 16:02:09 So, it doesn't appear that there is a follow on meeting. 16:02:15 hogepodge: just read the above email, I can respond to that 16:02:39 since current topic is much larger topic, do people have time to go over flagged tests? 16:03:26 * markvoelker has a few minutes more, but has already reviewed most of them in gerrit 16:03:40 hogepodge: ah, no worries, thats my confusion 16:03:41 lemme just throw out an idea here: for capabilities Defcore wants that have no tests, we could write testcases, that are essentially usecases, that could then be implemented after accepted 16:03:42 markvoelker is always excellent with reviewing things 16:03:59 anyone else has looked at the flagged tests? #link https://review.sopenstack.org/#/q/status:open+project:openstack/defcore,n,z 16:04:52 I will review with updates. I have an engineer working on patching broken tests and can mark those. 16:05:01 thank you hogepodge 16:05:07 need to correct link... move . to fromb efore s to after 16:05:07 eglute: typo in that link. Make that: https://review.openstack.org/#/q/status:open+project:openstack/defcore,n,z 16:05:12 If we can fix this cycle I'd rather not re-add them. 16:05:49 thanks rockyg and markvoelker 16:05:53 markvoelker, yeah that one 16:05:55 Speaking of flag reviews, we should really make a decision on this one so Chris's patches don't get held up further: https://review.openstack.org/#/c/190751/ 16:06:14 not sure how that link works for me 16:06:25 #topic flagged tests 16:07:01 markvoelker hogepodge zehicle_irl i am ready to merge https://review.openstack.org/#/c/190751/ if everyone is good with it 16:07:37 need to review 16:07:58 thank you zehicle_irl 16:08:17 #action zehicle_irl review https://review.openstack.org/#/c/190751/ 16:08:58 how about this one? https://review.openstack.org/#/c/189961/ 16:09:20 any further comments? 16:09:43 eglute: works for me as soon as we get 190751 worked out 16:09:49 thanks markvoelker 16:10:09 any further comments on https://review.openstack.org/#/c/189927/ 16:11:48 I'll get off my duff and review the list this week. Sorry. Been swamped. 16:11:55 thank you rockyg 16:12:03 That one is currently being fixed in tempest upstream, so I'd prefer to wait on it. 16:12:18 hogepodge can you comment with -1 on it? 16:12:41 Just left a comment. 16:12:48 thank you 16:13:20 any other opened issues that could be merged? or need more discussion? 16:13:49 there are others that are opened, but for the sake of time, i rather not go over every opened issue right now 16:14:01 I need to step out, other obligations to attend to. 16:14:08 i am available in defcore irc later today. 16:14:28 thank you everyone, please comment on the hogepodge email and review issues 16:14:41 i will close the meeting unless there are any last minute comments 16:14:58 #endmeeting