16:00:59 #startmeeting interopwg 16:01:00 Meeting started Wed Aug 16 16:00:59 2017 UTC and is due to finish in 60 minutes. The chair is eglute. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:01:01 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:01:04 The meeting name has been set to 'interopwg' 16:01:06 o/ 16:01:13 #chair markvoelker 16:01:14 Current chairs: eglute markvoelker 16:01:18 #chair hogepodge 16:01:18 Current chairs: eglute hogepodge markvoelker 16:01:18 o/ 16:01:24 Hello Everyone! 16:01:27 #topic agenda 16:01:30 o/ 16:01:47 today's agenda, please update as needed: #link https://etherpad.openstack.org/p/InteropVertigo.11 16:02:01 o/ 16:02:09 o/ 16:02:12 #topic ptg 16:02:42 PTG less than a month away! Please let us know you are coming by updating etherpad for PTG: https://etherpad.openstack.org/p/InteropDenver2017PTG 16:02:49 also, please add proposed topics 16:03:14 o/ 16:03:16 #topic 2017.09 Guideline 16:03:32 I'm double booked this morning so my responses may be a bit delayed 16:04:13 Since the August board meeting was cancelled, we will ask the board to approve the next guideline during the September board meeting 16:04:29 therefore, the next guideline will be 2017.09 rather than 2017.08 16:04:30 o/ 16:04:40 * markvoelker makes note to rename it 16:04:58 eglute could you explain a little bit on the reason for the naming ? 16:04:58 #action markvoelker to rename 2017.08 to 2017.09 16:05:03 2017.09 16:05:15 zhipeng we name the guidelines based on when they are approved, 16:05:21 ah right 16:05:23 :)thx 16:05:25 year.month 16:05:29 :) 16:05:49 yeah, very simple convention, makes naming things easy 16:06:51 thank you zhipeng for submitting bug report 16:06:53 #link https://storyboard.openstack.org/#!/story/2001156 16:07:01 good catch, wonder what happened there 16:07:41 :) 16:07:47 something we catch during testing 16:08:02 we need to fix this, any takers? 16:08:08 not sure if it is a bug or we misunderstand something 16:08:20 eglute I could do it 16:08:22 i think they should be unique 16:08:25 zhipeng thank you!! 16:08:32 actually not 16:08:35 so I could remove those duplicated capabilites ? 16:08:50 If I understand some tests are overloaded, but that looks like an error. The fix should be to look at the code base and correspond the test names to the idempotent_id 16:08:50 in RefStack we do not match output to ID 16:08:59 we match them to the FQN 16:08:59 #action zhipeng to fix duplicate test IDs 16:09:40 okey I will submit a patch on this 16:09:47 zhipeng what hogepodge said. 16:09:52 zhipeng thank you!!! 16:10:11 any other questions or comments on the next guideline? 16:10:12 duplicated capabilities should not be removed. Now, we may be in a situation where if the tests were being skipped in testing we might want to flag them for the current guidelines. 16:10:30 see the explaination of FQN vs ID in https://bugs.launchpad.net/refstack/+bug/1709323 16:10:34 Launchpad bug 1709323 in refstack "refstack-client removes tests which could be found by id (renamed tests)" [Undecided,New] 16:10:36 hogepodge got it :) 16:10:50 catherineD thx will take a look at it 16:11:13 these are actually testing 2 different feature using the same test method 16:11:26 it is like test the same thing in IPv4 vs IPv6 16:11:54 catherineD but different tests should have different ids, correct? 16:12:11 that's a case of overload 16:12:17 no ID is define at the test methid level 16:12:21 eglute: in the code they don't 16:12:29 Interop-WG define test at the test case leve 16:12:33 the tests use different input parameters. 16:12:51 see the explaination in https://bugs.launchpad.net/refstack/+bug/1709323 16:12:52 Launchpad bug 1709323 in refstack "refstack-client removes tests which could be found by id (renamed tests)" [Undecided,New] 16:12:53 We might want to ask the tempest team if we should introduce wrapper methods 16:13:07 thanks catherineD 16:13:44 anything else related to this? 16:14:01 RefStack dioes not use test ID for pass/fail evaluation at all . Tests are also mapping with FQM from testlist. 16:14:11 thank you catherineD 16:14:16 good to know 16:14:54 #topic Extension programs 16:15:03 thank you hogepodge for submitting patches 16:15:22 hogepodge question: when do you want to get those approved? 2017.09 or 2018.01? 16:16:05 * markvoelker is leaning toward advisory by 2017.09 given that we have the extra month, personally 16:16:08 eglute: I'd like to present them to the board as advisory for 2017.09, with the intention to refine them for official approval in 2018.01 16:16:15 ++ 16:16:24 eglute: refine them with community and vendor feedback 16:17:14 ok, then should the capabilities be as "advisory"? right now all are required. 16:17:20 There's good discussion happening right now, which I want to encourage, but I'd also like them to be on track for board approval in early next year. I think that strikes a good balance between expediency and community review. 16:17:34 eglute: probably. I struggled with how to handle this the first time around. But yes, I think that's the right way to go 16:17:45 I can send up new patches. 16:18:27 yeah, we should decide how new programs should be handled. advisory is good, and if someone wants early designate/heat badge, they should pass all the advisory capabilities 16:18:39 I'd also like to bring up the notion of criteria. Right now the only criteria I have is PTL suggested, but I'm wondering if during the advisory period if we need a new set of criteria for extensions. 16:19:39 hogepodge: I was noodling that a little this week too as I'm looking at verticals (for example: we've already been talking to OPNFV and I have some requirements suggested by ONAP now too) 16:19:49 hogepodge is that what you call achievements? 16:20:52 markvoelker seems like ONAP is on a different level from OpenStack ? 16:20:53 eglute: yes 16:21:06 eglute: see also: https://github.com/openstack/interop/blob/master/doc/source/schema/2.0.rst#criteria 16:21:25 zhipeng: Yes, but they consume OpenStack via the multi-vim project. 16:22:04 ah understood 16:22:31 i think for criteria we should try to use the ones as we use for main guidelines, except as it relates to that particular project 16:23:10 markvoelker hogepodge has something called "achievements" in the json, but i wasnt sure what it tied to in teh schema 16:23:39 part of our confusing lexicon, we say achievements when a capability achieves the criteria 16:23:47 so there's a synonym there 16:24:04 hogepodge true 16:24:35 hogepodge how is that tied to schema 16:24:45 hogepodge: I had noticed you added the "ptl_proposed" achievements in those patches and was trying to think if using it as a way to define a source ("who wants this") made sense 16:25:17 heh 16:25:47 markvoelker: I couldn't think of any other criteria. I asked project leaders for APIs they, as project leaders, thought represented the interoperable set according to our requirements (not admin, single user, etc) 16:25:58 perhaps ptls could score those capabilities on how important they are 16:26:10 E.g. in the NFV vertical program case I might have some stuff we want to denote as "OPNFV needs this" and others as "ONAP needs this" or even specifically "ONAP multi-vim needs this". 16:26:14 schema 2.0 was designed to allow for different sets of criteria to be used 16:26:49 PTLs should know what capabilities will need to be used across clouds 16:26:53 But I'm not really sure that's what we want to use for achievements...my gut feel is to use similar criteria to what we use for Powered, but scope them to appropropriate audience 16:26:58 and since the guideline is what's approved I put the criteria in the document so that they get pulled along with the approval. But I'm entirely happy to work on new criteria 16:27:08 repurpose old, and so on 16:27:16 markvoelker hogepodge i think it would be good to have some official documentation/justification for including capabilities in these new programs. 16:27:43 E.g. if a thing is needed by both OPNFV and ONAP it's widely deployed (among clouds used for NFV) 16:27:48 markvoelker i agree on " to use similar criteria to what we use for Powered, but scope them to appropropriate audience" 16:27:56 (kinda poor example but you get the picture) 16:28:08 eglute: I would like the guidelines to contain all of the information, but I'd happily document it elsewhere 16:28:25 hogepodge i mean something similar to current scoring process 16:28:52 we have all the capabilities scored, and comments on some why they were scored that way 16:29:00 sure, I just don't think that the criteria for the core programs is 100% applicable to the extension or vertical programs 16:29:05 so that next time we look, we have a reference point 16:29:23 hogepodge not even if it is scoped down? 16:29:39 eglute: scoped down, yes 16:30:37 i think it would have to be scoped to that particular project. For example, for every cloud that deploys Heat, do most have X feature? 16:30:38 I'm hearing that I need to update the two reviews and submit a third with proposed extension criteria? 16:32:08 we need something that would justify why a set of capabilities are included 16:32:22 It's just so difficult to measure something like that, and my inclination is to drop a criteria like that. It's an extension which is a special case, and since it's optional being more strict on what has to be there makes more sense. I don't want a bunch of extensions that only add things in name 16:32:47 hogepodge: If we're going with the "same criteria, scoped to appropriate audience" then I'm not sure we need to doc them in a third patch...that's pretty much what we've presented to the BoD and community several times now. 16:32:49 right now the justification is that project leaders have identified the capability as what's required for interoperable 16:33:22 hogepodge i think it is good enough for getting them as advisory 16:33:42 markvoelker: sure, but if we want external documentation that says something about why we chose a set of criteria for extensions we will have a one to many situation :-D 16:33:46 but going forward, we will need some paper trail of why those particular capabilities were chosen 16:35:36 hogepodge: If we're using the same criteria (but scoped) then we already have most of the doc though. 16:35:57 markvoelker: ok 16:36:51 E.g. I'd be fine with a smallish doc that basically boils down to something along the lines of "same as https://github.com/openstack/interop/blob/master/doc/source/process/CoreCriteria.rst but with the audience scoped to the project in question" 16:37:45 +1. 16:38:20 I'm not a big fan of "ptl_proposed" as I think it's too course. The PTL probably has good reasons for proposing a capability, and I think if we use the core criteria (scoped) then it gives them a chance to back it with reasons rather than just being one person's opinion. 16:38:49 markvoelker: ok 16:39:27 (me saying ok like that makes it sound way more terse than I intend) :-D 16:39:33 =) 16:40:04 hogepodge i think we can work on those extra capabilities between now and next year, when the new programs become required 16:40:21 i think we should be able to get it approved as advisory next month 16:41:10 Especially since there's a conversation around heat that I was to give the full space to breathe 16:41:56 heat: #link https://review.openstack.org/#/c/490648/ 16:42:21 i agree, good discussion there 16:43:58 anybody want to share their thoughts on heat or designate programs? 16:44:43 eglute, yes thanks guys to propose heat out:) 16:45:11 I think heat and designate are great add-ons to start with 16:45:29 Generally it seems like a reasonable start on both fronts, though personally I'd like to spend some time picking through some tooling around some of them. Just elbow grease+time. =) 16:45:31 we do want as much discussion as possible at this stage 16:46:00 markvoelker what kind of tooling do you think we should work on? 16:46:30 I think that's the intent of advisory, and we're in a good place to fulfill that intent 16:47:19 Well, just generally I'm looking for justifications that the capabilities are useful, and tooling is often a good indicator: e.g. if tools that use (for ex.) Designate support a bunch of API's but not others, maybe we need to look a little more closely at those that no tools use and see if they're actually good, useful candidates. 16:47:48 The very fact that people are suggesting them is a good start, so just looking for some more backing for those opinions 16:47:52 hogepodge i agree, good start for advisory 16:47:58 markvoelker +1 16:48:03 So basically the typical scoring process. =) 16:48:57 markvoelker: sure - thats what we did when we suggested our (Designates) list 16:49:02 ++ 16:50:39 mugsie: did you folks keep notes or something along the way? Would love to take a peek if so. 16:50:47 mugsie thanks for working on the designate capabilities. we are just looking for additional documentation on why these particular capabilities were chosen. something like this: https://github.com/openstack/interop/blob/master/working_materials/scoring.txt 16:51:28 unfortunatly not- we did it in person in boston 16:51:32 All i have is https://etherpad.openstack.org/p/BOS-Forum-DNS-InterOp 16:51:37 Ah, ok. No worries. 16:51:37 and https://etherpad.openstack.org/p/DNS-InterOp-Capabilities 16:52:47 so everyone ok with proposing heat and designate add-ons as advisory to the board in September? 16:53:15 mugsie do any of the tests require admin? do you know? 16:53:30 they do not 16:53:45 mugsie great, thank you 16:54:09 its is just 2 endpoints - creating / update / deleting a DNS zone and modifying Records 16:55:04 mugsie thank you. seems like it covers essential interop for DNS 16:55:08 time check five minutes 16:55:16 ok, so everyone ok with proposing heat and designate add-ons as advisory to the board in September? 16:55:31 Yep (pending the iterations hogepodge mentioned) 16:55:45 I can get to work on those for next week 16:55:48 +1 16:55:50 thank you hogepodge 16:56:12 mugsie: if you can offer any insight into scoring, that would be really great 16:56:50 sure - I will have a read of the methodolgy and comment in the review 16:56:56 mugsie let us know if you would like to understand scoring better, i can help 16:57:07 ricolin: same with you 16:57:14 great - if I have any questions, I will shout 16:57:27 mugsie perfect! 16:57:35 hogepodge, sure 16:58:00 since we are almost out of time, everyone, please be sure to review any outstanding patches 16:58:02 Thank you everyone, I think this is a really bold and important step forward, and I'm inspired by the interest in seeing it be successful. 16:58:08 as well as this RefStack patch: 16:58:13 https://review.openstack.org/#/c/480298/ 16:58:27 hogepodge mugsie ricolin thank you so much for working on this 16:58:46 any last words? 16:58:47 Veryvery quickly on the NFV vertical front: last week I spoke with some folks from ONAP multi-vim. They have a rough list of stuff they need from OpenStack here that folks may want to look over: https://wiki.onap.org/pages/viewpage.action?pageId=8229235 16:59:00 eglute, NP it takes entire heat team to make it:) 16:59:01 I'll have that boiled down into actual API's by the time we get to Denver. 16:59:10 thank you markvoelker 16:59:19 ricolin very true! 16:59:49 #action everyone look at https://wiki.onap.org/pages/viewpage.action?pageId=8229235 before next meeting 17:00:05 ping us on interop channel or directly if needed. 17:00:08 thanks everyone! 17:00:12 #endmeeting