01:02:37 <markvoelker> #startmeeting DefCore 01:02:38 <openstack> Meeting started Thu Jun 11 01:02:37 2015 UTC and is due to finish in 60 minutes. The chair is markvoelker. Information about MeetBot at http://wiki.debian.org/MeetBot. 01:02:39 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 01:02:41 <openstack> The meeting name has been set to 'defcore' 01:02:48 <purp> o/ 01:02:52 <markvoelker> #chair eglute 01:02:53 <openstack> Current chairs: eglute markvoelker 01:02:56 <hughhalf> . 01:02:56 <zehicle> o/ 01:03:04 <markvoelker> #chair zehicle 01:03:04 <openstack> Current chairs: eglute markvoelker zehicle 01:03:05 <eglute> thanks, it didnt like me then. 01:03:33 <zehicle> #link https://etherpad.openstack.org/p/DefCoreFlag.3 01:03:34 <markvoelker> no problem....I'll turn it over to you to drive now. =) 01:03:40 <eglute> thank you markvoelker 01:03:49 <zehicle> roll call pleasde 01:04:11 <eglute> o/ 01:04:12 <zehicle> please indicate your time zone also, I'd like to make sure that we're accomplishing our objective 01:04:16 <zehicle> o/ Central 01:04:20 <eglute> o/ CST 01:04:23 <markvoelker> o/ EDT 01:04:34 <dwalleck> o/ CST 01:04:38 <hughhalf> o/ AEST - GMT+10 (Canberra Australia) 01:04:45 <hogepodge> o/ PDT 01:05:03 <purp> PDT 01:05:13 <purp> o/ PDT 01:05:27 <eglute> someone emailed the list and said this time does not work for him, he is in GMT +12 01:05:35 <zehicle> yy, that's Robert 01:05:44 <eglute> y 01:06:04 <zehicle> ok, I'm thinking to follow the agenda on the ether pad 01:06:27 <zehicle> please make additions/changes there and we'll incorporate it 01:06:38 <eglute> sounds good 01:06:42 <zehicle> #topic mid-cycle meeting 01:07:44 <zehicle> I'd like to have use close (or have a plan to close) the location & timing 01:07:51 <zehicle> do we have quorum to do that here? 01:07:54 <purp> Checked in with mtreinish; he's looking at late July/early August. Trying to sync with infra midcycle 01:08:27 <zehicle> We've got some block out times: OSCON & OpenStack BoD 01:08:32 <eglute> zehicle regarding quorum, we had some people last week that said they would travel, some are not here right now 01:08:38 <zehicle> yy 01:08:55 <zehicle> I suspect that we cannot close it tonight. if so, can we get it down to two choices? 01:09:04 <zehicle> maybe 3 01:09:11 <purp> Seems reasonable. 01:09:20 <eglute> my concern with waiting too long is that then we are at timeline July 27 (Summit - 3 months) 01:09:24 * zehicle gets his calendar out 01:09:53 <eglute> BoD is on July 28th (Tuesday), oscon week before that 01:10:13 <purp> Week of July 6? 01:10:18 <purp> Too early? 01:10:19 <zehicle> I'm on vacation th week after 01:10:26 <zehicle> and traveling the week of July 6 01:10:47 <zehicle> So, 7/13 week 01:10:57 <zehicle> Could we plan it like this: 01:11:02 <zehicle> 1) SJC week of 7/13 01:11:15 <zehicle> 2) PDX week of 7/20 01:11:24 <zehicle> 3) ATX week of 7/27 01:11:36 <zehicle> with a target dates of Wed-Thurs 01:11:40 <eglute> works for me. 01:11:48 <purp> I can't make 13 Jul - 13 Aug. But we knew that. 01:12:01 <eglute> so doodle to the mailing list with the dates? 01:12:11 <zehicle> that' 01:12:31 <eglute> #action eglute send out doodle to the mailing list with date options 01:13:14 <zehicle> #topic v1.3 schema 01:13:40 <zehicle> so we merged the v1.3 schema with some discussion still potentially open 01:14:02 <eglute> yes, i think there were some things that were missed. 01:14:14 <zehicle> markvoelker, were you OK w/ the latest HACKING or did we jump the gun? 01:14:23 <eglute> lots of comments on https://review.openstack.org/#/c/185158/, so if something was missed, please submit a new patch 01:14:41 <hogepodge> #link https://review.openstack.org/#/c/185158/ 01:14:52 <markvoelker> So, I think the thing we may have overlooked was the bit where we removed the ability to drop tests completely... 01:15:08 <eglute> i think there was at least one thing missed, need to find the new patch that was created that called it out 01:15:10 <eglute> yeah 01:15:27 <zehicle> It looks like we've started the discussion on the new patch 01:15:28 <eglute> #link https://review.openstack.org/#/c/189961/ 01:15:39 <eglute> ^^ new patch 01:15:40 <markvoelker> I wasn't sure if that was intentional or not though 01:16:02 <eglute> not intentional! 01:16:04 <hogepodge> yup, we need to be able to drop tests from next. I had proposed a change to do that, but it didn't make it in. 01:16:31 <hogepodge> which is part of the point of next, to resolve outstanding flags. 01:16:32 <markvoelker> Ok, so that should be relatively easy to correct 01:16:58 <markvoelker> Shall I submit a patch for it, or does someone already have one in the works? 01:17:01 <eglute> the previous patch ended up with too many comments and i could not tell that something was still not resolved. so yes, we need to fix it 01:17:25 <hogepodge> markvoelker: you should feel free to do that 01:17:39 <eglute> #action markvoelker submit a patch to fix process for removing tests 01:17:42 <hogepodge> I don't have one in the works that's ready to go 01:17:43 <zehicle> I'd like to make sure that the patch w/ the flags stays about flags 01:17:50 <eglute> +1 01:17:58 <zehicle> do we have tests that need to be removed? where did they go? 01:18:13 <markvoelker> zehicle: Yes, see https://review.openstack.org/#/c/189961/ 01:18:15 <hogepodge> zehicle: yes, there are tests that are hypervisor specific 01:18:43 <zehicle> soundn't we keep the tests but flag them? 01:18:54 <hogepodge> and this https://review.openstack.org/#/c/189979/ 01:19:04 <hogepodge> In next? No. 01:19:05 <zehicle> I thought that was the purpose of the flags? 01:19:09 <eglute> i think the hypervisor one should be dropped, based on the discussions 01:19:13 <zehicle> I'm worried about having tests in the system that don't show up 01:19:27 <markvoelker> zehicle: these are tests for capabilities that never should have been DefCore in the first place b/c they fail Core Criteria 01:19:38 <zehicle> +1 on that 01:19:39 <markvoelker> So we could keep them...forever... 01:19:45 <zehicle> except that they would be flags 01:19:45 <hogepodge> My understanding is that a required test is something we want, and that the actions on a flag are to fix or remove. 01:19:50 <markvoelker> But that would just create clutter and we'd have to re-evaluate them every cycle 01:19:59 <markvoelker> Whcih seems like a waste of resources 01:20:04 <zehicle> not if the flag reason was clear 01:20:11 <zehicle> OR... could we put them into a capability? 01:20:25 <zehicle> I'm worried that we're going to have them pop up every cycle for review 01:20:35 <eglute> in hypervisor case, it would not meet the common use case 01:20:38 <zehicle> I'd rather make a decision about them one time and let it ride 01:20:45 <hogepodge> confusing to end users. I've been telling vendors that a flag means a capability is required, but has a problematic test. 01:21:03 <zehicle> that still seems like a "compute-hypervisor-vendor-specific" capability 01:21:12 <markvoelker> zehicle: so we had recently decided that we were going to re-evaluate all flagged tests at the beginning of every cycle. 01:21:12 <zehicle> then the cap is NOT required 01:21:13 <hogepodge> zehicle: if that's the case I misunderstood the purpose then 01:21:13 <purp> This seems to point up the need for multiple flag types. No? 01:21:25 <zehicle> I' 01:21:27 <markvoelker> So if we just leave them in the flag list indefinitely we're going to be re-evaling them indefinitely 01:21:40 <markvoelker> And that list is going to get longer and longer 01:21:42 <zehicle> I'm happy to discuss.... my understanding was that we'd eventually list _every_ test 01:21:43 <hogepodge> zehicle: I thought that a capability couldn't be removed in a cycle, but could in the next. 01:21:58 <zehicle> hogepodge, so we can flag them and move them 01:22:10 <hogepodge> zehicle: but it shouldn't be in the required capability list 01:22:21 <markvoelker> zehicle: I don't think it's feasible to list every test. That's a moving target. 01:22:25 <dwalleck> Does it make sense to have capabilities that are driver specific? As in, if you are using a certain driver, within that context the expected behaviors are <things> 01:22:28 <hogepodge> zehicle: moving I'm ok with too 01:22:31 <markvoelker> Especially when you consider non-tempest tests 01:22:58 <zehicle> markvoelker, eventually, it would be good to have them all tracked somehow since we want the data from all of them 01:23:23 <zehicle> dwalleck, we can have all sorts of capabilities. it's helpful to users to map functionality 01:23:24 <markvoelker> zehicle: I'm saying that eventually will never come though....the tests are moving much faster than we are. 01:23:27 <zehicle> we just don't have to require them 01:23:36 <hogepodge> we have a mechanism for listing all of the tests, it's the repository the test comes from 01:24:10 <eglute> is this a topic we should leave for mid-cycle? 01:24:16 <zehicle> I'm OK w/ this suggestion - I have some concerns based on earlier postures 01:24:49 <zehicle> if we're thinking it's not a big deal then I'm OK to remove 01:24:58 <zehicle> since we have a lot of tests that are not covered 01:25:09 <zehicle> I just want to make sure that we don't keep trying to re-add them 01:25:10 <eglute> i think for now, i vote +1 to remove 01:25:30 <markvoelker> So, how about this: I'll just propose the patch and we can discuss it there. I'm happy to be overruled if folks think differently 01:25:44 <purp> markvoelker: +1 01:25:48 <eglute> works for me 01:25:50 <markvoelker> (in fact, I thought I had been overruled when the schema change with the procedure change landed! =p) 01:26:01 <markvoelker> #action markvoelker to propose patch for test removal 01:26:06 <zehicle> no, we wanted to split the discussion 01:26:16 <zehicle> that's why we want to make sure to cover it here 01:26:32 <zehicle> because we suspected that the non v1.3 issues had not been resolved 01:26:40 <eglute> +1 on splitting discussions in patches as appropriate. otherwise important issues get lost 01:27:02 <zehicle> based on discussion here, it seems like v1.3 is OK 01:27:09 <markvoelker> I'll get that patch up tomorrow morning so we can get moving on it and have decision soonish so Chris's patches can move forward or be refactored 01:27:16 <zehicle> and we're aware that some of the issues in that patch need further discussion 01:27:24 <eglute> yes 01:27:32 <zehicle> mission accomplished 01:27:53 <zehicle> other discussion on v1.3 patch and related skeltons unearthed? 01:28:19 * zehicle thinks we unblocked a lot of good discussion around flagging 01:28:20 <hogepodge> need a proper validating schema? 01:28:20 <eglute> the skeletons might be hiding in the next topic 01:28:32 <hogepodge> Also, gate job to ensure json is parsable? 01:28:37 <hogepodge> and correct? 01:28:42 <zehicle> #topic capabilities subdivisions 01:28:54 <zehicle> hogepodge, +1 01:29:12 <zehicle> getting more critical w/ each addition 01:29:53 <zehicle> Van was not going to make this meeting 01:30:01 <zehicle> I had two items related 01:30:10 <eglute> right, i think we skipped this topic last week as well 01:30:12 <zehicle> 1) who/when can we get the capabilities subsets done 01:30:15 <zehicle> ? 01:30:21 <zehicle> does someone have that list? 01:30:39 <hogepodge> It's in a google sheet that I'm sure I can dig up a link for given enough time. 01:30:55 <zehicle> that has the correct subdivisions? 01:31:02 <zehicle> hogepodge, will doing that break all your patches? 01:31:02 <hogepodge> Van and Catherine have been the primary contributors, and both aren't here (Catherine should be back next week) 01:31:14 <eglute> #link https://docs.google.com/spreadsheets/d/15Fkt2n95WPPe7CYH-0WIKz3dhMeW3FW2gl5aSlamEeY/edit#gid=6 01:31:26 <eglute> right, i think Van will be as well 01:31:43 <hogepodge> zehicle: I'm not worried about the patches, they can be rewritten if the schema updates. I wanted to reclassify the tests for discussion, and figured those patches would be long lived and require rebasing 01:32:19 <zehicle> ok 01:32:48 <eglute> so skip this topic for now? 01:32:50 <zehicle> so, can we get that subdivision done? 01:33:15 <hogepodge> It's probably a matter of automation based on the list, so yes? 01:33:15 <zehicle> I'm worried about it causing downstream work if we delay 01:33:33 <zehicle> if you've got the material in that format 01:33:44 <zehicle> it's really not that much data to tweak 01:34:01 <zehicle> I'm thinking about 30 minutes tops 01:34:13 <zehicle> so, likely less effort than automation 01:34:39 <zehicle> the cap list is not the question - it's the test membership 01:34:49 <zehicle> that's what was missing from the earlier patch 01:35:00 <eglute> hogepodge is this something you can work on with Van and Catherine? 01:35:12 <hogepodge> I can 01:35:21 <eglute> thank you hogepodge 01:35:44 <zehicle> ok, I see the list now 01:35:51 <eglute> #action hogepodge work with Van and Catherine to subdivide capabilities 01:36:41 <zehicle> depending on the flags, they may actually be able to merge cleanly 01:37:10 <zehicle> the second part was just the we had a discussion about how to score capabilities in IRC 01:37:17 <zehicle> in the #openstack-defcore channel 01:38:01 <zehicle> the short version is that Van suggested a way to vote that may work 01:38:11 <zehicle> we'll need to try it sometime soon 01:38:18 <eglute> if everyone is looking at the spreadsheet, voting should work 01:38:19 <markvoelker> zehicle: got a link or a date when that happened? Was probably while I was on PTO and I'd love to read... 01:38:32 <zehicle> hogepodge, are there any new capabilities under consideration? 01:38:38 <markvoelker> I can dig in eavesdrop if not 01:38:42 <eglute> markvoelker i think that happened over the phone? 01:38:50 <hogepodge> zehicle: not that I know of immediately 01:38:51 <zehicle> it was IRC 01:39:05 <zehicle> yesterday I think - around 4pm Central 01:39:32 <markvoelker> Ok, I'll dig and post a link if I can turn it up before end of topic 01:39:33 <zehicle> was a big burst of activity including patches around that time 01:39:34 <markvoelker> thanks 01:39:45 <eglute> i missed then as well 01:39:48 <dwalleck_> I'd be glad to help with capabilities as well. I should have the bandwidth 01:39:54 <zehicle> wanted to make sure that people were are we'd dicussed it - we'll need to formally doc it before we try it 01:40:00 <eglute> thank you dwalleck_ ! 01:40:22 <zehicle> dwalleck_ and hogepodge I think you've got all the data you need to subdivide from the spreadsheet 01:40:39 <hogepodge> markvoelker: https://etherpad.openstack.org/p/defcore_scoring is a log of that conversation 01:40:45 <markvoelker> #link http://eavesdrop.openstack.org/irclogs/%23openstack-defcore/%23openstack-defcore.2015-06-09.log.html#t2015-06-09T20:04:03 01:40:52 <hogepodge> markvoelker: or that too :-) 01:41:00 <markvoelker> #link https://etherpad.openstack.org/p/defcore_scoring 01:41:04 <markvoelker> Thanks hogepodge 01:41:11 <purp> Thanks hogepodge 01:41:13 <zehicle> next topic.... 01:41:20 <zehicle> #review hacking file (flag tests) 01:41:24 <zehicle> #topic review hacking file (flag tests) 01:41:42 <zehicle> 20 minutes remaining and this could be the rest.... 01:41:45 <zehicle> other topics first? 01:42:22 * zehicle reserves last few minutes to review the choice of time 01:42:33 <eglute> i think we need to resolve this... dont think other topics will be short either 01:42:49 <purp> This is the one I missed last week, yes? 01:43:09 <eglute> yes i think so 01:43:11 <zehicle> purp, it's a long runniung thread 01:43:20 <eglute> there is a patch somewhere, let me find it 01:43:22 * purp is jealous of the stereo zehicles. 01:43:26 <markvoelker> To be clear, are we talking about this? https://review.openstack.org/#/c/188661 01:43:39 <zehicle> yy 01:43:43 <zehicle> #link https://review.openstack.org/#/c/188661/ 01:43:43 <markvoelker> #link https://review.openstack.org/#/c/188661 01:43:57 <eglute> yes, thanks markvoelker 01:44:10 <markvoelker> So IMHO there's not that much that's really controversial in the patch. I made a few suggestions in the review. 01:44:18 <zehicle> oh, can we cover #link https://review.openstack.org/#/c/182105/ first? 01:44:20 <markvoelker> The big question for me is: is the list intended to be exhaustive? 01:44:44 <markvoelker> zehicle: sure 01:44:45 <zehicle> markvoelker, I think so. that's a good catch 01:44:53 <zehicle> ok, back to the 2015A patch for a minute 01:45:08 <markvoelker> #link https://review.openstack.org/#/c/182105/ 01:45:20 <hogepodge> Oh, that's mine. 01:45:31 <zehicle> I wanted to discuss my objections and make sure that we were on the same page 01:45:47 <zehicle> yy, you were adding flag details to the 2015A process 01:46:07 <zehicle> to an extent, we did not add those details there on purpose 01:46:16 <zehicle> since they were left to DefCore to manage 01:46:33 <zehicle> I'm open to discussion that they should be part of the broader process 01:46:50 <zehicle> but it would have to be 2015B in that case and go back to the board 01:47:13 <markvoelker> Yeah, I think this one predates some of the recent changes to HACKING that came in with schema 1.3. 01:47:25 <markvoelker> Much of this is now covered there 01:47:40 <zehicle> you are correct markvoelker 01:47:48 <markvoelker> E.g. rule D307 covers what is required when flagging a test 01:47:53 <hogepodge> I'd like something to make flagging a harder thing to do 01:48:09 <hogepodge> if that's covered in hacking, fine. 01:48:24 <zehicle> that was the goal of getting written down in hacking 01:48:41 * zehicle hacking is not the most obvious name for rules of engagement 01:48:53 <markvoelker> hogepodge: D307 basically says "everything in the current schema's flagging section is required", so I think we're good there 01:49:09 <markvoelker> I'm happy to obtain different opinions though. =) 01:49:14 <hogepodge> markvoelker: I can't find what's in the schema though. I don't think it's documented 01:49:33 <markvoelker> hogepodge: https://github.com/openstack/defcore/tree/master/schema 01:49:53 <hogepodge> markvoelker: ah, ok 01:49:53 <zehicle> hogepodge, I moved it so we could track versions 01:50:08 <zehicle> since I was changing a lot of comments when I was fixing the readme 01:50:51 <zehicle> I just wanted to resolve if we felt there was a process change beyond hacking required 01:51:46 <hogepodge> hacking needs more guidance along the lines of what I wrote, but is that part of what markvoelker is going to send up as part of his action item? 01:52:19 <markvoelker> hogepodge: guidance about when a flag can be removed, or...? 01:52:22 <zehicle> I have no issues adding the rules 01:52:27 <zehicle> my concern was about where 01:52:47 <zehicle> 2015B would be the right place if we wanted it at the board level 01:53:14 <zehicle> Hacking is right if we think we can hold the line 01:53:27 <eglute> some flagging guidance should be in 2015B i think 01:53:32 <eglute> but maybe not as detailed 01:53:40 <markvoelker> zehicle: I'm of the opinion that this level of detail isn't of interest to the Board. But then, I'm not on the Board. =) 01:53:49 <hogepodge> plus info about removing tests in next (or not) 01:53:50 <zehicle> +1 01:54:13 <zehicle> there are some items that the process leaves to defcore 01:54:24 <dwalleck_> hackings have always been a good "how do I work within this project" place to me 01:54:25 <markvoelker> hogepodge: I'll definitely hit the removal of tests in .next in my patch 01:54:26 <eglute> so sounds like everyone in favor of when to flag and what, when to remove to put hacking doc? 01:54:26 <purp> Agree. 01:54:36 <zehicle> test membership and flagging are both in that category 01:54:54 <eglute> should we create a separate flagging process doc? 01:55:04 <zehicle> I 01:55:12 <zehicle> I'm ok w/ them in hacking - since it's central 01:55:17 <eglute> ok 01:55:23 <zehicle> let's collect them there and see if we need more 01:55:23 <markvoelker> eglute: I thought about that, but came around to kind of liking it in HACKING 01:55:35 <hogepodge> five minute warning 01:55:47 <zehicle> as long as we all agree with the location, the name of the file works for me 01:55:48 <markvoelker> Basically b/c I suspect the flagging process is an area where people not routinely involved in DefCore will want to interact with us 01:55:55 <eglute> markvoelker ok, that works for me if we have consensus. most important to have them somewhere :) 01:56:04 <markvoelker> and HACKING is, as dwalleck_ pointed out, where to look for info about how to interact with a project 01:56:17 <zehicle> easy enough to xlink in the readme 01:56:29 <eglute> works for me then 01:56:34 <zehicle> ok, my topic on that patch took most of the time. sorry 01:56:39 <purp> +1 HACKING 01:56:53 <zehicle> markvoelker, +1 on adding a "list is comprehensive" statement 01:57:10 <zehicle> we don't want flag to be allowed because we did not think of a reason to block them 01:57:25 <zehicle> if a new reason surfaces then we can discuss 01:57:35 <zehicle> except for pixie dust 01:57:43 <zehicle> not up for discussion 01:58:10 <eglute> flag neutron: because of pixie dust. 01:58:14 <zehicle> #topic is this time working? 01:58:31 <zehicle> just a check 01:58:32 <markvoelker> zehicle: I'm also curious about the difference b/t D401 and D402. Didn't really seem to be one? 01:58:45 <purp> Hard for PDT ... family dinner hour. 01:59:09 <zehicle> did we get additional audience based on the time? 01:59:16 <eglute> no, less people 01:59:17 <purp> markvoelker: I read D401 as "inadequately tests" and D402 as "test fails inappropriately" 01:59:20 <markvoelker> time is ok for me...note that we have an AI to re-eval this in ~a month to see if it's working 01:59:27 <zehicle> dwalleck_, you are our rep from alternate time zone 01:59:36 <eglute> dwalleck_ is in CST 01:59:40 <dwalleck_> yup 01:59:42 <zehicle> oh, sorry 01:59:52 <purp> That's pretty alternate. I've been there. 01:59:59 <dwalleck_> Though the nice thing about this time is that it's not possible to conflict with anything else :) 02:00:00 <eglute> we do have someone from Australia 02:00:07 <zehicle> who was our 02:00:13 <hogepodge> hughhalf: is in australian time zone 02:00:14 <dwalleck_> well, work conflict 02:00:17 <markvoelker> purp: hrm. I guess I read both as "the test is borked". =) 02:00:23 <zehicle> it was hughhalf 02:00:29 <eglute> hughhalf what do you think? 02:00:42 <hogepodge> AEST - GMT+10 (Canberra Australia) 02:00:43 <hughhalf> Sorry, stepped away, one mo please 02:00:55 <zehicle> I think you've answered the question 02:01:10 <hughhalf> Doorbell rang! :) 02:01:15 <purp> Heh. 02:01:16 <eglute> ok, we will try couple more times and then re-evaluate i think! 02:01:20 <zehicle> eglute, +1 02:01:27 <hogepodge> any meetings following this? please tell us to vacate if so. 02:01:27 <eglute> we agreed to give it a month 02:01:28 <hughhalf> So yes, this time is ok for me 02:01:29 <zehicle> just wanted to test it at the end 02:01:35 <zehicle> not suggesting a change 02:01:55 * hughhalf nods 02:02:00 <purp> #action purp will ping Robert to see what times work better (due before end of June) 02:02:02 <zehicle> we're done 02:02:12 <eglute> thank you everyone! 02:02:16 <markvoelker> zehicle: did you want me to propose a new patchset for https://review.openstack.org/#/c/188661/3/HACKING.rst or did you want to? 02:02:24 <zehicle> any addition discussion, we'll be on regular channel 02:02:24 * hughhalf notest that fwiw a time that will work for Robert will likely work for most Oz folk too. 02:02:25 <hughhalf> thanks all 02:02:37 <zehicle> markvoelker, go ahead 02:02:41 <zehicle> I was just getting it started 02:02:48 <zehicle> #endmeeting