16:00:40 <hogepodge> #startmeeting defcore
16:00:41 <openstack> Meeting started Wed Dec  9 16:00:40 2015 UTC and is due to finish in 60 minutes.  The chair is hogepodge. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:43 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:46 <markvoelker> o/
16:00:46 <openstack> The meeting name has been set to 'defcore'
16:00:51 <dwalleck> o/
16:01:04 <purp> o/
16:01:13 * markvoelker notes that both zehicle and eglute have sent apologies for absence today
16:01:50 <hogepodge> #chair markvoelker
16:01:51 <openstack> Current chairs: hogepodge markvoelker
16:01:55 <markvoelker> #link https://etherpad.openstack.org/p/DefCoreRing.5 Agenda
16:02:17 <hogepodge> Good morning everybody
16:02:27 <hogepodge> (evening for those on the other side of the planet)
16:02:31 <dwalleck> howdy :-)
16:02:53 <hogepodge> #topic Linux as required capability
16:03:27 <hogepodge> markvoelker: has a great wrapup in the etherpad, so he can sum up and lead this topic
16:03:42 <markvoelker> Sure
16:04:04 <markvoelker> So, the conversation around the flag request we've been discussing at the past few meetings has now been discussed by the BoD and the TC
16:04:14 <catherineD> o/
16:04:36 <markvoelker> While there are a lot of differing opinions about how to handle it, one thing has emerged pretty clearly: we want Capabilities that are required to be explicit, not implicit.
16:05:03 <markvoelker> Oracle has asked for this, the Board and TC like it, it's probably the one thing that has universal agreement in this whole discussion. =)
16:05:29 <jlk> hurray for agreement
16:05:33 <markvoelker> There are a number of things that we can make explicit, and the TC discussed about three of them yesterday.
16:05:59 <markvoelker> Monty is currently drafting a TC resolution to state the TC position on this, and he and I chatted briefly yesterday too
16:06:29 <markvoelker> Based on those three things, I'm drafting up some scoring patches for 2016.07 so the proposed Capabilities can get a fair hearing
16:06:50 <markvoelker> (there links in the etherpad if you want to see some super rough drafts: https://etherpad.openstack.org/p/DefCoreRing.5 )
16:07:02 <markvoelker> Two notes:
16:07:21 <markvoelker> 1.)  There may be other things we propose Capabilities for that stem from this discussion, but these are a start.
16:08:15 <markvoelker> 2.) I don't expect universal agreement on all of these (some Board members in particular expressed views that don't necessarily align with some of these), but it should give them an open hearing.
16:08:52 <markvoelker> So, all that said: I'll get those into Gerrit soonish, but in the meantime we need to actually decide what to do with the Oracle patch.
16:09:08 <purp> #link http://eavesdrop.openstack.org/meetings/tc/2015/tc.2015-12-08-20.01.html TC meeting summary from yesterday
16:09:27 <markvoelker> My suggestion is we wait for the TC to set up their resolution for reference, then we can cast our +1/-1 votes and be done with that bit.
16:09:47 * markvoelker stops typing furiously so other people can talk if they have things to say =p
16:10:16 <purp> #link http://lists.openstack.org/pipermail/openstack-tc/2015-December/001088.html Monty's email upon which the TC resolution will be based
16:10:48 <hogepodge> It's ultimately up to the chairs if they want it to land, but it sounds like the consensus isn't there.
16:11:11 <purp> I'll note that it seems to be the majority opinion of the TC that the Oracle patch should not be merged.
16:11:15 <purp> (perhaps strong majority)
16:11:59 <hogepodge> I proposal would be to write a test that explicitly boots linux (probably cirros, since it's tiny and available on multiple architectures), then build out a capability from that.
16:11:59 <markvoelker> hogepodge: Sure, I'd just like the chairs (and everyone else who cares) to actually record their votes fi they haven't yet.
16:12:46 <markvoelker> hogepodge: YEah, a test for "can boot linux" is probably a little nuianced, frankly.  Unless "can boot linux" is actually "can boot linux on x86_64" or some such.
16:13:25 <dwalleck> markvoelker: +1 on any x86 OS
16:13:35 <SammyD> +1 on any x86
16:13:58 <purp> hogepodge: I'd have likely said Debian, not strongly attached tho
16:14:44 <purp> #link https://review.openstack.org/#/c/244782/ Oracle patch (for convenience in summary notes)
16:14:46 <dwalleck> I think any hypervisor that can boot  an x86 image should be able to boot Linux. I'm not sure what being more specific buys
16:15:31 <hogepodge> markvoelker: really, just using the exiting image configs (and updating the documentation to say "linux images" gets us a huge part of the way there. Lots of tests boot images and test capabilities.
16:15:31 <purp> It begs an interesting point: what about other architectures? For example, Datacentred are running an ARM-based OpenStack cloud.
16:16:00 <markvoelker> dwalleck: The thing is that we need a test to prove the capability works.  So the question becomes "what the image you boot to prove you can run Linux"?
16:16:06 <hogepodge> purp: mordred seemed to feel that until we can discover architecture, x86 should be required. I disagree with that.
16:16:49 <purp> hogepodge: I'm kinda with you, a bit with him. I think his view is pragmatic (what can we do now) vs. architectural (what will we be required to do in the future)
16:17:04 <markvoelker> purp: exactly.  There's some dissent on whether the real intent is a Linux userland (in which case a test needs to be somewhat modular, but have some way to verify Linux has been booted"
16:17:09 <mordred> well
16:17:12 <dwalleck> markvoelker: we can solve the test problem. I've proposed a solution that I've implemented elsewhere https://review.openstack.org/#/c/255023/
16:17:19 * purp chuckles
16:17:19 <mordred> the architecture thing is mostly me word quibbling
16:17:27 <jlk> heh
16:17:29 <purp> ... as one does.
16:17:29 <mordred> I think a user should be able to boot a kernel
16:17:33 <dwalleck> That makes guest instance testing extendable
16:17:35 <hogepodge> purp: it seems unnecessarily exclusionary to me, and harms good community members (like datacentred)
16:17:41 <jlk> right, I'm thinking the wording means a lot when it comes to containers and bare metal
16:17:41 <catherineD> Linux is one of the  x86 image  but  not all Linux  images are x86 base
16:17:45 <mordred> I mention architecture because if someone runs a pure openpower cloud right now
16:17:56 <mordred> and I download an ubuntu image from ubuntu and upload it
16:17:57 <mordred> it's not going to work
16:17:59 <jlk> I didn't catch if TC felt that containers / BM are not enough to be OpenStack(TM)
16:18:03 <mordred> and we don't have a GREAT way of letting me know that
16:18:10 <mordred> but honestly - this is not even _close_ to a real problem
16:18:34 <mordred> and datacenterd is an awesome cloud (I have an accout there)
16:18:45 <mordred> so - that portion of my rambling was not intended ot be exclusionary
16:18:56 <mordred> just pointing out we're missing, as openstack, a way to communicate that information well to our users
16:18:58 <markvoelker> jlk: those who spoke up basically said "full virt is necessary to be OpenStack" (someone scream if I'm putting words in people's mouths)
16:19:10 <mordred> yah. that's the position
16:19:14 <mordred> well
16:19:16 <mordred> or bare metal
16:19:20 <jlk> markvoelker: fair enough.
16:19:24 <mordred> basically "you need to be able to boot a kernel"
16:19:28 <hogepodge> markvoelker: I heard the same thing (also knocking out bare metal clouds)
16:19:32 <mordred> n
16:19:34 <mordred> no
16:19:36 <mordred> that was clarified
16:19:43 <purp> jlk: didn't get that it specifically excluded containers, but that just containers is not enough.
16:19:57 <mordred> bare metal clouds are _not_ knocked out or excluded - they are explicitly important to eveyone there
16:20:18 <mordred> russell spoke poorly, sean corrected him and he agreed on the full virt
16:20:38 <russellb> +1
16:20:46 <mordred> (btw - I'll be pushing up a resolution with all of this spelled out today)
16:20:51 <jlk> purp: sure, it'd exclude a container only cloud
16:20:57 <mordred> yes. it will do that
16:21:07 <jlk> I'm okay with that though, as long as the TC wording is clear
16:21:12 <markvoelker> Right, so here's what I'll suggest: I'll finish those capability patches and post them.  Then we can iterate in gerrit until we hit rough consensus.
16:21:18 <mordred> \o/
16:21:38 <hogepodge> mordred: dhellmann: asked me to not conflate virtualization with bare metal, fwiw. Regardless, bare metal can't pass interop tests as it stands.
16:22:12 <purp> markvoelker: yes, would suggest we loop in some folks from Datacentred on the review for https://github.com/markvoelker/defcore/commit/2dc712042714ab9aa619517739c6177028c14078
16:22:36 <markvoelker> purp: Sure.  I imagine some members of the BoD will have things to say about some of them too.
16:22:51 <mordred> hogepodge: sure. I think we have work to do on the bare metal side ... but I think it's _intent_ that bare metal be a thing that one can get from nova
16:22:59 <markvoelker> So once they go up in gerrit I'll send them out to the usual channels and people can add whomever they want to solicit responses from to the Gerrit review.
16:23:15 <purp> markvoelker: Yup. Want to be sure we specifically call attention to that one for DC'd.
16:23:21 <mordred> hogepodge: while I do not think it is intent that container-only clouds that can't boot os images are a thing we expect to be first-class citizens
16:23:51 <dwalleck> hogepodge: Why wouldn't a bare metal one pass? If it's treated like another flavor, it's just another guest os
16:24:12 <mordred> dwalleck: there are some of the capabilties that the ironic driver doesn't fully do yet
16:24:13 <hogepodge> mordred: agree. To me it's fertile ground for a new component (like compute and storage are components)
16:24:25 <mordred> hogepodge: yup.
16:24:29 <markvoelker> dwallek: off the top of my head baremetal fails some of the volume attachement tests IIRC
16:24:39 <dwalleck> mordred: I know create image doesn't, but many do
16:24:48 <mordred> (also, datacentred runs x86 cloud AND arm)
16:25:05 <mordred> dwalleck: yah. I think create image is one of the trickiest ones, since, you know, not so much with there being a chance of that one working :)
16:26:01 <hogepodge> We're 25 minutes in. Are there any other comments on this topic?
16:26:15 <markvoelker> #action markvoelker to put up patches in gerrit
16:27:04 <markvoelker> #topic Flag tests requiring multiple tenants and users
16:27:14 * markvoelker yields the floor to hogepodge
16:27:28 <hogepodge> #link https://review.openstack.org/#/c/253138/
16:27:52 <hogepodge> That review currently is there for discussion (I wouldn't expect it to be merged without a significant pruning)
16:28:16 <hogepodge> Part of the goal of an interop standard is to let users verify results on their own.
16:28:30 <hogepodge> Currently that can be done, but it's difficult as you need tenant seperated accounts.
16:28:32 * purp reads
16:29:26 <hogepodge> I'm making an argument that testing tenant isolation isn't necessarily an interop capability (although it is a valuable security capability), and we do users a disservice by requiring it as part of testing.
16:29:35 <markvoelker> Yeah, in general I like the idea of making things more end-user-doable (is that a word?) but the tradeoff as proposed is too large.
16:30:24 <markvoelker> For example, it cites the costs you'd incur running this on a public cloud.  While that is super important, it bears mention that on a lot of private clouds (62% of OpenStack installs based on the last user survey) that cost is likely way less.
16:30:34 <purp> markvoelker: +1
16:30:45 <hogepodge> It also makes it difficult for the Foundation to independently verity test results. I didn't think it would be a problem, but I keep getting feedback from the third-party ci testing that cheating is a problem. For public clouds, it would be nice to ease accountability without having to swipe two credit cards. for tenant seperated accounts.
16:31:14 <markvoelker> So, another avenue for helping with costs came up in another discussion....
16:31:45 <markvoelker> Particularly for public clouds, we could potentially have centralized testing administered by the Foundation (say, via refstack).
16:32:09 <markvoelker> With source posted for the tests (e.g. version of tempest they're using, etc) so end users can verify it
16:32:34 <markvoelker> And presumably those public clouds might have incentive to offer the Foundation reduced cost/free accounts to perform the testing.
16:33:08 <purp> markvoelker: on that last, I don't think we want reduced cost or free accounts.
16:33:46 <purp> Don't want to be marked or observed while testing Volkswagon's cloud. ;D
16:34:16 * markvoelker always wants free accounts to public clouds but nobody ever phones up and gives them to me...sigh
16:34:21 <markvoelker> purp: heh
16:34:24 <purp> But in all seriousness, we don't want an account that can be made special — if there's a concern about cheating, I want to buy two gift cards, swipe both, and test manually.
16:34:51 <dwalleck> If the only tests that require multiple users are authorization tests, then as has been mentioned before, is authorization inside a project really a capability?
16:35:13 <markvoelker> purp: ok, reasonable thing to debate.  Generally though, the thing is this: if the main issue here is the cost to run tests on public cloud, there are perhaps ways to deal with that that don't involve removing tests/capabilities from the Guidelines.
16:35:30 <markvoelker> dwalleck: quite possibly, yes
16:35:43 <purp> markvoelker: Totally agree that we don't want to remove them unless necessary.
16:35:50 <catherineD> dwalleck: authorization tests are not the only ones needed 2 isolated tenants
16:35:50 <purp> hogepodge: what's the overall cost, roughly?
16:35:54 <hogepodge> I've spoken with qa, and they think that a bunch of those tests can be refactored. Right now they require tenant isolation because it's part of a generic class setup, but not all need it
16:36:15 <hogepodge> purp: based on Dreamhost around $200/mo
16:36:27 <hogepodge> purp: that's to pay for enough vms across two tenants.
16:36:27 <purp> Okay. 10 public clouds?
16:37:06 <hogepodge> So that list I posted can be pruned down.
16:37:27 * dwalleck spent way more than $200 last May
16:37:30 <markvoelker> purp: more like about 20 total in the Marketplace, but not all of those may attempt to be OpenStack Powered
16:37:41 <hogepodge> Also, if we know which tests require tenant isolation, we can have a shadow guideline that excludes them, and can be used as a partial independent verification.
16:37:47 <purp> hogepodge: I completely agree that things which don't require multitenancy should be pared down as much as possible via refactoring.
16:38:49 * markvoelker notes 22 minutes remaining and wonders if we should just note "please add comments to review" and move on at this point
16:39:05 <hogepodge> dwalleck: yeah, I feel that pain as an individual swiping my card rather than as a corporation, and I'm getting reimbursed. I guess I'm kind of cheap? :-D
16:39:11 <hogepodge> markvoelker: agree
16:39:32 <purp> Let's do. I'll say that I'm more interested in finding a way to cover costs of independent testing than I am of optimizing to reduce it at this point.
16:39:37 <markvoelker> #action everyone please add your comments to the gerrit review https://review.openstack.org/#/c/253138
16:39:39 <dwalleck> hogepodge: That was my individual card :-(
16:39:46 <purp> For example, if there was a fee for TM use.
16:39:50 <purp> I'll follow up there.
16:39:57 <hogepodge> #topic Remove "flag all tests clauses"
16:40:02 <markvoelker> #link https://review.openstack.org/#/c/234824/
16:40:24 <dwalleck> purp: +1
16:40:30 <SammyD> purp: +1
16:40:53 <markvoelker> I think the commit message more or less sums this one up.  Basically the assertion here is that the "you can't flag all tests in a capability" clause we have today isn't needed and in fact may just be a hinderance to using good judgement
16:41:03 <hogepodge> +1
16:41:30 <markvoelker> So, here again: happy to entertain commentary now, but I think we just need to get people to actually comment on the review
16:41:32 <hogepodge> with the goal of exercising good judgment and possibly reintroducing once the guideline has stabilized a bit more
16:42:02 <hogepodge> but with the introduction of lots of new projects scheduled for 2016, I expect a lot of flagging against new capabilities
16:42:09 <hogepodge> (just based on experience)
16:42:40 <purp> hogepodge: generally agree. Will comment on review. =]
16:43:39 <markvoelker> hogepodge: frankly I probably wouldn't even bother reintroducing it unless it proves to be necessary.  I'm having a hard time imagining a future where it's necessary.
16:43:47 <markvoelker> But we can deal with that later. =)
16:43:58 <markvoelker> Ok, move on?
16:44:19 <purp> markvoelker +1 move on
16:44:22 <markvoelker> Next topic is sort of related anyway actually
16:44:34 <markvoelker> #topic DefCore Capabilities should have more than one test
16:44:37 <markvoelker> #link https://review.openstack.org/#/c/233814/
16:45:15 * purp chuckles.
16:45:18 <markvoelker> I think hogepodge and I have sort of both come down on the side that this is another unecessary restriction, and one that would be violated 17 times as proposed anyway.
16:45:33 <markvoelker> But obviously there is room for more opinions. =)
16:45:34 <hogepodge> yup, it's nice in principle, bad in practice
16:46:14 <markvoelker> Partly I think this came about because of the "you can't request to flag all tests in a capability" rule, so if that goes away we may not even really need this.
16:46:57 <catherineD> markvoelker: does that mean that you will end up with capability with no test?
16:47:10 <markvoelker> catherineD: nope, it means that one-test capabilities are ok
16:47:26 <markvoelker> which is the current state of affairs
16:47:27 <purp> It seems like the core of this and the last are: "We want to name capabilities that are important, they may have few or buggy tests, and we want to make sure we get more and better tests."
16:48:36 <purp> And generating interest in making better tests is hard.
16:48:38 <markvoelker> purp: yeah, with the added stipulation that it seems at least some in the community feel that the minimum bar for a capability should be that it has multiple tests associated with it.
16:49:06 <purp> markvoelker: I'm probably in that camp, though I need to introspect a bit to know why
16:49:34 <purp> It seems like DefCore is saying "these are the most important areas of OpenStack to get right"
16:49:43 <markvoelker> purp: one counterpoint to which has been: if the technical contributors in the community feel a thing is adequately tested with one test and it becomes widely adopted, why should DefCore set a different bar than devs and users?
16:49:48 <purp> And I like my most important code to be very well tested.
16:50:13 <markvoelker> purp: I definitely think we all like well tested code. =)
16:50:32 <hogepodge> To me there's also a sense of "does this thing work?" And some things are easy to check. Like, "give me a token". You know if you get a token or not.
16:50:39 <purp> markvoelker: My counter to that argument would be, "Most users never look at the tests, and so can't be assumed to be satisfied. And most devs aren't as interested in making test code as they are in making feature code."
16:50:58 <hogepodge> And that one test may actually do a lot to make sure the thing is working.
16:51:05 <dwalleck> I love writing test code :-)
16:51:15 <purp> dwalleck: ++ me too. =D
16:51:29 <markvoelker> hogepodge: Right, some of the image tests come to mind.  I think there's one test that creates an image, boots it, and does some other stuff all in one test for example
16:51:37 <purp> hogepodge: so a given test shouldn't necessarily do a lot of things
16:52:08 <SammyD> purp: +1 for test cases being atomic
16:52:10 <purp> markvoelker hogepodge: like the Oracle example — implictly covers many things, and should be refactored into more than one test.
16:52:51 <purp> And, super-often, as an OS dev faced with "make more tests or make code to scratch my itch" I'll choose the latter.
16:53:06 <purp> Partly, I think we exist to urge the former in the important places.
16:53:17 <purp> </soapbox>
16:53:59 <purp> I'll comment in the review after I think a bit. I'm inclined to say we knowingly violate the one-test principle, shouldn't, and want to improve that situation if we can.
16:53:59 <hogepodge> With a few minutes left, I propose that we wrap up by posting links to the discussion items we're not going to make it to today.
16:54:30 <hogepodge> #topic outstanding reviews (backlog for next week)
16:54:31 * markvoelker is fine with that
16:54:45 <hogepodge> #link https://review.openstack.org/#/c/226980/
16:54:49 <hogepodge> review for adjusting weights
16:54:59 <hogepodge> #link https://review.openstack.org/#/c/224868/
16:55:07 <hogepodge> review for adding cutoff score field to guideline
16:55:17 <hogepodge> #link https://review.openstack.org/#/c/232128/
16:55:29 <hogepodge> review for recurring testing recommendations
16:55:44 <hogepodge> #topic open discussion
16:56:02 <purp> Thanks, markvoelker and hogepodge, for all the work you do (and for stepping in to run things today.
16:56:03 <hogepodge> any last bits anyone wants to add?
16:56:36 <purp> Just an apology that I've not been very present for the past several months and have remedied the conflict.
16:56:41 <hogepodge> thank purp and markvoelker and everyone else. always lots of good discussions here
16:56:49 <purp> And for talking so much today. But I'm not sure that will change in the future. =]
16:57:00 <markvoelker> In particular I'd like to call out https://review.openstack.org/#/c/226980/ for comments as I'd love to get that settled before we go into our next round of scoring
16:57:25 <purp> markvoelker: when is that?
16:57:33 * markvoelker high fives everyone who provides input including purp and hogepodge
16:57:35 <hogepodge> markvoelker: I'm thining that we might want a new scoring category
16:57:51 <markvoelker> purp: well, I have an AI to post patches adding capabilities as a result of the Oracle discussion this week. =)
16:58:11 <hogepodge> markvoelker: I need to give it a bit more thought though. Something along the lines of testability or usage for interoperability. Needs thinking.
16:58:39 <hogepodge> (as in, it's not well formed enough yet in my mind)
16:58:42 <markvoelker> hogepodge: sure, would be happy to hear thinking on that
16:59:06 <hogepodge> Thanks everyone.
16:59:34 <markvoelker> #endmeeting