14:59:43 <eglute> #startmeeting defcore
14:59:44 <openstack> Meeting started Wed Aug 12 14:59:43 2015 UTC and is due to finish in 60 minutes.  The chair is eglute. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:59:45 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:59:48 <openstack> The meeting name has been set to 'defcore'
15:00:01 <markvoelker> o/
15:00:09 <gema> o/
15:00:22 <eglute> Hello everyone, wave \o o/ if you are here for defcore meeting
15:00:33 <hogepodge> o/
15:00:50 <eglute> Etherpad #link https://etherpad.openstack.org/p/DefCoreFlag.10
15:01:12 <eglute> #topic agenda
15:01:41 <eglute> please take a look at the agenda and add/update as needed
15:02:01 <catherine_> o/
15:03:22 <eglute> #topic midcycle recap
15:03:47 <eglute> thank you everyone for coming to the midycle in Austin and enjoying the nice TX summer
15:04:14 <eglute> hogepodge and Rob wrote a couple blog posts recaping things accomplished at the midcycle
15:04:26 <eglute> #link http://superuser.openstack.org/articles/openstack-continues-to-strenghten-its-commitment-to-interoperability
15:04:38 <eglute> #link http://robhirschfeld.com/2015/08/11/defcore-update-slowly-taming-the-interop-hydra/
15:05:05 <eglute> i think we were very productive during midcycle, and settled a lot of issues.
15:05:22 <eglute> i wont go over everything here, but let me know if you have questions
15:05:34 <eglute> if not, we will move to the next item
15:05:35 <auld> 0/
15:06:11 <galstrom> o/
15:06:25 <hogepodge> I added a followup from a meeting working item at the end of the agenda.
15:06:38 <eglute> thank you hogepodge, so it
15:06:50 <eglute> saw it
15:07:10 * eglute cannot spell today
15:07:52 <eglute> #topic ID new capabilities
15:08:49 <eglute> so we have 76 days till Tokyo, and 15 days before we need to have new capabilities identified
15:09:19 <markvoelker> eglute: I've added the names of folks who volunteered to be point on IDing new capabilities to the etherpad.
15:09:28 <eglute> markvoelker thanks- how is that going
15:09:33 <hogepodge> I've fallen down on following through with that. It's on my todo list for the next few days.
15:09:41 <hogepodge> Before the ops mid-cycle
15:10:03 <eglute> i am ok using this time to talk about capabilities since it is the most important item on the agenda
15:10:06 <markvoelker> For my part I'll have netowrking mostly hammered out ~middle of next week and should have made a dent in Nova by week's end.
15:10:08 <hogepodge> (except for keystone which is small)
15:10:19 <markvoelker> Nova should be easier since it already has some capabilties.
15:10:50 <catherine_> for the new capabilities for Tokyo, do we mean for Network or for all other project as well ..
15:11:37 <markvoelker> catherine_: with each Guideline we evaluate potential new Capabilities from all of OpenStack.  If there are good candidates, we score them.
15:11:38 <catherine_> since we have decided at the midcycle that the next project to add to core is Neutron ...
15:12:15 <catherine_> ic ..
15:12:19 <markvoelker> cateherine_: So the next Guideline could contain more new stuff than Neutron capabilties, yes.  If good candidates are ID'd by the point persons listed in the etherpad and they score high enough.
15:12:34 <hogepodge> neutron is getting special consideration, but we could add glance capabilities if they enhance compute in some way
15:13:13 <eglute> hogepodge have seen people ask for new capabilities?
15:13:41 <catherine_> ok in that case I will work on a patch for Heat ...
15:13:52 <eglute> also, what are you seeing in this round? are people getting ready for the summit yet, as far as you can tell?
15:14:17 <eglute> catherine_ I don't think Heat would be considered for DefCore, at least not now
15:14:49 <markvoelker> eglute: I think it's worth looking into at least.  Based on user survey results it's almost as widely deployed in prod as Swift now.
15:14:50 <hogepodge> eglute: no, I did see the the current glance tests we have specifically target v2 by name, which is worrisome given the current state of the world (many v1 deployments and what appears to be an incomplete implementation of v2)
15:14:51 <markvoelker> eglute: http://a4.res.cloudinary.com/hqq9ey1mh/image/upload/c_limit,w_793/v1431734254/ndz1dxvsq12unsmwhjo8.jpg
15:15:11 <markvoelker> It may not score high enough, but at least we'll know where it stands.
15:15:41 <eglute> markvoelker you are right! that is interesting. where is that info coming from?
15:16:01 <markvoelker> eglute: http://superuser.openstack.org/articles/openstack-users-share-how-their-deployments-stack-up
15:16:10 <markvoelker> E.g. the user survey.
15:16:14 <eglute> markvoelker thanks!
15:16:39 <eglute> hogepodge sounds like we need to re-evaluate glance?
15:17:19 <hogepodge> eglute: it's weirdly classified. Right now we only run against the proxy, but it was reclassified as v2 only.
15:17:24 <hogepodge> (nova proxy)
15:17:56 <hogepodge> eglute: had I been more careful about evaluating that reclassification I would have -1'ed it, but I didn't
15:18:16 <eglute> ok, do we need to take another look at glance then
15:18:56 <hogepodge> eglute: but all of that is based on nova proxy, which in practice uses v1 in the backend and can use v2 or v1 in the front end
15:18:56 <gema> someone did ask on our tailgaters meeting about glance and being able to use it without admin
15:19:02 <markvoelker> eglute: I don't see why we wouldn't at least examine it.  It's fairly widely deployed so it seems reasonable that there are capabilties there that app developers might want to depend on for interoperability.
15:19:12 <gema> I can confirm I can list images in v1 and v2 of glance's api without admin
15:19:18 <gema> (in case it helps)
15:19:25 <eglute> thanks gema
15:19:28 <gema> np
15:19:37 <markvoelker> I'm sort of punting to the point people on the etherpad to do that eval and if they think there's reasonable candidates, propose scores for them so we can discuss via gerrit.
15:19:51 <eglute> markvoelker +1
15:19:54 <hogepodge> markvoelker: +1
15:20:52 <eglute> hogepodge since you have glance, can you re-evaluate and propose new?
15:21:00 <hogepodge> yes
15:21:04 <eglute> thanks
15:21:28 <markvoelker> Oh, and speaking of that: as of right now the only scoring proposal we have is the WIP networking one, so I haven't started the ball rolling on scoring meetings yet....
15:21:56 <eglute> markvoelker that's ok! also, we could use this time slot as well
15:22:08 <markvoelker> ONce I see those come in I'm happy to chair a weekly meeting or scoring discussions per earlier discussion, just didn't want to call meetings with nothing to discuss yet. =)
15:22:28 <eglute> markvoelker works for me!
15:22:47 <markvoelker> #info Scoring proposals need to be submitted by Aug. 27 per 2015A timelines
15:23:14 <markvoelker> #info Scoring itself should be finished by Sep 27.
15:23:14 <eglute> markvoelker i talking scoring needs to be done by Sept. 27?
15:23:50 <markvoelker> eglute: correct.  By Aug 27 we just need the proposals submitted to Gerrit.  Then we've basically got a month to review them.
15:24:04 * eglute thanks markvoelker
15:24:36 <markvoelker> So, also on the topic of scoring....
15:25:04 <markvoelker> At the midcycle when we adopted the new scoring format, folks asked for a script to calculate the total scores for each Capability
15:25:07 <eglute> so designated sections fall under scoring, correct? we dont have specific timelines for defining those
15:25:12 <markvoelker> And some folks asked for it to produce a CSV file as well
15:25:29 <markvoelker> I posted a script that does that here, which could use some reviews: https://review.openstack.org/210674
15:25:49 <eglute> thanks markvoelker
15:26:08 <markvoelker> #link https://review.openstack.org/210674 Scoring script
15:26:10 <eglute> #action everyone review scoring script #link https://review.openstack.org/210674
15:26:29 <eglute> #topic scoring
15:27:21 <markvoelker> eglute: getting back to your question on designated sections: I had sort of envisioned that being part of the capabilties scoring
15:27:23 <markvoelker> So basically....
15:27:48 <eglute> there is also a scoring format for review, everyone please take a look at it: #link https://review.openstack.org/#/c/210080/
15:28:03 <markvoelker> When we find that we've scored something high enough to add it to a Guideline, the patch to .next.json would also include additions to the designated sections if the new capabilities aren't covered already
15:28:31 <eglute> +1, works for me
15:28:56 <eglute> i think we need scoring section in our hacking file
15:29:16 <markvoelker> eglute: Sure.  I can work that up once we actually approve the scoring script.
15:29:46 <eglute> #action  markvoelker propose patch to hacking file after scoring script and format approved
15:30:10 <eglute> any other comments/questions regarding scoring?
15:30:42 <markvoelker> eglute: just one trivial one =)
15:30:53 <eglute> do tell!
15:31:25 <markvoelker> Folks may notice the scoring script has a -1 from jenkins on it at the moment.  Apparently the gate is currently having issues, so I'll mind rechecks for it.
15:31:27 <markvoelker> The script itself is fine, so don't let that stop you from reviewing it.
15:32:05 <eglute> cool, i was wondering about that
15:32:31 <markvoelker> eglute: if you look at the console log for the failure, jenkins is basically not finding it's venv.  Nothing to do with the change itself.
15:33:01 <eglute> poor jenkins :)
15:33:43 <eglute> talking of patches: everyone please take a look at this, i would like to merge this after Jenkins starts behaving again: https://review.openstack.org/#/c/210686/
15:34:21 <eglute> #topic require vendors to submit configuration
15:34:30 <eglute> #link https://review.openstack.org/#/c/207209/
15:34:42 <eglute> markvoelker has made comments on the patch
15:35:00 <eglute> would like to hear people's feedback on the subject
15:35:08 <eglute> markvoelker, can you summarize?
15:35:25 <markvoelker> eglute: sure.  So, I tried to capture some of the objections that we talked through at the midcycle
15:35:44 <markvoelker> Basically I'm not convinced that the data we're collecting is actually useful for the objective we're trying to meet
15:36:01 <gema> yeah, I was wondering what's the objective
15:36:20 <markvoelker> The patch says the objective is:
15:36:21 <eglute> hogepodge can you speak to that
15:36:40 <markvoelker> "The objective is for users to understand the requirements for deployable products and to know the resources available on hosted products."
15:36:51 <hogepodge> From an interop perspective the backing technologies are important to know.
15:37:14 <hogepodge> It varies based on what the product is
15:37:17 <markvoelker> But what we actually collect is information about one very specific configuration that was used to run one set of tests which require no scale or redundancy
15:37:46 <markvoelker> So that's not really informative at all.  Worst case, it's actually misleading.
15:37:56 <catherineD|2> We also discussed during midcycle that some of those info maybe confidential ...
15:38:04 <hogepodge> We use the information differently from distributions (where the user can reconfigure to be non-interop) and from private/public clouds.
15:38:20 <gema> vendors are public clouds or are they also integrators of openstack?
15:38:43 <hogepodge> Right now you need to know if a vendor is running kvm or xen, because that's a interoperability decision.
15:38:48 <hogepodge> For example.
15:38:55 <markvoelker> gema: this covers anyone that wants to call themselves an "OpenStack Powered" product
15:39:01 <gema> markvoelker: ok
15:39:16 <hogepodge> The policy is problematic because we have a wide use case to cover with it. The Foundation definitely wants that information though.
15:39:18 <gema> markvoelker: but integrators deploy different clouds for different usecases (in terms of configuration)
15:39:35 <markvoelker> hogepodge: but if I'm Red Hat or Ubuntu or Mirantis, I can let you configure any of those things.  THe fact that I happened to run my defcore tests on KVM doesn't tell you that.
15:39:39 <gema> so I am not sure how we would go about submitting that data unless the details are generic such as kvm/xen
15:39:54 <markvoelker> If you want information about what's supported, the vendor's docs are probably a much better place to find it.
15:40:04 <markvoelker> As opposed to looking at just one small deployment
15:40:08 <gema> yep, I can speak for ubuntu on this subject :D
15:40:30 <gema> markvoelker: +1
15:40:34 <hogepodge> It tells you how a vendor got to interop, and acts as guidance. An end user can do whatever they want.
15:41:20 <markvoelker> hogepodge: but it only tells you *one* configuration that was used to meet the Guideline.  It may be completely non production worthy, and there may be many other ways to get there.
15:41:25 <hogepodge> We've been asked if using a OpenStack Powered Platform distribution in new product immediately qualifies that product for the mark. The answer is a strong "no" for the reasons you mentioned."
15:41:55 <markvoelker> I would really hate for this to turn into "what you tested DefCore on is your default reference architecture" because that really does an injustice to all the other ways to configure a great cloud
15:42:08 <hogepodge> markvoelker: It tells you that you can get to interoperability.
15:42:19 <gema> hogepodge: what does that mean?
15:42:26 <eglute> hogepodge how about different configurable deploys that would be prod worthy and pass defcore? sounds to me like vendors would want to share all their configs then as passing defcore
15:42:27 <markvoelker> hogepodge: passing the tests tells you that. =)
15:42:39 <catherineD|2> strickly speaking we would achieve ultimate interops if we do not have to really care about the implementation
15:42:52 <hogepodge> as a cloud deployer, I hope I would be testing my cloud
15:43:08 <gema> hogepodge: you may be testing many clouds , in fact you do, as a cloud deployer
15:43:13 <hogepodge> I don't think there's anything onerous about asking "how did you get to these results".
15:43:26 <eglute> well, we deploy multiple diff configs for private cloud.
15:43:27 <markvoelker> eglute: I'd argue that they can't necessarily do that in the results of a single test run.  E.g. if I'm Ubuntu there are probably dozens of ways to get a production-grade interoperable cloud.
15:43:48 <eglute> markvoelker i agree,
15:43:50 <markvoelker> eglute: if they want to tell people about what they recommend, they probably publish reference architectures
15:43:58 <markvoelker> eglute: which may be usecase-specific
15:44:08 <eglute> right, i know we have multiple ref architectures
15:44:12 <catherineD|2> markvoelker: I feel like we are discussing about reference configuration ...
15:44:33 <catherineD|2> reference architecture and configuration ...
15:44:37 <markvoelker> catherineD|2: that's my fear
15:45:02 <eglute> i agree here...
15:45:13 <hogepodge> markvoelker: for distributions a deployer will have an architecture in mind.
15:45:49 <catherineD|2> hogepodge: that would be their default deployment ...
15:45:56 <eglute> hogepodge for a single distribution there can be multiple architectures
15:46:01 <markvoelker> hogepodge: as a deployer I would absolutely not want to get that from one test run that features no scale testing and was usecase-agnostic.
15:46:07 <hogepodge> markvoelker: but say RedHat says "these results are from a basic packstack deployment" and SuSE says "this is a four node HA test deplyment", that actually says a lot, even though you could do more or less with either
15:47:06 <hogepodge> So you're arguing that because the configuration may be weak we should just ignore that and not ask anything about it? We're shining daylight into the testing process.
15:47:18 <eglute> well, not just week
15:47:20 <gema> hogepodge: I think I see your point, we are doing some other compliance testing and we have to certify certain drivers and we are forced to run the tests on the biggest machine that this certification is going to be valid for
15:47:40 <gema> hogepodge: or else the run doesn't mean anything and you cannot really tell if the cloud is "compliant"
15:47:41 <markvoelker> I'm arguing that if the goal is to allow "users to understand the requirements" this completely fails at it.
15:48:40 <markvoelker> Or put another way: the stated goal here isn't to shine daylight into the testing process, according to the patch. =)
15:48:42 <eglute> markvoelker i agree
15:49:29 <markvoelker> To be clear: I'm not opposed to us collecting info on principal.  I just don't think we're accomplishing the stated intent.
15:49:37 <hogepodge> then we can update the patch to have better wording, but I feel strongly that vendors need to say something about their test environment that gives us an indication of what they're actually running and what users can expect
15:49:56 <eglute> should we start with optional reporting of this things?
15:50:07 <hogepodge> optional means not doing it
15:50:30 <gema> hogepodge: I can deploy a test cloud with a reference architecture for us and run the tests and that cloud will never be used by a customer and a production cloud based on that architecture will likely be much more powerful/reliable than my test cloud
15:50:32 <catherineD|2> hogepodge: for a public vendor they maynot want to disclose exact info ...
15:50:38 <hogepodge> How many vendors ran the advisory tests? Two come to mind immediately, maybe three.
15:51:38 <eglute> hogepodge you have the visibility into who runs the advisory tests
15:51:40 <hogepodge> catherineD|2: understandable, and I think we try to account for that
15:51:46 <markvoelker> hogepodge: to be fair, the instructions we gave them showed how to run just the required tests
15:52:05 <markvoelker> E.g. https://github.com/openstack/defcore/blob/master/2015.04/procedure.rst
15:52:11 <hogepodge> markvoelker: no, I mean people who were runnning defcore tests before they were required
15:52:23 <hogepodge> before 2015.03
15:52:59 <hogepodge> markvoelker: so what do you want to see on that patch?
15:54:21 <markvoelker> hogepodge: given the stated intent of the patch, I don't think a single refstack run is a good way to accomplish it.  I'm not sure what a better way is--perhaps a place in the Marketplace to show reference architectures or something.
15:54:47 <markvoelker> (and their corresponding test results)
15:54:59 <eglute> +1
15:55:02 <hogepodge> markvoelker: we want to include deployment information in the marketplace, requirements and backing technologies that do make a difference.
15:55:25 <markvoelker> hogepodge: That sounds like a thing that would be very useful
15:55:29 <hogepodge> So midonet vs ovs may not make a difference, but floating vs assigned ips may for example
15:55:43 <galstrom> hogepodge: is this not more of a function of the type of information you want reported not being discoverable via the APIs?
15:55:47 <hogepodge> kvm vs xen vs vmware definitely makes a difference
15:56:15 <catherineD|2> hogepodge: I agree on that one ..
15:57:08 <hogepodge> galstrom: the completely discoverable cloud is the platonic ideal we're reaching for :-D
15:57:22 <eglute> ok so sounds like we need more discussion on this and another patch
15:57:40 <galstrom> hogepodge: i agree.. but I dont think you are going to get vendors to optionally report this type of data
15:58:04 <hogepodge> Yes, we can take the discussion to the patch and we can propose a rewording of the goals and implementation.
15:58:07 <catherineD|2> I think the problem area is D505, D506 and D508
15:58:10 <eglute> hogepodge markvoelker how about a ML discussion on this?
15:58:18 <hogepodge> sure
15:58:26 <eglute> or a new patch and then discussion :)
15:58:28 <hogepodge> We have two minutes.
15:58:31 <eglute> 1
15:58:32 <catherineD|2> eglute: +1 on ML discussion ...
15:58:43 <markvoelker> sure, either is fine
15:58:58 <eglute> this way, we can get more feedback from others not in this meeting
15:59:05 <eglute> we are almost at time, any last words?
15:59:12 <hogepodge> In the last 60 seconds, take a look at https://review.openstack.org/#/c/164865/
15:59:21 <auld> ML?
15:59:29 <eglute> #action markvoelker hogepodge work on new patch for https://review.openstack.org/#/c/207209/ and start ML
15:59:31 <hogepodge> tl;dr, the functional tests from swift require administrator access
15:59:36 <catherineD|2> RefStack is now under Infra namespace .... next step is actually move the code ...
15:59:36 <galstrom> auld: Mailing List
15:59:53 <notmyname> hogepodge: I need to respond to that. what do you mean by "admin" access?
16:00:21 <hogepodge> notmyname: privileged account
16:00:21 <eglute> hogepodge i think we will need to postpone that till next meeting, as i think it will need more discussion
16:00:38 <notmyname> hogepodge: I'll check, but I don't think that's true
16:00:39 <eglute> #action everyone review https://review.openstack.org/#/c/164865/ before next meeting and comment on patch
16:00:42 <auld> :)thanks
16:00:52 <eglute> we are at time, thank you everyone!
16:00:52 <galstrom> auld: np
16:00:56 <hogepodge> notmyname: I need to check again too.
16:01:05 <eglute> #endmeeting