15:00:31 <zns> #startmeeting satori
15:00:32 <openstack> Meeting started Mon Apr  7 15:00:31 2014 UTC and is due to finish in 60 minutes.  The chair is zns. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:36 <openstack> The meeting name has been set to 'satori'
15:01:39 <zns> Good morning/afternoon! Who's here?
15:01:45 <samstav> o/
15:01:45 <gondoi> o/
15:01:51 <gusmaskowitz> o/
15:01:55 <gusmaskowitz> howdy folks
15:01:56 <jasonpgignac> o/
15:02:32 <zns> Cool. I think others are out today, so we have quorum.
15:02:42 <zns> #topic action items
15:03:06 <zns> Reviewing action items: #1 address possible duplication between 1295391 and 1293670 - DONE
15:03:18 <zns> #action zns start ML discussion & blueprint for built-in python data plane discovery
15:03:22 <zns> Not done.
15:03:53 <zns> gondoi - we didn't "discuss netloc_info and ip_info" ... but did you "create blueprint"?
15:04:01 <gondoi> yes
15:04:11 <zns> Cool. DONE. Thank you!
15:04:23 <zns> #action caleb_ Developer docs for doing discovery using Python rather than the command line
15:04:26 <gondoi> https://blueprints.launchpad.net/satori/+spec/ip-and-netloc-info
15:04:27 <zns> assuming not done.
15:04:38 <zns> #link https://blueprints.launchpad.net/satori/+spec/ip-and-netloc-info
15:04:38 <hhoover> o/ (sorry for lateness)
15:04:43 <zns> Hi hhoover
15:05:00 <zns> manage the agenda and the Thursday cut-off for topics - partially done.
15:05:28 <zns> I updated the meeting docs, but I did not send out a note last week on the ready agenda items.Sorry about that. I'll put it in my book for this week.
15:05:48 <zns> gusmaskowitz: create a spec and blueprint for Triple-O - status?
15:05:54 <gusmaskowitz> DONE.
15:06:03 <zns> Nice.
15:06:13 <zns> Do you have a link?
15:06:18 <gusmaskowitz> I wont say it's a GREAT blueprint…. but it's there
15:06:29 <zns> That counts.
15:06:30 <gusmaskowitz> now for the discussion.
15:06:44 <zns> gusmaskowitz: link?
15:06:57 <gusmaskowitz> https://wiki.openstack.org/wiki/Satori/Specialize-in-Tripple-O-Discovery
15:07:14 <zns> #link https://wiki.openstack.org/wiki/Satori/Specialize-in-Tripple-O-Discovery
15:07:16 <zns> Thanks
15:07:19 <zns> gondoi: adding blueprint/spec URLs next to these topic items and adding them to next weeks agenda?
15:07:58 <gondoi> ?
15:08:00 <gusmaskowitz> zns that was the wiki link sorry. This is the blueprint. https://blueprints.launchpad.net/satori/+spec/satori-specialize-in-tripple-o
15:08:11 <zns> #link  https://blueprints.launchpad.net/satori/+spec/satori-specialize-in-tripple-o
15:08:31 <zns> gondoi: that was an action item (including me to help!) from last week.
15:08:43 <gondoi> oh yes
15:08:44 <gondoi> done
15:08:50 <zns> Great.
15:09:02 <gondoi> there is a blueprint, but i did not create the spec urls to the wiki
15:09:09 <gondoi> links in the meeting notes
15:09:31 <zns> Cool.
15:09:50 <gondoi> sorry i mean agenda
15:10:10 <zns> Does anyone want to discuss a specific blueprint or spec from https://wiki.openstack.org/wiki/Meetings/Satori ?
15:10:33 <zns> Given I didn't send an email out, I'm not going to assume the topics have been read by all...
15:10:58 <samstav> https://blueprints.launchpad.net/satori/+spec/satori-minimal
15:11:11 <zns> I did write this spec https://wiki.openstack.org/wiki/Satori/DiscoverySchema on Thursday, but did not send it out.
15:11:37 <zns> samstav: I know you spent a lot of time on that one.
15:12:00 <samstav> Unfortunately so. I have a couple things to mention
15:12:00 <zns> # topic Discuss https://blueprints.launchpad.net/satori/+spec/satori-minimal
15:12:44 <zns> samstav: go ahead...
15:13:13 <gondoi> 140 characters or less :P
15:13:19 <samstav> This is a description of where I left off
15:13:21 <samstav> https://gist.github.com/smlstvnh/9863169
15:13:47 <zns> multimedia. Love it. samstav raising the bar.
15:14:15 <samstav> I would bore you to tears going into the details, but most of them are there. Currently looking for alternatives to the initial implementation idea.
15:14:45 <samstav> The attempt also resulted in an bug + patch to pbr :
15:14:47 <samstav> patch: https://review.openstack.org/#/c/84218/
15:15:10 <samstav> bug: https://bugs.launchpad.net/pbr/+bug/1300381
15:15:48 <samstav> Mr. Hellman seems skeptical, but I'm still waiting for a reply on the bug discussion
15:16:20 <zns> samstav: what alternative are you leaning towards? Or what direction do you want to take this to?
15:17:21 <samstav> The idea is, you can create a package's environment so that it can be extended, but it's hard to do it the other way around, I found.
15:17:57 <zns> And that conflicts with that we want the default package to be the full one with all dependencies... right?
15:18:02 <samstav> We were trying to create a "base" package with full dependencies, and a "minimal", alternative package with *fewer dependencies.
15:18:22 <samstav> zns exactly
15:18:50 <zns> The conflict arises when we use the same package for those two use cases. So are we going to need to have two packages?
15:19:15 <samstav> We either need two packages, or for the "base" package to be the minimal one
15:19:53 <zns> I prefer the first option, since the minimal package is for a very precise use case that is not the most common one.
15:20:03 <zns> i.e. two packages
15:20:26 <samstav> OK, that is definitely the quickest solution as well
15:21:15 <zns> The use case being, just for the record, for those who want to run satori for data plane discovery on a local machine. So they don't want to have to install nova client and paramiko with their complex dependencies...
15:21:28 <samstav> How will we handle the namespacing? e.g. if I have satori-minimal installed AND satori installed in the same environment?
15:22:16 <zns> good point. Not only the package names should be different, but potentially the app as well.
15:22:46 <zns> Unless we have a special entry point and packge name: mini-satori
15:22:51 <samstav> My assumption is that you will get the same code installed, but import statements will be try;except [ed], and that they have the same namespace
15:23:09 <samstav> or is that ^^ chaos
15:23:13 <zns> What happens if I have satori installed and then I run pip install satori-minimal?
15:23:44 <zns> It should be a noop if we're going to have both be supported...
15:23:49 <samstav> right. Is that possible?
15:23:54 <zns> And that feels jacky.
15:24:02 <zns> hacky!
15:24:08 <samstav> hmmmmmm.
15:24:28 <zns> Consider then running pip uninstall satori-minimal. Wouldn't that break satori?
15:24:42 <samstav> Well, that depends on how pip handles these packages.
15:25:01 <zns> Unless satori depends on satori-minimal and that's how the packages are builtt?
15:25:04 <samstav> Can packages have a different name on pypi but the same application-package name?
15:25:46 <zns> I believe it is possible. I don't know how clean that will be for folks installing/uninstalling packages.
15:25:57 <samstav> satori depends on satori-minimal… sure.
15:26:13 <samstav> that sounds right, right?
15:26:24 <gondoi> i like that
15:26:40 <gondoi> satori-mini being a dep of satori
15:26:45 <zns> It sounds like it would work. I don't know about if it is the "right" solution. Too subjective of a term :-)
15:26:58 <zns> Me too. Sounds like the better path.
15:27:00 <gondoi> what about calling it satori-local
15:27:03 <samstav> It's not always obvious, unless you're dutch.
15:27:07 <zns> samstav: do you want to explore that?
15:27:37 <samstav> satori-local is probably better if that is its primary use case
15:27:37 <zns> +1 satori-local
15:27:44 <samstav> +1
15:28:32 <zns> # agreed we'll handle the local use case by creating a satori-local package and having satori depend on it to minimize duplication of code and keeping packages clean.
15:28:39 <zns> #agreed we'll handle the local use case by creating a satori-local package and having satori depend on it to minimize duplication of code and keeping packages clean.
15:28:56 <samstav> Sounds good. I'll explore it
15:29:07 <zns> Thanks.
15:29:34 <zns> Can we discuss https://wiki.openstack.org/wiki/Satori/DiscoverySchema? Any objections?
15:29:47 <gondoi> sounds good to me
15:29:49 <zns> #topic Schema https://wiki.openstack.org/wiki/Satori/DiscoverySchema
15:30:18 <zns> I have some specific questions to bring up, but fire away if you have some too...  first question:
15:30:54 <zns> How do you feel about the "errors" array? Is it intuitive? Any objections?
15:31:20 <gondoi> i like it.. do we care what order they happened in?
15:31:38 <zns> The idea is that not all errors mean discovery should fail. We can have partially successful discoveries.
15:31:46 <jasonpgignac> I imagine the user will, optionally, want some sort of Stack Trace.
15:32:17 <gondoi> or possibly additional contextual data around the error, like what was being executed
15:32:23 <zns> gondoi: I can't think of a use case. Especially since some operations might happen in parallel. Can you think of a use case?
15:32:24 <hhoover> jasonpgignac: I would want to know the WHY around an error
15:32:39 <zns> jasonpgignac: how about that gets added if --debug is turned on?
15:33:15 <jasonpgignac> hhoover: I agree in concept. However, my experience of error handling is that many errors occur that we as the devs do not expect. These errors, we don't KNOW the why. So we cannot communicate it to the user.
15:33:24 <zns> I don't think stack traces should be provided in output by default.
15:33:26 <samstav> Now --debug is affecting not only how satori runs but the discovery results ?
15:33:28 <gondoi> zns: no i don't think so unless it's a timing issue... but I can't think of any off the top of my head
15:34:14 <samstav> not "Now", I mean "Then".   That might be odd. But a separate option might be overkill. Hmm.
15:34:34 <jasonpgignac> zns: I imagine you will have a lot of people irritated at running a five minute discovery, having it error, not knowing what to do with the error because it provides limited information, and having to wait 5 minutes to run it again, with debug on.
15:35:11 <jasonpgignac> zns: This is a technical tool, with a target market of people who not only can read a stacktrace, but would prefer to have the data vailable in the case of an error, I would think?
15:35:24 <hhoover> maybe the error messages just need to be very descriptive. "could not log in" != "server unreachable"
15:35:56 <zns> Even technical tools don't output stack traces unless you have debugging turned on. Give me an example of one that does?
15:36:09 <jasonpgignac> hhoover: DEFINITELY more descriptive, and perhaps we can just make the carefully defined errors show iwth no stacktrace, but an unexpected one be more descriptive?
15:36:11 <zns> hhoover: +1
15:36:25 <zns> Good error messages/handling.
15:36:33 <jasonpgignac> This would require a lot of work from the devs to make sure they catch ONLY the VERY narrow situation they'e planned for. And write a lot of error handlers.
15:37:00 <samstav> Can we think for a second about *not* including error information directly inside the discovery results?
15:37:19 <samstav> But rather, *always* write a log somewhere, much like pip does?
15:37:38 <samstav> Unless the error information is a valid piece of discovery results
15:37:53 <zns> samstav: how would that work when satori is used as a library?
15:37:55 <gondoi> that would be good, then you can dump whatever you want in the logs and not bother the user
15:37:57 <jasonpgignac> samstav: That's a great idea. Then write a little "There was errors in this discovery. For more info see foo/satori-broke-sorry.log
15:37:59 <hhoover> It could say something like, errors occurred, check satori.log for info
15:38:05 <hhoover> or some such
15:38:16 <hhoover> +1 jasonpgignac
15:38:27 <samstav> hhoover yes that is exactly what I was thinking, esp. when I referenced pip
15:38:33 <gondoi> zns: thinking about the library use case
15:38:36 <samstav> ins good question
15:38:40 <samstav> zns* ^
15:39:08 <jasonpgignac> If you use it as a library, then we shoudl raise errors.
15:39:11 <gondoi> maybe we can only put the log file work in the shell.py related code?
15:39:15 <zns> We could return a tuple, with errors and other non-results in the other variable.
15:39:18 <jasonpgignac> And let the developer wrapping it catch them if they wish.
15:39:33 <zns> jasonpgignac: that defeats the use case of supporting partially successful discoveries.
15:39:58 <jasonpgignac> zns: No, it just gives the user control over whether they would LIKE a partially successful discovery.
15:40:14 <jasonpgignac> Libraries should fail loudly and give freedom of handling to the implementer.
15:40:31 <jasonpgignac> IMHO
15:40:31 <zns> So what is returned if a discovery succeeds partially?
15:41:28 <zns> gondoi: I like that. Then one response comes from the discovery code and shell.py strips out the errors?
15:41:32 <jasonpgignac> Then, let the user receive the error, log it, handle it, or deal with as they wish, and decide what to do with the parts that worked.
15:41:59 <gondoi> yes and logs to file the excess info for reference later
15:42:13 <zns> jasonpgignac: How would you raise an error and return valid results at the same time?
15:42:21 <gondoi> ~/.satori/logs/satori.log (possibly)
15:42:42 <jasonpgignac> Have an error that returns the partially completed results as a property of itself?
15:43:05 <zns> jasonpgignac: -1. Does not feel pythonic to me...
15:44:05 <zns> I don't think we should be returning valid results inside errors. Furthermore, how would that work when there are multiple errors? Feels like the wrong direction...
15:44:14 <jasonpgignac> zns: A function that returns data to me on a partial failure feels highly idiosyncratic to me.
15:44:27 <zns> I prefer gondoi's approach. Strip errors out in shell.py
15:45:38 <jasonpgignac> 'sfine. Just the way I see it. Failures shoudl look like failures. I should not have to check if something partially failed. That feels like tricking the end user.
15:45:46 <jasonpgignac> The user of the library, I mean.
15:46:36 <jasonpgignac> I mean, that sounds like, maybe, an OPTION I could feed in?
15:46:55 <samstav> So, in shell.py,  we strip out errors from the discovery results, but if satori is being used as a library, we leave tracebacks/error messages inside the discovery return body?
15:47:08 <jasonpgignac> Like, "satoriobjectthing.discover(ignore_errors=true)
15:47:09 <samstav> Not sure if I followed all of that correctly.
15:47:28 <jasonpgignac> But the default, to me, would be false.
15:47:38 <jasonpgignac> But I can submit to the opinion of the majority, that's fine.
15:47:39 <zns> samstav: yes.
15:48:23 <samstav> The code/fns used to strip out those errors should be exposed to the user of the library so that they can do the same.
15:48:51 <jasonpgignac> I have another question, too, but can wait until we are done with this part
15:49:03 <zns> jasonpgignac: I'm just concerned that if we raise an error on every failure, we're not going to have a useable library. Think about how we've been using this at Rackspace. We provide a lot of useful information even if we can't access one server in a topology.
15:49:55 <jasonpgignac> zns: I can  have a more thorough conversation on what I think woudl be the solution to that, but it might be bigger than this meeting.
15:50:30 <samstav> I.e. it should be part of satori's python API. I am re-considering my opinion that errors/tbs shouldn't be part of the discovery results, because there really isn't a better place for them. Esp. from a python API perspective, I would like to have access to those tracebacks and error context right where the data would have otherwise been.
15:51:32 <zns> jasonpgignac: I don't believe "Libraries should fail loudly and give freedom of handling to the implementer" translates to "Always raise an exception" and never return partial results. We're returning errors clearly. And in some cases, when we're doing a sequence of steps, we can return a grape of what succeeded and what did not.
15:52:45 <jasonpgignac> zns: I see your point, I think that essentially the problem is that you are not running and returning the results of one mehtod - you are running a number of interrelated methdos with essentially independent result sets that are then integrated into a related set of data.
15:53:03 <jasonpgignac> What you have are essentialyl a set of nested promises, some of which you may break.
15:53:04 <zns> samstav: I agree. And if --debug is true, maybe the errors list returned would also have tracebacks. Then a library caller can parse those results.
15:53:38 <jasonpgignac> As a library user, I would like to be able to decide which of those promises I can live without being fulfilled, and which are entirely essential, in a simple, default way.
15:53:59 <jasonpgignac> So, a set of nested promise handlers, more or less.
15:54:21 <jasonpgignac> If I DON'T provie a handler, though, I don't think the right thing to is assume I dont' care if a given piece of data works.
15:55:19 <zns> jasonpgignac: we've not built in the ability to specify what you want yet. Right now, satori decides that. At some point, that could be a feature.
15:55:49 <gondoi> is this something we may want to finalize in #openstack-satori if we have anything else to quickly discuss?
15:55:52 <zns> 5 minutes left...
15:56:23 <zns> How about I take this feedback and come back with an updated proposal next week?
15:56:27 <jasonpgignac> zns: Yes, as stated, I think this conversation is probably bigger than this meeting. I am willing to defer to the majority opinion, or to table until we have mroe time.
15:57:31 <gondoi> i would like to see a more final proposal... maybe we can mull over it and find a time to discuss before the next meeting?
15:57:41 <jasonpgignac> agreed
15:58:37 <zns> OK. I'll rev the proposal and send that out. We can then set up a time to discuss.
15:58:45 <gondoi> sounds good
15:58:54 <zns> OK. Thank you for the good discussion. We're out of time for today.
15:59:03 <samstav> cool
15:59:06 <gondoi> thanks zns
15:59:08 <zns> #endmeeting