15:00:25 <zns> #startmeeting satori
15:00:25 <openstack> Meeting started Mon Mar 24 15:00:25 2014 UTC and is due to finish in 60 minutes.  The chair is zns. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:26 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:29 <openstack> The meeting name has been set to 'satori'
15:01:04 <zns> Hi - who's here for the Satori team meeting?
15:01:06 <caleb_> o/
15:01:11 <walker_> o/
15:01:12 <gusmaskowitz> o/
15:01:20 <samstav> o/
15:01:45 <gondoi> o/
15:02:15 <zns> Welcome! Thank you for joining. We don't have new items on the agenda. We'll review action items and if anyone has new topics, please have them ready for after that.
15:02:22 <zns> #topic action items
15:02:41 <zns> I assume the beer purchases are scheduled for Atlanta.
15:03:00 <caleb_> :)
15:03:06 <walker_> ha
15:03:16 <zns> gondoi: nova client requirements - I saw the pull request. All done? Anything else on that?
15:03:46 <walker_> zns: can we come back to that one? gondoi lied about being present
15:04:10 <zns> walker_: OK. I'll assume it is closed given his patch.
15:04:16 <adrian_otto> hi
15:04:16 <zns> samstav: implement localhost blueprint - done, right?
15:04:23 <zns> HI adrian_otto
15:04:30 <samstav> Indeed. Though the gates are cracking the whip
15:04:41 <gondoi> done
15:04:44 <zns> OK. I'll work on that with you today.
15:04:48 <zns> thanks, gondoi
15:04:59 <zns> samstav: plugin install options?
15:05:42 <zns> file bug for satori not working with ip address - done
15:06:00 <zns> reporting, while samstav types.
15:06:26 <samstav> There were some discussions floating around this week. Some regarding subclassing setuptools, or something completely different: using the package's "version" to indicate which satori package to pull down
15:06:37 <zns> implement plugins own establishing the connection to start with, but we provide a built-in ssh module that they can use - partially done. I'm closing the action item and letting the blueprint continue tracking this. We've done a lot of work on this with ohai-solo and the localhost feature.
15:07:06 <caleb_> samstav: Is there a blueprint for that?
15:07:47 <samstav> When I say subclassing setuptools, I really mean adding a Command or Feature (these are classes in setuptools) which could be passed to `python setup.py install` directly, OR through pip via `pip install ____ --install-options="--our-option-here'" `
15:08:09 <gondoi> zns: which bug ID is the IP bug?
15:08:15 <zns> I think there's a topic for discussion on this one. WRT dependencies, setup, per, etc...
15:08:40 <zns> bug 1293670
15:08:40 <samstav> There is no blueprint for that yet zns. That could be an action item for me this week.
15:08:48 <zns> https://bugs.launchpad.net/satori/+bug/1293670
15:09:25 <zns> Would that fall under https://blueprints.launchpad.net/satori/+spec/satori-minimal?
15:09:31 <samstav> s/could be passed to/could be targeted/
15:09:39 <zns> That's where we're talking about managing dependencies.
15:09:44 <gondoi> zns: if that hasn't been worked on, it may be a duplicate of https://bugs.launchpad.net/satori/+bug/1295391
15:09:55 <gondoi> let's table it for a second, sorry
15:10:30 <zns> gondoi: looks like a dupe, yes.
15:10:53 <zns> #action ins address possible duplication between 1295391 and 1293670
15:11:03 <zns> Damned autocorrect....
15:11:10 <zns> #action zns address possible duplication between 1295391 and 1293670
15:11:16 <samstav> ACK blueprint satori-minimal, though the brainstorming is leaning towards a generic solution for indicating the "type" or "edition" of the satori package
15:11:25 <gondoi> i have code submitted for it, we can review
15:11:38 <zns> cool
15:11:42 <zns> #topic dependencies and sertup
15:11:52 <zns> #topic dependencies and setup
15:12:16 <zns> We have a few possible blueprints around dependencies. Let me describe the issues...
15:13:11 <zns> Running pip install satori and then satori localhost does not need dependencies on paramiko or nova client. Those are heavy dependencies that cause fragility.
15:13:54 <caleb_> if a sysinfo plugin is needed for that use case, is it really all that minimal?
15:14:16 <zns> For example, paramiko requires pycrypto which requires python-dev packages and compilation that sometimes can fail. If we want pip install satori to be as reliable as possible for a localhost install, then these dependencies are problematic.
15:14:19 <caleb_> I'd argue that the ohai-solo package is much more heavy than paramiko and nova client.
15:14:40 <caleb_> ^ I retract my paramiko comment
15:15:17 <zns> But the ohai-solo plugin in satori is just python class, no external dependencies. And the package is self-contained, so installing it almost never fails.
15:15:49 <caleb_> I'm with you.
15:15:56 <zns> The ohai-solo one, that is. Credit to walker_  - that thing is solid and easy to install.
15:16:12 <walker_> except on Friday. :D
15:16:28 <zns> What happened Friday?
15:16:34 <walker_> .....um, nothing....
15:16:49 <walker_> Breakdown in the CI/CD pipeline for the packages
15:16:56 <walker_> but, it is all better now
15:18:01 <walker_> I would say that we should make installing things for the most common use case as easy as possible
15:18:28 <samstav> If localhost discovery is the most common use case, I must have missed something
15:18:32 <zns> walker_: did that break any installs?
15:18:47 <zns> On hosts..
15:18:55 <walker_> If localhost is easy to install, but the most common usage requres a bunch of flags, etc. Then we are not doing what we should be doing
15:19:03 <walker_> zns: no
15:19:15 <samstav> walker_ bingo
15:19:26 <zns> walker_: ok. So still reliable! Even on Fridays :-)
15:19:29 <zns> So the question before us is how do we support a minimal install for localhost discovery, and also support a full install with ssh and control plane install. One idea was to have a separate package in pypi. That could show up as a different name (ex. satori-full) or a specially tagged version (pip install satori==full)
15:20:01 <gondoi> sooooo.. someone is probably going to shoot me.. but i would say ansible
15:20:13 <walker_> everyone take a drink
15:20:38 <caleb_> gondoi: Go on.
15:20:38 <samstav> 🔫
15:20:43 <zns> And which one is the default. Minimal? And then from within satori we can install other packages. Ex. 'satori list packages' and 'satori install full'
15:20:44 <gondoi> actually.. that may not be true... i need to review it's full dependency tree
15:20:44 <samstav> don't shoot guns. don't be violent.
15:20:44 <gondoi> but
15:20:56 <gondoi> you can pip install ansible and run localhost discovery
15:21:16 <caleb_> samstav: The Australian version
15:21:19 <gondoi> no need for worry about external packaging and installations
15:21:24 <zns> gondoi: so... are you saying we should abandon satori and just use ansible?
15:21:35 <gondoi> i have a WIP going that's not working yet
15:21:43 <gondoi> haha......... no
15:21:46 <walker_> zns: I feel ya - and understand that localhost should be minimal; however, I'm not sure we understand the most common usage yet. Right now, localhost is not a common use-case at all
15:22:04 <gondoi> https://blueprints.launchpad.net/satori/+spec/ansible-when-present
15:22:29 <gondoi> that could change to just ansible as a discovery option
15:22:35 <walker_> Maybe the full, minimal, decaf, no-fat options should come once we have a better understanding of popular usage
15:22:44 <gondoi> right now the discovery data returned just needs to be translated to our schema
15:23:18 <gondoi> also, ohai stuff is working now, it just feels "heavy" in terms of dependencies
15:23:22 <walker_> also +1 to gondoi's Ansible stuff - that could result in a much lighter weight solution (although it requres paramiko doesn't it? /troll)
15:23:35 <zns> We have received a request for localhost. So maybe we just publish a minimal version for localhost and assume the default use case is not on local - and therefore should wrk with all the things (nova client, paramkio) installed?
15:23:43 <gondoi> that's why i said i need to review it's dependencies.. i'm sure it probably does
15:23:49 <samstav> satori-solo ?
15:23:53 <walker_> boooo
15:24:27 <zns> gondoi: I think the ansible blueprint doesn't change the discussion. We would still need to decide if we should have a minimal dependency version of satori.
15:24:28 <caleb_> I like full to be the default and minimal to be the reduced fat soy sugar free with 2 pumps version
15:24:41 <zns> And doesn't ansible depend in ssh? Does it use paramiko?
15:25:24 <walker_> zns: gondoi is unsure if it requires paramiko
15:25:29 <walker_> but it does use SSH
15:25:42 <zns> +1 to full being the default.
15:25:53 <walker_> +1 as well
15:26:01 <gondoi> looks like the answer is yes it does https://github.com/ansible/ansible/search?q=paramiko&ref=cmdform
15:26:23 <gondoi> it uses it's own ssh, it would not use the ssh module satori has
15:26:42 <gondoi> i think this is getting distracting from our current topic
15:26:46 <walker_> yeah
15:26:58 <zns> gondoi: even if we use ansible, we would still need to talk about nova client as dependency for control plane discovery, right? Or does ansible do control plane discovery? And if it does, does it not use nova client?
15:27:02 <walker_> zns: I'm afraid multiple packages is going to be confusing
15:27:18 <gondoi> ansible would be data plane
15:27:24 <zns> Any objections to the full being the default?
15:27:44 <samstav> ansible's ssh module: https://github.com/ansible/ansible/blob/c9fcbf7bdd46b493a1349662f92be63b66412007/lib/ansible/runner/connection_plugins/ssh.py
15:27:59 <gondoi> can we solve for a single step install process with full?
15:28:03 <zns> #agreed the full version (with dependencies) will be the default install of satori.
15:28:04 <gondoi> pip intall satori
15:28:27 <gondoi> right now it's potentially 3 step correct? depending on the OS
15:28:40 <zns> For localhost use cases, what does everyone think of having a package in pypi tagged with local. `pip install satori==local`
15:28:47 <samstav> It doesn't implement the ssh protocol, it just opens pipes through Popen
15:29:04 <caleb_> zns: I'm cool with that.
15:29:07 <walker_> zns: That seems sane
15:29:22 <walker_> what impact does that have with publishing packages?
15:29:27 <zns> gondoi: you mean also install ansible and/or ohai-solo ashen running pip install satori?
15:29:50 <zns> #agreed for localhost use cases, we will have a package in pypi tagged with local. `pip install satori==local`
15:30:22 <caleb_> do we know how we're going to get ohai-solo installed in that scenario?
15:30:29 <gondoi> no i mean apt-get install python-dev
15:30:31 <gondoi> pip install satori
15:30:43 <gondoi> or even apt-get install build-essentials
15:31:05 <zns> I don't know how that is handled in python, usually...
15:31:32 <zns> Is this a packaging question? Would that be apt-get install satori?
15:32:10 <caleb_> If I would have know this was going to be a packaging discussion I would have been sick today :)
15:32:34 <gondoi> well i saw some chatter about needing to install python-dev on 12.04 ubuntu, for example
15:32:51 <zns> caleb_: What's the benefit of installing ohai-solo with satori. The current method of downloading it when needed is working fine. It's actually not causing us any problems (that I've seen so far).
15:33:12 <caleb_> zns: I'm sorry, you're right.
15:34:43 <zns> I thin we're confusing the data plane components  with the dependencies in satori that are needed for remote data and control  plane discovery. We don't have a problem with data plane plugins (ohai-solo or ansible).
15:35:15 <caleb_> I think installing deps on remote machiens as a part of discovery makes sense
15:35:37 <caleb_> it begins to feel odd when the deps for localhost aren't handled at install time
15:35:42 <caleb_> IMO
15:35:46 <zns> So, problem #1: unnecessary dependencies for the local use case. Solved with pip install satori==local
15:36:16 <zns> problem #2: pip install needing additional packages (ex. python-dev).
15:36:53 <zns> I think problem #2 is a packaging problem.
15:37:12 <samstav> Problem #2 is very common
15:37:24 <zns> caleb_: you mean dependcies like ohai-solo?
15:37:35 <caleb_> zns: Yes. but only in the localhost scenario
15:37:47 <samstav> a python package is responsible for itself. if we obligate satori for managing system packages, thats a different ball game.
15:38:04 <caleb_> samstav: Agreed. Maybe its just something to address in the docs
15:38:31 <samstav> like gondoi said, it would just mean there are 2 additional steps
15:39:04 <zns> caleb_: the only weirdness is if I install satori==local and then try to do remote discovery and that fails. Probably with "You have installed local satori. Please re-install full" or something like that. Does that describe the oddness?
15:39:16 <caleb_> No
15:39:36 <caleb_> `sudo pip install satori==local`
15:39:40 <caleb_> satori localhost
15:39:51 <caleb_> (fails because you're not root, cannot install ohai-solo package)
15:40:15 <zns> That almost never happens (refer back to kudos to walker_ )
15:40:35 <caleb_> it will always fail if its executed as a user that cannot install a package
15:40:57 <caleb_> so you need to `sudo satori localhost` the first time to get the package installed
15:41:04 <caleb_> and then `satori localhost` from then on
15:41:10 <samstav> I have not implemented specifying a user when running localhost
15:41:11 <caleb_> unless a new version is rolled out
15:41:20 <caleb_> then it would error again
15:41:24 <samstav> so what caleb_ said is accurate. it is ran as the user running satori
15:41:27 <zns> caleb_: I see. Is the story here that an admin installs satori and then a user runs it and can't install ohai-solo?
15:41:34 <caleb_> zns: Yes.
15:41:41 <zns> Dumb admin?
15:41:53 <zns> jk.
15:42:15 <zns> So do we document tha for an admin?
15:42:27 <zns> Or do we build it in to the install?
15:43:12 <samstav> ins I'm not sure what you're asking
15:43:14 <zns> Wouldn't it be weird if pip install satori installed a ruby package?
15:43:16 <samstav> zns ^^
15:43:53 <samstav> It would absolutely be weird
15:43:56 <caleb_> Yes.
15:44:03 <caleb_> I think it just needs to be documented.
15:44:17 <caleb_> and maybe we shouldn't assume that we can always update the package
15:44:20 <zns> So then is the only answer to document it so an admin also installs ohai-solo?
15:44:30 <caleb_> +1
15:44:40 <zns> Or we include a basic localhost discovery module in python inside satori?
15:45:24 <samstav> zns come again?
15:46:30 <zns> If sudo pip install satori (local or full) should just wor when run by a user, then we should add data plane discovery in python in satori.
15:46:43 <zns> /s/wor/work
15:47:34 <zns> And therefore make ansible, and ohai-solo totally optional.
15:48:04 <samstav> Yes. Is that ambitious considering the work has been done with ohai-solo, and requires a minimal and sometimes unnoticeable barrier?
15:49:19 <zns> I don't feel it's about ambition. I hate to duplicate that work. But if we're doing this in openstack and therefore we need python and there isn't a python dependency we can take, then we're almost forced to do it...
15:50:09 <gusmaskowitz> as a future user of Satori I think it is completely acceptable to give me a choice of data-plane discovery methods. If only one exists for the first year and it's Ruby based then so what.
15:50:24 <samstav> It takes ambition to put time into things only because you are forced to
15:50:31 <gondoi> time check 10 mins remaining
15:50:37 <gusmaskowitz> there will be data-plane discovery plugins - correct ?
15:51:07 <gondoi> gusmaskowitz: as i see it, correct
15:51:17 <zns> And, before someone says it, devstructure blueprint has a lot of dependencies (like git), so it doesn't solve the problem. Although we could explore contributing to it to make it lighter and re-use its data plane discovery pieces (if they suffice). Then we add it to our requirements.txt and pip install satori will always work...
15:52:10 <samstav> k. for the record, I +1 the idea of a "pure python" data plane discovery and satori working out of the box
15:52:20 <zns> gusmaskowitz: yes, but then you know separate setup is needed for the plugins.
15:52:43 <zns> So you wouldn't expect pip install satori to comes with plugins.
15:52:47 <gusmaskowitz> I also +1 a pure python approach at install time for the record.
15:52:56 <gusmaskowitz> we are short in time. perhaps this is a mailing list discussion
15:53:09 <gondoi> i agree with samstav that a pure python solution will be nice to have
15:53:24 <zns> caleb_?
15:53:52 <caleb_> I'm onboard. Good docs will get plugins setup
15:53:58 <caleb_> thats an action though
15:53:58 <zns> gusmaskowitz: yeah. We can take it to the ML.
15:54:17 <zns> To start built-in python data plane discovery?
15:54:44 <zns> #action zns start ML discussion & blueprint for built-in python data plane discovery
15:55:24 <zns> For the record, I don't love the idea of duplicating the work that exists in ohai.
15:55:59 <samstav> I also *really* don't love it.
15:56:17 <samstav> I just waaant it.
15:56:18 <caleb_> I dont think its the most pressing issue ATM
15:56:23 <zns> But if satori can become a light, awesome discovery module that maybe ansible and other python tools use then satori is just the python versions of ohai.
15:56:52 <zns> OK. To be continued on the ML.
15:57:02 <zns> Anything else?
15:57:07 <gondoi> tests are broken
15:57:15 <gondoi> pythonwhois is failing to install from git
15:57:19 <gondoi> i am willing to take that action item
15:57:30 <zns> caleb_: maybe for Rackspace, but for satori as a project, I'd say this is critical.
15:58:05 <zns> #action gondoi Fix pythonwhois is failing to install from git
15:58:07 <zns> Thanks
15:58:08 <caleb_> I think we should add a new requirement to user list announcement: Developer docs for doing discovery using Python rather than the command line.
15:58:26 <samstav> caleb_ agreed
15:58:40 <caleb_> I can take that as an action
15:58:52 <zns> #action caleb_ Developer docs for doing discovery using Python rather than the command line.
15:59:09 <zns> caleb_: I'm not sure what that means, so we can sync up offline.
15:59:34 <zns> Lots of discussion. Did I miss any action items for anyone?
15:59:57 <zns> Going once....
16:00:05 <zns> Gone!
16:00:09 <zns> OK. Thanks, everyone!
16:00:13 <zns> #endmeeting