15:00:25 #startmeeting satori 15:00:25 Meeting started Mon Mar 24 15:00:25 2014 UTC and is due to finish in 60 minutes. The chair is zns. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:26 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:29 The meeting name has been set to 'satori' 15:01:04 Hi - who's here for the Satori team meeting? 15:01:06 o/ 15:01:11 o/ 15:01:12 o/ 15:01:20 o/ 15:01:45 o/ 15:02:15 Welcome! Thank you for joining. We don't have new items on the agenda. We'll review action items and if anyone has new topics, please have them ready for after that. 15:02:22 #topic action items 15:02:41 I assume the beer purchases are scheduled for Atlanta. 15:03:00 :) 15:03:06 ha 15:03:16 gondoi: nova client requirements - I saw the pull request. All done? Anything else on that? 15:03:46 zns: can we come back to that one? gondoi lied about being present 15:04:10 walker_: OK. I'll assume it is closed given his patch. 15:04:16 hi 15:04:16 samstav: implement localhost blueprint - done, right? 15:04:23 HI adrian_otto 15:04:30 Indeed. Though the gates are cracking the whip 15:04:41 done 15:04:44 OK. I'll work on that with you today. 15:04:48 thanks, gondoi 15:04:59 samstav: plugin install options? 15:05:42 file bug for satori not working with ip address - done 15:06:00 reporting, while samstav types. 15:06:26 There were some discussions floating around this week. Some regarding subclassing setuptools, or something completely different: using the package's "version" to indicate which satori package to pull down 15:06:37 implement plugins own establishing the connection to start with, but we provide a built-in ssh module that they can use - partially done. I'm closing the action item and letting the blueprint continue tracking this. We've done a lot of work on this with ohai-solo and the localhost feature. 15:07:06 samstav: Is there a blueprint for that? 15:07:47 When I say subclassing setuptools, I really mean adding a Command or Feature (these are classes in setuptools) which could be passed to `python setup.py install` directly, OR through pip via `pip install ____ --install-options="--our-option-here'" ` 15:08:09 zns: which bug ID is the IP bug? 15:08:15 I think there's a topic for discussion on this one. WRT dependencies, setup, per, etc... 15:08:40 bug 1293670 15:08:40 There is no blueprint for that yet zns. That could be an action item for me this week. 15:08:48 https://bugs.launchpad.net/satori/+bug/1293670 15:09:25 Would that fall under https://blueprints.launchpad.net/satori/+spec/satori-minimal? 15:09:31 s/could be passed to/could be targeted/ 15:09:39 That's where we're talking about managing dependencies. 15:09:44 zns: if that hasn't been worked on, it may be a duplicate of https://bugs.launchpad.net/satori/+bug/1295391 15:09:55 let's table it for a second, sorry 15:10:30 gondoi: looks like a dupe, yes. 15:10:53 #action ins address possible duplication between 1295391 and 1293670 15:11:03 Damned autocorrect.... 15:11:10 #action zns address possible duplication between 1295391 and 1293670 15:11:16 ACK blueprint satori-minimal, though the brainstorming is leaning towards a generic solution for indicating the "type" or "edition" of the satori package 15:11:25 i have code submitted for it, we can review 15:11:38 cool 15:11:42 #topic dependencies and sertup 15:11:52 #topic dependencies and setup 15:12:16 We have a few possible blueprints around dependencies. Let me describe the issues... 15:13:11 Running pip install satori and then satori localhost does not need dependencies on paramiko or nova client. Those are heavy dependencies that cause fragility. 15:13:54 if a sysinfo plugin is needed for that use case, is it really all that minimal? 15:14:16 For example, paramiko requires pycrypto which requires python-dev packages and compilation that sometimes can fail. If we want pip install satori to be as reliable as possible for a localhost install, then these dependencies are problematic. 15:14:19 I'd argue that the ohai-solo package is much more heavy than paramiko and nova client. 15:14:40 ^ I retract my paramiko comment 15:15:17 But the ohai-solo plugin in satori is just python class, no external dependencies. And the package is self-contained, so installing it almost never fails. 15:15:49 I'm with you. 15:15:56 The ohai-solo one, that is. Credit to walker_ - that thing is solid and easy to install. 15:16:12 except on Friday. :D 15:16:28 What happened Friday? 15:16:34 .....um, nothing.... 15:16:49 Breakdown in the CI/CD pipeline for the packages 15:16:56 but, it is all better now 15:18:01 I would say that we should make installing things for the most common use case as easy as possible 15:18:28 If localhost discovery is the most common use case, I must have missed something 15:18:32 walker_: did that break any installs? 15:18:47 On hosts.. 15:18:55 If localhost is easy to install, but the most common usage requres a bunch of flags, etc. Then we are not doing what we should be doing 15:19:03 zns: no 15:19:15 walker_ bingo 15:19:26 walker_: ok. So still reliable! Even on Fridays :-) 15:19:29 So the question before us is how do we support a minimal install for localhost discovery, and also support a full install with ssh and control plane install. One idea was to have a separate package in pypi. That could show up as a different name (ex. satori-full) or a specially tagged version (pip install satori==full) 15:20:01 sooooo.. someone is probably going to shoot me.. but i would say ansible 15:20:13 everyone take a drink 15:20:38 gondoi: Go on. 15:20:38 🔫 15:20:43 And which one is the default. Minimal? And then from within satori we can install other packages. Ex. 'satori list packages' and 'satori install full' 15:20:44 actually.. that may not be true... i need to review it's full dependency tree 15:20:44 don't shoot guns. don't be violent. 15:20:44 but 15:20:56 you can pip install ansible and run localhost discovery 15:21:16 samstav: The Australian version 15:21:19 no need for worry about external packaging and installations 15:21:24 gondoi: so... are you saying we should abandon satori and just use ansible? 15:21:35 i have a WIP going that's not working yet 15:21:43 haha......... no 15:21:46 zns: I feel ya - and understand that localhost should be minimal; however, I'm not sure we understand the most common usage yet. Right now, localhost is not a common use-case at all 15:22:04 https://blueprints.launchpad.net/satori/+spec/ansible-when-present 15:22:29 that could change to just ansible as a discovery option 15:22:35 Maybe the full, minimal, decaf, no-fat options should come once we have a better understanding of popular usage 15:22:44 right now the discovery data returned just needs to be translated to our schema 15:23:18 also, ohai stuff is working now, it just feels "heavy" in terms of dependencies 15:23:22 also +1 to gondoi's Ansible stuff - that could result in a much lighter weight solution (although it requres paramiko doesn't it? /troll) 15:23:35 We have received a request for localhost. So maybe we just publish a minimal version for localhost and assume the default use case is not on local - and therefore should wrk with all the things (nova client, paramkio) installed? 15:23:43 that's why i said i need to review it's dependencies.. i'm sure it probably does 15:23:49 satori-solo ? 15:23:53 boooo 15:24:27 gondoi: I think the ansible blueprint doesn't change the discussion. We would still need to decide if we should have a minimal dependency version of satori. 15:24:28 I like full to be the default and minimal to be the reduced fat soy sugar free with 2 pumps version 15:24:41 And doesn't ansible depend in ssh? Does it use paramiko? 15:25:24 zns: gondoi is unsure if it requires paramiko 15:25:29 but it does use SSH 15:25:42 +1 to full being the default. 15:25:53 +1 as well 15:26:01 looks like the answer is yes it does https://github.com/ansible/ansible/search?q=paramiko&ref=cmdform 15:26:23 it uses it's own ssh, it would not use the ssh module satori has 15:26:42 i think this is getting distracting from our current topic 15:26:46 yeah 15:26:58 gondoi: even if we use ansible, we would still need to talk about nova client as dependency for control plane discovery, right? Or does ansible do control plane discovery? And if it does, does it not use nova client? 15:27:02 zns: I'm afraid multiple packages is going to be confusing 15:27:18 ansible would be data plane 15:27:24 Any objections to the full being the default? 15:27:44 ansible's ssh module: https://github.com/ansible/ansible/blob/c9fcbf7bdd46b493a1349662f92be63b66412007/lib/ansible/runner/connection_plugins/ssh.py 15:27:59 can we solve for a single step install process with full? 15:28:03 #agreed the full version (with dependencies) will be the default install of satori. 15:28:04 pip intall satori 15:28:27 right now it's potentially 3 step correct? depending on the OS 15:28:40 For localhost use cases, what does everyone think of having a package in pypi tagged with local. `pip install satori==local` 15:28:47 It doesn't implement the ssh protocol, it just opens pipes through Popen 15:29:04 zns: I'm cool with that. 15:29:07 zns: That seems sane 15:29:22 what impact does that have with publishing packages? 15:29:27 gondoi: you mean also install ansible and/or ohai-solo ashen running pip install satori? 15:29:50 #agreed for localhost use cases, we will have a package in pypi tagged with local. `pip install satori==local` 15:30:22 do we know how we're going to get ohai-solo installed in that scenario? 15:30:29 no i mean apt-get install python-dev 15:30:31 pip install satori 15:30:43 or even apt-get install build-essentials 15:31:05 I don't know how that is handled in python, usually... 15:31:32 Is this a packaging question? Would that be apt-get install satori? 15:32:10 If I would have know this was going to be a packaging discussion I would have been sick today :) 15:32:34 well i saw some chatter about needing to install python-dev on 12.04 ubuntu, for example 15:32:51 caleb_: What's the benefit of installing ohai-solo with satori. The current method of downloading it when needed is working fine. It's actually not causing us any problems (that I've seen so far). 15:33:12 zns: I'm sorry, you're right. 15:34:43 I thin we're confusing the data plane components with the dependencies in satori that are needed for remote data and control plane discovery. We don't have a problem with data plane plugins (ohai-solo or ansible). 15:35:15 I think installing deps on remote machiens as a part of discovery makes sense 15:35:37 it begins to feel odd when the deps for localhost aren't handled at install time 15:35:42 IMO 15:35:46 So, problem #1: unnecessary dependencies for the local use case. Solved with pip install satori==local 15:36:16 problem #2: pip install needing additional packages (ex. python-dev). 15:36:53 I think problem #2 is a packaging problem. 15:37:12 Problem #2 is very common 15:37:24 caleb_: you mean dependcies like ohai-solo? 15:37:35 zns: Yes. but only in the localhost scenario 15:37:47 a python package is responsible for itself. if we obligate satori for managing system packages, thats a different ball game. 15:38:04 samstav: Agreed. Maybe its just something to address in the docs 15:38:31 like gondoi said, it would just mean there are 2 additional steps 15:39:04 caleb_: the only weirdness is if I install satori==local and then try to do remote discovery and that fails. Probably with "You have installed local satori. Please re-install full" or something like that. Does that describe the oddness? 15:39:16 No 15:39:36 `sudo pip install satori==local` 15:39:40 satori localhost 15:39:51 (fails because you're not root, cannot install ohai-solo package) 15:40:15 That almost never happens (refer back to kudos to walker_ ) 15:40:35 it will always fail if its executed as a user that cannot install a package 15:40:57 so you need to `sudo satori localhost` the first time to get the package installed 15:41:04 and then `satori localhost` from then on 15:41:10 I have not implemented specifying a user when running localhost 15:41:11 unless a new version is rolled out 15:41:20 then it would error again 15:41:24 so what caleb_ said is accurate. it is ran as the user running satori 15:41:27 caleb_: I see. Is the story here that an admin installs satori and then a user runs it and can't install ohai-solo? 15:41:34 zns: Yes. 15:41:41 Dumb admin? 15:41:53 jk. 15:42:15 So do we document tha for an admin? 15:42:27 Or do we build it in to the install? 15:43:12 ins I'm not sure what you're asking 15:43:14 Wouldn't it be weird if pip install satori installed a ruby package? 15:43:16 zns ^^ 15:43:53 It would absolutely be weird 15:43:56 Yes. 15:44:03 I think it just needs to be documented. 15:44:17 and maybe we shouldn't assume that we can always update the package 15:44:20 So then is the only answer to document it so an admin also installs ohai-solo? 15:44:30 +1 15:44:40 Or we include a basic localhost discovery module in python inside satori? 15:45:24 zns come again? 15:46:30 If sudo pip install satori (local or full) should just wor when run by a user, then we should add data plane discovery in python in satori. 15:46:43 /s/wor/work 15:47:34 And therefore make ansible, and ohai-solo totally optional. 15:48:04 Yes. Is that ambitious considering the work has been done with ohai-solo, and requires a minimal and sometimes unnoticeable barrier? 15:49:19 I don't feel it's about ambition. I hate to duplicate that work. But if we're doing this in openstack and therefore we need python and there isn't a python dependency we can take, then we're almost forced to do it... 15:50:09 as a future user of Satori I think it is completely acceptable to give me a choice of data-plane discovery methods. If only one exists for the first year and it's Ruby based then so what. 15:50:24 It takes ambition to put time into things only because you are forced to 15:50:31 time check 10 mins remaining 15:50:37 there will be data-plane discovery plugins - correct ? 15:51:07 gusmaskowitz: as i see it, correct 15:51:17 And, before someone says it, devstructure blueprint has a lot of dependencies (like git), so it doesn't solve the problem. Although we could explore contributing to it to make it lighter and re-use its data plane discovery pieces (if they suffice). Then we add it to our requirements.txt and pip install satori will always work... 15:52:10 k. for the record, I +1 the idea of a "pure python" data plane discovery and satori working out of the box 15:52:20 gusmaskowitz: yes, but then you know separate setup is needed for the plugins. 15:52:43 So you wouldn't expect pip install satori to comes with plugins. 15:52:47 I also +1 a pure python approach at install time for the record. 15:52:56 we are short in time. perhaps this is a mailing list discussion 15:53:09 i agree with samstav that a pure python solution will be nice to have 15:53:24 caleb_? 15:53:52 I'm onboard. Good docs will get plugins setup 15:53:58 thats an action though 15:53:58 gusmaskowitz: yeah. We can take it to the ML. 15:54:17 To start built-in python data plane discovery? 15:54:44 #action zns start ML discussion & blueprint for built-in python data plane discovery 15:55:24 For the record, I don't love the idea of duplicating the work that exists in ohai. 15:55:59 I also *really* don't love it. 15:56:17 I just waaant it. 15:56:18 I dont think its the most pressing issue ATM 15:56:23 But if satori can become a light, awesome discovery module that maybe ansible and other python tools use then satori is just the python versions of ohai. 15:56:52 OK. To be continued on the ML. 15:57:02 Anything else? 15:57:07 tests are broken 15:57:15 pythonwhois is failing to install from git 15:57:19 i am willing to take that action item 15:57:30 caleb_: maybe for Rackspace, but for satori as a project, I'd say this is critical. 15:58:05 #action gondoi Fix pythonwhois is failing to install from git 15:58:07 Thanks 15:58:08 I think we should add a new requirement to user list announcement: Developer docs for doing discovery using Python rather than the command line. 15:58:26 caleb_ agreed 15:58:40 I can take that as an action 15:58:52 #action caleb_ Developer docs for doing discovery using Python rather than the command line. 15:59:09 caleb_: I'm not sure what that means, so we can sync up offline. 15:59:34 Lots of discussion. Did I miss any action items for anyone? 15:59:57 Going once.... 16:00:05 Gone! 16:00:09 OK. Thanks, everyone! 16:00:13 #endmeeting