20:00:04 #startmeeting heat 20:00:05 Meeting started Wed May 22 20:00:04 2013 UTC. The chair is shardy. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00:06 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:00:08 The meeting name has been set to 'heat' 20:00:13 #topic rollcall 20:00:23 shardy here 20:00:25 here (mostly) 20:00:34 o/ 20:00:37 I'm awake 20:00:43 here 20:00:47 hello 20:00:53 Travis Tripp here 20:00:55 o/ 20:01:01 hi 20:01:28 o/ 20:01:35 hi all 20:01:41 #topic Review last week's actions 20:01:49 zaneb awake/alive :) 20:02:08 #info stevebaker stevebaker to send ML email re backwards-compatibility 20:02:08 sdake_: I'm awake 20:02:13 hi! 20:02:18 hi 20:02:18 hi! 20:02:21 o/ 20:02:24 I saw this sent today 20:02:35 sent, haven't checked for replies yet 20:02:42 hi 20:02:42 please add replies if you have strong opinions on the subject :) 20:02:46 stevebaker: I wonder if you might want to send it to the general openstack ML 20:02:58 there was one reply stating it will be possible in future versions of openstack to use past versions 20:02:58 stevebaker: as that one is full of users, rathe than developers 20:03:03 #info shardy PoC HOT patch 20:03:10 true 20:03:13 So I started looking at this, but not posted anything yet 20:03:31 #action shardy PoC HOT patch 20:03:46 was waiting for the converged template, which brings me to.. 20:03:57 #info randallburt, tspatzier provide converged DSL/HOT example 20:04:06 so, we discussed this a bit 20:04:08 * SpamapS starts drum roll 20:04:14 lol 20:04:33 and I think based on our conversations today, we may want to change tacks and go the zane route of a couple new bps for some specifics 20:04:52 rather than try to get an entire format agreed upon beforehand 20:05:02 randallburt: sounds good, and I'd really like us to start making some progress on the Provides BPs soon 20:05:04 +1 million 20:05:16 Providers that is 20:05:24 shardy: should have some folks starting on that in the next week or so 20:05:39 randallburt: Do you have resources to look at those, the current assignee has been fairly quiet ;) 20:05:40 its all about the journey, not the destination ;) 20:05:40 actually, next week 20:05:40 randallburt: not sure this depends on zane's proposal. We should be able to agree on a hello world kind of template without that 20:05:46 shardy: yes 20:05:55 he will be less quiet soon 20:06:02 randallburt: Ok, great, sounds good 20:06:10 Would be good to have this PoC HOT patch to be able to start some coding 20:06:27 shardy I'll back randallburt up.. will get the resources going asap.. we are rolling off or previous comittments. 20:06:37 s/or/our 20:06:46 So, should we still be commenting on the template alternatives posted or will there be a new place? 20:06:49 based on some discussions today tspatzier, I think the new bps will help there 20:06:57 starting with Providers, then maybe Environments next 20:06:58 tspatzier: OK, I'll still post a PoC draft patch, which we can discuss, figure out where the additional template translation may work 20:07:14 can you summarize the new BPs? 20:07:17 we already do something somewhat similar with the YAML/CFN reformatting 20:07:22 and when we might see them? 20:07:27 TravT: I'll do one more review to gerrit and we can work from bps from there 20:07:38 #link https://blueprints.launchpad.net/heat/+spec/provider-resource 20:07:38 TravT: comments would still be welcome 20:07:42 so bps are fine, but there is at least a template that is the goal to deploy, right? 20:07:52 shardy: will do another push after meeting 20:08:01 SpamapS: that was my thinking as well 20:08:03 like, we've settled on one to be 0.1 of the examples? 20:08:20 tspatzier: Ok. Just got back from 3 weeks travel and found them this morning. 20:08:25 SpamapS: I personally would prefer initially to focus on adding the missing stuff to the internal model 20:08:32 then polish the syntax later 20:08:39 shardy: 100% agree 20:08:51 shardy: agreed, afraid we'd get stuck otherwise 20:08:52 yea we need provider + env 20:09:05 shardy: I suggest that missing stuff be exercised by a "good enough" template that can be improved on as the holes are found. 20:09:14 SpamapS: let's be clear here, design is not only bottom-up or top-down, both must happen _simultaneously_ 20:09:26 yeah totally 20:09:44 Like we know there are things that will need to be done in the core to support things that are not yet done in the interface (templates). 20:09:56 zaneb: yep, but we could test the new bottom-up features via a superset CFN syntax 20:10:11 we don't actually *need* a whole new DSL syntax to test the functionality 20:10:30 although I know many people are interested in working towards having one 20:10:30 +1 for a superset CFN until there is at least a feature-parity-capable HOT template format settled on. 20:10:43 +1000 for a superset syntax 20:10:55 zaneb: careful, you'll run out of +1's 20:11:01 shardy: agreed. the more I work with it, the more I'm convinced we'll get there eventually 20:11:06 Ok, sounds like we have agreement, hooray! ;) 20:11:09 * mordred hands zaneb a bucket of +1's 20:11:12 then you become that guy who has to -1 everything just to get back to even. :) 20:11:15 shardy: agree, but having a picture of how the new features would surface in the DSL would help understanding if we can parse the new format easily in the future and if it looks intuitive 20:11:18 zaneb: can use some of mine 20:11:43 tspatzier: sure, that discussion should continue, and is valuable 20:11:49 tspatzier: exactly 20:12:05 I just think we should start work on the bottom-up stuff now, before we start running out of time.. :) 20:12:09 we need the code and the template syntax to co-evolve with feedback from each other 20:12:11 h1 is only a week away.. 20:12:17 which brings me to the next topic.. 20:12:32 Yeah, land what you can when you can. Don't build up all the delta in a massive "early in I cycle" patch bonanza 20:12:50 just looking at the template syntax won't work; just looking at the code will create a mess 20:13:06 Yep, incremental steps towards the end goal 20:13:18 #topic h1 bugs/blueprints 20:13:44 #link https://launchpad.net/heat/+milestone/havana-1 20:13:55 I'll be focusing on my quantum/vpc bugs 20:14:08 Anyone got anything they don't think will land in time? 20:14:14 h1 is only a week away 20:14:23 afraid so, will adjust everything at the end of this week 20:15:06 jpeeler: no problem, just bump any to h2 which won't get completed 20:15:09 shardy have some infra patches that haven't passed review yet 20:15:18 but tested on devstack seem to work 20:15:35 parallel launch is dependent on folks reviewing the patches 20:15:42 seems like h1 is looking pretty good really. 20:16:05 sdake_: sounds good - do you need reviews from us? 20:16:20 zaneb: Ok, how many more patches in the queue? 20:16:31 i had to rebase one because of pbr, I may need to rebase others 20:16:32 about 20 20:17:23 zaneb: Ok, well lets see if we can shorten the review cycle a bit over the next few days 20:17:32 #action everyone to do reviews ;) 20:17:46 there's 4 up there now if there are any takers :) 20:17:48 zaneb are they in gerrit, I don't see 20 patches from you 20:17:48 moar +1's and +2's 20:18:14 sdake_: no, I haven't submitted the whole series because rebases make it too difficult to follow 20:18:24 zaneb makes sense 20:18:52 Ok, one other thing re h1 20:19:16 Can everyone do a bit of testing next week, try to make sure we have a milestone release which works? 20:20:00 we may be close to having some heat tempest gating 20:20:34 well done 20:20:52 stevebaker: sounds great! Still be good for us all to do some basic smoke tests tho (ie not just unit tests) 20:21:16 yup 20:21:27 anything else on h1 or bugs/bps in general? 20:21:42 should get to ceilometer alarms in h2 20:21:57 asalkeld: awesome :) 20:22:16 alarming functionality nearly all in ceilometer (eglynn just doing the last bit) 20:22:43 asalkeld: how hard is the migration going to be from the heat perspective? 20:22:54 Seems like we'll have to rip out a fair amout of stuff 20:23:05 (or make optional rather) 20:23:06 well thinking on having both in place 20:23:13 +1 20:23:22 * SpamapS always tests tip, so far things look great. :) 20:23:27 +1 20:23:27 for people that already have deployments 20:23:33 asalkeld: for how long? 20:23:43 1 release? 20:23:46 though I don't test watches or autoscaling :p 20:23:59 asalkeld: i can help with the ceilometer rework 20:24:00 thinking of ppetit 20:24:19 thanks m4dcoder 20:24:20 asalkeld: I'm thinking of how we do the engine-scaling stuff 20:24:25 SpamapS: do you regularly test stack update? 20:24:38 having to support periodic tasks as well makes it far more difficult 20:24:40 zaneb, we will get a web hook 20:25:04 thinking of still inserting a watch as a placeholder 20:25:05 I can help with Ceilometer work as well 20:25:05 asalkeld: that's fine for ceilometer, but for supporting the old way? 20:25:22 yea, got to keep it there 20:25:31 at least optionally 20:25:43 asalkeld: actually zaneb and I discussed the engine-scale-out problem and realized that it gets much simplified without all the engine periodic tasks 20:25:46 I'll tread carefully 20:26:02 shardy, yea it does 20:26:10 asalkeld: or we could say "no scaling out of heat-engine if you use the old autoscaling" 20:26:18 yip 20:26:19 that works for me 20:26:28 zaneb: that sounds like a pretty reasonable compromise 20:26:41 won't be pretty, but it is reasonable :D 20:26:48 ~. 20:27:02 yea, don't want to make users pissed off 20:27:04 shardy: sorry, coffee shop wifi booted me off 20:27:27 shardy: I do test update quite a bit, but only updating metadata and occasionally adding new instance resources 20:28:05 SpamapS: Ok, cool - I'm just thinking of test exposure for my now-quite-bug update interface refactor/cleanup 20:28:25 Anything else or shall we move on? 20:28:31 all good 20:28:34 shardy: hopefully stevebaker's tempest stuff will include updatestack? 20:28:49 * SpamapS still thinks humans are the worst choice for testing things. 20:28:51 SpamapS: it will when you write one :P 20:29:09 hah 20:29:14 SpamapS: s/testing/checking/ 20:29:19 stevebaker: sweet, I actually really want to flesh out that testing. 20:29:42 I guess the point is really asking people who actually *use* the features if they're happy that they still work 20:29:49 Oh, BTW, if you want to test Heat on Rackspace hardware, we have $500/mo accounts for you:; iopenedthecloud.com 20:29:54 but yep, automated tests ftw 20:30:02 Unrelated to topic, please excuse. 20:30:05 thanks fsargent 20:30:11 already applied 20:30:19 fsargent: thanks, saw that link earlier 20:30:37 #topic Removal of tools/openstack scripts 20:30:51 So I've raised a bug to remove tools/openstack* 20:30:54 what is promting this? 20:31:03 \o/ 20:31:08 I thought you guys used it 20:31:23 I suspect what is prompting that is the untenable situation it presents. 20:31:31 its a maintenance burden 20:31:33 wont be using it for long because of lib changes 20:31:36 Heat doesn't need to maintain a "deploy openstack" script. :-P 20:31:43 asalkeld: the reason is we've got lots of users coming in asking for RDO help (having installed via packstack), and their configs are broken because tools/openstack does things slightly differently 20:31:56 So why not just use packstack 20:31:57 I find it tools/openstack unusable already on F17 20:32:01 is my suggestion 20:32:04 I'd love to see us maintain a VM image people can boot and try Heat on 20:32:07 ya 20:32:08 * SpamapS points at diskimage-builder ... 20:32:51 I actually have used the script quite a lot, but it seems like time to kill it IMO (now we have packstack and RDO) 20:32:52 is there an ubuntu version of packstack? 20:32:59 that was my next question - 20:33:00 juju ;) 20:33:11 packstack would likely run on ubuntu 20:33:12 (that people use) 20:33:13 what's "the" way to do all-in-one installs on Ubuntu? 20:33:35 apt-get install all-the-things 20:33:36 crickets 20:33:52 can we just link to a procedure someone else publishes? 20:34:05 Thats what I would recommend. 20:34:21 or devstack, which we cover already 20:34:29 SpamapS: I'm not an ubuntu user, can you suggest such a link? 20:35:01 stevebaker: the idea is instructions for devstack, RDO/packstack, and $ubuntu-packaged-stuff 20:35:14 although we're not packaged for ubuntu yet are we? 20:35:18 shardy: no, because I use devstack or diskimage-builder's boot-stack 20:35:30 which both use git 20:35:38 shardy: the documentation tells how to install openstack on ubuntu 20:35:39 and are probably too crazy for most people. :) 20:35:55 shardy: the debian packages likely work fine on Ubuntu 20:36:18 in fact, heat is in saucy 20:36:19 https://launchpad.net/ubuntu/+source/heat 20:36:25 #link http://docs.openstack.org/grizzly/basic-install/apt/content/ 20:37:02 I can practically hear people googling;) 20:37:06 shardy ^ 20:37:11 actually I have a bookmark :) 20:37:15 sdake_: Ok, I was thinking of a simple, scripted, beginner friendly script or tool 20:37:42 shardy, we don't have to find it now 20:38:20 asalkeld: Ok, I just wanted to clarify if such a thing exists before deleting tools/openstack_ubuntu 20:38:29 shardy: there is no such tool unless the deb/ubuntu community writes one 20:38:47 it's fair to say that people have been successfully installing openstack on Ubuntu for quite some time without our help 20:39:07 Ok, well are we agreed we should remove the scripts and update the docs then? 20:39:18 sure 20:39:23 zaneb: I get the impression almost all of our ubuntu users are on devstack 20:39:24 Right, the packages are very good and do a lot of setup for you. 20:39:59 #topic Open discussion 20:40:14 Anyone have anything to discuss? 20:40:20 infrastructure are wanting to use heat againt rackspace and hpcloud 20:40:20 this provider/environment idea is going to really help to abstract the 20:40:21 differences in API/identity 20:40:31 clarkb, ... 20:40:34 o/ 20:40:50 tell us your wows 20:41:03 woes? 20:41:16 yip that one 20:41:16 those too 20:41:52 there are a couple places that I think we want to start using Heat. For managing our pool of devstack-gate slaves and to manage an elasticsearch cluster. I think mordred would like to see us use heat for all the things but these are some concrete goals we can focus on 20:42:23 this requires running VMs in rackspace and hpcloud and they provide some restrictions that are fun to work with 20:42:49 neither provide a way for self service VM image uploads so we can only boot the images they provide or snapshots that we create 20:42:51 Rackspace also wants to run Heat on Rackspace public cloud… we will be having to work through some of those fun challenges ourselves. 20:43:00 this might make a nice concrete (and productive) goal for the provider stuff 20:43:17 for one thing, we can develop alternative middleware pipelines for whatever auth weirdness 20:43:34 both also have funny keystone. Rackspace has special auth stuff iirc and hpcloud returns extra data 20:43:36 asalkeld: you mean envionments? 20:43:44 well both 20:43:53 so basically, you'd have a template resource for doing things one way when you run heat in Rackspace, but a different resource template for running heat at HP? 20:43:53 env + provider 20:43:56 and rackspace does not use cloud-init 20:44:14 asalkeld: ah, k, cool 20:44:18 fwiw I am not against running two different heats if I need to 20:44:18 short term it should be possible to hard-configure a heat to 1 external cloud 20:44:25 one to talk to rackspace and one to talk to hpcloud 20:44:40 this may end up being better for sanity 20:44:46 Yeah, keith_bray, we were discussing this yesterday evening on #heat. Hence fsargent offering the hookup 20:45:18 have you guys done anything on this yet? 20:45:27 yeah, its fun building an image to just install cloud-init only to snapshot it for use on other things 20:45:33 Yeah... 20:45:40 I'll ask people about cloud-init 20:45:44 I'm not familiar with it personally 20:45:55 i know there's been talk about it, but not sure where things are or if they are atm 20:46:04 OK. What are alternatives to cloud-init? 20:46:04 can we run with out cloud-init? 20:46:05 fsargent, let's link up efforts. 20:46:08 * stevebaker has to go. kthxbye 20:46:09 chef 20:46:10 We could add an instance type that pushes heat's stuff in via SSH 20:46:15 puppet, salt, etc 20:46:21 asalkeld: not really 20:46:24 oh keith_bray you're a racker, got it. 20:46:29 since I assume the SSH key is somehow magically injected to allow users to get into their instance 20:46:41 we could support a mode where cfn-init reads metadata via the API instead of userdata 20:46:52 but then you need something to run cfn-init 20:47:01 uh, i don't think we have metadata running either 20:47:01 but it seems like not having cloud-init is probably a fairly temporary limitation of Rackspace's cloud 20:47:03 SpamapS: you would think that but I think we do an initial login with passwords (that doesn't mean ssh with keys isn't possible just not happening today) 20:47:12 (though I thought that last summer when it went public...) 20:47:28 clarkb: key, passwords, w'ever ;) 20:47:32 you can inject keys, though 20:47:42 as files or personality 20:47:51 point being, Heat isn't totally helpless here 20:48:17 yeah, assuming the config is normalized…. I suspect that cloud-init (or something akin to that) is the 'better' option. Better is, of course, the enemy of 'done'. 20:48:59 All Heat wants to do after asking nova to boot a server is feed in some JSON to initialize its tools 20:49:25 and a starting script 20:49:26 clarkb: remind me where you are? 20:49:29 So, in theory, you can have a custom resource plugin that ssh's in and puts the metadata where you want it, and runs userdata. 20:49:29 atm heat requires cloud-init and nova ec2 metadata to work properly, but we could support some other mode 20:49:38 so, couldn't you spin a server and install cloud-init and cfn-tools then snapshot it and use that for testing from then on? 20:49:43 fsargent: I am at HP but spend all my time hacking on the openstack ci/infra stuff 20:49:47 Ok 20:49:50 oh yeah, metadata. 20:49:59 but spending any time on this is a waste IMO, since RS should just follow the entire rest of the industry and support cloud-init. 20:50:03 the question is, do we really need to? 20:50:10 so how does rackspace handle userdata passed to the nova API at the momemt? 20:50:23 +2 to RS growing cloud-init 20:50:26 SpamapS: We'll do what we can. 20:50:33 tend to agree we should avoid special casing the codebase 20:50:34 SpamapS RS is working on it.. your point is valid. 20:50:38 #link http://www.openlogic.com/wazi/bid/188106/Bootstrapping-an-Ubuntu-Server-on-Rackspace-Using-Cloud-Init-and-Fog 20:50:47 read that :) 20:50:52 Not the ideal way to do it. 20:51:03 not a lot of alternatives, though 20:51:12 how does that solve the meta data server problem? 20:51:31 it doesn't. you have to use local "no-cloud" data injected via personality 20:51:44 any ways I think we really need to support infra - they would be an awesome user 20:51:58 agreed 20:51:58 asalkeld: obviously I agree but I am also biased :) 20:52:03 :) 20:52:29 and it is highlighting real problems we need to solve 20:52:45 agreed, we just need to understand exactly what changes (if any) are needed 20:53:00 shardy, we need a wiki/bp 20:53:04 but if we special case for rs, and hp, and all the other vendors, code base becomes hard to maintain 20:53:13 and a plan and people 20:53:20 sdake_: yeah. I am not sure I would want to special case to do this 20:53:26 asalkeld: care to take an action? ;) 20:53:34 sure 20:53:49 #action asalkeld wiki/bp re infra heat usage 20:53:53 sdake_: I am thinking that if a few things like being able to boot snapshots and some way around cloud-init were possible that would be sufficient 20:53:57 sdake_ hopeing for no special casing 20:54:11 clarkb willing to help with snapshot, think that is a fine feature 20:54:29 not sure how to get around cloud init - maybe a plugin for bootstrapping 20:54:53 that leaves us with figuring out keystone, but it sounds like we can have middleware that handles this 20:54:59 are we saying the images can't have cloud-init, or that the data can't be passed via user/metadata? 20:55:14 there is userdata 20:55:19 shardy: can't be passed via metadata 20:55:24 atm images dont have cloud init and can't pass via metadata server 20:55:42 booting snapshots allows us to add cloud init 20:55:44 (only boot time) 20:55:46 sdake_: but we could use some other cloud-init datasource? 20:56:11 boot time is how we do it now, so fine 20:56:12 shardy perhaps a plugin that the cloud provider could provide - just speculation there tho 20:56:14 ie it doesn't *have* to be Ec2DataSource, we could make that configurable via cloud-config? 20:56:37 Ok, I guess we need to better understand the limitations and investigate 20:56:46 so it has personalities 20:56:46 shardy I think the way rs cloud works is you have to pass the data prior to boot 20:57:01 can't we have a /etc/rc.init 20:57:10 that installs cloud-init 20:57:17 and runs it manually 20:57:19 ? 20:57:45 technical details... 20:57:59 Yep, we should follow up via bp/wiki I guess 20:58:00 asalkeld: not portably 20:58:01 * SpamapS drops off early 20:58:08 nearly out of time 20:58:10 asalkeld: should work i think 20:58:18 would be interesting to try anyway 20:58:53 Agreed. If there's BP/wiki, that should make it easier for some of us rackers to prod the right people. 20:59:10 keith_bray: ^^ 20:59:12 Ok, times up, thanks all! 20:59:17 #endmeeting