20:00:04 #startmeeting heat 20:00:05 Meeting started Wed May 15 20:00:04 2013 UTC. The chair is shardy. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00:06 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:00:08 The meeting name has been set to 'heat' 20:00:14 #topic rollcall 20:00:23 holla 20:00:24 hello all 20:00:25 howdy 20:00:28 hi 20:00:28 hi all 20:00:30 shardy here 20:00:36 Hi! 20:00:37 o/ 20:00:38 o/ 20:00:50 hello 20:00:55 hey 20:01:17 asalkeld around? 20:01:44 #topic Review last week's actions 20:01:49 hi 20:01:56 sorry bit late 20:02:16 hey asalkeld 20:02:20 heya 20:02:22 #info stevebaker to send ML email re backwards-compatibility 20:02:31 stevebaker: did that happen? 20:02:36 It appears that I haven't written that email yet. Please add it as an action again 20:02:43 Ok, np 20:02:55 #action stevebaker stevebaker to send ML email re backwards-compatibility 20:03:08 #info all to look at dsl examples and provide feedback 20:03:30 that's been done some, but more review would be greatly appreciated. 20:03:47 So I know the DSL discussions are ongoing, but anyone got any specific comments or requests in that area? 20:03:55 yeah, I've not made any comments, but I've been looking at 'em and pondering 20:03:58 we also had a discussion today about merging some proposals and pushing hard on getting a "fina" single wordpress template sorted. 20:04:25 I've agreed to post a PoC patch showing how we could possibly do an engine level translation from the proposed HOT format to the internal model 20:04:38 maybe an action for me and Thomas to get that done soonest? 20:04:41 I'll post a draft/wip patch so we can all comment 20:04:47 hopefully by next week 20:05:01 #action shardy PoC HOT patch 20:05:10 shardy: I'm still green in that area, but let me know how/if I can help 20:05:42 and as randallburt said, we're pushing for a consensus on a basic template to test with 20:05:43 is there a reason we're trying to adapt our internal model to the template format and not the other way around? 20:06:17 #action randallburt, tspatzier provide converged DSL/HOT example 20:06:41 zaneb: because our current internal model lacks some capabilities that are desired 20:07:06 also it's based on aws 20:07:08 zaneb: and we hope it will be additive and (relatively) transparent to the CFN 20:07:11 zaneb: that's a good question - my understanding is we're trying to identify the gaps in the internal model 20:07:14 SpamapS: oh, I know that 20:07:24 randallburt: +1 20:07:50 *hope* ;) 20:07:57 I mostly see there just being some new internal resources and a few things abstracted so that both cfn and HOT can use them in slightly different ways. 20:07:58 ideally CFN can inject into HOT but HOT will be easier to use and more powerful 20:08:02 zaneb: I see the template-syntax thing as being a byproduct of the process, really it's a way for us to hack on the abstractions IMO 20:08:14 zaneb: I think the plan would not be to change the internal model radically for this first HOT patch, but let it evolve over time, basically to enrich it for new features as randallburt and SpamapS said 20:08:37 tspatzier: agree with that 20:08:50 Yep, and to figure out where/how we can do that in a low-risk way 20:09:05 just concerned that starting top-down with the format and then trying to squeeze that into an implementation is making life hard for ourselves 20:09:07 which is the point of us starting to look at how we can do that with some code 20:09:19 If there's something to spend time on now, it is making sure HOT is forward-flexible so whatever we support in the first release we can keep supporting for a long time. 20:09:19 composability and substitutability let people do cool things, and safe if we can do it incrementally which is the plan 20:09:29 zaneb: yeah, why we hope to put some code to it soonest 20:09:32 zaneb: I'm hoping when we start actually looking at code the process may reverse somewhat 20:09:52 ok 20:09:56 moving on 20:10:01 is there a place that describe this HOT format for new folks like me to catch up? 20:10:03 I think getting something concrete started comming from bottom up can help to shape the DSL instead of just doing conceptual work for too long 20:10:05 ie we'll figure out the gaps, add stuff, then work out a sane way to expose it via templates 20:10:28 #link https://review.openstack.org/28598 20:10:35 m4dcoder: ^^ 20:10:39 m4dcoder: not yet, but would be a good idea as we validate the simple templates before moving on to more complex stuffs 20:11:00 ok, lets move on shall we? 20:11:07 +q 20:11:10 #topic Reorganize resources into heat/engine/resources/OS, heat/engine/resources/AWS 20:11:12 or 1 rather 20:11:40 this came out of a irc chat spamaps and I had last week 20:11:54 Sounds like a good idea 20:12:08 is this the "Taking the AWS out of Heat"? 20:12:11 zaneb: will it be a problem for the plugin code? 20:12:12 Indeed 20:12:15 one issue with this 20:12:26 some of the code is shared 20:12:38 I think that is a great idea, but how to handle volume.py, which now has both AWS::EC2::Volume and native cinder resource type? 20:12:39 I was thinking there would also be a common? 20:12:41 e.g. CinderVolume inherits from Volume (the AWS one) 20:12:49 probably break them up 20:12:51 zaneb, I started working on changing that 20:13:15 Yeah, we should still have common base classes where it makes sense to, or modify the AWS resources to inherit and override stuff from the native ones 20:13:16 we could have a common base or something 20:13:20 Creating utility functions instead of inheritance 20:13:24 so spamaps and I had this discussion about inheritance as well 20:13:35 It adds a tiny bit of duplication that we can leave with I think 20:13:41 Is this bringing any value? 20:13:50 asalkeld: +1 exactly 20:13:54 and we both agreed we dislike that model :) which is why this was added to the agenda so people could vote on whether we really want inheritance in resources or not 20:14:00 Yes it brings the ability to turn off AWS 20:14:04 isn't there heaps of bp we could be working on? 20:14:05 which is desired by deployers 20:14:13 can we have pythonic package names? heat.engine.resources.aws.ec2.volume 20:14:21 stevebaker_: +1 20:14:23 SpamapS: how? by deleting all the python files? 20:14:24 filter by not 'AWS::' 20:14:35 asalkeld: yes, exactly 20:14:47 asalkeld if there are bps you want to work on work on them 20:14:50 Thats a fair point. 20:14:57 the way to turn off aws is to filter out all plugins that start with AWS:: at registration time 20:15:05 this came out of the original native server resource 20:15:29 so but it is not strictly neccessary 20:15:30 which lists inheritence as a goal 20:15:38 (move the code about) 20:16:05 is anybody against reorganizing python classes to match resource type names? 20:16:25 Hm 20:16:29 filter = 1 line in config file, vs. selectively installing modules 20:16:44 zaneb those are orthogonal 20:16:47 I think there are advantages long term (assuming we're aiming to have a full set of native resources) in having the AWS ones inherit/share implementation 20:16:59 zaneb: I didn't mean by selectively installing modules, I was more thinking of abstraction to make it clear where AWS specific things were happening. 20:17:03 zaneb: you could still install AWS resources, just not enable them 20:17:10 zaneb: but the filter is superior, so I like that better. 20:17:14 shardy: they could do that and still be in separate files though 20:17:28 clearly we want to filter, but also putting them in different places makes the code more sanitary 20:17:35 stevebaker_: Yes, agreed 20:17:49 I am -1 on inheriting from native resources being the way they work, but +1 for filtering. 20:17:51 I guess this is really part of the abstract-aws BP? 20:18:04 #link https://blueprints.launchpad.net/heat/+spec/abstract-aws 20:18:07 I'm not against better code organisation 20:18:27 but as a development priority... it isn't even on the list 20:18:29 i am -1 on inheriting too - inheritance evil 20:18:36 The point is simply to be able to document/use/deploy Heat w/o the AWS 20:18:38 anyone volunteering for the reorg? 20:18:42 zaneb it is in grizzly-2 20:18:50 abstract-aws doesn't mean "in the code abstract it" it means "in the project, abstract it" 20:18:51 zaneb: I agree, low priority, but if someone wants to contribute the patch, fine 20:18:52 i just added it because we had a discussion on irc about it 20:19:09 sdake: it isn't on *my* list ;) 20:19:16 zaneb: its on mine 20:19:30 and by "it" I mean making clear the line between AWS and Heat 20:19:31 one of the resource types was on mine as well 20:20:04 the "how" of that is not all that important to me. 20:20:23 but I appreciate you guys setting me on a path toward filter vs. reorg code base. :) 20:20:34 here is another way to frame the question - what is the harm of putting the aws stuff in one dir and the os stuff in another dir 20:21:05 sdake: no harm, but what is the benefit? My "it helps you turn AWS off" argument doesn't really work :p 20:21:08 sdake: I guess the question is more, is it worth the effort involved 20:21:21 shardy, +1 20:21:25 it helps devs understand what is what - readability issue 20:21:28 it makes it obvious where to find the code for a resource type 20:21:31 also code churn 20:21:31 otherwise the code is spread all over the place 20:21:40 which currently it is not (obvious) 20:21:59 sdake: I'm not against better code organisation 20:22:02 SpamapS: +1 20:22:48 sdake: Ok, cool, seems like nobody's against the reorg, just nobody's particulary keen to do it themselves ;) 20:22:48 in my mind its not about turning aws off and on, its about helping developers come up to speed on the code base and add more native resource types 20:23:11 makes sense 20:23:13 sdake: that's worth a lot 20:23:50 maybe this is something we do immediately after havana-1 (or another release) 20:23:54 i think spamaps _was_ keen :) 20:24:01 stevebaker_ +1 20:24:08 to reduce the disruption from code churn 20:24:12 Might still get motivated. 20:24:45 Ok, agree lets leave this until after h-1 and see if anyone wants to post a patch 20:25:02 can be under abstract-aws BP since we have separate BPs for each native resource? 20:25:04 shardy when is h1? 3 weeks? 20:25:09 two weeks 20:25:15 s/a patch/a lot of small patches/ please 20:25:43 May 30 - havana-1 20:25:44 So if anyone has anything they thing is/isn't going to land, please move the milestone target in LP :) 20:26:25 in particular jpeeler and stevebaker_ you have a lot of bugs so please bump those you think will slip 20:26:38 * shardy should've created an agenda item for this..oops 20:26:40 I'll look 20:26:52 Anyway, shall we move on? 20:26:53 shardy: is there a particular commit date so to speak? 20:27:38 jpeeler: well bear in mind reviews have been slow lately so I'd say anything not posted by the end of next week will probably slip, but we'll see how it goes I guess 20:29:08 #topic heat-templates target audience 20:29:35 So this was prompted by a discussion with pfreund in #heat 20:29:38 yes 20:30:17 wanted to get peoples ideas about the targe audience of heat-templates, ie should it be purely for heat examples, or should there be a (probably unsupported) "contrib" area or something 20:30:30 been meaning to reply to the thread... 20:30:35 I wrote an e-mail in openstack-dev list 20:30:37 ie to try to get a community of testers and template authors interested in heat 20:30:53 I think we should make it really obvious that the heat-templates repo exists.. 20:30:59 and tell all users about it.. 20:31:09 but stay out of defining policy for it just yet 20:31:09 One concern I have is the idea that we provide supported "production ready" templates, which I think we should not 20:31:31 but encouraging reuse and collaboration has to be good right? 20:31:33 My thoughts on this is that each top-level directory of heat-templates should have a readme describing what the target audience for those templates should be. for cfn/ the audience is something like "people exploring heat's capabilities" 20:31:34 shardy, Why not? 20:31:35 any thoughts? 20:31:37 shardy: right, I think we'll get to a place where having a collection of those is a good thing, but we shouldn't do it yet. 20:31:58 therve: because it would involve effort that we as a team cannot currently afford 20:32:05 production templates could have security implication responsibilites we don't want 20:32:11 You need a whole community of motivated users to make such a collection work. 20:32:23 jpeeler: agreed 20:32:32 shardy: I disagree a bit in that I don't think its unreasonable to expect that the templates we use for validating heat will deploy on a stock install of the service 20:32:34 shardy, OK, it's not a disagreement, just a resource issue 20:32:58 SpamapS: yep, I guess the question I'm asking is is heat-templates the place for that collection? 20:33:21 randallburt: the "test/demo/validation" templates would absolutely be expected to work 20:33:26 yea, so don't discourage production templates, just have a way of saying if each template is example/poc/whatever 20:33:45 if someone turns up with a production template and wants to maintain it in heat-templates I think we should let them 20:33:46 this is about, should we allow loads of user templates, ie too many to reasonably test and maintain 20:33:50 asalkeld exactly 20:33:53 shardy: probably not. To be really scalable the collection needs to be more distributed than a single git repo can do. 20:33:54 #idea let these live in people's githubs. we keep a wiki page linking to them. 20:34:07 And also should we talke responsibility for something people will use in production 20:34:13 therve: it's a question of, "do we want to be responsible for supporting this?" 20:34:20 zaneb, I see. 20:34:26 I guess we can wait for the problem to come up :) 20:34:27 therve: and the answer is "oh, G*d no" 20:34:41 When I said production ready, it was more meaning "templates who can be copied for real production template", still has an example, but not only a functionnality per template 20:34:49 zaneb: lol, exaaactly ;D 20:34:50 still as 20:34:51 Users who want to reduce duplication of effort as Heat gains adoption will probably want to be responsible for it. 20:35:04 i think heat-templates should be for development/validation purposes of hea. there should be a separate public template catalog for the communities to contribute their templates. 20:35:11 zaneb: but hosting somebody's templates doesn't necessarily imply that we're supporting them 20:35:16 zaneb, Sure, but if someone comes tomorrow and say "Here's a great template to deploy X", there is no reason to refuse it. 20:35:26 right, and that's totally cool 20:35:42 therve: and we should point to it but not own/maintain it 20:35:48 stevebaker_, +1 20:35:48 one reason not to do it, is the review burden 20:35:57 There is no warranty of any kind for sure 20:35:57 therve: as long as they also say "and I'll fix the bugs reported against it…" 20:36:11 stevebaker_, therve: right, provided we have a way of distinguishing supported from unsupported (or at least "less supported" 20:36:22 asalkeld, I think it's a good problem to have :) 20:36:31 asalkeld: that is true, we've been struggling with that lately 20:36:41 shardy: that could be stated explicitly in the readme 20:37:36 we'd need to make it clear in each template explicitly as well i think 20:37:51 we could require people to put the name of the author in the template 20:37:52 Ok, well there's a thread on openstack-dev "Introducing heat-templates", feel free to weigh in there 20:38:01 just as well yaml supports comments :) 20:38:25 stevebaker_: yah, I can't actually believe aws used a format without them 20:38:35 stevebaker_: absolutely single biggest advantage over json 20:38:51 who needs comments anyways ;) 20:39:01 Should we move on and let the discussion continue on the ML? 20:39:20 re catalog -- the way jekyll (markdown) does it is nice -- wiki-style list people can update at the bottom of http://jekyllrb.com/docs/plugins/ -- quality and discussion is at the bottom 20:39:44 #topic Open discussion 20:39:55 shardy: oi, one more item 20:40:07 stevebaker_: there is? 20:40:13 * shardy refreshes his browser 20:40:18 #undo 20:40:19 Removing item from minutes: 20:40:46 #topic Linking to downloadable tripleo built images 20:41:12 so tripleo are autobuilding some heat-cfntools enabled images 20:41:17 http://jenkins.tripleo.org:8080/job/autobuilt-images/elements=ubuntu%20vm%20heat-cfntools/ 20:41:26 http://jenkins.tripleo.org:8080/job/autobuilt-images/elements=fedora%20vm%20heat-cfntools/ 20:41:33 which I've been using for my tempest development 20:41:33 Yes plesae try them! :) 20:41:43 #link http://jenkins.tripleo.org:8080/job/autobuilt-images/elements=fedora%20vm%20heat-cfntools 20:42:07 might need a version number in the link 20:42:07 ubuntu one is pretty good, fedora one has a bunch of pending reviews and might need a bit for fixing 20:42:25 asalkeld: ? 20:42:28 elements=fedora18x86_64 20:42:37 not just fedora 20:42:45 but when fedora is ready I'd like to propose that switch all links to prebuilt images to these ones 20:42:57 asalkeld: we use the latest always :) 20:42:59 +1 on the idea 20:43:02 +1 20:43:03 stevebaker_: Who will maintain them? 20:43:05 about to switch ubuntu to 13.04 20:43:09 +1 too btw ;) 20:43:33 but would be more than happy to accept changes to have multiple versions if thats what is needed 20:43:37 shardy: jenkins will ;) 20:43:52 anything that gets us away from having to maintain an image repo gets my +100 provided the alternative images work 20:43:58 stevebaker_: that is the correct answer :) 20:44:01 seriously, if they're part of tempest gating we will all be maintaining them on every commit 20:44:09 ...however 20:44:21 Those images are built from github.com/stackforge/diskimage-builder and github.com/stackforge/tripleo-image-elements 20:44:29 stevebaker_: I mean who will own the template definitions etc, where are they, gotta link? 20:44:44 SpamapS: pre-empted my question :D 20:44:49 today I got this comment on a review https://review.openstack.org/#/c/28650/ 20:45:03 shardy: tripleo-image-elements heat-cfntools is our element 20:45:43 so eventually this image building will result in some blessed images being hosted from stack.openstack.org 20:46:24 stevebaker_: Ok, sounds good :) 20:46:35 Right, these images will be used not only in gating tempest/heat, but hopefully in the tripleo gate for all of openstack.. eventually 20:46:41 * shardy will not miss watching oz fail after running for 10mins 20:46:43 until that happens I am suggesting: 20:46:45 1. link to jenkins.tripleo.org as soon as the fedora image is good enough (and delete our old prebuilt images) 20:46:47 2. link to stack.openstack.org when that happens 20:47:43 Can we maintain multiple fedora versions? That link doesn't specify what Fedora version? 20:48:08 ie atm F17 works with heat but F18 doesn't (unless you use updates-testing cloud-init which I pushed) 20:48:21 how about epel and scientific linux as well? 20:48:33 Its F18 20:48:39 so I think maintaining images for all non-eol versions of platforms we care about would be good 20:48:44 I think tripleo are receptive to building different images, but I don't know what the upper limit is ;) 20:48:46 "patches accepted" 20:48:48 s/we/we or our users/ 20:49:12 DIB_CLOUD_IMAGES=${DIB_CLOUD_IMAGES:-http://mattdm.fedorapeople.org/cloud-images/} 20:49:15 DIB_RELEASE=${DIB_RELEASE:-Fedora18} 20:49:16 SpamapS: as mentioned above, F18 is broken until my cloud-init update gets promoted from updates-testing 20:49:48 those both can be overriden so.. bring it on. :) 20:49:58 shardy: maybe a fedora-updates-testing dib element is needed 20:50:02 so we need a F17 version as well, and every release, there's always overlap while we work through the latest round of upgrade-brokenness 20:50:27 unfortunately, mattdm does not make F17 images 20:50:31 so... 20:50:33 and i386 and x64 versions of each... 20:50:43 stevebaker_: I think we need images available for all non-eol versions, but an updates-testing image could also be good I guess 20:50:54 * shardy is a bit scared of updates-testing in general 20:51:03 don't be scared its just crack 20:51:09 dont run updates testing in images 20:51:12 it breaks things badly 20:51:13 SpamapS: fyi fedora 19 will have an official openstack image download 20:51:18 trust me been there done that 20:51:20 stevebaker_: \o/ 20:51:58 stevebaker_: link? and will it have heat-cfntools? 20:52:01 Ok well please do discuss needs in #tripleo and we have a bug tracker too launchpad.net/diskimage-builder 20:52:31 I had one other topic that I wanted to bring up 20:52:31 SpamapS: OK, other than the single-fedora-version this all sounds good 20:52:49 shardy: regarding bp on UpdateStake support for ScalingPolicy, what's the expected behavior? Any example? Is it just handling the update of a ScalingPolicy resource? I see there's update for cloud watch alarm and ASG already. 20:52:50 anyone got anything else, 8mins left 20:52:51 https://github.com/stackforge/tripleo-image-elements 20:52:53 https://github.com/stackforge/diskimage-builder 20:52:58 Namely that Heat needs to be more responsive to the OpenStack underneath it. 20:53:14 m4dcoder: shall we discuss that in #heat after the meeting? 20:53:19 ok 20:53:33 I wanted to float the idea that "UpdateStack" would re-create any underlying resources that have gone missing 20:53:58 SpamapS: out of interest, why would they go missing? 20:53:59 meaning, if I nova delete an instance, and then UpdateStack comes along and sees that instance id is gone.. it will create a new instance. 20:54:08 shardy: out of band reasons 20:54:27 out-of-control admins ;) 20:54:39 I'm not sure UpdateStack is the proper place for this 20:54:44 So for instance lets say during the initial bring up the instance was stuck 20:54:55 It seems it would be desirable to maintain the state all the time. 20:55:01 bad mirror, or bad compute host.. basically "s*** happens" 20:55:26 therve: I agree, we can't be expected to recover from all insane out-of-band hackery which may happen 20:55:32 actually 20:55:34 yes we can 20:55:38 hmmm 20:56:05 I don't see why Heat should be completely rigid 20:56:19 shardy, Well I'm saying we should restart the instance. Just not in UpdateStack 20:56:27 SpamapS: sounds worthy of discussion anyway - care to raise a BP? I know we have a bug about restarting a failed update, but this is something different 20:56:31 shardy: the alternative is I have to rename the resource, so that heat will re-create it. 20:56:34 related to this, I've been thinking about a stack abandon/adopt 20:56:41 therve: we already can via the IHA mechanism 20:56:50 abandon is like a stack delete which doesn't delete the resources 20:56:52 shardy: its all going to the problem of resiliency to external factors, and yes I think a BP is in order. :-P 20:57:10 * therve looks it up 20:57:18 stevebaker_: yeah I like that idea, that would also work 20:57:23 stevebaker_: you can already set a DeletionPolicy on resources 20:57:26 adopt will recreate a stack based on existing (possibly abandoned) resources 20:57:27 ok I'll write up a formal proposal 20:57:45 have already started working on some poc patches 20:57:48 SpamapS: I'm in favor of resiliency, I just think it should be something you explicitly enable rather than a behind-the-scenes feature of UpdateStack, cos it's actually a recovery action 20:57:48 SpamapS: UpdateStack = neat idea -- would having a way to write idempotent templates achieve the same end ? 20:58:35 SpamapS: IMO it actually fits a heat stack create retry pattern better, where you can re-create a stack based on the current definition 20:58:47 alexheneveld: thats just the thing.. I think the current templates should be idempotent. :) 20:59:03 but we're nearly out of time, so can followup elsewhere.. 20:59:05 agreed 20:59:07 well sometimes i want 2 wordpresses :) 20:59:12 #endmeeting