20:00:21 <shardy> #startmeeting heat
20:00:23 <openstack> Meeting started Wed Sep 18 20:00:21 2013 UTC and is due to finish in 60 minutes.  The chair is shardy. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:24 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:00:26 <openstack> The meeting name has been set to 'heat'
20:00:33 <shardy> #topic rollcall
20:00:35 <SpamapS> o/
20:00:38 <bgorski> o/
20:00:41 <stevebaker> \o
20:00:43 <jpeeler> hi
20:00:49 <spzala> hi
20:00:53 <MikeSpreitzer> hi
20:01:21 <tspatzier> Hi
20:01:26 <m4dcoder> o/
20:01:34 <therve> Hi!
20:01:39 <radix> hi!
20:01:45 <radix> hello from cloudy austin!
20:02:12 <shardy> therve, sdake around?
20:02:16 <stevebaker> zaneb?
20:02:26 <therve> shardy, Hi :)
20:02:30 <shardy> stevebaker: zaneb is ill I think
20:02:32 <SpamapS> hello from sunny seattle
20:02:49 <radix> SpamapS: heh heh
20:02:53 <shardy> Ok then, lets get started :)
20:02:59 <shardy> #link https://wiki.openstack.org/wiki/Meetings/HeatAgenda
20:03:00 <SpamapS> no joke.. the sun has peeked out
20:03:19 <shardy> #topic review last weeks actions
20:03:24 <shardy> #link http://eavesdrop.openstack.org/meetings/heat/2013/heat.2013-09-11-20.00.html
20:03:44 <shardy> #info randallburt to move Rackspace resources to /contrib directory
20:04:00 <shardy> So this is posted as a draft review:
20:04:03 <sdake_> howdy
20:04:16 <sdake_> @linuxcon but in time for the meeting
20:04:35 <radix> randall is unfortunately out sick again today
20:05:07 <shardy> #link https://review.openstack.org/#/c/46410/
20:05:33 <shardy> radix: OK, well I think, as mentioned in the review, that we need more details about the test issues he's encountering
20:05:47 <stevebaker> Draft is a sucky feature, it should be marked as WIP
20:05:54 <shardy> so if randallburt is not around lets move on
20:05:57 <shardy> stevebaker: agree
20:05:59 <radix> I recall he said that it had to do with python path.
20:06:02 <radix> but that's all I know
20:06:46 <MikeSpreitzer> https://docs.google.com/document/d/1cR3Fw9QPDVnqp4pMSusMwqNuB_6t-t_neFqgXA98-Ls
20:06:53 <MikeSpreitzer> woops
20:06:56 <MikeSpreitzer> wrong paste
20:07:01 <shardy> radix: I think moving it from /contrib to heat/contrib may solve that
20:07:06 <MikeSpreitzer> I can't follow your link to review
20:07:28 <SpamapS> MikeSpreitzer: thats because it is a private draft
20:08:01 * stevebaker remembers *why* it is a sucky feature
20:08:10 <shardy> #action randallburt to repost review as WIP not draft
20:08:26 <shardy> #topic RC1 Bug status
20:08:59 <shardy> #link https://launchpad.net/heat/+milestone/havana-rc1
20:09:16 <shardy> So we're supposed to be getting the list of open bugs down to zero, to cut RC1
20:09:23 <stevebaker> ouch
20:09:28 <shardy> is anyone else getting pretty concerned about the number of bugs?
20:09:34 <SpamapS> o/
20:09:41 <SpamapS> drop all << High now
20:09:46 <shardy> IMO we need to think about bumping everything non-essential to Icehouse
20:10:19 <shardy> so please review the list, and @heat-core, set anything you're sure we need to High, or we'll probably defer it
20:10:24 <SpamapS> way too many medium priority for an rc
20:10:41 <shardy> btw I think we should stop spending so much time on the properties docs stuff
20:10:53 <SpamapS> we can probably try to land the in progress ones, but definitely not the ones that haven't been started
20:10:54 <shardy> it's worthwhile, but there's real bugs (and testing) which I think are more important
20:10:58 <shardy> thoughts?
20:11:17 <shardy> SpamapS: I agree, but we should try to complete the high ones
20:11:28 <stevebaker> yeah, shall we leave them as medium but only bump them if they are not done?
20:11:35 <SpamapS> shardy: yeah, all of the High's deserve another week at least. :)
20:11:51 <shardy> stevebaker: I think the problem is, the trigger for RC1 isn't a date, it's when we reach bugs==0
20:12:04 <shardy> so we should be focussing on fixing all the Highs, and testing
20:12:08 <SpamapS> I feel that we should bump them entirely. If we finish all of the High's, we should focus on test/doc.
20:12:11 <sdake_> does the priority for blueprints not also apply to bugs?
20:12:12 <shardy> and deferring everything else
20:12:14 <stevebaker> ok
20:12:24 <shardy> sdake_: we're in Feature Freeze, so no BPs?
20:12:52 <sdake_> what i mean is blueprints implemented a new process whereby we decide to accept/defer based upon priority
20:12:55 <sdake_> we can do the same with bugs
20:13:02 <stevebaker> btw if a bug is assigned to someone, but it is not marked as In Progress then I think it is ok to poach. agreed?
20:13:05 <shardy> stevebaker: I know you were driving the docs stuff, and I think it's valuable, but do you think deferring some (what isn't in progress) is reasonable?
20:13:10 <shardy> given the time constraints..
20:13:22 <stevebaker> shardy: yes, lets get in what we can
20:13:25 <shardy> stevebaker: agree
20:13:46 <radix> I thought we decided that docs could still be worked on post-rc1?
20:13:46 <radix> and still get into havana?
20:14:11 <shardy> radix: they can, probably, but we need to get all the actual bugs fixed for RC1
20:14:17 <radix> yeah, ok
20:14:19 <shardy> so we can test and find the remaining bugs ;)
20:15:45 <shardy> Ok I'll defer any non-high bugs tomorrow, and please take any remaining high bugs, as there are a couple unassigned
20:16:13 <shardy> SpamapS: are you working on bug #1209492 ?
20:16:14 <uvirtbot> Launchpad bug 1209492 in heat "Users can fill up the events table" [High,Triaged] https://launchpad.net/bugs/1209492
20:17:48 <shardy> Ok, I guess not ;)
20:18:02 <shardy> anyone else have anything else to raise re RC1 or the Havana release?
20:18:12 <stevebaker> SpamapS: could you check if https://bugs.launchpad.net/heat/+bug/1214227 is addressed by https://review.openstack.org/#/c/43149/ ?
20:18:15 <uvirtbot> Launchpad bug 1214227 in heat "Failure to download TemplateURL in nested stack is never reported" [High,Triaged]
20:18:51 * stevebaker stares at the back of SpamapS's head
20:20:06 <shardy> Ok, lets move on
20:20:12 <shardy> #topic open discussion
20:20:22 <shardy> Anyone have anything to raise?
20:20:26 <MikeSpreitzer> Lots
20:20:30 <therve> Trusts :/
20:20:31 <stevebaker> I'd like to talk blueprint hot-software-config
20:20:38 <SpamapS> stevebaker: in theory that should address it
20:20:40 <radix> summit sessions?
20:20:43 <radix> should we talk about those?
20:20:47 <stevebaker> shardy: who has the talking stick?
20:20:58 <shardy> Yeah we still need more summit proposals, 9 so far
20:21:04 <SpamapS> apologies for not responding.. diskimage-builder seems to be deleting peoples' /dev
20:21:10 <stevebaker> ouch
20:21:31 <radix> ok I'm going to go create some RIGHT NOW
20:21:37 <therve> shardy, Don't we have 9 slots? :)
20:22:02 <shardy> therve: anything specific, other than testing and bug-fixing needed for trusts?
20:22:11 <stevebaker> So I'm thinking of writing a full spec for https://blueprints.launchpad.net/heat/+spec/hot-software-config to kick of discussion in heat-core on how to implement components.
20:22:13 <spzala> stevebaker: last I heard, nanji had a good progress on hot-software-config
20:22:19 <shardy> therve: Yeah, I think we'll have a shortlist then pick 9 :)
20:22:45 <therve> shardy, I'm trying to debug it. It seems you can't get a token from a token obtained via trusts, and we seem to try to do that
20:22:57 <shardy> stevebaker: wiki page for software-config would be great, catalyst for summit discussions
20:23:14 <stevebaker> spzala: yes he has, but I have some issues with how it is currently being modeled, and I'm more than happy for nanjj to continue once a spec has been agreed on
20:23:19 <tspatzier> stevebaker, I started a wiki page some time back: https://wiki.openstack.org/wiki/Heat/Software-Configuration-Provider
20:23:28 <shardy> therve: Yeah, so as discussed earlier I may have messed up there, we just have to work out the right fix
20:23:28 <radix> stevebaker: I would like to see more explanation of why we need "components" as a separate concept from resources
20:23:33 <spzala> stevebaker: +1
20:23:35 <tspatzier> I'd be happy to work with you on that
20:23:36 <radix> stevebaker: not here in the meeting, just documented somewhere.
20:23:54 <stevebaker> tspatzier: ok, I'll check it out. But fair warning it will be a slightly different take on what is there currently
20:24:11 <stevebaker> radix: yes, I'll include a section on that
20:24:15 <radix> thanks
20:24:29 <tspatzier> stevebaker, that's ok. Good to have some discussion going on this topic
20:24:32 <shardy> therve: as you're discovering, trusts aren't exactly easy :\
20:24:35 <stevebaker> as a related thing, I wrote this https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config
20:25:03 <stevebaker> ...also to provoke discussion - I expect we'll still be talking about this at summit
20:25:07 <shardy> stevebaker: related to that, I think we should have a BP for better cloud-config integration
20:25:26 <shardy> ie so you can boot instances and customize them with no in instance tools except cloud-init
20:25:36 <shardy> (which is already somewhat possible, although limited)
20:25:38 <stevebaker> shardy: that wiki page might be exactly that
20:25:44 <shardy> stevebaker: k, cool
20:25:45 <tspatzier> I think I actually submitted some design session proposals on this :-)
20:25:59 <shardy> tspatzier: Ok, cool, we're aligned then :)
20:26:00 <stevebaker> tspatzier: yes, we'll need a session
20:26:41 <stevebaker> shardy: could you harvest those links for the minutes?
20:26:53 <MikeSpreitzer> What about the relationship between heat and the discussions in the nova scheduler group?
20:26:58 <shardy> tspatzier: I hit bug #1224111 today, if possible can you please take a look?
20:26:59 <uvirtbot> Launchpad bug 1224111 in heat "HOT cannot be used as a provider template" [High,Confirmed] https://launchpad.net/bugs/1224111
20:27:20 <shardy> tspatzier: HOT related, I think I know roughly what needs to be done, but any input appreciated :)
20:27:29 <shardy> #link https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config
20:28:18 <tspatzier> shardy, I will have a look. Might get in touch with you on IRC to talk about your initial thoughts if you already have some
20:29:22 <shardy> tspatzier: Ok, thanks, basically we need template_resource resources parameters mangled in the same way as HOT resource properties
20:29:56 <tspatzier> shardy, ok thanks. I'll check that out.
20:30:35 <shardy> #link https://wiki.openstack.org/wiki/Heat/Software-Configuration-Provider
20:30:41 <radix> so is the plan to decide what to talk about *at* the summit? or will there be voting beforehand?
20:30:53 <shardy> stevebaker: were those all the links?
20:30:58 <radix> (or whatever decision-making process)
20:31:14 <stevebaker> shardy: yes, thanks
20:31:21 <shardy> radix: we'll vote in one of these meetings shortly before summit
20:31:26 <radix> ok cool
20:31:36 <stevebaker> radix: not voting beforehand, just discussion and an attempt at concensus
20:31:42 <MikeSpreitzer> Any answers about cross-stack references?
20:31:45 <radix> sounds good to me
20:31:53 <shardy> radix: yeah, sorry s/vote/discuss
20:32:09 <radix> MikeSpreitzer: sorry I haven't been following it
20:32:10 <shardy> get conesensus then I approve
20:32:20 <tspatzier> stevebaker, on the software config thing: once you have a look at the wiki page I started and think we need to start over radically different, let's get in touch and work on a common page. I want to avoid having multiple conflicting/competing pages.
20:32:50 <stevebaker> tspatzier: sounds good, maybe etherpad rather than wiki for a start
20:33:15 <tspatzier> stevebaker, makes sense. Or there is also an attached discussion page. Whatever you think is best
20:33:42 <stevebaker> ok
20:34:24 <MikeSpreitzer> (I should be waiting for shardy to direct our attention, right?)
20:34:28 <shardy> Ok, any other topics anyone wants to raise?
20:34:38 <MikeSpreitzer> Yes
20:34:49 <shardy> MikeSpreitzer: this is open discussion, so go for it ;)
20:35:06 <MikeSpreitzer> I have been having some discussion with scheduler folks that I think is relevant to heat too
20:35:22 <MikeSpreitzer> Not sure how many of you follow both.
20:35:31 <shardy> MikeSpreitzer: I've been following that, there's definitely a discussion to be had at summit
20:35:39 <shardy> MikeSpreitzer: I've been following your openstack-dev posts
20:35:44 <MikeSpreitzer> My interest is in holistic scheduling, I think it is relevant to both communities
20:35:50 <stevebaker> MikeSpreitzer: is this the placement patch which didn't make feature freeze?
20:36:02 <MikeSpreitzer> No, I'm not that close to a patch
20:36:08 <stevebaker> ok
20:36:26 <therve> MikeSpreitzer, I'm not sure how much it impacts Heat to be honest
20:36:30 <MikeSpreitzer> I'm looking for a bit of consensus on architecture.
20:36:32 <therve> Nova should grow some features
20:36:41 <therve> We'll be able to use them once it's exposed
20:37:09 <MikeSpreitzer> Holistic scheduling is about scheduling a whole template at once, including all resources of all types potentially… that does not sound like nova to me.
20:37:39 <radix> I can maybe see a reason to separate out the scheduler to be more generally useful in other contexts... maybe
20:37:39 <therve> Okay
20:37:42 <MikeSpreitzer> (I qualify "potentially" because some scheduling can and should be deferred to lower silos — when it has no interactions with other scheduling)
20:37:45 <radix> but I have not thought much about it
20:37:47 <shardy> MikeSpreitzer: I think if the nova scheduler has to start caring about dependencies (between instances) then the logic should be in Heat
20:38:05 <radix> some concrete use cases would be nice
20:38:10 <shardy> but if it's "just" placement hints, then nova seems like an OK place to do it
20:38:14 <MikeSpreitzer> Yes, I think there is a need here for something that is prior to nova/cinder/neutron/...
20:38:29 <MikeSpreitzer> Hadoop is one
20:38:34 <shardy> otoh I did voice conerns about scope creep of this feature at the Havana summit..
20:38:43 <shardy> (nova group scheduling)
20:38:50 <MikeSpreitzer> You want some VMs placed near each other, and Cinder volumes placed near them (if you are using direct attached storage)
20:39:05 <stevebaker> MikeSpreitzer: so each api would need to take scheduler hints? Our native resources can use whatever gets exposed in the api
20:40:11 <MikeSpreitzer> Yes, each resource API should have a way to be told what the placement should be.  In fact, many already do.
20:41:10 <MikeSpreitzer> The nova group scheduling is explicitly intended to be the basis for more development, but I think what we really need is not Compute specific
20:41:52 <therve> The fact is, we can't do anything about placement without support from the underlying projects
20:41:58 <MikeSpreitzer> of course
20:42:24 <stevebaker> MikeSpreitzer: so unless heat has to interact directly with a scheduler, this doesn't seem too controversial from a heat point of view
20:43:30 <shardy> It could even make life easier, ie implement ServerGroup resource which can automatically span compute nodes, instead of trying to do it ourselves in InstanceGroup
20:43:36 <MikeSpreitzer> A holistic scheduler is something that would logically be prior to heat, if heat is merely infrastructure orchestration.  OTOH, if Heat grows to include software coordination, that is something that should be prior to holistic infrastructure scheduling.
20:44:15 <shardy> MikeSpreitzer: do you have a link to a wiki page describing your vision of this?
20:44:16 <MikeSpreitzer> So holistic infrastructure scheduling would go in the middle of an expanded heat.
20:44:40 <MikeSpreitzer> https://docs.google.com/document/d/1hQQGHId-z1A5LOipnBXFhsU3VAMQdSe-UXvL4VPY4ps/edit#
20:45:03 <shardy> #link https://docs.google.com/document/d/1hQQGHId-z1A5LOipnBXFhsU3VAMQdSe-UXvL4VPY4ps/edit#
20:45:12 <MikeSpreitzer> That's something I threw  together before yesterday's scheduler meeting, expecting it to mainly provide pictures to talk to.  It is not good as a standalone doc.
20:45:43 <stevebaker> ok, we'll take a look. thanks
20:46:00 <jasond`> regarding the multi-engine design, i'm thinking about implementing a locking abstraction that will use a database lock by default (option 1 at https://etherpad.openstack.org/vJKcZcQOU9) but have an optional zookeeper driver that can be used for production deployments (option 4 at https://etherpad.openstack.org/vJKcZcQOU9).  this would keep ZK out of the dependencies, which was the major objection to it.  any thoughts?
20:46:15 <MikeSpreitzer> As I said, it does not stand up to independent study too well.  I cited it as a response to a request for pictures.
20:46:44 <stevebaker> jasond`: sounds good
20:47:28 <shardy> jasond`: sounds like we may need another summit session on this ;)
20:47:57 <jasond`> shardy: i won't be at the summit :/
20:48:11 <jasond`> but randall should be able to discuss it
20:48:12 <MikeSpreitzer> I'm lost, to what does that etherpad respond or elaborate?
20:48:13 <radix> summit all the things!
20:48:47 <shardy> MikeSpreitzer: allowing use of multiple heat-engine processes
20:48:58 <shardy> jasond`: that's unfortunate :(
20:49:09 <MikeSpreitzer> I mean, is there a blueprint or something for that?
20:49:23 <jasond`> MikeSpreitzer: https://blueprints.launchpad.net/heat/+spec/multiple-engines
20:49:28 <stevebaker> MikeSpreitzer: it is a discussion of options for a race condition bug
20:49:46 <MikeSpreitzer> OK, I'll look into that.
20:50:03 <kebray> jasond still worth you submitting the session topic :-)   we'll let Randall proxy for you as speaker.  shardy, it's travel budget.  it's really unfortunate.  I'd send all my devs if I could.
20:50:06 <jasond`> MikeSpreitzer: feel free to update the etherpad with any comments
20:50:37 <jasond`> kebray: sure, i'll submit it
20:50:38 <MikeSpreitzer> Anyway, if you folks are agreeing with the outline of [software coordination -> holistic infrastructure scheduling -> infrastructure orchestration -> lowlevel APIs] then I am happy.
20:50:47 <shardy> #link https://blueprints.launchpad.net/heat/+spec/multiple-engines
20:51:01 <shardy> #link https://etherpad.openstack.org/vJKcZcQOU9
20:51:19 <shardy> MikeSpreitzer: I think we need more information (and/or to read your doc)
20:51:44 <MikeSpreitzer> As I said, the doc is not great, I was expecting mainly discussion.
20:52:07 <radix> MikeSpreitzer: try writing up something a bit more holistic that we can look at maybe
20:52:18 <MikeSpreitzer> OK
20:52:26 <shardy> MikeSpreitzer: I'll review the doc, but I think what's needed is a top-level overview at this point, perhaps create a wiki page in wiki.openstack.org?
20:52:36 <radix> yeah, agreed with shardy
20:53:19 <shardy> Ok, 8 mins, anything else anyone wants to raise?
20:53:33 <MikeSpreitzer> OK.  The hardest part is going to be knowing where to stop.  There is a lot to say, and I have not worked it all out.
20:54:34 <MikeSpreitzer> Well, I have this little question about cross-stack references
20:54:53 <shardy> MikeSpreitzer: I replied to your ML post earlier
20:54:55 <MikeSpreitzer> Suppose you create a stack that implements a shared service, and then later create a stack that uses the service …
20:55:20 <MikeSpreitzer> the response about nested stacks?  I mean stacks that are separate, not nested.
20:55:34 <stevebaker> yes, this will not be an unusual scenario
20:56:26 <shardy> I think you'd still take the output of one and feed it in as a parameter to another, no?
20:56:43 <shardy> only via the API rather than an internal reference?
20:56:49 <shardy> (template-internal)
20:56:49 <MikeSpreitzer> That might be an approach...
20:57:05 <MikeSpreitzer> it would mean using IDs from the lower layers, rather than referring to things by their template names
20:57:25 <stevebaker> yes, refs for most resources are their uuids
20:57:56 <kebray> MikeSpreitzer not sure I fully understand... but, wondering if this is similar to a use case I care about, creating (or adding to) a stack resources that already exist, just by passing in their IDs.
20:58:21 <MikeSpreitzer> Yes, I think it is related.
20:58:28 <shardy> MikeSpreitzer: atm I think it's the only approach, but yeah you could maybe allow HOT's get_attr to optionally take a stack ID
20:58:29 <kebray> So, doing create resource with creating the resource :-)
20:58:32 <MikeSpreitzer> Let me elaborate  a little on my case...
20:58:50 <stevebaker> only 2 minutes left, this should be moved to #heat
20:59:01 <MikeSpreitzer> OK
20:59:18 <shardy> Yup, time's up, thanks all!
20:59:25 <shardy> #endmeeting