20:00:16 <stevebaker> #startmeeting heat
20:00:17 <openstack> Meeting started Wed Mar 26 20:00:16 2014 UTC and is due to finish in 60 minutes.  The chair is stevebaker. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:18 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:00:21 <openstack> The meeting name has been set to 'heat'
20:00:22 <skraynev> o/
20:00:26 <stevebaker> #topic rollcall
20:00:28 <zaneb> greetings
20:00:30 <asalkeld> o/
20:00:41 <asalkeld> (hi from Raleigh)
20:00:50 <bgorski> o/
20:00:54 <shardy> o/
20:00:57 <tspatzier> hi
20:01:22 <mspreitz> o/
20:01:55 <stevebaker> no actions last week
20:02:09 <stevebaker> #topic Adding items to the agenda
20:02:18 <stevebaker> #link https://wiki.openstack.org/wiki/Meetings/HeatAgenda#Agenda_.282014-3-26_2000_UTC.29
20:03:07 <stevebaker> anything to add to the agenda?
20:03:40 <zaneb> fixing the gate?
20:03:56 <zaneb> or leave that for offline
20:04:00 <randallburt> sorry I'm late
20:04:09 <stevebaker> I'll add it
20:04:11 <skraynev> zaneb: you mean pep8? or something else?
20:04:33 <shardy> skraynev: the config generation which is currently broken
20:04:33 <zaneb> skraynev: yeah, the config file is borked
20:04:49 <stevebaker> #topic Icehouse rc-1 bug burndown list\
20:04:55 <skraynev> shardy, zaneb: ok, thanks guys ;)
20:05:07 <stevebaker> 3 bugs to go, lets quickly go through each one
20:05:28 <stevebaker> https://bugs.launchpad.net/heat/+bug/1291091
20:05:29 <uvirtbot> Launchpad bug 1291091 in heat "dangerous secgroup rule created when the "SourceSecurityGroupName" property string doesn't refer to an existing security group" [High,In progress]
20:05:36 <stevebaker> https://review.openstack.org/#/c/81558/
20:05:45 <stevebaker> this needs reviews as soon as gate is fixed
20:05:58 <stevebaker> should be good to go
20:06:31 <stevebaker> https://bugs.launchpad.net/heat/+bug/1243992
20:06:32 <uvirtbot> Launchpad bug 1243992 in heat "Race on delete between OS::Neutron::Subnet and OS::Nova::Server->networks->uuid" [High,In progress]
20:06:41 <stevebaker> https://review.openstack.org/#/c/82394/
20:06:49 <stevebaker> needs an approve
20:07:26 <stevebaker> https://bugs.launchpad.net/heat/+bug/1296859
20:07:27 <uvirtbot> Launchpad bug 1296859 in heat "stack-list throws an error if any stack returned contains a template error" [Medium,Triaged]
20:07:47 <stevebaker> sdake has a migration script pending to fix this particular validation error
20:08:38 <stevebaker> I've kicked a few bugs, is there anything that people want to pull back into rc1?
20:09:36 <shardy> stevebaker: No, but IMO we should have an rc2 for https://bugs.launchpad.net/heat/+bug/1297761
20:09:37 <uvirtbot> Launchpad bug 1297761 in heat "No validation of top-level template keys" [Undecided,New]
20:09:45 <shardy> and any other stuff obviously :)
20:10:00 <stevebaker> we could be ready for cutting rc1 at the end of this week <-- ttx
20:10:14 <tspatzier> stevebaker: I started working on this today.
20:10:23 <stevebaker> yeah I'm going to assume we will want an rc2
20:10:28 <ttx> stevebaker: awesmoe
20:10:32 <ttx> or some
20:10:49 <stevebaker> o for oresome
20:10:59 <zaneb> stevebaker: lol
20:11:04 <stevebaker> #topic Should we sync latest oslo.db code?
20:11:14 <stevebaker> #link https://review.openstack.org/#/c/76539/
20:11:31 <stevebaker> is syncing oslo db too high risk for rc2?
20:11:44 <sdake> stevebaker sorry was busy with interviews, migration patch should be ready to roll in couple hours
20:11:52 <asalkeld> yes, it's quite different
20:12:16 <zaneb> stevebaker: sounds late & risky to me
20:12:40 <asalkeld> they are starting to remove all globals
20:12:50 <asalkeld> so requires a bit of work
20:13:00 <asalkeld> (sessions etc)
20:13:04 <stevebaker> ok, lets do it in early juno
20:13:10 <sdake> imo changing anything in ocmmon is not ideal at this point
20:13:21 <asalkeld> ocaml :)
20:13:31 <stevebaker> #agreed sync oslo db in early juno
20:14:02 <shardy> +1
20:14:17 <stevebaker> #topic Defining designated sections for Heat core definition
20:14:26 <stevebaker> this one is a doozy
20:14:41 <randallburt> I want section 7.
20:15:12 <stevebaker> so there has been back and forth between the TC and the board about how to define what is "core" OpenStack
20:15:35 <zaneb> so, do we only have to explicitly list pluggable stuff that we want in core?
20:15:53 <zaneb> or do we have to list all of the non-pluggable stuff + the pluggable stuff we want in core?
20:16:04 <stevebaker> the board have asked each project to define their designated sections, and I shall tell you what that means
20:16:14 <zaneb> (I think there was some discussion about this, but I forget the outcome, if any)
20:16:17 <asalkeld> kinda depends on what is defined as core in the other projects
20:16:58 <stevebaker> *if* heat was part of OpenStack core, what parts of heat would need to be installed for a cloud to claim that they "have heat"
20:17:11 <stevebaker> #link https://wiki.openstack.org/wiki/Governance/CoreDefinition
20:17:14 <asalkeld> we can't say resource x is core if the feature is not core in it's project
20:17:26 <randallburt> seems to me, Heat "core" should be HOT, CFN, and the "native" and CFN resources that correspond to the "core" openstack services (whatever they may be).
20:17:38 <asalkeld> -1 to cfn
20:17:38 <stevebaker> here is where our results can go
20:17:42 <stevebaker> #link https://etherpad.openstack.org/p/openstack-designated-sections
20:17:48 <zaneb> asalkeld: but we could say if you have that feature, you have to have the resource plugin for it too
20:17:52 <randallburt> asalkeld:  I wouldn't argue
20:17:52 <sdake> -1 to non-native resources
20:18:11 <sdake> core is the minimum
20:18:16 <stevebaker> so I just want to start the discussion now, and lets whack a first draft into https://etherpad.openstack.org/p/heat-designated-sections
20:18:17 <zaneb> randallburt: no need for cfn to be core IMO
20:18:19 <sdake> we dont want to force cfn and non-native on folks
20:18:43 <randallburt> so openstack native api, and resource plug-ins that support the other services defined as "core"?
20:18:56 <asalkeld> sounds good
20:19:05 <asalkeld> what about cfn-tools?
20:19:06 <stevebaker> randallburt: that is the gist I've been thinking of
20:19:11 <zaneb> yep, and the HOT parser
20:19:16 <shardy> I agree native api and resources, but we'll have to fix the gaps where we still require cfn stuff
20:19:23 <randallburt> cfn-tools isn't *required* to make heat work
20:19:26 <stevebaker> asalkeld: that is up to the user's images
20:19:37 <randallburt> stevebaker:  +1
20:19:40 <zaneb> shardy: true, we'll have to require WaitCondition and the like
20:19:42 <asalkeld> stevebaker, just saying we should be explict
20:19:44 <sdake> cfn-tools is not part of hte integrated release randallburt
20:19:52 <sdake> to be core, it has to be integrated first
20:19:52 <stevebaker> so lets get on that etherpad and brain-dump a list
20:20:01 <randallburt> k
20:20:06 <shardy> zaneb: provide native signalling in general, e.g from ceilometer which currently requires ec2tokens
20:20:20 <shardy> that's the main gap AFAIK
20:20:47 <sdake> i thought to hvae a designed section you had to have functional CI in place
20:21:02 <sdake> stevebaker did I parse that wrong?
20:22:02 <stevebaker> ah yes, designated sections are validated by tempest tests, so you should be able to run tempest against a cloud to check for compliance
20:23:37 <stevebaker> how about this for designated
20:23:46 <stevebaker> heat-api
20:23:48 <stevebaker> heat-engine
20:23:50 <stevebaker> all OS::Heat::* resources
20:23:52 <stevebaker> OS::* resources which correspond with the APIs of designated sections of other projects
20:24:33 <zaneb> can we insert "non-deprecated" in there?
20:25:07 <stevebaker> clouds are free to reimplement resource types, as long as they match the schema of ours, since that will pass the tests
20:25:18 <stevebaker> zaneb: yes, because its not long enough yet ;)
20:25:18 <asalkeld> can we add "core" to our resource versioning system?
20:25:23 <shardy> stevebaker: sounds good, but mentioning tempest, I guess there'll be a core/non-core refactor there, as currently we have a mix
20:25:31 <skraynev> stevebaker: what about AWS:: resources which are used as parent class for OS:: resources ?
20:25:33 <asalkeld> so we can just dump this info out
20:25:51 <stevebaker> shardy: I'm guessing there will be some attribute tagging on tests
20:26:19 <stevebaker> skraynev: I think the parent relationship should be inverted on those resources
20:26:21 <zaneb> skraynev: not required - making your own implementation is OK. besides, you don't have to register the AWS:: resources
20:26:22 <shardy> skraynev: IMO, in the medium term we need to refactor those out, and move to a model where most AWS resources are provider templates based on native resources
20:27:06 <stevebaker> "OS::* resources which correspond with the non-deprecated APIs of designated sections of other projects"
20:27:28 <sdake> we also dont want to require deprecated OS:: resources
20:27:50 <stevebaker> sdake: I'll just add that to replaceable sections
20:28:03 <sdake> wfm
20:28:38 <skraynev> shardy: I think that will be better to have one base class, which will be used OS ans AWS resources both.
20:28:48 <afarrell> hello, first time here - looking to contribute to heat. specifically, tosca translation... ps - sorry for the interruption.
20:29:00 <stevebaker> afarrell: \o
20:29:13 <shardy> skraynev: well that's basically what I said, only the AWS resources become provider templates not actual plugins
20:29:16 <stevebaker> afarrell: you should talk to tspatzier
20:29:29 <afarrell> thanks steve
20:29:45 <shardy> skraynev: e.g AWS::EC2::Instance becomes a provider template containing an OS::Nova::Server resource
20:29:55 <tspatzier> afarrell: yeah, let's talk. but maybe not hijack the meeting for this ;-)
20:29:56 <stevebaker> well, I was expecting this topic to take the entire meeting, there appears to be resounding consensus
20:30:11 <afarrell> tspatzier - no worries, thanks!
20:30:29 <skraynev> shardy: agree, it make sense.
20:30:34 <sdake> i have 1 q
20:30:40 <stevebaker> I'll paste that etherpad into https://etherpad.openstack.org/p/openstack-designated-sections
20:30:46 <randallburt> stevebaker:  we can argue some more if it makes you feel better.
20:30:46 <sdake> the wait conditions as they exist today, doesn't that require the cloudwatch api?
20:31:01 <stevebaker> they require the cfn API, yes
20:31:15 <sdake> ok I guess we need to fix that :)
20:31:17 <randallburt> stevebaker:  which we'll have to fix if we include them in "core", yes?
20:31:28 <shardy> sdake: Yeah, that's part of the native signaling I mentioned
20:31:30 <sdake> so we can't require waitconditions in the core
20:31:51 <sdake> need to add an exception for waitconditions - since our dependency is not core
20:31:52 <stevebaker> well, the waitcondition is AWS::, so they are just as replacable as the API they use
20:31:56 <randallburt> we can always leave them out in the first go, they're handy but you can do cool stuff without them.
20:31:56 <zaneb> stevebaker: is this an immediate thing, or targeting some future release? A lot of cfn stuff may drop out over time, but be required right now
20:32:13 <sdake> stevebaker good point - back to lala land forme
20:32:22 <stevebaker> but software deployments can use native heat api *or* cfn
20:32:22 <shardy> sdake: I started working on some native alterntives, but to some extent you can replace them with stevebaker's SoftwareDeployment resources, which already support native signalling
20:32:48 <shardy> stevebaker: \o/ SoftwareDeployments ftw :)
20:33:14 <sdake> if only someone would write a hip blog about softwraedeployments :)
20:33:21 <shardy> I still think we may need a pure signalling resource for some use-cases, but it's definitely not super high priority
20:33:23 <stevebaker> signal_transport=HEAT_SIGNAL ;)
20:33:47 <asalkeld> any thoughts about moving software configs into a seperate repo?
20:33:54 <asalkeld> (like the autoscaling)
20:34:09 <asalkeld> (an aside I know)
20:34:09 <shardy> asalkeld: like the autoscaling?
20:34:14 <tspatzier> sdake: going to play around with software deployment and hope to produce some more samples and even docs around how to use it.
20:34:28 <stevebaker> asalkeld: software configs could be stored in glance, which would leave a *tiny* deployments api
20:34:38 <sdake> tspatzier that would be great - people wont know how to use it without docs :)
20:34:39 <stevebaker> asalkeld: which may as well stay in heat
20:34:47 <asalkeld> stevebaker, I mean the code
20:34:59 <asalkeld> no biggie
20:35:17 <zaneb> stevebaker, asalkeld: does it have a separate endpoint in the catalog?
20:35:22 <sdake> i think we need to sort out getting autoscaling into a new repo before we shave off more work :)
20:35:34 <asalkeld> zaneb, I think it should
20:35:43 <stevebaker> asalkeld: I hadn't considered that. its just part of the heat api currently
20:35:56 <randallburt> asalkeld, zaneb templates will, software cofigs would too I suppose.
20:36:08 <zaneb> asalkeld: inclined to agree it should, but *does* it? ;)
20:36:23 <asalkeld> no
20:36:58 <shardy> sdake: why do we want autoscaling in a separate repo, when there has been zero progress on the native autoscaling API over an entire cycle?
20:37:12 <tspatzier> are you talking about all the hooks that are currently going into heat-templates? or more of the software config implementation?
20:37:18 <shardy> I'm very happy we have the native resources, but IMO splitting repos is premature
20:37:50 <stevebaker> tspatzier: that is good point, I've been assuming that heat-templates is the best place for the hooks to live
20:37:53 <zaneb> shardy: 2 cycles by my count, but hopefully that won't last forever. I don't think sdake is suggesting we split it _now_
20:37:55 <sdake> shardy my point was we haven't tackled autoscaling, why would we tackle software deployments as a separate repo
20:38:23 <shardy> sdake, zaneb: k, just wanted to check there wasn't some decision I missed ;)
20:38:27 <asalkeld> yeah, it's a matter of people
20:38:34 <tspatzier> stevebaker: yes, agree. I don't see other parts to be split out at the moment.
20:38:39 <shardy> I'm not opposed to splitting stuff when it makes sense
20:39:01 <shardy> asalkeld: That's my main argument for not splitting things out, lets focus in one place rather than fragment effort
20:39:15 <shardy> when it becomes clear there's an obvious split, split the repos
20:39:22 <stevebaker> shardy: +1
20:39:30 <asalkeld> just re: hoha about application/infrastructure seperation
20:39:32 <shardy> and develop things in a way that would be sympathetic to a future separation
20:39:47 <asalkeld> shardy, sure
20:39:59 <asalkeld> but maybe a good time for a new endpoint
20:40:04 <asalkeld> to make that easy
20:40:57 <stevebaker> #topic Teh gates is borken
20:41:22 <asalkeld> reminds me to go fix solum sample config
20:41:30 <zaneb> I put up a patch to fix the gate
20:41:55 <zaneb> it modifies an oslo-incubator file without an upstream change
20:41:58 <zaneb> discuss.
20:42:17 <sdake> i'm pretty sure when I did that I got 2 -1 votes :)
20:42:20 <shardy> zaneb: so can you explain why the oslo folks wouldn't take your patch?
20:42:35 <stevebaker> zaneb: do you want to append https://review.openstack.org/#/c/81558/ to that so we get a head-start on merging it?
20:42:45 <asalkeld> is this the client config sections?
20:42:45 <shardy> I'm -2 on it unless we move it out of openstack/common
20:42:56 <zaneb> shardy: it was a different patch that they rejected
20:43:03 <dhellmann> if you have oslo patches you need to land for rc1, please let us know in #openstack-oslo
20:43:14 <stevebaker> can't we just have a change which runs generate_sample.sh
20:43:37 <sdake> generate_sample.sh calls a python file stevebaker
20:43:39 <sdake> generator.py
20:43:47 <sdake> generator.py is pos imo :)
20:44:03 <dhellmann> sdake: :-(
20:44:18 <sdake> sorry dhellmann but thtas how I see it :(
20:44:31 <zaneb> dhellmann: I'll drop in and discuss with you after this
20:44:39 <shardy> zaneb: since we have dhellmann here, can you explain the issues, and why we can't just get the stuff into oslo?
20:45:06 <zaneb> shardy: basically, because the fix for us breaks Nova
20:45:07 <sdake> ya an accounting of the nova viewpoint woudl be helpful
20:45:10 <zaneb> and vice-versa
20:45:16 <shardy> zaneb: If we *have* to fork temporarily to unblock the gate, lets pull the generator code out of the heat oslo tree completely IMO
20:45:25 <zaneb> I'm fine with that
20:45:34 <vijendar> ls
20:45:34 <dhellmann> zaneb: thank
20:45:57 <dhellmann> sdake: patches welcome, keep your attitude
20:46:16 <zaneb> so to get a fix that works for Nova too involves some roundabout changes to either python-keystoneclient, nova or both
20:46:28 <vijendar> sorry.. my last message was by mistake (wrong terminal)
20:47:22 <zaneb> it's not clear what the Right way to fix it is, and we don't have time to figure it out while the gate is broken
20:47:37 <shardy> who'd have thought generating a config file could be so hard :(
20:48:08 <shardy> zaneb: sounds like we fork with a temporary fix then work on the long-term solution for early Juno?
20:48:26 <zaneb> shardy: the way that olso.config registers options is kinda... nasty. problems were probably inevitable
20:48:49 <zaneb> yep, I think that's the way to go
20:48:53 <shardy> sounds odd that we have conflicting requirements to Nova, I thought the point of us all using oslo.config etc was that it's all common
20:49:32 <zaneb> shardy: I can explain the details after, but TBH it's really boring
20:49:46 <asalkeld> shardy, well in this case we are happy this isn't a library
20:49:50 <stevebaker> ok, lets fork to heat.common for now
20:49:57 <zaneb> long story short, python-keystoneclient is doing it wrong and Nova is caught up in it
20:50:48 <shardy> Maybe all the projects should stop generating config from keystoneclient, and just import a (generated, by the keystoneclient tree) sample snippet
20:50:57 <mspreitz> zaneb: could that be the cause of https://bugs.launchpad.net/nova/+bug/1289135 ?
20:50:59 <uvirtbot> Launchpad bug 1289135 in python-cinderclient "cinderclient AmbiguousEndpoints in Nova API when deleting nested stack" [Undecided,New]
20:51:06 <asalkeld> shardy, that is possible
20:51:08 <zaneb> mspreitz: no
20:51:13 <shardy> The coupling between keystoneclient and all-the-projects seems to be the main issue, we keep getting borken by them
20:51:19 <asalkeld> you can apend static config
20:51:21 <shardy> s/them/it
20:51:23 <sdake> the way our client section is generated is suboptimal as well
20:51:52 <asalkeld> one other issue is you can't exclude files
20:52:00 <sdake> i'll fix that up once we get a generator that works
20:52:10 <asalkeld> otherwise we could work around this
20:52:28 <zaneb> sdake: what would you change?
20:52:53 <sdake> zaneb i'll submit a patch for review when generator is working - you can comment there
20:53:14 <sdake> it does some wacky global manipulation
20:53:24 <zaneb> I'm curious now, because it might affect the way I want to fix the generator ;)
20:53:41 <asalkeld> zaneb, the problems i found:
20:53:42 <sdake> i had a patch at one time but i'm not sure if I sitll hav eit
20:53:50 <sdake> i couldn't get it to work with the generator so abaonded it
20:53:56 <asalkeld> the __eq__ operator in oslo.config
20:54:13 <asalkeld> does not compare value
20:54:29 <asalkeld> also you can't have an option in multiple groups
20:54:34 <zaneb> asalkeld: it does now! that's the cause of this bug
20:54:57 <asalkeld> ok
20:55:15 <stevebaker> #topic open discussion
20:55:20 <stevebaker> 5 minutes
20:55:21 <sdake> harestarter
20:55:23 <zaneb> yeah, registering a single option in multiple groups would be nice
20:55:33 <sdake> rationales for deprecating?
20:56:12 <sdake> zaneb shardy seem to have the thought we should deprecate
20:56:20 <sdake> like to understand the technical reasons
20:56:26 <zaneb> please please please let's deprecate it
20:56:28 <shardy> sdake: it is broken, we don't want to maintain it, and what it does is probably possible via other means
20:56:39 <skraynev> stevebaker: 1 question. if we have default value in property schema should it be used during update ?
20:56:47 <shardy> sdake: It does a weird update, without properly managing dependences, not using the update code
20:57:15 <shardy> it deletes resource when you aren't expecting it, and is generally not going to actually do what the user really wants
20:57:16 <sdake> i think people want a ha restart feature
20:57:29 <sdake> is the answer to deprecate or make it work properly?
20:57:35 <zaneb> deprecate.
20:57:36 <sdake> or is the issue people don't want a ha restart feature
20:57:58 <zaneb> the issue is that it doesn't belong in Heat
20:58:14 <asalkeld> minstral
20:58:18 <shardy> sdake: I think the same is possible either via an AutoScaling group os size 1 with an associated alarm, or via stack convergence triggered by a signal
20:58:20 <stevebaker> skraynev: good question. it might make best sense to use the previous value as the default
20:58:57 <shardy> sdake: If users want the functionality I have no problem with us providing it, but the implementation is not something we want to maintain, IMO
20:59:00 <sdake> if it doesn't belong in heat then where hsould it belong?
20:59:02 <zaneb> stevebaker: that makes no sense to me
20:59:28 <skraynev> stevebaker: hm it's confusing to me
20:59:30 <sdake> ok i've heard the technical implementation is wrong - that can be fixed - i've heard it doesn't belong in heat - would like to understand where it belongs
20:59:35 <zaneb> stevebaker: options are None or the actual default
21:00:15 <skraynev> stevebaker: when you had some list of something  and then update with None - you will not get any changes..
21:00:24 <zaneb> sdake: in Murano, defined as a Mistral workflow on top of your Heat stack
21:00:32 <shardy> sdake: we could make OS::Nova::Server optionally respond to a signal and rebuild, or have a subclass resource which does that
21:00:34 <randallburt> zaneb:  ugh.
21:00:50 <skraynev> time ... is over
21:00:53 <stevebaker> skraynev: good point. I retract!
21:00:57 <stevebaker> #endmeeting