20:01:05 <zaneb> #startmeeting heat
20:01:05 <openstack> Meeting started Wed Jul 15 20:01:05 2015 UTC and is due to finish in 60 minutes.  The chair is zaneb. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:01:07 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:01:10 <openstack> The meeting name has been set to 'heat'
20:01:34 <zaneb> #topic roll call
20:01:40 <zaneb> I am here
20:01:42 <randallburt> o/
20:01:46 <pas-ha> o/
20:02:18 <Kairat> o/
20:02:47 <zaneb> fyi stevebaker is away this week, so I drew the short straw
20:03:29 <pas-ha> is it rly only 4 of us here? o_O
20:03:31 <skraynev> o/
20:03:43 <zaneb> I know ryansb is here because I spoke to him like 4 minutes ago
20:04:04 <zaneb> well, let's get started
20:04:11 <zaneb> #topic Adding items to agenda
20:04:17 <zaneb> #link https://wiki.openstack.org/wiki/Meetings/HeatAgenda#Agenda_.282015-07-015_2000_UTC.29
20:04:27 <zaneb> nice of somebody to create an agenda
20:04:43 * zaneb hasn't read scrollback from last week's 3am meeting yet
20:04:54 <zaneb> any other topics?
20:05:15 <randallburt> nope. looks like a short one today...
20:05:28 <zaneb> my favourite kind :)
20:05:29 <pas-ha> not from me, thanks everyone who's answered on my ML yesterday
20:05:43 <zaneb> #topic important reviews
20:05:45 <randallburt> agreed
20:05:54 <zaneb> #link https://etherpad.openstack.org/p/heat-reviews
20:06:40 <randallburt> designate stuff merged so I took them off the list
20:06:57 <zaneb> cool
20:07:11 <zaneb> what do we normally do in this part of the meeting?
20:07:25 <ryansb> talk about important reviews
20:07:39 <zaneb> exhort everyone to review harder?
20:07:51 <ryansb> or beg them to
20:07:53 <randallburt> yep
20:08:06 <ryansb> exhort, beg, wheedle, encourage, etc
20:08:39 <randallburt> so many red exes on the convergence patches
20:09:07 <Kairat> I have 2 questions that I wonder this week
20:09:25 <zaneb> randallburt: they probably all need a rebase after the mock 1.1 funness
20:09:27 <Kairat> The first is related to convergence
20:10:02 <randallburt> zaneb:  yeah. someone needs a swift kick for that one
20:10:12 <Kairat> On scale we faced withe following problem: when creating a big stack heat is ddosing nova
20:10:49 <zaneb> Kairat: more than it was with non-convergence?
20:10:50 <Kairat> big = a lot of instances in one stack
20:10:50 <randallburt> Kairat:  so that we can keep on topic of the patches for the moment: there's a lot of green on https://review.openstack.org/#/c/153235/. Any reason its still not approved?
20:11:05 <Kairat> yep, it is not convergence
20:11:24 <Kairat> I know we are developing some batching for resource groups
20:11:41 <skraynev> randallburt: I personally don't know ;)
20:12:30 <zaneb> looks like most of the priority reviews are specs...
20:12:46 <zaneb> does that mean nobody is having trouble getting their code reviewed?
20:12:56 <zaneb> or everyone is blocked on specs? :D
20:13:04 <skraynev> zaneb: I have added one about deprecation changes (implementation Hidden status)
20:13:19 <randallburt> skraynev:  its approved now
20:13:33 <skraynev> zaneb: everyone scares to do something with convergence without you :)
20:13:45 <zaneb> :(
20:14:13 <skraynev> randallburt: thank you ! :)
20:14:25 <skraynev> zaneb: it';s just my guess ;)
20:14:28 <randallburt> skraynev:  np
20:14:51 <zaneb> ok, let's move on. Kairat, we'll get to you topic in open discussion
20:15:02 <zaneb> #topic high-priority bugs
20:15:16 <Kairat> zaneb, ok, thanks
20:15:24 <zaneb> #link http://bit.ly/1FnhaIK
20:15:33 <skraynev> zaneb: note, when I copy-pasted agenda - I didn't check link (I hope it works)
20:15:44 <zaneb> worked for me
20:16:07 <skraynev> fuh. it's good news for me :)
20:17:42 <skraynev> https://bugs.launchpad.net/heat/+bug/1301320 should we changes some statuses for this bug ?
20:17:43 <openstack> Launchpad bug 1301320 in heat "There are no accessibilities for the backup_stack" [High,Triaged]
20:17:46 <zaneb> how many of these bugs are *actually* in progress I wonder
20:18:06 <skraynev> or it's enough to have tag: fixed-by-convergence ?
20:18:22 <zaneb> I think that's probably enough
20:18:30 <zaneb> we could drop it back to Medium
20:18:43 <zaneb> since realistically we will never fix this except with convergence
20:21:10 <skraynev> zaneb: got it
20:21:35 <pas-ha> I even wonder why "we do not have API to access backup stack"? access - yes (just append *), manipulation - no :)
20:22:53 <zaneb> looks like the fix for https://bugs.launchpad.net/heat/+bug/1465204 has actually merged
20:22:53 <openstack> Launchpad bug 1465204 in heat "convergence: resources not using the correct template on update" [High,Fix committed] - Assigned to Anant Patil (ananta)
20:24:07 <zaneb> pas-ha: the backup stack doesn't look much like any stack a user might recognise. I don't think making it visible through the standard API would be a Good Thing
20:24:38 <pas-ha> AFAIR it is visible - stack-show <stackname>*
20:24:50 <pas-ha> if I'm not mistaken
20:25:31 <Kairat> just because backup_stack stored in DB
20:25:49 <pas-ha> btw re bug https://bugs.launchpad.net/heat/+bug/1409814 - do we care for Icehouse any more (since the branch is no longer there..)
20:25:49 <openstack> Launchpad bug 1409814 in heat "unit tests fail on stable/icehouse: "AttributeError: 'module' object has no attribute 'neutronclient'"" [High,Triaged] - Assigned to Pradeep Kumar Singh (pradeep-singh-u)
20:25:52 <zaneb> how did https://bugs.launchpad.net/heat/+bug/1468916 get moved to in progress? I don't see any patch for it...
20:25:52 <openstack> Launchpad bug 1468916 in heat "stack arn is too big for shorter ceilometer resource_id column" [High,In progress] - Assigned to Kairat Kushaev (kkushaev)
20:27:01 <Kairat> zaneb, https://review.openstack.org/#/c/196123/
20:27:28 <zaneb> Kairat: thanks, added as a comment
20:28:17 <zaneb> pas-ha: in theory we don't support Icehouse any more, so we can deep six that one
20:29:42 <pas-ha> yes, we even have nowhere to merge any potential fix already :)
20:30:13 <skraynev> pas-ha: need to restore new branch ? ;)
20:30:24 <skraynev> s/new/old :)
20:30:35 <pas-ha> yay, Infra would be super-happy :)
20:31:02 <zaneb> pas-ha: it never affected master anyway, so I deleted Heat from affects (so it's icehouse only now)
20:31:33 <zaneb> does the branch really go away altogether?
20:31:38 * zaneb did not know that
20:32:13 <pas-ha> branch - yes, tags do live
20:32:27 <zaneb> interesting
20:32:34 <zaneb> ok, let's move on
20:32:40 <zaneb> we got one off the list at least :)
20:32:51 <zaneb> #topic Show attribute base patch
20:32:58 <zaneb> #link https://review.openstack.org/#/c/195609/
20:33:06 <zaneb> skraynev: I assume you added this topic :)
20:33:15 <skraynev> zaneb: right :)
20:33:31 <skraynev> just want to ask people to review it.
20:33:32 <zaneb> you have the floor
20:34:16 <skraynev> need more feedback about two base patches on review :)
20:34:28 <zaneb> added it to the etherpad
20:34:38 <skraynev> zaneb: thx
20:35:49 <skraynev> Unfortunately we can not use some auto-generic method now (it's the reason, why I should add _show resource function to each resource)
20:36:06 <zaneb> saw one thing I can comment on :)
20:36:31 <skraynev> maybe I am wrong, but there are a lot of clients with their custom logic (on show)
20:36:38 <zaneb> does that call the whole idea into question?
20:37:22 <skraynev> zaneb: not sure, I want to upload one cross-project spec with changing all client :)
20:37:37 <skraynev> I just afraid, that it takes more time, then current approach
20:37:54 <skraynev> currently it looks expensive only for us :)
20:37:59 <zaneb> ah right, and then after that we could have a generic method?
20:38:22 <skraynev> zaneb: correct, but IMO it may takes more time.
20:38:42 * zaneb is personally ok with that
20:39:19 <zaneb> the resource plugin interface is a public API we need to maintain backwards compat on
20:39:25 <skraynev> however last comment from Divakar give me a good idea about using get... for most part of resources.
20:39:32 <zaneb> so I'd rather be conservative about what we add to it
20:40:17 <skraynev> zaneb: I think, that common _show_resource method is not bad.
20:40:45 <skraynev> I just will re-write it for some "special" resources :)
20:41:01 <zaneb> "special" :)
20:41:25 <skraynev> zaneb: who don't want to make normal utils in clients :)
20:41:26 <zaneb> ok, shall we move on to the next topic?
20:41:30 <skraynev> yes
20:41:39 <zaneb> #topic batching of large templates
20:41:44 <zaneb> Kairat: go
20:41:45 <pas-ha> skraynev, seems you would also have to add a class attribute "entity" like client.<entity>.get()
20:42:33 <Kairat> So we facing with problems on scale when launching large stacks (without convergence)
20:42:53 <Kairat> large  = 60 inst and more
20:43:13 * pas-ha notes - as copy-pasted verbatim into template
20:43:14 <skraynev> pas-ha: make sense (suggest to discuss it tomorrow ;) )
20:43:48 <Kairat> It turns out that we are ddosing nova (and perhaps other services) and it cannot manage a lot of get requests
20:44:08 <zaneb> which is frightening in itself :D
20:44:17 <pas-ha> actually what we found is that stack validation is the most troublesome part - it is synchronous
20:44:22 <Kairat> I am interesting how we can manage such cases in convergence
20:44:39 <Kairat> because heat will be faster
20:44:46 <Kairat> with the new architecture
20:45:00 <zaneb> actually, I think Heat will be slower
20:45:10 <skraynev> pas-ha: i did not check it for master honestly (only for juno)
20:45:14 <pas-ha> since validation can make "lots" of API calls to e.g. Glance
20:45:29 <Kairat> pas-ha, cashing can help us
20:45:47 <skraynev> Kairat, pas-ha: with validation constraints - yes
20:46:16 <Kairat> zaneb, even if it will be slower the problem stays
20:46:29 <zaneb> Kairat: agreed
20:46:30 <pas-ha> skraynev, it is synchronous in master https://github.com/openstack/heat/blob/master/heat/engine/service.py#L707-L710
20:46:55 <zaneb> convergence phase 1 doesn't really make a big difference either way to this
20:46:57 <pas-ha> Kairat, no, slower means more delays between API calls, less ddosing
20:47:16 <skraynev> pas-ha: I though, that we skipped it there (for master)...
20:47:38 <skraynev> pas-ha: but may be I mixed it with some patch on review
20:47:42 <zaneb> pas-ha: that might get from 60 to like 65
20:47:42 <pas-ha> no, only for non-reslovable right-now references
20:48:42 <zaneb> I wish we had some way of typing parameters
20:49:10 <pas-ha> so, as I was saying, for large stack validation may take too long, and rpc call from api to create simply times out
20:49:13 <skraynev> zaneb: and all of these requests will happen in one moment ? or one by one with some delay ?
20:49:15 <zaneb> so that e.g. once we had validated something as a glance.image it would have that type and always be accepted as such
20:49:43 <zaneb> skraynev: depends on the template
20:49:57 <skraynev> pas-ha: correct, but I suppose (caching should solve it) or you want suggest something else ? :)
20:50:29 <zaneb> pas-ha: that's interesting. ryansb - that's probably why we had to increase the timeout in TripleO
20:50:36 <pas-ha> btw, in resource-group + environment, how many times the e.g image parameter of the inner template will get validated?
20:50:39 <skraynev> zaneb: in worst case - 100+ OS::Nova::Servers without resource group
20:50:50 <Kairat> pas-ha, we don't know if nova will work when creating stack and polling nova
20:51:20 <Kairat> pas-ha, IMO, even if validation will pass we might face with some problems when polling
20:51:23 <zaneb> skraynev: I was saying that any slow-down in Heat from convergence would have a very marginal impact on the rate of requests to Nova
20:51:44 <pas-ha> zaneb, understood
20:52:16 <zaneb> so one thing we have talked about is having batching on create (not just update) for scaling groups
20:52:25 <skraynev> zaneb: hm. so potential issue will be still there :)
20:52:33 <zaneb> though obviously that would only help here if y'all were using a scaling group
20:52:51 <Kairat> we have some specs for resource group
20:53:03 <Kairat> * batching for resource group
20:53:03 <pas-ha> IMO everyone creating 100+vms should use at lease resource group :)
20:53:22 <zaneb> Kairat: ramishra is working on a patch for that I know
20:54:09 <skraynev> pas-ha: agree, however I am not sure, that we can forbid use copy-pasted code :(
20:54:21 <zaneb> Kairat: https://review.openstack.org/#/c/194052/15
20:54:57 <zaneb> skraynev: we can, by timing out before it finished validation ;)
20:55:01 <chuprykov> zaneb: isn't this just refactoring?
20:55:39 <zaneb> chuprykov: no. I think the refactoring comes later in the series
20:56:23 <Kairat> pas-ha, i am wondering if many users will create stacks in parallel
20:56:25 <chuprykov> zaneb: i have spec on batching https://review.openstack.org/#/c/183506/
20:56:43 <skraynev> zaneb: yes :) small timeout makes deploying huge stack  painful :)
20:56:51 <pas-ha> Kairat, that's also surely true
20:58:00 <chuprykov> zaneb: dont seems that ramishra's patch add batching on create
20:58:11 <zaneb> not on create, no
20:58:35 <zaneb> that would be a separate change required for both AutoscalingGroup and ResourceGroup
20:58:40 <chuprykov> just usual rolling_update
20:58:57 <Kairat> So it seems the problem is still here and it is quite complex)
20:59:02 <zaneb> Kairat: we have 2 minutes. did you have another topic?
20:59:09 <zaneb> 1 minute
20:59:12 <pas-ha> not so short meeting after all :)
20:59:19 <Kairat> Nope
20:59:28 <Kairat> That's all from me
20:59:30 <skraynev> pas-ha: we break expectations :)
20:59:42 <zaneb> ok, let's wrap up and decamp to #heat
20:59:48 <zaneb> thanks everyone!
20:59:51 <Kairat> Thanks
20:59:58 <zaneb> #endmeeting