19:02:43 <lifeless> #startmeeting tripleo
19:02:44 <openstack> Meeting started Tue Dec 17 19:02:43 2013 UTC and is due to finish in 60 minutes.  The chair is lifeless. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:02:45 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:02:48 <openstack> The meeting name has been set to 'tripleo'
19:02:51 <lifeless> SpamapS: got your UTC clock on ? :)
19:02:56 <lifeless> #topic agenda
19:03:03 <lifeless> bugs
19:03:04 <lifeless> reviews
19:03:04 <lifeless> Projects needing releases
19:03:04 <lifeless> CD Cloud status
19:03:04 <lifeless> CI virtualized testing progress
19:03:06 <lifeless> Insert one-off agenda items here
19:03:08 <lifeless> moving TripleO UI under Horizon codebase
19:03:11 <lifeless> open discussion
19:03:14 <lifeless> #topic bugs
19:03:16 <lifeless> #link https://bugs.launchpad.net/tripleo/
19:03:16 <lifeless> #link https://bugs.launchpad.net/diskimage-builder/
19:03:18 <lifeless> #link https://bugs.launchpad.net/os-refresh-config
19:03:21 <lifeless> #link https://bugs.launchpad.net/os-apply-config
19:03:23 <lifeless> #link https://bugs.launchpad.net/os-collect-config
19:03:26 <lifeless> #link https://bugs.launchpad.net/tuskar
19:03:28 <lifeless> #link https://bugs.launchpad.net/tuskar-ui
19:03:31 <lifeless> #link https://bugs.launchpad.net/python-tuskarclient
19:04:10 <jtomasek> hi
19:04:24 <lifeless> well done - all bugs triaged now
19:04:32 <lifeless> still one incomplete we haven't chased to ground
19:04:55 <lifeless> criticals
19:05:01 <lifeless> https://bugs.launchpad.net/tripleo/+bug/1254246
19:05:06 <lifeless> https://bugs.launchpad.net/tripleo/+bug/1254555
19:05:11 <lifeless> https://bugs.launchpad.net/tripleo/+bug/1261253
19:05:50 <lifeless> I think we can close the first one, it's fixed in neutron
19:06:04 <rpodolyaka1> the first one should have been fixed by now, but the proposed fix revealed another interesting issue with DB schema migrations in Neutron
19:06:17 <lifeless> oh! it's marked fix committed
19:06:48 <rpodolyaka1> yeah, but this one is needed too https://review.openstack.org/#/c/61677/1
19:06:53 <rpodolyaka1> *to be merged first
19:07:20 <rpodolyaka1> long story short: ML2 migrations have been broken in Neutron for a long time and worked by accident
19:07:26 <lifeless> arghhh
19:07:29 <lifeless> *fun*
19:07:30 <lifeless> ok
19:07:33 <lifeless> so it stays open
19:07:44 <lifeless> the second - can someone agitate about that in Neutron? it's marked 'low'
19:07:52 <lifeless> which I really find a bit bizarre
19:07:57 <jog0> anteaya: ^
19:08:13 <marios> lifeless: i can ping enikanorov__ about it tomorrow
19:08:15 <anteaya> o/
19:08:36 <lifeless> marios: interestingly it says 'symptoms fixed but issue remains'
19:08:58 <lifeless> perhaps we should try removing clints workaround, and if that works, close it in tripleo ?
19:09:10 <anteaya> rpodolyaka1: can you be available after the meeting to discuss this in -neutron?
19:09:24 <rpodolyaka1> anteaya: sure
19:09:40 <anteaya> thanks
19:10:08 <rpodolyaka1> lifeless: marios: enikanorov told me he had fixed only one particular issue leading to problems with policies, though there might be others
19:10:14 <lifeless> rpodolyaka1: ah
19:10:26 <marios> rpodolyaka1: ah k
19:10:30 <lifeless> rpodolyaka1: so should we try removing the bandaid?
19:10:33 <rpodolyaka1> lifeless: though, I agree, we should try to remove workaround
19:10:49 <rpodolyaka1> lifeless:  at least to provide more information of errors we have
19:11:07 <enikanorov> looks like a good plan to me
19:11:09 <lifeless> is there a volunteer here to try that (not you rpodolyaka1 :P) or should we ask on the list?
19:11:40 <marios> sure i can give a go
19:11:44 <enikanorov> matrohon: btw i'm here
19:11:59 <lifeless> #action marios to try removing workaround for bug 1254555
19:12:47 <lifeless> bug 1261253 we can workaround very easily - it's just a matter of manually installing d2to1 into the mirror, and we document other cases of that already in the pypi element; after that we can downgrade the bug to a medium
19:13:17 <lifeless> see bug 1222306 for another example
19:13:27 <lifeless> any volunteers here, or should I ask on the liset?
19:13:55 <marios> o/
19:14:20 <lifeless> #action marios to document workaround in pypi element for bug 1261253
19:14:33 <lifeless> Any other bug business?
19:14:51 <dkehn> sorry for being late
19:14:56 <lifeless> #topic reviews
19:14:59 <lifeless> #link
19:15:00 <lifeless> http://russellbryant.net/openstack-stats/tripleo-openreviews.html
19:15:03 <lifeless> erm
19:15:07 <lifeless> #link http://russellbryant.net/openstack-stats/tripleo-openreviews.html
19:15:42 <lifeless> 
19:15:43 <lifeless> Stats since the last revision without -1 or -2 :
19:15:43 <lifeless> Average wait time: 0 days, 5 hours, 28 minutes
19:15:43 <lifeless> 1rd quartile wait time: 0 days, 2 hours, 31 minutes
19:15:43 <lifeless> Median wait time: 0 days, 3 hours, 42 minutes
19:15:45 <lifeless> 3rd quartile wait time: 0 days, 5 hours, 27 minutes
19:15:49 <lifeless> So, still in good shape. \o/
19:15:53 <marios> nice
19:16:07 <lifeless> Any discussion needed around reviews? People happy with the quality, helpfulness etc that they are receiving?
19:16:37 <dprince> happy
19:16:45 <lsmola_> very happy :-)
19:17:16 <lifeless> ok, cool
19:17:28 <lifeless> #topic projects needing releases
19:17:39 <lifeless> We've landed code -> we need to do a release of projects.
19:17:44 <lifeless> Can I have a volunteer?
19:17:56 <rpodolyaka1> o/
19:17:59 <rpodolyaka1> :)
19:18:06 <lifeless> #action rpodolyaka1 to release all the things
19:18:22 <lifeless> Any discussion points around releases?
19:19:56 <lifeless> #topic CD Cloud status
19:20:08 <lifeless> SpamapS: you here?
19:20:23 <lifeless> (He may be stuck in face to face meetings today)
19:21:13 <lifeless> going to take that as no :)
19:21:14 <lifeless> ok so
19:21:21 <lifeless> we're now back to somewhat reliable
19:21:26 <lifeless> but we found some major
19:21:28 <lifeless> issues
19:21:47 <lifeless> a) we were missing -o pipefail in a number of places, which with tee leads to undetected failures
19:22:08 <lifeless> specifically we were failing to build the noncompute image for weeks, deploying wiht the old one
19:22:29 <lifeless> we found this out when nova broke compat between the noncompute and compute images :)
19:22:47 <lifeless> fixing that lead to a cascade of small fixes that we used the two-reviewers for CD rule to land
19:22:57 <lifeless> as noone else was around,a nd we were down
19:23:10 <lifeless> so - please use -o pipefail :)
19:23:31 <lifeless> b) cinder was basically never working, we have no idea how we ever succeeeded with it included; fixed now.
19:23:55 <lifeless> The preserve-ephemeral patchset is now complete enough to experiment with!
19:24:36 <dprince> lifeless: unless we have a test or something in the overcloud that uses Cinder it could still break at any time right?
19:24:38 <lifeless> See the patch to devtest_overcloud (I0efa8f52864f49ccdb885f6f655c732c951b3f7a) for references.
19:24:50 <lifeless> dprince: less so, but yes.
19:25:07 <lifeless> dprince: we know all the current failures because we fixed the reporting chain so we detected them
19:25:39 * dprince likes checks in multiple places
19:25:40 <lifeless> I'm sure rpodolyaka1 & jog0 & whoever else is hacking on the preserve ephemeral patchset would love folk to try early adoption
19:25:52 <lifeless> I've shoehorned it into the current undercloud
19:26:04 <lifeless> to get real world experience
19:26:09 <rpodolyaka1> test it, break it, review it :)
19:26:13 <jog0> ++
19:26:16 <lifeless> good news - it deploys; bad news - we're not trying the new codepath entirely yet.
19:26:42 <lifeless> Need to land I0efa8f52864f49ccdb885f6f655c732c951b3f7a first
19:27:09 <lifeless> Anything else on the CD cloud status?
19:28:00 <lifeless> #topic virtualised testing
19:28:16 <lifeless> Anyone have news on zis?
19:28:52 <lifeless> dprince: pleia2: ?
19:28:59 <dprince> Well. I've started pushing some things to the incubator to get parity w/ tripleO CI.
19:29:00 <pleia2> I might lose my spot in this conference room in a couple minutes, so I'll be quick
19:29:15 <dprince> Soon I'll rip tripleo CI aport and have it use the devtest scripts.
19:29:16 <pleia2> https://review.openstack.org/#/c/61052/ is the main patch from derekh that I'm reviewing
19:29:19 <pleia2> (and testing)
19:29:59 <dprince> And derekh just about has the test worker stuff in the bag.
19:30:02 <lifeless> ok, cool
19:30:08 <pleia2> I believe derekh was going to look into confirming that the networking that works ok now, still works with the overlay network
19:30:42 <dprince> Specific to our Red Hat test environment we are still working on getting a small set (about 30) public IPs.
19:31:00 <dprince> that is mostly it I think.
19:31:01 <dprince> done.
19:31:10 <pleia2> that's it for me too
19:31:52 <pleia2> dprince: you and derekh available for a sync up tomorrow?
19:32:10 <lifeless> dprince: ok - hey the RH environment should be in the CD cloud section :)
19:32:28 <lifeless> dprince: but thanks!
19:32:43 <lifeless> ##topic moving TripleO UI under Horizon codebase
19:32:45 <lifeless> pleia2: I am!
19:32:53 <pleia2> lifeless: yay
19:32:55 <dprince> lifeless: okay, we can add it
19:33:05 <lifeless> #topic moving TripleO UI under Horizon codebase
19:33:07 <lsmola_> ok
19:33:11 <dprince> pleia2/lifeless: I'll send out an invite for the normal time.
19:33:15 <lifeless> dprince: thanks
19:33:19 <lifeless> lsmola_: tag:)
19:33:29 <pleia2> dprince: great
19:33:39 <lsmola_> so i have sent email with 2 plans, merging under Horizon program and merging directly to horizon codebase
19:34:23 <jomara> we will know more after today's meeting
19:34:26 <lifeless> cool
19:34:28 <lsmola_> we are leaning to merging directly to the codebase, although we have some conditions and development can be slightly slower
19:34:31 <lifeless> is there anything you need from us here?
19:34:39 <lifeless> or any concerns you want to talk about?
19:34:43 <lsmola_> though it will be all done properly upstream :-)
19:34:56 <jcoufal-mob> I don't think there are triplo related concerns
19:35:05 <lsmola_> not really I think, it will all be discussed on Horizon meeting today
19:35:12 <jcoufal-mob> Horizon meeting is more important today :)
19:35:17 <jcoufal-mob> Yup
19:35:23 <lsmola_> :-)
19:35:30 <lifeless> ok
19:35:48 <lifeless> Now I see there's a new item on the wiki page
19:35:56 <lifeless> lsmola_: please put them in the list at the top as well :)
19:36:01 <lifeless> #topic After heat stack-create init operations (lsmola)
19:36:06 <lsmola_> ok
19:36:09 <lifeless> "Regarding this discussion http://www.mail-archive.com/openstack-dev@lists.openstack.org/msg11671.html Does this initialization belongs to Heat template or to Tuskar-API? shardy says: but it is possible to update the configuration subsequently using cfn-hup, or os-collect-config/os-apply-config, which read updated resource Metadata and apply it "
19:36:15 <lsmola_> I wasnn't sure
19:36:22 <lifeless> dunno if that all came through - there should be a trailing " if it did
19:37:13 <lsmola_> so
19:37:36 <lsmola_> basically tuskar was calling these init scripts after the stack-create
19:37:56 <lsmola_> though I am not sure if that is the right approach
19:38:06 <lifeless> Ok
19:38:24 <lsmola_> It's an initialization step and it should be part of stack-create
19:38:27 <lifeless> so this is probably something to tease out on the list - I mean, I can say why its the way it is today for the CLI
19:39:13 <lsmola_> so are there any concerns with packing it to Heat or occ and oac?
19:39:14 <lifeless> we have a principle in the design that one-time things - basically API calls - should be externally orchestrated, not done by local machine scripts
19:39:26 <lifeless> all of this setup stuff is in that category.
19:39:41 <lifeless> Consider deployment of an HA setup. Which machine - and only one can do it - should run pki_setup
19:39:58 <lifeless> Ditto initial neutron setup
19:40:03 <lifeless> and keystone registrations
19:40:44 <lsmola_> hm
19:40:59 <lsmola_> not sure if i am right, but we will be using different heat templates for HA setup right?
19:41:29 <lifeless> Not sure at this point
19:41:32 <lifeless> I'd like to avoid that
19:41:42 <lifeless> just merge in with a count for the control plane of N != 1
19:41:50 <lsmola_> hm
19:41:56 <lifeless> Remember that we build the undercloud by doing one node and handing over then scaling up
19:42:12 <lsmola_> because you have to define also resources like load balancers, and thing like that in Heat template right?
19:42:25 <lifeless> for a given stack, yeah
19:42:38 <lifeless> We can divide these tasks into two categories
19:42:44 <lifeless> there are things that we /should/ be automating
19:42:45 <lsmola_> so it can be kind of hard to define all option to one template
19:42:50 <lifeless> like endpoint registration
19:42:58 <lsmola_> but rather have multiple tested templates with different setup
19:43:14 <lifeless> by which I mean that if an endpoint moves it needs to be re-registered
19:43:33 <lsmola_> right
19:43:44 <lifeless> but even thats not 100% clear - with VIPs you register once and any subsequent registration is deliberate and orthogonal to redeployment stuff
19:43:54 <lifeless> the other category is human tweaking
19:43:56 <lifeless> like network setup
19:44:29 <lifeless> until we have baremetal neutron to give outbound automatic policy stuff, the public network setup is entirely a matter of driving the Neutron API in the overcloud once
19:44:41 <lifeless> which users can do via the Admin tab of the deployed overcloud
19:45:09 <lifeless> Either way, none of these things need to be done on changes to the cloud, it's just initial bringup
19:45:31 <lsmola_> hm
19:46:01 <lsmola_> so i take it there is not place in heat templates, that is good for this init scripts?
19:46:14 <lifeless> heat doesn't know how to orchestrate APIs that it's deploying
19:46:23 <lifeless> only how to orchestrate APIs that provide it with resources
19:46:39 <lifeless> the 'software config' work that is ongoing is tangentially related, but not the same.
19:47:03 <lifeless> If we could get rid of the SSH for keystone, it would be all API all the time
19:47:17 <lsmola_> hmm ok
19:47:18 <lifeless> and then we could focus on addressing that in Heat
19:47:30 <lsmola_> we will put it separately then
19:47:34 <lifeless> anyhow - lets say: it is an issue, it should be fixed, but how isn't clear; -> the list
19:47:41 <lifeless> IMNSHO :)
19:47:48 <lsmola_> an we will discuss with heat guys, how to make this happen
19:48:01 <lifeless> Well, also a broader tripleo discussion
19:48:07 <lsmola_> lifeless: ok, cool
19:48:09 <lifeless> meetings aren't a good place to get everyones thoughts
19:48:15 <lifeless> like - this has been just you and me :)
19:48:25 <lifeless> no SpamapS, no ng, ...
19:48:26 <lsmola_> hehe, ok, point taken :-)
19:48:45 <lifeless> #topic open discussion
19:49:13 <lifeless> 12 minutes, get it while it's hot
19:50:10 <dkehn> cue crickets
19:50:59 <lifeless> and
19:51:01 <lifeless> #endmeeting