Tuesday, 2016-04-05

*** akshai has quit IRC00:01
*** shakamunyi has joined #tripleo00:03
*** shakamunyi has quit IRC00:09
*** rhallisey has quit IRC00:14
*** bvandenh has quit IRC00:25
*** bvandenh has joined #tripleo00:25
*** shakamunyi has joined #tripleo00:31
*** thrash is now known as thrash|g0ne00:33
*** julim has quit IRC00:43
*** shakamunyi has quit IRC00:47
*** dmacpher has quit IRC00:53
*** tiswanso has joined #tripleo00:59
*** tiswanso has quit IRC00:59
*** tiswanso has joined #tripleo01:00
*** shakamunyi has joined #tripleo01:04
*** yamahata has joined #tripleo01:05
*** dtrainor has quit IRC01:41
*** akshai_ has quit IRC02:07
*** Marga_ has quit IRC02:11
*** apetrich has quit IRC02:43
*** ramishra has joined #tripleo02:53
*** coolsvap has quit IRC03:01
*** Marga_ has joined #tripleo03:01
*** Marga_ has quit IRC03:06
*** saneax_AFK is now known as saneax03:08
*** Marga_ has joined #tripleo03:19
*** coolsvap has joined #tripleo03:24
*** coolsvap_ has joined #tripleo03:25
*** coolsvap_ has quit IRC03:25
*** tiswanso has quit IRC03:38
*** links has joined #tripleo03:39
*** ccamacho has quit IRC03:41
*** ccamacho has joined #tripleo03:44
*** r-mibu has quit IRC03:49
*** cwolferh has quit IRC03:50
*** masco has joined #tripleo04:02
*** akuznetsov has joined #tripleo04:09
*** ibravo has quit IRC04:10
*** akuznetsov has quit IRC04:13
*** cwolferh has joined #tripleo04:14
*** r-mibu has joined #tripleo04:29
*** tzumainn has quit IRC04:30
*** ramishra has quit IRC04:52
*** ramishra has joined #tripleo04:57
*** numans has joined #tripleo05:09
*** ramishra has quit IRC05:11
*** numans has quit IRC05:22
*** numans has joined #tripleo05:36
*** jcoufal has joined #tripleo05:45
*** rcernin has joined #tripleo05:51
*** oshvartz has joined #tripleo05:56
*** cwolferh has quit IRC05:57
*** bvandenh has quit IRC06:09
*** leanderthal|afk is now known as leanderthal06:22
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Tempest for tripleo CI  https://review.openstack.org/29584406:23
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Run tempest tests with periodic jobs  https://review.openstack.org/29703806:23
*** liverpooler has joined #tripleo06:24
*** mikelk has joined #tripleo06:24
*** mcornea has joined #tripleo06:25
*** liverpooler has quit IRC06:26
*** liverpooler has joined #tripleo06:26
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Tempest for tripleo CI  https://review.openstack.org/29584406:27
*** rlandy has quit IRC06:29
*** yamahata has quit IRC06:33
*** shivrao has quit IRC06:34
*** rdopiera has joined #tripleo06:51
*** ramishra has joined #tripleo06:52
*** rdopiera has quit IRC06:56
*** cwolferh has joined #tripleo06:58
*** shivrao has joined #tripleo06:59
*** rdopiera has joined #tripleo07:10
*** zoli|gone is now known as zoliXXL07:11
zoliXXLgood morning07:12
*** florianf has joined #tripleo07:12
*** aufi has joined #tripleo07:19
*** jaosorior has joined #tripleo07:21
*** apetrich has joined #tripleo07:23
*** jpena|off is now known as jpena07:24
*** apetrich has quit IRC07:25
*** apetrich has joined #tripleo07:26
*** qasims has joined #tripleo07:26
*** Marga_ has quit IRC07:29
*** dmacpher has joined #tripleo07:34
*** pcaruana has joined #tripleo07:38
*** shivrao_ has joined #tripleo07:38
*** shivrao has quit IRC07:40
*** shivrao_ is now known as shivrao07:40
*** ifarkas has joined #tripleo07:44
*** jistr has joined #tripleo07:44
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Remove individual service certificates  https://review.openstack.org/30149307:44
*** rasca has quit IRC07:46
*** jaosorior has quit IRC07:46
*** jaosorior has joined #tripleo07:47
*** rasca has joined #tripleo07:48
*** shardy has joined #tripleo07:48
*** dtantsur|afk is now known as dtantsur07:49
*** cwolferh has quit IRC07:51
*** paramite has joined #tripleo07:53
*** mbound has joined #tripleo07:54
*** ohamada has joined #tripleo07:54
*** sshnaidm has quit IRC07:55
*** mbound has quit IRC08:00
*** jistr has quit IRC08:01
*** jistr has joined #tripleo08:01
*** mbound has joined #tripleo08:06
*** gfidente has joined #tripleo08:08
*** gfidente has quit IRC08:08
*** gfidente has joined #tripleo08:08
*** shivrao has quit IRC08:10
*** jprovazn has joined #tripleo08:16
jaosoriormarios: Hey dude, when you have time can you check this commit out? https://review.openstack.org/#/c/299475/ It's a bit harder to digest :/08:18
mariosjaosorior: o/ willdo08:21
openstackgerritMerged openstack-infra/tripleo-ci: Add scripts to generate tripleo.org website  https://review.openstack.org/30045208:22
*** ebalduf has quit IRC08:23
*** panda has quit IRC08:26
jistrhave we seen the IPv6 job pass on stable/liberty yet?08:26
*** panda has joined #tripleo08:26
*** akrivoka has joined #tripleo08:27
jistri'm asking because maybe if we want to merge to s/l and we have non-ha + ha passed, we should merge maybe?08:27
jistryeah i don't see a single /liberty green on -upgrades job in cistatus http://tripleo.org/cistatus.html08:28
jistrmarios, gfidente: so should we merge this? https://review.openstack.org/#/c/301168/08:28
jistri don't think the recheck is going to do it08:29
jistri mean it's not going to be greener08:29
*** Marga_ has joined #tripleo08:29
* jistr checks what's the scoop with IPv6 jobs on liberty08:29
mariosjistr: not sure what the status of that upgrades job is gfidente will know more08:29
*** sshnaidm has joined #tripleo08:30
jistrcurrently i think upgrade doesn't test upgrades, it tests IPv6, right?08:30
mariosjistr: nice having time to do reviews and such stuff isn't it :) /me bites tongue multiple times08:30
jistrhehe :)08:31
jistrwell, we have a few things we need to get backported, so i'm looking mainly how to get us there08:31
jistrnot sure if rechecks will cut it if we're talking about -upgrades job08:31
rdopierahi there, I wonder if I could help with anything?08:31
mariosrdopiera: sheep man o/ long time stranger08:32
jistrrdopiera: hi!08:32
rdopierayeah, I was lost in the nova land08:33
mariosrdopiera: upstream meeting is today if you want https://wiki.openstack.org/wiki/Meetings/TripleO08:33
jistrand our main problem is unstable CI i think08:34
jistrso any debugging on that front is always helpful08:34
*** Marga_ has quit IRC08:34
mariosjistr: speaking of which the upgrades job on that review is failing on an attribute error http://logs.openstack.org/68/301168/1/check-tripleo/gate-tripleo-ci-f22-upgrades/48e1a0c/console.html#_2016-04-04_22_12_55_99908:34
jistrmarios: so it *does* try to do overcloud update http://logs.openstack.org/68/301168/1/check-tripleo/gate-tripleo-ci-f22-upgrades/48e1a0c/console.html#_2016-04-04_22_12_55_99908:35
rdopieraunstable ci, why does it always have to be unstable ci08:35
jistrok that's not going to help on recheck. my 2 cents is that we should just merge to s/l without upgrades job green, until we see it pass for the first time08:35
*** chlong has quit IRC08:35
jistrgfidente, marios ^^08:35
mariosjistr: yeah, i mean, really we should make it non-voting until it does pass.08:38
mariosjistr: we spoke briefly with gfidente about it last week so i know he is/was working on it but don't know what else he got pulled into etc08:39
*** derekh has joined #tripleo08:39
jistryea... i saw it passing on master once or twice08:39
jistrbut never on liberty yet08:39
*** electrofelix has joined #tripleo08:40
*** dmacpher has quit IRC08:41
gfidentejistr marios I think we can merge the undercloud fix for now08:41
gfidenteI will be investigating the failures in the upgrades job08:41
jistrgfidente: +108:41
gfidentelooks like the ha passing rate is also very low08:41
jistri'd like to get the backport going, and this is a low risk change, esp. given that we've seen the other jobs pass08:41
gfidenteso looks like it's time for some ci love08:41
gfidentejistr yeah it's also restoring what we used to do before anyway08:42
openstackgerritMerged openstack/tripleo-heat-templates: PLUMgrid Neutron integration  https://review.openstack.org/29915108:42
jistrgfidente: yea08:42
shardymarios: Hey, I didn't see any objections on the ML re the exception so I landed ^^08:44
marioso/ gfidente08:44
shardythe liberty backport looks fine too other than the upgrades job timing out08:44
mariosshardy: yeah makes sense i wrote along the same lines revisited yesterday08:44
shardyso I guess we just have to wait on that passing08:44
shardymarios: cool, I was hoping for a more affirmative +1 re the exception, but I guess no objections will have to do ;)08:44
mariosshardy: well, i one question/issue was 'who else would need an exemption' wrt integrations work... we can take that to mean no-one will feel hard done by cos they didn't get their thing landed into liberty08:46
gfidenteshardy on the upgrade job08:46
gfidentehave any idea why this08:47
gfidenteis happening only on the update attempt?08:47
shardymarios: Yeah, it's tricky but my take is folks had an entire cycle to get things landed into liberty08:47
openstackgerritMerged openstack/instack-undercloud: Set mariadb open_files_limit to -1 (unlimited)  https://review.openstack.org/30116808:47
shardyand we have to draw the line due to the mitaka backport policy08:47
gfidenteI checked the deploy_args / update_args and they are identical except for the -puppet-registry.yaml which is only given on the update attempt but it's the first environment passed08:47
shardywe can retain the ability to vote on any exceptional backports, but I'm thinking they will become increasingly less likely when we get beyond the Mitaka GA08:48
*** dtantsur is now known as dtantsur|brb08:49
*** florianf has quit IRC08:50
shardygfidente: Hmm, yeah that looks wrong - have you managed to reproduce locally?08:52
shardyit should just be reading an output from OS::TripleO::Services08:52
*** florianf has joined #tripleo08:53
openstackgerritCarlos Camacho proposed openstack-infra/tripleo-ci: Update deprecated commands in tripleo.sh for --overcloud_delete  https://review.openstack.org/30108708:54
gfidenteshardy no I don't see it locally08:54
openstackgerritCarlos Camacho proposed openstack-infra/tripleo-ci: Update deprecated commands in tripleo.sh for --overcloud_delete  https://review.openstack.org/30108708:55
*** Marga_ has joined #tripleo08:56
shardythat's the undercloud heat error - I think we reverted a patch last week which fixes it, sec08:57
openstackgerritJiri Stransky proposed openstack/instack-undercloud: Set mariadb open_files_limit to -1 (unlimited)  https://review.openstack.org/30151808:57
shardygfidente: I suspect we're pinned to a heat version that doesn't have that revert ^^08:59
shardySo unless the current pin has been recently promoted, I guess we'll need a tmprevert09:00
openstackgerritJiri Stransky proposed openstack/instack-undercloud: Set mariadb open_files_limit to -1 (unlimited)  https://review.openstack.org/30152109:00
*** ramishra has quit IRC09:03
ccamachojistr, good morning! quick question, after thinking about how to update the commands for Heat in tripleo.sh without duplicating to much code... and adding backwards compatibility with Liberty... I have created a small snippet to redefine the openstack command, do you think is a good idea to do it in this way?09:04
ccamachoIm testing it with different combinations...09:05
shardyccamacho: Morning!09:05
ccamachoshardy good morning!09:05
shardyccamacho: The heat cli commands are deprecated for Mitaka, but I don't think it's urgent to switch to the openstack CLI09:05
*** ramishra has joined #tripleo09:05
shardyccamacho: do you have a patch up already?09:05
ccamachoshardy, yeahp, this one: https://review.openstack.org/#/c/301087/09:06
shardyccamacho: Looks OK, but note re the TODO, we can't remove it after mitaka releases because tripleo.sh is now in the tripleo-ci repo, which needs to support all branches (not just master)09:10
shardyI suspect we'll support the old heat CLI for longer than we'll support e.g stable/liberty in TripleO, so the other option is just to delay this change until we EOL liberty09:11
shardygetting coverage of the new CLI interface is probably a good thing tho :)09:11
*** Marga_ has quit IRC09:12
ccamachoshardy, I saw it interesting as In my case I'm facing with to many tools doing the same and there is already code using the openstack CLI in tripleo.sh09:14
shardyYeah, it could be argued we should have an openstack overcloud delete CLI as it's more consistent09:16
*** mkovacik has joined #tripleo09:16
shardybut then we'll have *another* way to delete the stack ;)09:16
openstackgerritCarlos Camacho proposed openstack-infra/tripleo-ci: Update deprecated commands in tripleo.sh for --overcloud_delete  https://review.openstack.org/30108709:17
openstackgerritMerged openstack/tripleo-heat-templates: Enable galera replication for Mariadb 10.1  https://review.openstack.org/30070609:18
*** ramishra has quit IRC09:20
*** ramishra has joined #tripleo09:22
openstackgerritCarlos Camacho proposed openstack-infra/tripleo-ci: Update deprecated commands in tripleo.sh for --overcloud_delete  https://review.openstack.org/30108709:22
*** florianf has quit IRC09:26
*** sambetts|afk is now known as sambetts09:34
openstackgerritsandhya proposed openstack/diskimage-builder: Add support for building images capable of UEFI  https://review.openstack.org/28778409:40
*** florianf has joined #tripleo09:41
jaosoriorshardy, marios, gfidente: Taking another thought at the skipping of the post-deployment https://review.openstack.org/#/c/250268 now that I think about it. It also generates the services passwords file... So when we switch to using puppet for the keystone endpoints... wouldn't we instead want an option that only skips that part? and not the password file generation for the post-deployment09:43
*** akuznetsov has joined #tripleo09:44
*** coolsvap has quit IRC09:45
openstackgerritsandhya proposed openstack/diskimage-builder: Add support for building images capable of UEFI  https://review.openstack.org/28778409:46
shardyjaosorior: are you sure - looking at the code we create the passwords in _update_parameters?09:46
shardywhich makes sense because you need them before starting the deployment09:46
shardyit's true that in a future split-stack model that also may become optional09:47
shardyjaosorior: it seems we've got some gross global stuff which should mean e.g OVERCLOUD_ADMIN_PASSWORD matches the parameters09:49
shardyhopefully we can kill that when we move to puppet configuring keystone09:49
jaosoriorshardy: nevermind. I got a bit confused by the fact that that dict is used by os-cloud-config09:49
jaosoriorso what happens is that the service users are created there, and use those passwords passed by os-cloud-config09:49
jaosoriorbut that can hopefully be removed by passing it all to puppet09:50
jaosoriorno problem09:50
shardycool - yeah stuff like the utils.get_password in _keystone_init is confusion09:50
shardyconfusing even09:50
jaosorioryeah dude09:50
jaosoriorshardy: We really need to get rid of that crap X_x09:50
*** coolsvap has joined #tripleo09:57
*** rasca has quit IRC10:00
shardyderekh: Hey, what's the current status of current-tripleo promotion, is it automated via the periodic job now?10:00
openstackgerritMerged openstack/instack-undercloud: Delete default flavors in Python  https://review.openstack.org/28623310:01
number80sorry, irc client going nuts10:01
derekhshardy: it will be automated once this is merged https://review.openstack.org/#/c/296363/10:01
*** mgould has joined #tripleo10:01
shardyderekh: Ok, looking10:02
shardyit seems we're pinned to a 2-week-old heat commit which lacks a revert that we need10:02
derekhshardy: but trunk hasn't passed for about 2 weeks, that has to be figured out befor an actual promote happens10:02
shardyderekh: Ok - think we need this before the upgrades job will work:10:03
*** rasca has joined #tripleo10:03
shardyref the issue gfidente highlighted earlier10:03
derekhshardy: ok, I havn't looked at why trunk isn't working but we wont get a promote until it does10:04
shardyderekh: OK, I'll propose a tmprevert in the meantime, thanks!10:05
derekhshardy: a temprevert in heat ? not sure that will work, checking10:06
shardyderekh: can we not do tmpreverts in tripleo-ci anymore?10:07
derekhshardy: we havn't been able to in a long time(except for puppet modules), as we should no longer need to because we've pinned the packages and only move pins to a working repo10:09
shardyderekh: ack, yeah I just realized that :(10:09
shardySo we have to fix master and promote, or temporarily force heat to build from source10:10
shardyI incorrectly assumed we still had code that could automagically flip DIB_INSTALLTYPE and build a delorean package10:11
derekhshardy: Yup, either fix trunk and promote or we could add heat to the list of whitelisted packages so its pulled from /current10:11
shardytesting with it whitelisted would be a good plan anyway I guess, let me try that10:11
derekhshardy: but that will leave us open to heat regressions, maybe this is ok as a temporary solution but its only putting off the inevitable of fixing trunk10:12
derekhshardy: yup, worth testing out at least10:12
shardyYeah, even if we don't merge it it'll at least give us some data re fixing master10:12
shardyI'm not sure the upgrades job can work at all without this revert tho10:13
*** electrofelix has quit IRC10:14
*** electrofelix has joined #tripleo10:14
*** dtantsur|brb is now known as dtantsur10:16
shardyderekh: it does raise an interesting question re the workflow around reverts10:16
shardye.g this patch was reverted because of TripleO failures reported around deletes10:17
derekhshardy: looking at the temprevert function at the moment, I think we could make it work for packages again, it would just need to add the name of the project to ZUUL_CHANGES so that toci knows to build it, thats I think all thats missing10:17
shardywe reverted it, and there's no easy way for TripleO to actually get the fix10:17
shardyderekh: cool, that'd be a nice option to have10:17
openstackgerritMiles Gould proposed openstack/tripleo-quickstart: Update URLs to reflect move to OpenStack org  https://review.openstack.org/30124910:17
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci: Test with latest master heat  https://review.openstack.org/30155410:18
derekhshardy: yup, we're not ontop of keeping track of the periodic job, we need to be aiming for a promote every night and if it doesn't we need to fix the problem so a promote happens the next night10:18
derekhshardy: we just shouldn't allow the promote to get more then a day or 2 behind10:19
shardyderekh: Yeah, topic for the meeting later I guess10:19
derekhshardy: yup10:19
shardyneed to find some folks with bandwidth to help keep that rolling10:19
jaosoriorHave you guys seen some errors like "Engine went down during stack create" ? While the pingtest is running10:26
shardyjaosorior: Yes, it normally means it got OOM killed on a controller without enough ram and zero swap10:28
*** akuznetsov has quit IRC10:28
jaosoriorshardy: I remember we reduced the number of workers for the undercloud. Have we done such a thing for the overcloud nodes?10:28
*** qasims has quit IRC10:31
derekhjaosorior: you can check the journal log to see if OOM kill anything10:34
shardyjaosorior: In CI, yes, but not by default10:35
jaosoriorshardy: yeah, in CI is fine. Was just wondering where that is being set.10:37
jaosoriorcause I see the definition of that template. But I don't find where it's actually used in CI10:38
shardyjaosorior: ^^10:41
jaosoriorshardy: Alright, thanks dude10:42
*** electrofelix has quit IRC10:42
openstackgerritDmitry Tantsur proposed openstack/diskimage-builder: Add clean-machine-id element and use it in the ironic-agent element  https://review.openstack.org/30157110:44
*** electrofelix has joined #tripleo10:47
*** dshulyak has joined #tripleo10:50
openstackgerritDmitry Tantsur proposed openstack/python-tripleoclient: Use clean-machine-id element when building the overcloud image  https://review.openstack.org/30157410:53
*** hewbrocca has quit IRC10:59
*** hewbrocca-afk has joined #tripleo10:59
*** hewbrocca-afk is now known as hewbrocca10:59
*** hewbrocca is now known as hewbrocca-afk11:00
*** hewbrocca-afk is now known as hewbrocca11:01
*** rhallisey has joined #tripleo11:04
*** chem has quit IRC11:04
*** chem has joined #tripleo11:06
*** Marga_ has joined #tripleo11:12
gfidenteI think we it's about time we track rechecks appending a bug?11:13
jistrshardy: hi, we hit a problem where we need to distinguish between stack-create and stack-update again. I'm thinking the easiest way to solve it might be to have tripleo-common/tripleoclient pass a specific parameter "StackOperation" that would be set to "CREATE" or "UPDATE", and listen for that parameter in t-h-t. (We cannot use the 'action:' conditional on resources, because we're interested in create/update status of the *top* stack, not11:17
jistrthe immediate parent stack of the resource.)11:17
*** Marga_ has quit IRC11:17
shardyjistr: can't you just look at DeployIdentifier and store it?11:18
shardyif it's not stored on the node, it's a create11:18
shardyotherwise, it's an update?11:18
*** akuznetsov has joined #tripleo11:18
jistrshardy: hmm like create another file in /var/lib/tripleo, similarly as we do with package manifests? yeah i think we could do that11:19
gfidenteI was thinking to move that in the shared functions11:19
*** nico_auv has joined #tripleo11:19
jistrshardy: except it's not going to work on the first update after introducing this mechanism11:19
shardyjistr: yeah, that may be the simplest solution11:19
shardyjistr: Hmm, yeah I'll have to think if there's any way we can work around that11:20
jistrshardy: because existing deployments won't have deployment identifier stored, even though stack-create has happened for them11:20
shardyjistr: well it is stored *somewhere*11:20
shardybut obtaining it is likely to be hacky11:20
* shardy drops off for lunch, bbiab11:20
openstackgerritMerged openstack/tripleo-heat-templates: Nova needs the proper volumes to use Cinder  https://review.openstack.org/29490811:21
*** florianf has quit IRC11:22
jistrshardy: true, it's stored in hiera. But IIRC it's stored in hiera even for deployments that are about to be created, so that won't help to distinguish between create and update i think. (Unless we put someting even before the deployment that writes out the hiera file. That might do it.)11:23
*** mbound has quit IRC11:24
jistrshardy: also such mechanism would be node-local, e.g. on controller replacement it would resolve to 'create' on one controller and to 'update' on other controllers. That might be desirable in different scenarios but i don't think it's very good for the one we're solving (post-puppet restart trigger).11:26
*** nico_auv has quit IRC11:26
*** akuznetsov has quit IRC11:28
openstackgerritRyan Hallisey proposed openstack/tripleo-heat-templates: Nova needs the proper volumes to use Cinder  https://review.openstack.org/30159011:29
openstackgerritRyan Hallisey proposed openstack/tripleo-heat-templates: Nova needs the proper volumes to use Cinder  https://review.openstack.org/30159211:30
openstackgerritMiles Gould proposed openstack/python-tripleoclient: Use Ironic API v1.11 to support ENROLL state  https://review.openstack.org/27220611:30
*** rodrigods has quit IRC11:32
*** rodrigods has joined #tripleo11:32
*** trown|outtypewww is now known as trown11:35
*** ibravo has joined #tripleo11:36
*** thrash|g0ne is now known as thrash11:36
jaosoriortrown: Hey dude, got some time to check this CR out? https://review.openstack.org/#/c/299475/711:36
trownderekh: is it possible for me to get the sqlite db that backs http://tripleo.org/cistatus.html11:36
trownjaosorior: looking11:36
derekhtrown: yes, gimme a minute11:37
trownk I will review jaosorior patch in the meantime :)11:37
*** florianf has joined #tripleo11:38
*** mbound has joined #tripleo11:39
*** mbound has quit IRC11:39
jaosoriorthanks dude :D11:39
*** mbound has joined #tripleo11:40
*** mbound has quit IRC11:40
derekhtrown: goodsquishy.com/downloads/tripleo-jobs.db11:41
derekhtrown: the periodic job is a separate file but I guess that the one you want?11:42
trownderekh: ya, I wrote a compare jobs function to show pass rate of 1 job conditoned on another job passing11:42
trownI think that is a reasonable approximation of real pass rate11:42
derekhtrown: yup11:43
trownbut running the get_data function only gives a small recent sample of data11:43
derekhtripleo-jobs.py used to also generate a report for recent data but we took it out at some stage11:44
trownderekh: is there any unique identifier that links jobs run in the same ci run? I am having to fudge the calculation of P(A) AND P(B)11:45
derekhtrown: you'll have to keep an eye out for jobs changing names in that db file (depending on how far back in time you go)11:45
trownby just taking min(P(A),P(B))11:45
trownya really only interested in making a 30day,15day,7day,1day table11:46
derekhtrown: let me check, iirc we have the ZUUL id in there somewhere11:46
derekhtrown: the zuul_ref colume on the table should be common for jobs triggered at the same time11:48
trownderekh: ah right, I could use that with a time window and get all jobs run on a patchset11:49
trownI was using that, but without the time window it gives all jobs for a review11:50
*** weshay has joined #tripleo11:50
derekhtrown: you shouldn't need a time window, it should be uniq to the trigger (I think)11:51
trownoh right, I think I was mistaking that last field for change-id, but I see it is Z*11:52
derekhtrown: occasionally you'll get multiple zuul_ref's for the same trigger, that where zuul restarted a job e.g. if it lost its ssh connection to the jenkins slave11:52
trownbut the other ones would not have SUCCESS or FAILURE, so that would be fine11:53
*** MaxPC has joined #tripleo11:54
derekhtrown: for example is you look at http://tripleo.org/cistatus.html right now you'll see some entries have multiple jobs against them, looks like a jenkins went down or something11:54
derekhtrown: yup, filtering by SUCCESS/FAILURE should get you what you want11:54
trownya, for stats we don't want ABORTED or None, I also have it filtered to remove jobs that only run the nonha job and changes to tripleo-ci since those skew things too11:55
derekhtrown: seems about right11:57
*** dprince has joined #tripleo11:57
openstackgerritMerged openstack-infra/tripleo-ci: Include Gnocchi endpoints in enable-tls  https://review.openstack.org/30115711:57
derekhtrown: also filter out every second jobs with FAILURE ;-)11:57
derekhtrown: just a joke, never mind me11:58
trownlol, got ya, 20% -> 40% just like that :)11:58
*** masco has quit IRC12:01
rook-lappyshardy: i see that jenkins gave that patch a +1 (network validation patch I submitted).. Once that gets through we can look at patching up the gw12:03
jistrheya, trying to look into some of the CI fails12:04
jistris it expected that we don't get host info tarballs from jobs that timed out?12:04
jistre.g. this one http://logs.openstack.org/69/300569/1/check-tripleo/gate-tripleo-ci-f22-ha/09831a4/12:04
jistrhas no undercloud/overcloud logs12:05
jaosoriorjistr: I think that's related. this one also timed out http://logs.openstack.org/99/292799/4/check-tripleo/gate-tripleo-ci-f22-ha/d18683f/ and it only has the logs for one of the controllers and the undercloud12:07
shardyjistr: Ah I thought we only set DeployIdentifier on update but I see that's not actually the case12:07
shardyI guess we can have a DeployAction parameter boolean based on stack_is_new easily enough12:08
shardyit's just seems a bit weird given that we should already know the action based on the API call12:08
shardyjust setting DeployAction=stack_is_new as a parameter default would probably solve your problem tho I think?12:08
*** electrofelix has quit IRC12:08
jistrshardy: yea that's what i suggested above, just called it StackOperation :)12:09
jistrbut DeployAction wfm too :)12:09
shardyjistr: Aha, either wfm ;)12:09
*** mbound has joined #tripleo12:11
jistrjaosorior: looks like the one you linked timed out while collecting the logs after a green test. Talk about bad luck :D12:13
trownwell statistically speaking... average luck :p12:13
jistrok so i guess it *can* currently expected that we don't have logs from jobs that timed out12:14
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Plans Dropdown in Deployment Plan page  https://review.openstack.org/29588212:14
jistrand we have timeouts that don't seem to work12:14
jistrthat one was supposed to wait_for 3 minutes12:14
jistrbut it waited ~28 minutes12:14
trownya, I thought we tried to solve that by pre-timing out so we always get logs12:14
jistrok... i'll see if i can fix that timeout ^^12:15
jistrthere's something wrong with the deployed overcloud, looks like crm_resource --wait never finishes waiting. We need to really kill it after 3 minutes, otherwise we'll never get logs from those stuck HA jobs.12:16
*** pradk has quit IRC12:17
*** eggmaster has joined #tripleo12:17
*** pradk has joined #tripleo12:17
jaosoriorjistr: well... knowing that it's green. +2? https://review.openstack.org/#/c/292799/ :P12:18
jistrtrown: ah yea. wait_for is for polling, but what we need there is to timeout a blocking call. i'll submit a fix12:18
*** sshnaidm has quit IRC12:19
*** egafford has joined #tripleo12:20
*** nico_auv has joined #tripleo12:22
*** jayg|g0n3 is now known as jayg12:23
*** snecklifter has quit IRC12:25
*** panda has quit IRC12:26
*** panda has joined #tripleo12:26
jistrjaosorior: any potential for mitaka/liberty backport on that one? wondering about what it might do with updates/upgrades12:27
jaosoriorjistr: Which one? The one I pointed you to?12:28
jaosoriorjistr: the /etc/hosts format change?12:28
*** aufi has quit IRC12:28
jistrjaosorior: yes12:28
jaosoriorjistr: Those names are not used anywhere yet12:29
jaosoriorjistr: Which is what I was asking gfidente about.12:29
jistrjaosorior: ack, thanks. +212:30
jaosoriorjistr: Thanks for checking it out dude12:31
jaosoriorjistr: How acquainted are you with the python-tripleoclient? Got a question regarding that12:31
jistrjaosorior: not much, but you can post the question and maybe thrash can chime in if i don't know12:32
thrashjaosorior: go for it.12:32
*** sshnaidm has joined #tripleo12:34
jaosoriorjistr, thrash: in the postconfig, if the KeystoneAdminVip and the KeystoneInternalVip are not set as outputs, it seems that it uses the public endpoint... why is that? Is that something that might actually happen? Cause in this CR I'm removing them https://review.openstack.org/#/c/299475/7/tripleoclient/v1/overcloud_deploy.py12:34
*** snecklifter has joined #tripleo12:34
jaosoriorI mean, removing that setting; not the output12:34
*** akshai has joined #tripleo12:35
*** paramite is now known as paramite|afk12:35
*** rlandy has joined #tripleo12:37
*** paramite|afk is now known as paramite12:38
*** liverpooler has quit IRC12:39
thrashjaosorior: let me look, but that is probably fine.12:39
jaosoriortrown and I were wondering what's up with that O_O12:39
thrashthat code should probably be removed in N anyway. I thought we deprecated all of that.12:40
thrashall of that should move to puppet.12:40
*** jpena is now known as jpena|lunch12:41
jaosoriorthrash: That is the case. But if support for configurable ports is needed (haven't checked the bz but I think it is) then this can be backported12:41
thrashjaosorior: I'll comment on the patch where you and trown were discussing.12:42
thrashjaosorior: rather than type it twice. :)12:43
thrashjaosorior: commented...12:43
openstackgerritJiri Stransky proposed openstack-infra/tripleo-ci: Fix timeout on crm_resource --wait  https://review.openstack.org/30162412:43
jistrtrown, derekh: hoping this should help ^12:44
jaosoriorthrash: Thanks dude12:45
openstackgerritCyril Lopez proposed openstack/tripleo-docs: The control placement is needed more information to aviod a no valid host found  https://review.openstack.org/30118512:49
*** julim has joined #tripleo12:52
*** sshnaidm has quit IRC12:53
hewbroccadprince: Would you be able to have a look at an os-net-config blocker12:58
hewbroccadsneddon: is on PTO today12:59
hewbroccaand it's kind of an emergency...12:59
openstackhewbrocca: Error: Error getting bugzilla.redhat.com bug #1323717: NotPermitted12:59
*** ibravo has quit IRC12:59
*** morazi has joined #tripleo13:04
*** sshnaidm has joined #tripleo13:08
*** wendar_ has joined #tripleo13:14
*** mgagne_ has joined #tripleo13:20
*** rhallisey has quit IRC13:21
*** gfidente has quit IRC13:21
*** rbrady has quit IRC13:21
*** jdob has quit IRC13:21
*** rook has quit IRC13:21
*** uncurlhalo has quit IRC13:21
*** ayoung has quit IRC13:21
*** dsneddon has quit IRC13:21
*** mgagne has quit IRC13:21
*** wendar has quit IRC13:21
*** saneax has quit IRC13:21
*** al has quit IRC13:21
*** jprovazn has quit IRC13:21
*** mcornea has quit IRC13:21
*** jcoufal has quit IRC13:21
*** bhunter has quit IRC13:21
*** marios has quit IRC13:21
*** pblaho has quit IRC13:21
*** pleia2 has quit IRC13:21
*** therve has quit IRC13:21
*** mburned has quit IRC13:21
*** florianf has quit IRC13:21
*** shardy has quit IRC13:21
*** Ryjedo has quit IRC13:21
*** tserong has quit IRC13:21
*** yuanying has quit IRC13:21
*** Erming__ has quit IRC13:21
*** NobodyCam has quit IRC13:21
*** isq_ has quit IRC13:21
*** fabbione has quit IRC13:21
*** sirushti has quit IRC13:21
*** morazi has quit IRC13:21
*** rlandy has quit IRC13:21
*** egafford has quit IRC13:21
*** mbound has quit IRC13:21
*** ohamada has quit IRC13:21
*** ifarkas has quit IRC13:21
*** rcernin has quit IRC13:21
*** thrash has quit IRC13:21
*** larsks has quit IRC13:21
*** hewbrocca has quit IRC13:21
*** coolsvap has quit IRC13:21
*** shakamunyi has quit IRC13:21
*** bswartz has quit IRC13:21
*** marcusvrn_ has quit IRC13:21
*** hjensas has quit IRC13:21
*** rwsu has quit IRC13:21
*** spredzy has quit IRC13:21
*** afazekas has quit IRC13:21
*** jroll has quit IRC13:21
*** trown has quit IRC13:21
*** ianw has quit IRC13:21
*** EmilienM has quit IRC13:21
*** mandre has quit IRC13:21
*** StevenK has quit IRC13:21
*** MaxPC has quit IRC13:21
*** weshay has quit IRC13:21
*** trozet has quit IRC13:21
*** tremble has quit IRC13:21
*** jtomasek has quit IRC13:21
*** zoliXXL has quit IRC13:21
*** kbyrne has quit IRC13:21
*** onovy has quit IRC13:21
*** larstobi has quit IRC13:21
*** adam_g has quit IRC13:21
*** Nakato has quit IRC13:21
*** dobson has quit IRC13:21
*** tristanC has quit IRC13:21
*** cmyster has quit IRC13:21
*** akshai has quit IRC13:21
*** nico_auv has quit IRC13:21
*** eggmaster has quit IRC13:21
*** dprince has quit IRC13:21
*** dshulyak has quit IRC13:21
*** rasca has quit IRC13:21
*** ramishra has quit IRC13:21
*** pcaruana has quit IRC13:21
*** rook-lappy has quit IRC13:21
*** openstackgerrit has quit IRC13:21
*** beagles has quit IRC13:21
*** pino|work has quit IRC13:21
*** pradk has quit IRC13:21
*** lazy_prince has quit IRC13:21
*** dmsimard has quit IRC13:21
*** zaneb has quit IRC13:21
*** mordred has quit IRC13:21
*** jpeeler has quit IRC13:21
*** andreaf has quit IRC13:21
*** HenryG has quit IRC13:21
*** julim has quit IRC13:21
*** jaosorior has quit IRC13:21
*** numans has quit IRC13:21
*** shadower has quit IRC13:21
*** leanderthal has quit IRC13:21
*** stevebaker has quit IRC13:21
*** SpamapS has quit IRC13:21
*** zigo has quit IRC13:21
*** bkero has quit IRC13:21
*** Hazelesque has quit IRC13:21
*** ChanServ has quit IRC13:21
*** mgould has quit IRC13:21
*** mkovacik has quit IRC13:21
*** paramite has quit IRC13:21
*** rdopiera has quit IRC13:21
*** david-lyle has quit IRC13:21
*** lifeless has quit IRC13:22
*** lynxman has quit IRC13:22
*** leifmadsen has quit IRC13:22
*** r-mibu has quit IRC13:22
*** social has quit IRC13:22
*** SlickNik has quit IRC13:22
*** igorbelikov has quit IRC13:22
*** jpena|lunch has quit IRC13:22
*** dtantsur has quit IRC13:22
*** Slower has quit IRC13:22
*** Ng has quit IRC13:22
*** tdasilva has quit IRC13:22
*** quack_quack has quit IRC13:22
*** CaptTofu has quit IRC13:22
*** rhefner has quit IRC13:22
*** cinerama has quit IRC13:22
*** panda has quit IRC13:22
*** rodrigods has quit IRC13:22
*** jistr has quit IRC13:22
*** mikelk has quit IRC13:22
*** ccamacho has quit IRC13:22
*** lucasagomes has quit IRC13:22
*** psanchez has quit IRC13:22
*** Tristitia has quit IRC13:22
*** bnemec has quit IRC13:22
*** crinkle has quit IRC13:22
*** rickflare has quit IRC13:22
*** sshnaidm has quit IRC13:22
*** snecklifter has quit IRC13:22
*** chem has quit IRC13:22
*** akrivoka has quit IRC13:22
*** oshvartz has quit IRC13:22
*** Goneri has quit IRC13:22
*** greghaynes has quit IRC13:22
*** athomas has quit IRC13:22
*** ryansb has quit IRC13:22
*** colonwq has quit IRC13:22
*** dweaver has quit IRC13:22
*** sambetts has quit IRC13:22
*** slagle has quit IRC13:22
*** derekh has quit IRC13:22
*** number80 has quit IRC13:22
*** jrist has quit IRC13:22
*** michchap has quit IRC13:22
*** adarazs has quit IRC13:22
*** nijaba has quit IRC13:22
*** lmiccini has quit IRC13:22
*** shanky has quit IRC13:22
*** george_goh has quit IRC13:22
*** bandini has quit IRC13:22
*** jayg has quit IRC13:22
*** ansiwen has quit IRC13:22
*** telnoratti has quit IRC13:22
*** jidar has quit IRC13:22
*** matbu has quit IRC13:22
*** saneax has joined #tripleo13:24
*** Marga_ has joined #tripleo13:24
*** sshnaidm has joined #tripleo13:24
*** morazi has joined #tripleo13:24
*** julim has joined #tripleo13:24
*** rlandy has joined #tripleo13:24
*** akshai has joined #tripleo13:24
*** snecklifter has joined #tripleo13:24
*** panda has joined #tripleo13:24
*** nico_auv has joined #tripleo13:24
*** egafford has joined #tripleo13:24
*** pradk has joined #tripleo13:24
*** eggmaster has joined #tripleo13:24
*** mbound has joined #tripleo13:24
*** dprince has joined #tripleo13:24
*** MaxPC has joined #tripleo13:24
*** weshay has joined #tripleo13:24
*** florianf has joined #tripleo13:24
*** rodrigods has joined #tripleo13:24
*** chem has joined #tripleo13:24
*** hewbrocca has joined #tripleo13:24
*** dshulyak has joined #tripleo13:24
*** rasca has joined #tripleo13:24
*** mgould has joined #tripleo13:24
*** coolsvap has joined #tripleo13:24
*** ramishra has joined #tripleo13:24
*** mkovacik has joined #tripleo13:24
*** derekh has joined #tripleo13:24
*** akrivoka has joined #tripleo13:24
*** jprovazn has joined #tripleo13:24
*** jistr has joined #tripleo13:24
*** ohamada has joined #tripleo13:24
*** paramite has joined #tripleo13:24
*** shardy has joined #tripleo13:24
*** jaosorior has joined #tripleo13:24
*** ifarkas has joined #tripleo13:24
*** pcaruana has joined #tripleo13:24
*** rdopiera has joined #tripleo13:24
*** mcornea has joined #tripleo13:24
*** mikelk has joined #tripleo13:24
*** oshvartz has joined #tripleo13:24
*** rcernin has joined #tripleo13:24
*** jcoufal has joined #tripleo13:24
*** numans has joined #tripleo13:24
*** r-mibu has joined #tripleo13:24
*** ccamacho has joined #tripleo13:24
*** shakamunyi has joined #tripleo13:24
*** rook-lappy has joined #tripleo13:24
*** bhunter has joined #tripleo13:24
*** quack_quack has joined #tripleo13:24
*** david-lyle has joined #tripleo13:24
*** Goneri has joined #tripleo13:24
*** Ryjedo has joined #tripleo13:24
*** trozet has joined #tripleo13:24
*** number80 has joined #tripleo13:24
*** greghaynes has joined #tripleo13:24
*** bswartz has joined #tripleo13:24
*** marcusvrn_ has joined #tripleo13:24
*** tremble has joined #tripleo13:24
*** openstackgerrit has joined #tripleo13:24
*** marios has joined #tripleo13:24
*** athomas has joined #tripleo13:24
*** pblaho has joined #tripleo13:24
*** tserong has joined #tripleo13:24
*** ryansb has joined #tripleo13:24
*** hjensas has joined #tripleo13:24
*** jtomasek has joined #tripleo13:24
*** jrist has joined #tripleo13:24
*** rwsu has joined #tripleo13:24
*** zoliXXL has joined #tripleo13:24
*** sambetts has joined #tripleo13:24
*** mburned has joined #tripleo13:24
*** kbyrne has joined #tripleo13:24
*** thrash has joined #tripleo13:24
*** yuanying has joined #tripleo13:24
*** tristanC has joined #tripleo13:24
*** colonwq has joined #tripleo13:24
*** lazy_prince has joined #tripleo13:24
*** michchap has joined #tripleo13:24
*** beagles has joined #tripleo13:24
*** onovy has joined #tripleo13:24
*** social has joined #tripleo13:24
*** pleia2 has joined #tripleo13:24
*** SlickNik has joined #tripleo13:24
*** larstobi has joined #tripleo13:24
*** lucasagomes has joined #tripleo13:24
*** spredzy has joined #tripleo13:24
*** shadower has joined #tripleo13:24
*** psanchez has joined #tripleo13:24
*** Erming__ has joined #tripleo13:24
*** igorbelikov has joined #tripleo13:24
*** dweaver has joined #tripleo13:24
*** adam_g has joined #tripleo13:24
*** dmsimard has joined #tripleo13:24
*** Nakato has joined #tripleo13:24
*** adarazs has joined #tripleo13:24
*** zaneb has joined #tripleo13:24
*** lifeless has joined #tripleo13:24
*** nijaba has joined #tripleo13:24
*** jpena|lunch has joined #tripleo13:24
*** lmiccini has joined #tripleo13:24
*** NobodyCam has joined #tripleo13:24
*** afazekas has joined #tripleo13:24
*** shanky has joined #tripleo13:24
*** Tristitia has joined #tripleo13:24
*** dtantsur has joined #tripleo13:24
*** Slower has joined #tripleo13:24
*** lynxman has joined #tripleo13:24
*** Ng has joined #tripleo13:24
*** therve has joined #tripleo13:24
*** matbu has joined #tripleo13:24
*** dobson has joined #tripleo13:24
*** mordred has joined #tripleo13:24
*** george_goh has joined #tripleo13:24
*** bnemec has joined #tripleo13:24
*** tdasilva has joined #tripleo13:24
*** jpeeler has joined #tripleo13:24
*** cmyster has joined #tripleo13:24
*** bandini has joined #tripleo13:24
*** jayg has joined #tripleo13:24
*** ansiwen has joined #tripleo13:24
*** andreaf has joined #tripleo13:24
*** leanderthal has joined #tripleo13:24
*** larsks has joined #tripleo13:24
*** pino|work has joined #tripleo13:24
*** ianw has joined #tripleo13:24
*** jroll has joined #tripleo13:24
*** trown has joined #tripleo13:24
*** sirushti has joined #tripleo13:24
*** fabbione has joined #tripleo13:24
*** isq_ has joined #tripleo13:24
*** EmilienM has joined #tripleo13:24
*** CaptTofu has joined #tripleo13:24
*** telnoratti has joined #tripleo13:24
*** rhefner has joined #tripleo13:24
*** stevebaker has joined #tripleo13:24
*** slagle has joined #tripleo13:24
*** HenryG has joined #tripleo13:24
*** mandre has joined #tripleo13:24
*** leifmadsen has joined #tripleo13:24
*** bkero has joined #tripleo13:24
*** StevenK has joined #tripleo13:24
*** wolfe.freenode.net sets mode: +v Ng13:24
*** SpamapS has joined #tripleo13:24
*** crinkle has joined #tripleo13:24
*** zigo has joined #tripleo13:24
*** jidar has joined #tripleo13:24
*** rickflare has joined #tripleo13:24
*** Hazelesque has joined #tripleo13:24
*** cinerama has joined #tripleo13:24
*** ChanServ has joined #tripleo13:24
*** wolfe.freenode.net sets mode: +o ChanServ13:24
*** tiswanso has joined #tripleo13:24
*** saneax has quit IRC13:24
*** saneax has joined #tripleo13:24
*** openstackstatus has quit IRC13:24
*** mgagne_ has quit IRC13:24
*** mgagne_ has joined #tripleo13:24
*** al has joined #tripleo13:25
*** dsneddon has joined #tripleo13:27
*** zoliXXL is now known as zoli|mtg13:27
*** sshnaidm has quit IRC13:27
*** gfidente has joined #tripleo13:28
*** openstackstatus has joined #tripleo13:28
*** ChanServ sets mode: +v openstackstatus13:28
*** uncurlhalo has joined #tripleo13:28
*** jdob has joined #tripleo13:28
*** ayoung has joined #tripleo13:29
*** rhallisey has joined #tripleo13:29
*** rbrady has joined #tripleo13:29
*** mgould has quit IRC13:29
*** mgould has joined #tripleo13:32
*** apetrich has quit IRC13:34
*** links has quit IRC13:35
*** trozet has quit IRC13:37
*** trozet has joined #tripleo13:37
*** tzumainn has joined #tripleo13:38
gfidenteayoung you around?13:39
slagleEmilienM: you merged this with failing CI: https://review.openstack.org/#/c/259568/13:40
slagleand it's a real failure13:40
slaglethe patch breaks updates and deletes13:40
dprincemichchap: I'm going to try and wire up your puppet-tripleo neutron work today. Nice job on that (complex) set of services13:41
*** saneax is now known as saneax_AFK13:41
michchapdprince: ta. I need to add plumgrid since it came after I wrote them13:41
michchapdprince: but otherwise I think it's looking ok13:42
*** sshnaidm has joined #tripleo13:42
dprincemichchap: cool13:42
dprinceslagle: is there a bug for the delete issue?13:43
dprinceslagle: we can revert if needed but this patch is kind of important to land13:43
dprinceI see it13:44
openstackLaunchpad bug 1566259 in tripleo "Upgrades job failing with ControllerServiceChain config_settings error" [Critical,Triaged]13:44
slagledprince: yea, it's the same error on delete13:44
gfidentethat is the upgrade job testing heat13:45
*** dtrainor has joined #tripleo13:45
dprinceslagle: okay, yeah I'm using heat master and I'm not seeing any delete issues13:45
numansi am seeing the same error while deleting an overcloud..13:48
numans openstack stack delete overcloud --yes13:48
numansERROR: The Referenced Attribute (ControllerServiceChain config_settings) is incorrect.13:48
*** dtrainor has quit IRC13:48
*** dtrainor has joined #tripleo13:49
shardydprince: AFAICT that's because we're using a 2-week old heat in CI13:50
*** sshnaidm has quit IRC13:50
shardywe already reverted the heat patch, but we're using the buggy one still13:50
hewbroccacan we move the pin for Heat to latest?13:50
*** jpena|lunch is now known as jpena13:51
dprinceshardy: +2 on your patch to bump the pin13:51
shardyhewbrocca: I was trying that here13:51
dprinceshardy: but it still isn't passing for other reasons?13:51
shardydprince: the upgrades job just took too long13:52
dprinceshardy: yeah, it looks like the upgrade was totally fine.13:52
shardyand the nonha job failed due to a glance error13:52
shardydprince: we were discussing this earlier as a topic for the meeting13:53
shardywe really need to figure out who has bandwidth to constantly fix things when the periodic job fails13:54
jistrhmm would be nicer if we could bump delorean repo pin instead of always pulling latest heat, no? IIRC we used to pull latest ironic and heat, but we stopped doing that because we were broken often. Or do i not recall this correctly?13:54
shardye.g we've not promoted for 2 weeks, so how can we improve that latency13:54
* jistr searches logs13:54
hewbroccashardy: I need to get you more folks to help :)13:54
dprinceshardy: this is the pain w/ not being a gate :/13:54
shardyjistr: Yes, well volunteered! :D13:54
jistrehm :D13:55
dprincejistr: way to step up dude :)13:55
*** mbound has quit IRC13:55
ayounggfidente, I'm more linear than round13:56
*** lblanchard has joined #tripleo13:58
shardydprince: Hey did you already start an etherpad with session ideas for Austin?13:58
shardyI was about to start one so we can discuss it in the meeting13:59
dprinceshardy: start one13:59
dprincemeeting time14:00
*** sshnaidm has joined #tripleo14:04
*** liverpooler has joined #tripleo14:05
*** liverpooler has quit IRC14:05
*** liverpooler has joined #tripleo14:05
*** zoli|mtg is now known as zoli|bb14:10
*** zoli|bb is now known as zoli|brb14:10
*** apetrich has joined #tripleo14:11
*** mbound has joined #tripleo14:15
*** jprovazn has quit IRC14:17
sshnaidmwhat is "wall time"?14:17
*** ramishra has quit IRC14:18
trowntotal job time14:18
sshnaidmtrown, thanks14:18
*** paramite has quit IRC14:19
openstackgerritMiles Gould proposed openstack/tripleo-quickstart: Update URLs to reflect move to OpenStack org  https://review.openstack.org/30124914:20
*** Marga_ has quit IRC14:20
dmsimardderekh: o/ can you let me know when you have a bit of time to talk about the tripleo symlink ?14:21
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Fix image caching logic  https://review.openstack.org/29953714:21
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Pre install packages on the instack image  https://review.openstack.org/29953814:21
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Upload the ironic-python-agent images to cache  https://review.openstack.org/30169814:21
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: [WIP] Use the cached ironic-python-agent images  https://review.openstack.org/30169914:21
*** julim has quit IRC14:21
*** rook has joined #tripleo14:21
derekhdmsimard: yup, lets do it after the tripleo meeting (on at the moment)14:21
*** ramishra has joined #tripleo14:22
openstackgerritAntoine Musso proposed openstack/diskimage-builder: dib-run-parts: make cp to target root more robust  https://review.openstack.org/21945314:27
*** dshulyak has quit IRC14:29
openstackgerritDmitry Tantsur proposed openstack/diskimage-builder: Add clean-machine-id element to remove /etc/machine-id  https://review.openstack.org/30157114:30
hewbroccadprince: For what it's worth, the os-net-config bug almost certainly has to be related to14:30
*** aufi has joined #tripleo14:30
openstackgerritAntoine Musso proposed openstack/diskimage-builder: dib-run-parts: make cp to target root more robust  https://review.openstack.org/21945314:30
dprincehewbrocca: yeah, could be. This was something I had wanted to kick the tires on a bit more :/14:31
hewbroccaI remember...14:32
hewbroccaWe could just revert this one and the following one that depends on it14:32
openstackgerritHonza Pokorny proposed openstack/tripleo-ui: Remove images and flavors tabs  https://review.openstack.org/30172014:36
*** ibravo has joined #tripleo14:42
*** dshulyak has joined #tripleo14:45
rookshardy -- is there something wrong with the patch that is holding up the +1 to workflow?14:48
*** myoung has joined #tripleo14:49
pradkgfidente, we're trying to debug an ipv6 issue in ci with adarazs and we seem to be hitting a dead end .. can you help ?14:50
adarazsgfidente: looks like the different services can't reach the mysql server during dbsync: http://logs.openstack.org/32/252032/39/check-tripleo/gate-tripleo-ci-f22-upgrades/3dda1df/console.html#_2016-04-05_10_12_51_93714:51
adarazswhich is a bit weird.14:51
pradkyea it fails with No route to host .. seems like a networking issue14:52
*** ibravo has quit IRC14:52
shardyrook: Only that it's not passed all the voting CI jobs14:52
*** ibravo has joined #tripleo14:53
shardywhich is most likely related to a current critical bug impacting the upgrades job14:53
*** mgould has quit IRC14:53
rookshardy i tried recheck -- but nada14:55
rookguess i need someone with +2 to recheck?14:56
*** mgould has joined #tripleo14:56
pradkshardy, whats the critical bug impacting upgrade jobs? is it same as what adarazs and I and talking above?14:57
*** chem has quit IRC14:57
*** zoli|brb is now known as zoli|wfh14:57
*** chem has joined #tripleo14:59
*** derekh is now known as derekh_brb14:59
dprinceshardy: totally fine with changing the agenda up BTW15:00
shardydprince: Cool, I've been thinking we could use a little more time for one-off items for a while15:00
dprinceshardy: I tried it before... I think some of the categories are likely go come up on a weekly basis15:00
dprinceshardy: like trown's item from last week. I was just going to go through those first15:01
shardydprince: ack, I guess we can maybe just figure out ways to timebox certain topics then15:01
dprinceshardy: essentially, anything that didn't get addressed previously goes first next week15:01
shardyin the heat meeting, we mostly focus on one-off items people want to discuss15:01
shardyso there's less focus on specs/bugs/ci etc15:01
shardybut we do have reasons for increased focus on those obviously15:02
dprinceshardy: yeah, it is a trade off15:02
trownI think bugs/CI is taking alot of the meeting time, and could pretty easily be split into another meeting15:02
trownwith a report (short) back to the main meeting15:02
shardytrown: Yeah, if we can get folks with time to commit to it then the CI sub-team is a great idea15:02
*** dtantsur is now known as dtantsur|brb15:03
trownwell, I think everyone agrees we need folks commited to it :) meeting or not15:03
trownthough we have always agreed on that I think15:04
shardyYeah, the message I am hearing is folks mostly don't have time to commit to it15:04
shardyso that's the first problem to solve15:04
dprinceshardy: yeah, if we can't land patches cause it is broken it is the thing to focus on15:04
*** pcaruana has quit IRC15:06
shardypradk: not the same issue AFAICS15:06
*** derekh_brb is now known as derekh15:06
*** lblanchard has quit IRC15:07
pradkshardy, hmm ok, we're a bit lost on what could be the problem with mysql not reachable here ..  http://logs.openstack.org/32/252032/39/check-tripleo/gate-tripleo-ci-f22-upgrades/3dda1df/console.html#_2016-04-05_10_12_51_93715:09
pradkshardy, you have any suggestions?15:09
*** jprovazn has joined #tripleo15:10
shardypradk: what do the overcloud logs say?15:10
pradkshardy, No route to host15:10
pradkshardy, so seems like mysql is not reachable when ipv6 is enabled but not exactly sure why.. nothing realy sticks out in the logs15:12
*** links has joined #tripleo15:14
gfidentepradk adarazs though I have seen overcloud deployed successfully in ci15:18
gfidentepradk adarazs where is that you're having the error?15:18
pradkgfidente, in upgrade ci job http://logs.openstack.org/32/252032/39/check-tripleo/gate-tripleo-ci-f22-upgrades/3dda1df/console.html#_2016-04-05_10_12_51_93715:19
adarazsgfidente: according to the logs it's resources.ControllerNodesPostDeployment.resources.ControllerOvercloudServicesDeployment_Step4.resources[0]15:19
pradkgfidente, fails with  DBConnectionError: (pymysql.err.OperationalError) (2003, \"Can't connect to MySQL server on 'fd00:fd00:fd00:2000::11' ([Errno 113] No route to host)\")15:20
adarazswell, it has like a zillion errors, but most of them is this. :)15:20
*** aufi has quit IRC15:20
pradkyea looks like services cant connect to mysql when running db sync.. so each services tries and ends up with the above error15:21
gfidenteI think this might be oom15:21
gfidenteit's single node deployment15:21
pradkhmm may be .. though i dont see anything related to memory in the logs15:23
pradkha and nonha jobs finish deploy without issues15:24
*** links has quit IRC15:25
openstackgerritIshant Tyagi proposed openstack/os-collect-config: Add insecure option to the cfn collector  https://review.openstack.org/28472515:25
*** mikelk has quit IRC15:25
*** mkovacik has quit IRC15:26
*** bvandenh has joined #tripleo15:31
*** Marga_ has joined #tripleo15:38
*** Marga_ has quit IRC15:38
*** Marga_ has joined #tripleo15:38
*** Marga_ has quit IRC15:39
rookshardy so, whats the gist of the CI issue? It isn't something specific with my patch is it?15:39
*** Marga_ has joined #tripleo15:39
*** dtantsur|brb is now known as dtantsur15:39
shardyrook: we're experiencing a variety of CI issues atm, the upgrades job in your case just took too long (because the CI environment is slooow atm) so the infra timeout killed it15:41
shardyso it's not related to your patch, sorry for the inconvenience15:42
shardyrook: there are other actual errors too, where folks are continually rechecking because some problems aren't deterministic15:42
shardythis all contibutes to further overloading the already slow environment unfortunately15:43
shardyrook: over the las two runs your patch passed all the voting jobs, so it's probably fine to merge it15:44
*** jaosorior has quit IRC15:46
*** leanderthal is now known as leanderthal|afk15:46
*** jaosorior has joined #tripleo15:47
openstackgerritMerged openstack/tripleo-heat-templates: Ping retry  https://review.openstack.org/29891915:48
*** apetrich has quit IRC15:48
rookshardy:  ah ok.15:50
rookthanks shardy15:51
*** oshvartz has quit IRC15:55
hewbroccadprince: Ugh... we can't revert those os-net-config patches unfortunately15:55
hewbroccawe're going to have to fix it15:56
bnemecdtantsur: Regarding a backport of https://review.openstack.org/#/c/298439/ , I'm not sure.  I would like to, but I'm not sure whether it qualifies as a bug fix or not.15:56
dprincehewbrocca: yeah. Those were added for midonet or something right?15:56
hewbroccaNuage :(15:56
dtantsurbnemec, it might fix booting from multipath volume, no?15:57
bnemecOn the one hand, there's no reason for this to not have worked in the past, so you could argue it's a bug.15:57
*** cwolferh has joined #tripleo15:57
hewbroccaamong others15:57
bnemecOn the other, it's not something we explicitly supported in the past, so it's kind of a new feature too.15:57
bnemecdtantsur: I guess I can propose the backports and if someone disagrees with them they can -1. :-)15:58
dtantsurbnemec, not rebuilding the ramdisk might also break hardware support for people, if they need to insert a module..15:58
dtantsurthough I don't have proofs for this15:58
*** yamahata has joined #tripleo15:59
bnemecdtantsur: Yeah, I think it's a good thing to be doing anyway.  I'll propose the backports, if someone objects we can discuss it on the review.15:59
snecklifterbnemec: any chance you can review https://review.openstack.org/#/c/291243/ again?16:01
*** jpeeler has quit IRC16:06
*** yamahata has quit IRC16:13
*** apetrich has joined #tripleo16:13
openstackgerritMerged openstack/tripleo-heat-templates: Filter for local nodes in check_resource function  https://review.openstack.org/30049416:14
zoli|wfhsee you tomorrow16:15
*** jcoufal has quit IRC16:17
*** derekh has quit IRC16:19
jistrtrown: this https://review.openstack.org/#/c/301624/ has already worked for the "bad case" -- it timed out (probably stuck) and we got the overcloud logs \o/ we'll need to verify the "good case" too before we merge16:20
trownjistr: saw that, nice one16:21
*** jpeeler has joined #tripleo16:23
*** jpeeler has quit IRC16:23
*** jpeeler has joined #tripleo16:23
*** apetrich has quit IRC16:25
*** panda has quit IRC16:26
openstackgerritPradeep Kilambi proposed openstack/puppet-tripleo: Map gnocchi vip to haproxy_listen_bind_param  https://review.openstack.org/30179816:26
*** panda has joined #tripleo16:27
openstackgerritChristopher Brown proposed openstack/os-net-config: Use interface operstate to determine nic status  https://review.openstack.org/29124316:27
snecklifterbnemec: thanks for review, have amended as per your comment16:29
*** sambetts is now known as sambetts|afk16:29
openstackgerritDmitry Ilyin proposed openstack/puppet-pacemaker: Merge with fuel-infra/puppet-pacemaker  https://review.openstack.org/29644016:33
hewbroccadprince: nice work on coming up with a patch16:33
dprincehewbrocca: yep, working on it now.16:34
dprincehewbrocca: give me a few and I think I'll have it16:35
*** trown is now known as trown|lunch16:43
jistrso analyzing the first HA logs we got from the HA job hang problem, services have a bunch of "WSREP has not yet prepared node for application use" errors when trying to talk to DB16:43
jistrgoogling says this can be a symptom of Galera cluster split16:43
jistrwe do check for clustercheck pass on deployment though16:43
jistrafter we merge this we'll get more logs for HA hanged jobs https://review.openstack.org/#/c/30162416:45
jistrso far i just rechecked it in hope we can get a green pass on that too16:45
*** wendar_ is now known as wendar16:45
*** jpena is now known as jpena|off16:46
*** tiswanso has quit IRC16:48
*** mbound has quit IRC16:48
*** tiswanso has joined #tripleo16:49
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Filter for local nodes in check_resource function  https://review.openstack.org/30182416:49
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Filter for local nodes in check_resource function  https://review.openstack.org/30182516:50
*** nico_auv has quit IRC16:51
jistrhmm this is also interesting http://fpaste.org/349948/87504214/16:51
*** apetrich has joined #tripleo16:51
jistrspecifically the VIP monitor and *start* failures16:51
jistrcould be network issues, or maybe also generally things being very overloaded?16:52
jistrlocally i've never had a failure where a VIP resource would fail to start16:52
*** sshnaidm has quit IRC16:53
*** lblanchard has joined #tripleo16:53
openstackgerritDan Prince proposed openstack/os-net-config: Add MASTER=bond SLAVE=yes to linux bond interfaces  https://review.openstack.org/30182716:53
dprincehewbrocca: ^^16:53
dprincemburned: ^^16:54
mburneddprince: ack, thanks16:54
gfidentejistr is it a single job you're looking at?16:56
*** eil397 has joined #tripleo16:56
gfidentejistr the saddest thing is that we actually get some greens https://review.openstack.org/#/c/300494/516:57
jistrgfidente: yes, we don't have logs for more jobs so far. There's a bug in tripleo-ci that these issues hang the whole CI job (waiting for crm_resource --wait indefinitely) and it then gets killed by the "main CI timeout", which results in not collecting machine logs. We need to merge the patch i linked above ^^ to get more logs for the hangs.16:58
gfidenteyes I saw taht16:58
*** akuznetsov has joined #tripleo16:59
*** sanjay__u has joined #tripleo16:59
*** akuznetsov has quit IRC16:59
hewbroccadprince: ROCK thanks16:59
*** ifarkas has quit IRC17:00
gfidentebut there is all sort of strange things, this seems to be failing when creating the admin roles17:00
gfidenteError: /Stage[main]/Keystone::Roles::Admin/Keystone_user[admin]: Could not evaluate: Command: 'openstack [\"token\", \"issue\", [\"--format\", \"value\"]]' has been running for more then 20 seconds!17:00
gfidenteas if things were just very slow17:01
*** lblanchard has quit IRC17:02
openstackgerritDan Prince proposed openstack/os-net-config: Add MASTER=bond SLAVE=yes to linux bond interfaces  https://review.openstack.org/30182717:02
jistrgfidente: yea, also corosync log looks bad http://paste.fedoraproject.org/349953/14598757/17:04
jistr"Corosync main process was not scheduled for 3309.2935 ms" heh17:04
gfidenteyeah it's full of those messages17:04
jistryeah several things point to general sluggishness being the cause of the CI failures17:05
jistrcc trown|lunch ^^17:05
gfidentejistr so it seems corosync is keeps reforming the cluster every time the token goes above the threshold17:06
*** oshvartz has joined #tripleo17:08
jistrmaybe we could try to increase it as the message suggests :)17:09
*** masco has joined #tripleo17:10
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Wait 20 minutes for tenant stack  https://review.openstack.org/30183217:11
jistrgfidente: i will submit a wip patch for now to test the token timeout increase approach17:12
*** sanjay__u has quit IRC17:12
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Wait 20 minutes for tenant stack  https://review.openstack.org/30183217:12
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Apply os-net-config with a script instead of element  https://review.openstack.org/27145017:12
*** apetrich has quit IRC17:13
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: [WIP] Increase corosync token timeout  https://review.openstack.org/30183617:15
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Apply os-net-config with a script instead of element  https://review.openstack.org/27145017:15
*** weshay has quit IRC17:17
jistrgfidente++ thanks for the help. WIP patch above ^^, we'll see17:20
*** mgould has quit IRC17:21
*** Marga_ has quit IRC17:22
*** dshulyak has quit IRC17:23
*** jistr has quit IRC17:24
openstackgerritBen Nemec proposed openstack/tripleo-heat-templates: Reload haproxy after injecting certs w/o pcmk too  https://review.openstack.org/30184017:26
*** akuznetsov has joined #tripleo17:28
*** shardy has quit IRC17:34
*** ramishra has quit IRC17:34
*** weshay has joined #tripleo17:36
*** shivrao has joined #tripleo17:38
*** jdob has quit IRC17:41
*** coolsvap has quit IRC17:41
*** jdob has joined #tripleo17:41
*** coolsvap has joined #tripleo17:42
*** davidlenwell has quit IRC17:42
*** mcornea has quit IRC17:46
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: TEST do not merge  https://review.openstack.org/30184917:47
*** julim has joined #tripleo17:48
*** davidlenwell has joined #tripleo17:49
*** ChanServ sets mode: +v davidlenwell17:49
*** bvandenh has quit IRC17:52
*** jdob has quit IRC17:53
*** jdob has joined #tripleo17:53
*** Marga_ has joined #tripleo17:53
*** mkovacik has joined #tripleo17:54
openstackgerritBen Nemec proposed openstack/instack-undercloud: Remove centos-cloud-repo element  https://review.openstack.org/30185117:54
*** coolsvap has quit IRC17:54
*** sshnaidm has joined #tripleo17:59
*** shivrao has quit IRC18:00
*** lblanchard has joined #tripleo18:01
openstackgerritDan Prince proposed openstack/os-net-config: Add MASTER=bond SLAVE=yes to linux bond interfaces  https://review.openstack.org/30182718:01
*** trown|lunch is now known as trown18:02
*** ohamada has quit IRC18:03
*** masco has quit IRC18:03
*** bvandenh has joined #tripleo18:06
*** dtantsur is now known as dtantsur|afk18:08
*** akuznetsov has quit IRC18:13
Gonerihi, I maintain a python lib that I use to do TripleO deployment. At this point, It's called python-rdo-m-helper and I'm about to rename it python-tripleo-helper. https://github.com/redhat-cip/python-rdo-m-helper18:19
*** paramite has joined #tripleo18:23
*** bvandenh has quit IRC18:24
*** dshulyak has joined #tripleo18:24
openstackgerritMerged openstack/tripleo-docs: Streamline the basic deployment flow  https://review.openstack.org/25919718:25
openstackgerritDan Prince proposed openstack/os-net-config: Bump hacking in test-requirements.txt  https://review.openstack.org/30186318:28
dprincebnemec: is that hacking error ^^^ something you've seen yet?18:29
dprincebnemec: https://bugs.launchpad.net/tripleo/+bug/156646318:29
openstackLaunchpad bug 1566463 in tripleo "os-net-config: hacking fails to install" [High,New] - Assigned to Dan Prince (dan-prince)18:29
*** akuznetsov has joined #tripleo18:30
*** egafford has quit IRC18:36
dprinceslagle: you wanna have a look at this so that if it passes we can land it18:37
dprinceslagle: https://review.openstack.org/#/c/301554/18:37
dprinceslagle: I'm actually wondering if we need both patches (yours and Steve's) in order to get somewhat more stable results18:37
dprinceslagle: and should we just go on and land them both if we see any of the jobs passing18:38
*** akuznetsov has quit IRC18:38
slagledprince: +2'd. i'm fine to push them through if we wanted to do that18:39
slaglei havent seen a single patch pass all 3 jobs yet this week18:39
dprinceslagle: last 3 pass I saw was my heat patch from this weekend to t-h-t (that I abandoned)18:40
*** rcernin has quit IRC18:41
dprinceslagle: okay, landed them both18:41
dprinceslagle: lets see what that does18:41
dprinceslagle: we are now at risk for other heat breakages though18:42
trowndprince: you had https://review.openstack.org/#/c/300594/1 yesterday :p18:42
openstackgerritRyan Hallisey proposed openstack/diskimage-builder: Workaround to build Fedora 23 images  https://review.openstack.org/30186618:42
openstackgerritMerged openstack-infra/tripleo-ci: Test with latest master heat  https://review.openstack.org/30155418:42
dprincetrown: oh, nice. I hadn't even noticed18:42
slagledprince: yea, i think we should revert shardy's patch once we get current-tripleo promoted18:42
dprinceslagle: ++18:43
trownbut there have been about 2 patches per day passing all three jobs which is pretty rough18:43
openstackgerritMerged openstack-infra/tripleo-ci: Wait 20 minutes for tenant stack  https://review.openstack.org/30183218:43
*** akuznetsov has joined #tripleo18:43
rhalliseyanyone know how to increase the size of the instack vm?18:49
rhalliseyon creation18:50
rhallisey30G is not enough in container land :)18:50
*** lblanchard has quit IRC18:51
slaglerhallisey: on creation, you'd have to hack instack-virt-setup18:52
slaglethere is a hardcoded "30" in there that you'd change18:52
trownrhallisey: you could try tripleo-quickstart :), it is configurable there18:52
slagleoh look at this guy with his fancy quickstart18:52
trownit is kind of like a giant container18:53
slagleafter creation, i think qemu-img resize will work, and then xfs_growfs when the image is booted18:53
rhalliseyslagle, ok I think that's the part I was missing18:53
trownya, that would probably be simpler18:53
rhalliseytrown, ya hardcord ftw18:53
rhallisey8 node virt-virt containerized HA cluster here we come :)18:56
jaosoriorbnemec: Hey dude, got time to check this one out? https://review.openstack.org/#/c/299475/19:05
jaosoriorand if someone wants to see some code removed from the loadbalancer manifest, here's some :D https://review.openstack.org/#/c/301493/19:05
*** akuznetsov has quit IRC19:11
bnemecdprince: I can't say that I have.  It isn't happening for me locally either.19:27
dprincebnemec: weird. I saw it on both Fedora 23 and Fedora 2219:28
dprincebnemec: maybe it is a Fedora thing19:28
dprincebnemec: did you try cleaning your .tox directory maybe19:28
dprincebnemec: anyways, don't spend to much time on it.19:28
*** pradk has quit IRC19:28
bnemecdprince: My VM is still on 21.  Maybe it's a dnf thing?19:28
dprinceprobably safe to bump that dependency anyways19:28
*** pradk has joined #tripleo19:29
bnemecI did tox -r19:29
*** ibravo2 has joined #tripleo19:29
dprincebnemec: yeah, Well I'll look into the code changes19:29
dprincebnemec: aside from the pep8 tests I needed this to fix py27 too19:30
dprincebnemec: I'll just carry it locally until I get the hacking issues fixed. May as well bump it I guess19:30
bnemecdprince: Yeah, I'm totally good with bumping our hacking version.19:30
*** julim has quit IRC19:30
* bnemec should probably move his dev vm to a supported version anyway...19:31
*** mburned has left #tripleo19:32
*** mburned has joined #tripleo19:32
openstackgerritRyan Hallisey proposed openstack/diskimage-builder: Workaround to build Fedora 23 images  https://review.openstack.org/30186619:33
*** akuznetsov has joined #tripleo19:34
*** jprovazn has quit IRC19:34
*** Goneri has quit IRC19:35
*** akuznetsov has quit IRC19:47
*** paramite has quit IRC19:49
*** ibravo2 has quit IRC19:53
*** jaosorior has quit IRC19:54
*** Goneri has joined #tripleo19:55
*** egafford has joined #tripleo19:55
*** dprince has quit IRC19:57
*** eil397 has left #tripleo20:07
*** akrivoka has quit IRC20:10
*** apetrich has joined #tripleo20:17
*** ibravo has quit IRC20:17
*** jayg is now known as jayg|g0n320:25
*** panda has quit IRC20:26
*** panda has joined #tripleo20:27
*** david-lyle_ has joined #tripleo20:28
*** david-lyle has quit IRC20:29
*** david-lyle_ is now known as david-lyle20:29
weshaytrown, ok.. have a undercloud-post role now, so I'm setup to try and get net-iso working w/ the outside roles20:33
trownweshay: nice I saw that repo go up20:36
weshayya.. the only diff w/ this one is that I'll start to use --skip-tags20:37
trownI think network isolation is a pretty fundamental requirement for dev environments, so I would love for that to get added to the tripleo-quickstart roles too20:37
weshaytrown, agree20:38
trownsince it will be a bit before it would consume the external roles20:39
*** dtrainor has quit IRC20:44
*** Goneri has quit IRC20:48
*** rhallisey has quit IRC20:49
*** rhallisey has joined #tripleo20:51
*** rhallisey has quit IRC20:52
openstackgerritxin wu proposed openstack/os-net-config: Set check_exit_code to False for ifup  https://review.openstack.org/30191220:53
*** ebalduf has joined #tripleo20:54
openstackgerritxin wu proposed openstack/os-net-config: Set check_exit_code to False for ifup  https://review.openstack.org/30191220:58
*** trown is now known as trown|outtypewww20:59
openstackgerritxin wu proposed openstack/os-net-config: Set check_exit_code to False for ifup  https://review.openstack.org/30191221:00
*** myoung is now known as myoung|afk21:02
*** florianf has quit IRC21:02
slaglelooking at a failed ha job21:02
slaglei assume it's not good that 2 of the galera nodes have partitioned21:02
bnemecSounds :-(21:03
slaglei wonder if this just another symptom of stuff running too slow21:03
bnemecMoar hardwares plz21:03
* bnemec should really get added to tripleo-ci-admins so he can log in to these systems21:04
bnemecI wonder if we just need to drop the number of testenvs again until we get more/better hardware.21:04
slaglewell this is all in the mysqld.log21:04
bnemecYeah, but I'm wondering what the bottleneck on the hardware is.21:05
*** dmacpher has joined #tripleo21:06
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Deploy Gnocchi as a Ceilometer metrics storage backend  https://review.openstack.org/25203221:08
slaglebnemec: i see some cpu throttled messages on this host. i'll have to check if the timestamps match up21:11
*** morazi has quit IRC21:13
*** hewbrocca is now known as hewbrocca-afk21:20
*** rhallisey has joined #tripleo21:23
*** tiswanso has quit IRC21:33
*** tiswanso has joined #tripleo21:34
*** tiswanso has quit IRC21:34
*** rbrady has quit IRC21:38
*** gfidente has quit IRC21:48
*** gfidente has joined #tripleo21:48
*** gfidente has quit IRC21:48
*** gfidente has joined #tripleo21:48
*** gfidente has quit IRC21:54
*** MaxPC has quit IRC21:59
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Use split cirros image for ping test  https://review.openstack.org/30195122:15
*** dshulyak has quit IRC22:23
*** leanderthal|afk has quit IRC23:13
*** Marga__ has joined #tripleo23:15
*** Marga__ has quit IRC23:15
*** Marga_ has quit IRC23:18
*** Marga_ has joined #tripleo23:25
*** leanderthal|afk has joined #tripleo23:25
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart: add facts for fedora  https://review.openstack.org/30117423:34
*** Goneri has joined #tripleo23:36
*** Marga_ has quit IRC23:37
*** akshai has quit IRC23:37
*** Marga_ has joined #tripleo23:44
*** Marga_ has quit IRC23:46
*** Marga_ has joined #tripleo23:46
*** Marga_ has quit IRC23:50
*** Marga_ has joined #tripleo23:51
*** Marga_ has quit IRC23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!