Friday, 2016-12-16

*** mhenkel has quit IRC00:04
*** dsneddon_ has joined #tripleo00:06
*** ayoung has quit IRC00:08
*** thrash has quit IRC00:09
*** trown|outtypewww has quit IRC00:09
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974200:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]00:10
*** ooolpbot has quit IRC00:10
*** bfournie has joined #tripleo00:11
openstackgerritIan Wienand proposed openstack/diskimage-builder: Install dracut-generic-config package  https://review.openstack.org/41154000:16
openstackgerritMikhail S Medvedev proposed openstack/diskimage-builder: Fix bootloader element on ppc  https://review.openstack.org/41154100:16
*** trown has joined #tripleo00:19
*** thrash has joined #tripleo00:20
*** thrash has quit IRC00:20
*** thrash has joined #tripleo00:20
EmilienMdsneddon: are we still on track for https://blueprints.launchpad.net/tripleo/+spec/tripleo-lldp-validation in ocata?00:31
*** rwsu has quit IRC00:32
openstackgerritMerged openstack-infra/tripleo-ci: Remove swapfile from undercloud  https://review.openstack.org/41032300:38
*** rwsu has joined #tripleo00:44
*** limao has joined #tripleo00:46
*** ayoung has joined #tripleo00:53
*** saneax is now known as saneax-_-|AFK00:58
openstackgerritjeck proposed openstack/puppet-tripleo: [TrivialFix] Fix typo error in comment  https://review.openstack.org/41155001:07
*** ctayal has quit IRC01:09
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974201:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]01:10
*** ooolpbot has quit IRC01:10
*** george_goh has quit IRC01:11
*** ctayal has joined #tripleo01:12
*** ctayal has quit IRC01:14
openstackgerritjeck proposed openstack/puppet-tripleo: [TrivialFix] Fix typo error in comment  https://review.openstack.org/41155101:16
*** mhenkel has joined #tripleo01:19
*** bana_k has quit IRC01:21
*** george_goh has joined #tripleo01:21
*** mhenkel has quit IRC01:24
*** sshnaidm is now known as sshnaidm|afk01:58
*** jeckersb is now known as jeckersb_gone01:59
*** jeckersb_gone is now known as jeckersb02:00
*** ctayal has joined #tripleo02:06
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974202:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]02:10
*** ooolpbot has quit IRC02:10
*** ctayal_ has joined #tripleo02:12
dsneddon_EmilienM: I think we've got good progress on reviews for LLDP data collection. If it's not fully polished by Ocata, at least most of the building blocks will be done.02:13
*** ctayal has quit IRC02:13
dsneddon_EmilienM: Enough to call the blueprint completed, I would guess.02:13
openstackgerritIan Wienand proposed openstack/diskimage-builder: Install dracut-generic-config package  https://review.openstack.org/41154002:19
*** fzdarsky_ has joined #tripleo02:24
*** fzdarsky|afk has quit IRC02:28
openstackgerritzhangyanxian proposed openstack/puppet-tripleo: Fix typo in endpoint.pp  https://review.openstack.org/41158802:39
openstackgerritzhangyanxian proposed openstack/puppet-tripleo: Fix typo in endpoint.pp  https://review.openstack.org/41158802:39
*** TSCHAK_ has quit IRC02:44
*** sai-out is now known as sai02:58
*** yamahata has quit IRC02:58
*** cwolferh has quit IRC03:01
*** thrash is now known as thrash|g0ne03:05
*** fragatina has quit IRC03:07
*** fragatina has joined #tripleo03:08
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974203:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]03:10
*** mhenkel has joined #tripleo03:10
*** fragatina has quit IRC03:12
*** fragatina has joined #tripleo03:12
*** ctayal_ has quit IRC03:12
*** mhenkel has quit IRC03:14
*** fragatin_ has joined #tripleo03:15
*** fragatina has quit IRC03:15
*** fragatin_ has quit IRC03:18
*** sudipto has joined #tripleo03:19
*** fragatina has joined #tripleo03:19
openstackgerritBrad P. Crochet proposed openstack/tripleo-docs: Reflect default usage of image build  https://review.openstack.org/40983903:19
*** fragatina has quit IRC03:24
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates: GATE TEST, please ignore  https://review.openstack.org/36544903:31
*** yamahata has joined #tripleo03:44
*** ramishra has quit IRC03:47
*** limao has quit IRC03:57
*** limao has joined #tripleo03:57
*** limao has quit IRC04:01
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974204:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]04:10
*** dmacpher is now known as dmacpher-afk04:26
*** udesale has joined #tripleo04:51
*** bana_k has joined #tripleo05:03
*** ooolpbot has joined #tripleo05:11
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:11
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974205:11
*** ooolpbot has quit IRC05:11
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]05:11
*** prateek has joined #tripleo05:13
*** links has joined #tripleo05:21
*** Vijayendra has joined #tripleo05:24
*** dsneddo__ has joined #tripleo05:25
*** fragatina has joined #tripleo05:26
*** dsneddon_ has quit IRC05:29
*** fragatina has quit IRC05:36
*** fragatina has joined #tripleo05:36
*** masco has joined #tripleo05:36
*** bana_k has quit IRC05:38
*** ctayal has joined #tripleo05:42
*** ramishra has joined #tripleo05:43
*** saneax-_-|AFK is now known as saneax05:44
*** jaosorior has joined #tripleo06:02
*** limao has joined #tripleo06:04
*** dmacpher-afk is now known as dmacpher06:05
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974206:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]06:10
*** ooolpbot has quit IRC06:10
*** pgadiya has joined #tripleo06:12
*** rcernin has quit IRC06:13
*** bana_k has joined #tripleo06:24
*** ctayal_ has joined #tripleo06:29
*** dsneddo__ has quit IRC06:29
*** ctayal has quit IRC06:30
*** rcernin has joined #tripleo06:37
*** lmiccini has joined #tripleo06:37
bandinimorning06:40
jaosoriorbandini: sup dude06:42
*** rcernin has quit IRC06:43
jaosorioralee: should we merge this https://review.openstack.org/#/c/408783/ ?06:43
*** ctayal_ has quit IRC06:44
*** matbu is now known as matbu|halfpto06:51
*** rcernin has joined #tripleo06:54
*** yprokule has joined #tripleo07:01
*** ramishra has quit IRC07:01
*** abehl has joined #tripleo07:05
*** ramishra has joined #tripleo07:06
*** paramite has joined #tripleo07:09
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974207:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]07:10
*** dsneddon_ has joined #tripleo07:14
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo: Initial pacemaker remote profile support  https://review.openstack.org/40096707:16
*** mhenkel has joined #tripleo07:16
*** leanderthal|afk is now known as leanderthal07:18
*** dsneddon_ has quit IRC07:19
*** bnemec has quit IRC07:21
*** abregman has joined #tripleo07:23
d0ugalNg: did you manage to figure it out?07:30
*** jpena|off is now known as jpena07:35
*** bnemec has joined #tripleo07:37
*** dsariel has quit IRC07:39
*** abregman has quit IRC07:39
*** aufi has joined #tripleo07:39
*** derekh has joined #tripleo07:40
derekhsshnaidm|afk: panda|Zz bnemec EmilienM new problem today, the instances nodepool is starting wont boot, and it keeps trying over and over07:42
jaosoriorX_X\07:42
derekhconsole log shows this http://chunk.io/f/188ecaea89224868bd091e9c9f13bca507:43
derekhI'm thinking nodepool has just switched onto a new image with problems (maybe a switch from 7.2 too 7.3)07:44
derekhto avoid the cloud being DOS'd I've run this "iptables -I INPUT -s 23.253.73.160 -j DROP"07:44
jaosoriorA start job is running for dev-disk...07:44
jaosoriorand it gets stuck there?07:44
derekhthis blocks out infra while we figure it out07:44
*** mcornea has joined #tripleo07:45
*** rasca has joined #tripleo07:45
derekhjaosorior: I havn't looked at any details07:45
derekhjaosorior: getting kids ready for school etc... I'll be back later07:45
derekhmainly I jumped in to tell people about the iptables rule so they wouldn't start debuging that07:46
jaosoriorderekh: thanks07:46
derekhttyl07:47
*** derekh has quit IRC07:47
*** iranzo has joined #tripleo07:54
*** iranzo has joined #tripleo07:54
*** tobias-fiberdata has quit IRC07:55
*** pcaruana has joined #tripleo07:56
*** florianf has joined #tripleo07:57
*** tesseract has joined #tripleo07:57
*** tesseract is now known as Guest3130407:58
openstackgerritDougal Matthews proposed openstack-infra/tripleo-ci: Update my feed to use the tripleo tag  https://review.openstack.org/41168708:06
openstackgerritDougal Matthews proposed openstack-infra/tripleo-ci: Use a more specific feed for my blog  https://review.openstack.org/41168708:07
*** fragatina has quit IRC08:09
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974208:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]08:10
*** shardy has joined #tripleo08:16
*** arxcruz has quit IRC08:20
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci: Implement major upgrade for Newton to Ocata  https://review.openstack.org/40483108:21
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add hook to generate metadata from service profiles  https://review.openstack.org/41133908:26
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add metadata settings for needed kerberos principals  https://review.openstack.org/41134008:26
jaosoriorshardy: this is the hook that I talked about yesterday https://review.openstack.org/#/c/411339/3  and this is an implementation of it https://review.openstack.org/#/c/411340/408:27
*** fzdarsky_ is now known as fzdarsky08:29
*** bana_k has quit IRC08:31
*** amoralej|off is now known as amoralej08:31
*** rlandy|bbl is now known as rlandy08:31
*** rlandy has quit IRC08:31
*** jaosorior has quit IRC08:32
*** ramishra has quit IRC08:32
*** jaosorior has joined #tripleo08:33
*** flepied has joined #tripleo08:33
*** pblaho has quit IRC08:42
*** hogepodge has quit IRC08:42
*** udesale has quit IRC08:44
*** hogepodge has joined #tripleo08:44
*** spredzy has joined #tripleo08:44
*** flepied has quit IRC08:49
*** ccamacho|out is now known as ccamacho08:55
*** jaosorior has quit IRC08:55
*** dsneddon_ has joined #tripleo08:55
Ngd0ugal: actually no, I decided to sleep instead of keep bashing at it :D08:56
Ngd0ugal: so if you have any hints, I would be very grateful!08:56
*** ohamada has joined #tripleo08:58
*** udesale has joined #tripleo09:00
*** lucas-afk is now known as lucasagomes09:04
*** jbadiapa has joined #tripleo09:06
*** dsneddo__ has joined #tripleo09:06
*** dsnedd___ has joined #tripleo09:08
*** dsneddon_ has quit IRC09:09
*** hewbrocca_afk is now known as hewbrocca09:10
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974209:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]09:10
*** tobias_fiberdata has joined #tripleo09:10
*** dsneddo__ has quit IRC09:12
*** dsneddon_ has joined #tripleo09:14
*** dsneddo__ has joined #tripleo09:16
d0ugalNg: Let me check I understand the issue. You are making changes to tripleoclient but then when you run "openstack overcloud deploy" (or another command) the changes don't seem to be there?09:16
d0ugalNg: Which command are you working on?09:16
Ngd0ugal: I'm adding a new command, and trying to get it to show up in the openstack command list09:17
d0ugalNg: ah, did you add it here? https://github.com/openstack/python-tripleoclient/blob/master/setup.cfg#L5709:17
Ngd0ugal: yep09:18
shardyNg: have you done pip install . in the tripleoclient git tree and/or done a delorean build of the tripleoclient tree?09:18
*** dsnedd___ has quit IRC09:18
Ngshardy: I did a setup.py install09:18
d0ugalYeah, that ^09:18
shardysetup.py install isn't enough to register the osc plugins IIRC09:18
d0ugalit should be :)09:18
shardylast time I tried, it wasn't09:18
d0ugalah, interesting. I wonder why.09:18
Ngaha!09:18
*** dsneddon_ has quit IRC09:19
Ngas always, thanks!09:19
*** lucasagomes is now known as lucas-brb09:19
*** flepied has joined #tripleo09:19
shardyNg: np, shout if that doesn't work :)09:19
Ngshardy: will do. I must owe you approx one metric brewery of beers by now ;)09:20
*** dsneddon_ has joined #tripleo09:21
d0ugalI think we all do09:21
openstackgerritAlfredo Moralejo proposed openstack-infra/tripleo-ci: Update packates after configuring openstack repos  https://review.openstack.org/41172509:24
shardyhehe, no worries, happy to help :)09:24
*** yamahata has quit IRC09:24
*** dsneddo__ has quit IRC09:24
openstackgerritAlfredo Moralejo proposed openstack-infra/tripleo-ci: Update packages after configuring openstack repos  https://review.openstack.org/41172509:25
shardyCan anyone give an update on the status of the broken ovb ha jobs?09:27
d0ugalThe last update I seen was from bnemec on https://bugs.launchpad.net/tripleo/+bug/164974209:28
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]09:28
*** athomas has joined #tripleo09:28
shardyOk https://review.openstack.org/#/c/411514 seems to be the partial workaround for that09:29
shardylets hope that passes the gate this time09:29
d0ugalshardy: doesn't look like it will09:30
d0ugalI was just looking at the status in zuul09:30
shardyugh09:31
* shardy thinks we should focus on multinode testing from now on09:32
*** panda|Zz is now known as panda09:33
pandashardy: we don't have a root cause yet, we are speculating that the te client is unable to receive the exit of the test runner, but we don't know why, and why only on ha jobs09:35
*** gfidente has joined #tripleo09:35
*** gfidente has quit IRC09:35
*** gfidente has joined #tripleo09:35
shardypanda: Ok, any ideas how we debug further?09:37
shardythe ha job enables network isolation, but it sounds like the problem is a layer below that?09:37
shardyunless our net-iso stuff breaks the network somehow09:38
*** zoli|gone is now known as zoli09:38
*** derekh has joined #tripleo09:41
derekh<shardy>Can anyone give an update on the status of the broken ovb ha jobs?09:42
derekhshardy: we got a new problem thismorning09:42
derekhshardy: all slaves that nodepool was bringing up are failing to boot09:43
shardyderekh: Ok.  Wow it's been a rough week for CI :(09:44
derekhshardy: ya, I think this might be related to the switch to centos 7.3 , that will be the 3rd 7.3 related thing...09:45
d0ugaldang09:45
derekhshardy: and we had a rh1 problem on monday09:45
*** dsneddon_ has quit IRC09:45
*** dsneddon has quit IRC09:46
*** dsneddon_ has joined #tripleo09:46
*** gfidente has quit IRC09:47
*** dsariel has joined #tripleo09:48
*** shinobu__ has joined #tripleo09:49
*** Vijayendra_ has joined #tripleo09:52
*** gfidente has joined #tripleo09:53
*** Vijayendra has quit IRC09:54
Ngshardy: pip install worked btw :)09:57
shardyNg: excellent :)09:58
*** chem has joined #tripleo09:59
d0ugalI really want to know why they are different :)09:59
*** pgadiya has quit IRC10:01
derekhsshnaidm|afk: panda EmilienM bnemec yup, that latest problem is related to CentOS 7.3 https://bugzilla.redhat.com/show_bug.cgi?id=140523810:05
openstackbugzilla.redhat.com bug 1405238 in util-linux "findmnt --target behaviour changed in 7.3, shows all mount-points in chroot" [Unspecified,New] - Assigned to kzak10:05
derekhinfra have the same problem on other clouds10:05
pandaderekh: nobody tested this before upgrading :(10:07
panda?10:07
*** milan has joined #tripleo10:07
*** chem has quit IRC10:08
*** chem has joined #tripleo10:08
derekhpanda: nodepool just builds a new image and uploads it to the clouds it uses, there isn't any test on it that I know of, in theory infra can just flick a switch to switch back to a old image if needed10:08
derekhpanda: but nobody on #infra at the moment to do it10:08
derekhbrb10:09
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974210:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]10:10
*** dsneddon_ has quit IRC10:11
*** pgadiya has joined #tripleo10:13
*** akrivoka has joined #tripleo10:14
*** dsariel has quit IRC10:15
sshnaidm|afkderekh, panda EmilienM bnemec https://review.openstack.org/#/c/410470/6 - not ideal, but could be also solution for "testenv timeouts"10:16
pandasshnaidm|afk: isn't the job marked as failure when client reaches time out ?10:18
*** openstackgerrit has quit IRC10:18
pandasshnaidm|afk:  seems to be working10:21
*** dsneddon has joined #tripleo10:26
*** ealcaniz has joined #tripleo10:26
sshnaidm|afkpanda, no, it passed there10:28
*** limao has quit IRC10:29
*** zoli is now known as zoli|lunch10:30
*** lucas-brb is now known as lucasagomes10:31
*** arxcruz has joined #tripleo10:32
*** florianf has quit IRC10:36
*** b00tcat has quit IRC10:36
*** numans has quit IRC10:38
*** florianf has joined #tripleo10:41
*** numans has joined #tripleo10:45
pandais there a single component that installs firewall rules on the undercloud, or every element takes care of it ?10:48
*** saibarspeis has joined #tripleo10:51
shardypanda: they're managed by puppet, via the same puppet-tripleo manifest we use for the overcloud10:51
shardyhttps://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.pp#L6610:51
*** openstackgerrit has joined #tripleo10:51
openstackgerritAlfredo Moralejo proposed openstack-infra/tripleo-ci: Update packages after configuring openstack repos  https://review.openstack.org/41172510:51
shardyhttps://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.yaml.template#L64610:51
shardypanda: ^^10:51
pandashardy: ok, and this https://github.com/openstack/tripleo-image-elements/blob/master/elements/geard/os-refresh-config/pre-configure.d/97-gearman-iptables is only used to build the overcloud image ?10:53
*** pblaho has joined #tripleo10:55
*** prateek has quit IRC10:59
*** b00tcat has joined #tripleo11:03
*** b00tcat has quit IRC11:03
*** b00tcat has joined #tripleo11:03
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974211:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165050311:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]11:10
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]11:10
*** milan has quit IRC11:16
*** chem has quit IRC11:18
*** tosky has joined #tripleo11:19
*** chem has joined #tripleo11:19
shardypanda: not sure about that one tbh - there's a bunch of element stuff which is no longer used, but we've not done the work to figure it all out and remove it11:20
openstackgerritLucas Alvares Gomes proposed openstack/tripleo-quickstart: VirtualBMC support for tripleo-quickstart  https://review.openstack.org/39970411:25
*** prateek has joined #tripleo11:27
shardyhrm, has something changed in how we write overcloudrc recently?11:31
shardymine is broken as it's pointing directly to the controller not the vip11:31
*** jaosorior has joined #tripleo11:38
jaosoriorshardy: hey, just saw your review on the metadata hook11:39
*** zoli|lunch is now known as zoli11:40
shardyjaosorior: yeah just had a couple of questions11:41
jaosoriorshardy: So, I initially had it in the service profiles since I was using the servername, but it turned out that's not necessary as we can assume that from the data nova gives out. However, regarding this https://review.openstack.org/#/c/411340/4/extraconfig/nova_metadata/krb-service-principals.yaml  the input parameters are actually specific to servers and roles11:41
jaosoriorshardy: for instance https://review.openstack.org/#/c/411340/4/puppet/services/aodh-api.yaml11:41
jaosoriorthat will end up giving telling the stack to pass that metadata for instances that deploy that service11:42
jaosoriorand that's needed, since we have composable services, and we can't sure that all the services are deployed together11:43
jaosoriorI haven't merged TLS for rabbitMQ, but that will be the same case as the apache services11:43
jaosoriorsay that we have a role that is dedicated to just the message broker, then that set of nodes would get the relevant metadata for the rabbitmq service (and thus will be able to set up the relevant service principals)11:44
*** jkilpatr has quit IRC11:44
shardyjaosorior: Yeah I get the per-service part, I'm just not clear why we need S::TripleO::ServerMetadataHook yet in every role11:45
jaosoriorand, for instance, the computes, don't need to get the metadata for apache, since it doesn't need it11:45
jaosoriorshardy: I could move it to services.yaml11:45
jaosoriorshardy: would that be a better place?11:45
*** thrash|g0ne is now known as thrash11:46
*** dsneddon has quit IRC11:48
shardyjaosorior: I guess I'm wondering why the per-service metadata you have in krb-service-principals can't go in a service template, then we don't need the new hook at all, only the wiring of the metadata_settings into each role11:49
shardyperhaps there's some later patch which will make that clear tho :)11:49
jaosoriorshardy: uhm... how could it be in a service template?11:50
*** milan|afk has joined #tripleo11:51
jaosoriorso basically krb-service-principals parses what the metadata_settings in the roles gives, having attempted to make it slightly generic so this wouldn't be the only use case. It subsequently outputs it in a format that the vendordata plugin expects11:52
jaosoriorshardy: in the service templates, we don't have visibility of all the services that are enabled per-role, so I wouldn't be able to do it there11:52
EmilienMhello11:53
shardyjaosorior: So there are other data formats we expect to support besides this one?11:53
*** dsneddon has joined #tripleo11:54
jaosoriorshardy: well, vendordata plugins are not a limited use-case. When we were discussing those in the summit it turned out a bunch of deployers were using those for very different things. So I thought it would be a good thing to support; that's why I attempted to make it generic11:54
EmilienMsshnaidm|afk, panda, derekh: /me reading backlog11:55
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: Postci tests trial and error  https://review.openstack.org/41118911:55
jaosoriorshardy: I could ditch the attempt to make it generic and just parse the output though, then I would rename it to something more specific to this vendordata plugin11:56
shardyjaosorior: Ok, fair enough.  I guess I started looking at the hard-coded references to eg haproxy and mysql and that made me think of defining the data with those services11:56
derekhEmilienM: tldr is that nodepool uploaded a new centos image that doesn't work (another centos 7.3 problem) https://bugs.launchpad.net/tripleo/+bug/165050311:56
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]11:56
shardyjaosorior: I'm OK with the current approach provided we actually need it to be generic and pluggable, I just wanted to clarify11:57
EmilienMderekh: I guess it's to early to have infra folks reverting the image11:57
shardythanks for the additional info11:57
derekhEmilienM: while we wait for somebody in infra to revert back to the old image I've blocked nodepools access to rh111:57
jaosoriorshardy: well, nobody has requested it yet, maybe I over-thought it.11:57
EmilienMderekh: I guess everything is blocked now11:57
derekhEmilienM: yup, I asked nobody there to do it yet11:57
derekhEmilienM: yup, jobs a just queueing up and not running11:57
jaosorior(talking about actual BZs)11:57
jaosoriorbut who knows11:57
EmilienMderekh: let me try to ping people11:58
EmilienMderekh: I'll be that guy today11:58
pandaderekh: in the meantime, is there something we could do to fix the 7.3 image ?11:58
*** akrivoka has quit IRC11:58
derekhpanda: I don't think so, its infras image we can't instruct nodepool to use a different one12:00
derekhpanda: infra ppl were working on a fix last night and deleted a broken image, but I guess forgot nodepool would build and upload a new one12:01
jaosoriorshardy: anyway, thanks for taking a look dude, I really appreciate your feedback; change the names as you suggested and parse the output in services.yaml, that way there's less changes in the actual roles templates12:04
*** tobias-fiberdata has joined #tripleo12:05
*** tobias_fiberdata has quit IRC12:06
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974212:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165050312:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]12:10
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]12:10
*** akrivoka has joined #tripleo12:13
*** dtantsur|afk is now known as dtantsur12:16
*** ramishra has joined #tripleo12:16
*** jkilpatr has joined #tripleo12:16
openstackgerritJustin Kilpatrick proposed openstack/tripleo-quickstart-extras: Add introspection with retries option  https://review.openstack.org/40367712:18
*** chem has quit IRC12:23
*** ohamada has quit IRC12:24
*** ohamada has joined #tripleo12:24
*** chem has joined #tripleo12:24
*** paramite_ has joined #tripleo12:27
openstackgerritLucas Alvares Gomes proposed openstack/tripleo-quickstart: VirtualBMC support for tripleo-quickstart  https://review.openstack.org/39970412:27
*** b00tcat has quit IRC12:27
*** pkovar has joined #tripleo12:29
*** pgadiya has quit IRC12:31
*** jeckersb is now known as jeckersb_gone12:33
*** lucasagomes is now known as lucas-hungry12:35
openstackgerritBrad P. Crochet proposed openstack/tripleo-common: Implement stack update as mistral actions  https://review.openstack.org/37951612:39
*** links has quit IRC12:42
jaosoriorbandini: do you know if we're running cinder-api with pacemaker?12:43
*** ohamada has quit IRC12:43
*** ohamada_ has joined #tripleo12:43
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: WIP -- Add pacemaker ansible module of composable upgrade  https://review.openstack.org/40339712:44
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Adds a step0 for pre upgrade-init checks  https://review.openstack.org/40863112:44
EmilienMjaosorior: pcs status?12:44
mariosmatbu|halfpto: sorry i rebased your one too //review.openstack.org/403397  - rebased onto shardys https://review.openstack.org/#/c/411310/ which removes the upgrade init from steps12:45
*** tremble has joined #tripleo12:45
*** zoli is now known as zoliXXL12:46
*** dprince has joined #tripleo12:46
*** ansmith has joined #tripleo12:46
*** pgadiya has joined #tripleo12:47
jaosoriorshardy: is there a way for me to merge two lists via Heat? or should I use yaql for that?12:47
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Adds a step0 for pre upgrade-init checks  https://review.openstack.org/40863112:48
bandinijaosorior: only cinder volume and cinder backup (of the cinder ones)12:49
bandinijaosorior: if you now look at master https://github.com/openstack/puppet-tripleo/tree/master/manifests/profile/pacemaker you see which profiles still exist. we removed the non-used ones12:50
dprinced0ugal: got a minute to chat about https://review.openstack.org/#/c/410970/12:50
d0ugaldprince: sure12:50
jaosoriorbandini: ah, ok, seems that cinder-api is still in t-h-t12:50
d0ugaldprince: I hadn't seen your response - reading.12:50
dprinced0ugal: well... we might should wait for rbrady-afk but he can catch up I guess12:50
bandinijaosorior: right we should remove the corresponding tht ones as well. I'll do12:50
d0ugaldprince: what/where is the undercloud installer? This is the Heat undercloud installer right? not instack?12:51
dprinced0ugal: yes12:51
dprinced0ugal: https://review.openstack.org/#/c/351351/12:51
dprinced0ugal: https://etherpad.openstack.org/p/tripleo-composable-undercloud12:51
dprinced0ugal: probably more than you'd like to read right now12:52
d0ugaldprince: thanks, I'll take a look later. I was aware of the work but hadn't been following it as much as I probably should have.12:52
dprinced0ugal: I think I found a similar example to what I'm trying to do though. It is similar to the tarball pattern12:52
dprinced0ugal: I think we still use the tripleo_common.utils tarball function in python-tripleoclient12:52
dprinced0ugal: and we also consume it via a Mistral workflow12:53
dprinced0ugal: I think this is a fine pattern for things that have to happen locally12:53
d0ugaldprince: Yeah, i wanted to remove that use12:53
d0ugaldprince: :)12:53
dprinced0ugal: but how can you? other than duplicating it?12:53
d0ugaldprince: I hadn't gotten that far yet12:54
dprinced0ugal: here is a paste file of my tripleo-undercloud-passwords.yaml file (that works today) http://paste.openstack.org/show/592619/12:54
dprinced0ugal: I need pretty much all of those12:55
*** dmarlin_ has quit IRC12:55
dprinced0ugal: and this function would be a shame to duplicate. Keeping in mind that instack already has its own logic for this we'd essentially have 3 different password generators in TripleO12:55
dprinced0ugal: in the short term anyway...12:55
dprinced0ugal: the point of this is all feedback loops, and eliminating old elements, and eventually instack too12:56
dprinced0ugal: in short, I think we will have some low level functions for things like passwords, tarballs that need to get used in multiple projects12:56
d0ugalI think the problem is that it is totally unclear what tripleo-common is. If it is going to be a shared library then we should move the workflows stuff out into it's own repo12:57
dprinced0ugal: and it would be a good idea if we could share them12:57
*** b00tcat has joined #tripleo12:57
dprinced0ugal: I totally support that12:57
d0ugalAt the moment it walks this weird line between being everything and nothing12:57
dprinced0ugal: but the functions we are debating here I think should probably stay in tripleo-common then12:57
d0ugaldprince: Agreed.12:58
*** ealcaniz has quit IRC12:58
*** ansmith has quit IRC12:58
d0ugalMoving everything out of tripleo-common sounds like hard work :)12:59
d0ugaland I don't really know where we draw the line, so I am not sure it will ever become clearer12:59
dprinced0ugal: I didn't say it had to happen. But in the meantime I don't think it is that bad to reuse some code from tripleo_common.utils externally either12:59
d0ugaldprince: yeah, I guess. I don't like it but I don't have a better solution for you today.12:59
*** matbu|halfpto is now known as matbu13:00
dprinced0ugal: I suppose we could document our limited use intent for the utils code in the readme13:00
*** weshay_bbiab is now known as weshay13:01
d0ugaldprince: it would be nice to rename "generate_overcloud_passwords"13:01
d0ugal(i.e. remove "overcloud")13:01
dprinced0ugal: I'd be happy to do that as part of this patch13:02
*** morazi has joined #tripleo13:02
d0ugaldprince: cool, that would clear the usage up a bit at least too13:02
d0ugaldprince: so you actually want SnmpdReadonlyUserPassword to be a random password?13:03
d0ugalbecause that is where it will be initially created?13:03
dprinced0ugal: the first time it is generated random is fine13:04
dprinced0ugal: thereafter this file would get loaded and it would get re-used13:04
d0ugaldprince: right, makes sense.13:04
* d0ugal considers starting a larger question about what tripleo-common should be on openstack-dev13:05
*** fultonj has joined #tripleo13:05
openstackgerritFlavio Percoco proposed openstack/python-tripleoclient: Add container support to undercloud  https://review.openstack.org/37267113:05
dprinceshardy: hey, I got one for you. Any ideas on how we might make deployed_server run firstboot scripts13:07
EmilienMflaper87: you got a sec to talk about https://review.openstack.org/#/c/372671/ ?13:07
dprinceshardy: I'd like the capability to share docker initialization and perhaps OVS initialization across the overcloud/undercloud efforst13:07
EmilienMdprince: same for you ^13:07
openstackgerritFlavio Percoco proposed openstack/python-tripleoclient: Add container support to undercloud  https://review.openstack.org/37267113:08
EmilienMdprince, flaper87: before we go further, I would like to see a tripleo CI job similar to our "undercloud-only" job but with containers. It would be something like "undercloud-only-container" or something.13:08
EmilienMdprince, flaper87: and I wouldn't be in favor to review this patch until we have this job in place.13:09
*** dmarlin_ has joined #tripleo13:09
openstackgerritMerged openstack/tripleo-quickstart-extras: Add validation parameters to overcloud deploy  https://review.openstack.org/40472613:09
openstackgerritMerged openstack/tripleo-quickstart-extras: Replace hardcoded stack user by ansible_user  https://review.openstack.org/40480013:10
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974213:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165050313:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]13:10
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]13:10
flaper87EmilienM: yup13:10
EmilienMdprince, flaper87: I could help you to create the job ubt i'll need your help too13:10
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add hook to generate metadata from service profiles  https://review.openstack.org/41133913:11
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add metadata settings for needed kerberos principals  https://review.openstack.org/41134013:11
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Introduce role-specific nova-server-metadata  https://review.openstack.org/41054513:11
dprinceEmilienM: ++ on the jobs. I think we need 2 of them13:11
dprinceEmilienM: one for containers and one for baremetal13:11
dprinceEmilienM: both will be useful, especially if we engage scenarios alongside of them I think13:12
EmilienMdprince: and it's super fast, the undercloud-only take 20 or 30 min max13:12
EmilienMso it's not a big deal to add one (it's single node)13:12
dprinceflaper87: see my question to shardy above about reusing the firstboot scripts. You will likely care about my bootstrapping madness13:13
dprinceEmilienM: fast is nice :)13:13
flaper87dprince: need to read backlog13:13
flaper87EmilienM: we're working on CI, it's not like it's not important it's just taken a bit more time because master was broken for containers13:13
flaper87I agree we shouldn't merge this w/o ci13:14
flaper87and I don't expect it to land before next year anyway13:14
EmilienMflaper87: I can bootstrap it on project-config13:14
dprinceflaper87: because you said that I'm gonna land your patch right now13:14
flaper87dprince: rofl13:15
flaper87dprince: mine depends on yours13:15
flaper87:D13:15
dprinceflaper87: its fine. I'll land that too and just squash them into one big commit13:15
*** rbrady-afk is now known as rbrady13:16
flaper87dprince: and ninja-approve all the things13:16
flaper87EmilienM: so, I'm good with that, I just think it's too early to setup a CI for this undercloud thing13:16
flaper87it's not going to land now, really13:16
EmilienMflaper87: what about experimental pipeline?13:16
flaper87there are things still changing13:16
flaper87EmilienM: that could work13:16
EmilienMbut how do you test your code?13:17
dprinceEmilienM: container aside. I'm game for a CI job for the baremetal version13:17
EmilienMdprince: it's good to be game for that, we need to actually do it :D13:17
flaper87EmilienM: experimental sounds good if we want to kickstart something13:17
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add hook to generate metadata from service profiles  https://review.openstack.org/41133913:17
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add metadata settings for needed kerberos principals  https://review.openstack.org/41134013:17
dprinceand I'm optimistic on the containers one too. Once we get the bootstrapping issues sorted out I think it'll come fairly quickly13:17
EmilienMdprince: I talked with weshay yesterday and the missing thing is the oooq patch in tripleo-ci so we can enable the container ci job13:17
EmilienMflaper87: I'll bootstrap something13:18
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras: deployment updates for containerized compute  https://review.openstack.org/40098613:18
flaper87EmilienM: cool, I'll use that to help adding more CIs13:18
EmilienMdprince: I'll be optimistic when I'll see it working in CI13:18
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates: WIP: Configure Kernel Args and Tuned and then reboot for Compute  https://review.openstack.org/41179713:19
skramajashardy: ^13:19
skramajashardy: i am still testing, will update by next week...13:20
*** jcoufal has joined #tripleo13:20
*** masco has quit IRC13:20
flaper87dprince: what exactly did you say to shardy  ?13:21
* flaper87 can't find the logs13:21
dprinceflaper87: the question was if he had any ideas about how we might use firstboot scripts (user_data) with deployed_server to bootstrap things like docker and OVS13:22
dprinceflaper87: I would like to bootstrap these things consistently across the undercloud/overcloud efforts as we need them13:22
dprinceflaper87: for the overcloud, now that we are using overcloud-full though we could use packages (or elements) for some of it I guess13:23
jpenajust fyi: https://review.openstack.org/411800 should fix the tripleo CI, amoralej told me it was failing in https://review.openstack.org/41172513:25
*** pradk has joined #tripleo13:25
weshayEmilienM, I'll ping panda and trown and see if we can get it merged today13:26
EmilienMweshay: merge what? all CI is broken now13:26
weshayugh.. /me looks13:26
panda... and it gets worse each day13:26
weshayha still?13:27
flaper87dprince: mmh, I'd rather not depend on things coming from overcloud-full. I don't think we do that now but just saying13:27
flaper87dprince: also, +1 for consistency13:27
*** dr_gogeta86 has joined #tripleo13:29
*** dr_gogeta86 has joined #tripleo13:29
*** fragatina has joined #tripleo13:29
*** pgadiya has quit IRC13:30
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add metadata settings for needed kerberos principals  https://review.openstack.org/41134013:31
openstackgerritMerged openstack/tripleo-quickstart-extras: prep-network: add support to IPv6 topology  https://review.openstack.org/40083713:32
*** jayg|g0n3 is now known as jayg13:32
*** rlandy has joined #tripleo13:33
pandaweshay: ha still yes, we have a workaround, but infra images were updated to 7.3 and they are not booting13:33
shardydprince: Hey, sorry been grabbing lunch - can we configure the server so that the user_data gets put into a local datasource which cloud-init then reads?13:33
dprinceshardy: perhaps, if that works13:34
shardyhttp://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html13:34
dprinceshardy: I was thinking a bit more streamlined with having a mechanism to pass-thru the shell scripts into a Heat script element which runs via os-collect-config directly13:35
shardysomething like that perhaps - I'm thinking we could use a SoftwareDeployment that runs very early to bootstrap the cloud-init data and re-run cloud-init?13:35
shardydprince: Sure, that just won't work with e.g cloud-config yaml etc13:36
shardywe could have occ collect the user_data, write it, then run cloud-init via an o-r-c script tho I guess13:36
dprinceshardy: yes, that might work13:37
shardykind of a weird way to run cloud-init but it might work13:37
openstackgerritMerged openstack/tripleo-validations: Don't rely on overcloudrc  https://review.openstack.org/40080013:37
dprinceshardy: weird is okay w/ me13:37
shardyhehe :)13:37
shardymaybe we could juggle the service start ordering, so cloud-init actually runs after the o-r-c script13:38
jaosoriortrown: hey, could you check this out https://review.openstack.org/#/c/401452/ ?13:38
trownjaosorior: sure13:39
*** numans has quit IRC13:39
pradkgfidente, Hi13:40
pradkgfidente, is there any more info you need from us regarding https://bugs.launchpad.net/tripleo/+bug/1646506 ? I did a fresh deploy yesterday with ceph and aodh/gnocchi works fine for me13:41
openstackLaunchpad bug 1646506 in tripleo "Gnocchi / Aodh fail to work when RBD backend is enabled" [High,Confirmed]13:41
pradkgfidente, I'm wondering if this is a timing issue?13:41
weshaypanda, ah ya.. saw that early this morning13:42
*** numans has joined #tripleo13:42
pradkgfidente, may be in this ci scenario we could try configuring ceph earlier ?13:43
dprinceflaper87: the heat noauth middleware patch landed. So that is one less patch :)13:43
openstackgerritMerged openstack/tripleo-quickstart-extras: Install gcc to use upstream PXE  https://review.openstack.org/41029313:44
*** jpena is now known as jpena|lunch13:45
*** amoralej is now known as amoralej|lunch13:47
flaper87dprince: yeah, noticed this morning13:47
*** rhallisey has joined #tripleo13:49
*** lucas-hungry is now known as lucasagomes13:49
*** fultonj has quit IRC13:49
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras: deployment updates for containerized compute  https://review.openstack.org/40098613:52
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras: add the containers prep role to the quickstart-extras  https://review.openstack.org/40098313:52
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras: Create tasks to install, run stackviz and collect logs  https://review.openstack.org/40078213:55
*** fultonj has joined #tripleo13:56
*** dsneddon has quit IRC13:57
*** [1]cdearborn has joined #tripleo13:59
*** ealcaniz has joined #tripleo13:59
*** chlong has quit IRC14:00
openstackgerritMerged openstack/tripleo-quickstart-extras: Baremetal undecloud role playbook fixes  https://review.openstack.org/41133614:00
*** sudipto has quit IRC14:01
*** numans has quit IRC14:02
*** numans has joined #tripleo14:06
*** Goneri has joined #tripleo14:07
*** Vijayendra_ has quit IRC14:07
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974214:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165050314:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]14:10
*** Vijayendra_ has joined #tripleo14:15
*** almondjoy has joined #tripleo14:15
*** zoliXXL is now known as zoli|brb14:16
*** almondjoy has quit IRC14:19
*** almondjoy has joined #tripleo14:20
*** trown is now known as trown|brb14:22
*** bogdando has quit IRC14:23
thrashd0ugal: fwiw, tripleo-workflows would be just fine with me. :)14:23
EmilienMrook: ^ see the 2 alerts14:24
rookat 071014:24
rooklooking14:24
*** ealcaniz has quit IRC14:27
*** Vijayendra_ has quit IRC14:27
socialanyone deploying rdo master and able to do introspection?14:27
*** tzumainn has joined #tripleo14:27
jaosoriorderekh: any idea what's upw itht his http://logs.openstack.org/14/411514/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/3136b48/console.html#_2016-12-16_14_29_04_708997 ?14:30
jaosoriorseems commits now are failing in seconds O_o14:30
d0ugalthrash: interesting, thanks - maybe I'll propose that. I do think it came up in the past. /cc rbrady14:30
derekhjaosorior: trying to find it at the moment14:30
derekhjaosorior: thats a good way to clear our queue ;-)14:31
jaosoriorhahaha well, true that14:31
*** lblanchard has joined #tripleo14:31
*** prateek has quit IRC14:31
derekhjaosorior: found it, one sec14:34
derekhjaosorior: I was beaten to it https://review.openstack.org/#/c/411800/14:36
*** amoralej|lunch is now known as amoralej14:36
*** prateek has joined #tripleo14:37
matbushardy: the CI is broken ?14:37
*** trown|brb is now known as trown14:37
matbui was looking at your changes14:37
*** prateek has quit IRC14:38
*** prateek has joined #tripleo14:39
shardymatbu: Yeah, ovb was already broken but now the multinode job seems broken too :(14:39
shardyhttp://tripleo.org/cistatus.html14:39
shardyNot figured out why yet14:39
EmilienMshardy: I'm looking at it now14:39
derekhOk, first order of business, recheck this https://review.openstack.org/#/c/411514/ as soon as this lands https://review.openstack.org/#/c/411800/14:39
derekhthen we're ok, right?14:40
EmilienMit's in puppet catalog14:40
EmilienMderekh: yes14:41
EmilienMwe should14:41
EmilienMpradk, amoralej, jpena|lunch: http://logs.openstack.org/87/411687/2/check/gate-tripleo-ci-centos-7-nonha-multinode/25c5d50/logs/var/log/undercloud_install.txt.gz#_2016-12-16_08_57_01_00014:41
shardyderekh: Yeah that's one issue, but there's also a problem with the undercloud install now14:41
EmilienMamoralej: is it the thing you're trying to fix with yum update?14:41
EmilienMshardy: the multinode problem is a 7.3 thing again14:41
amoralejhttps://review.openstack.org/#/c/411725/14:41
derekhbummer14:41
amoralejyes14:41
EmilienMshardy: we need to run yum update before running puppet14:41
shardyEmilienM: :(14:42
EmilienMderekh: https://review.openstack.org/#/c/411725/ too14:42
*** Vijayendra has joined #tripleo14:42
shardyEmilienM: Ok, I guess that should be easy enough for multinode14:42
amoralejbut we need to get https://review.openstack.org/#/c/411800/14:42
openstackgerritAndrey Shestakov proposed openstack/diskimage-builder: Fix dhcp-all-interfaces for ubuntu-minimal xenial  https://review.openstack.org/40772514:42
*** tremble has quit IRC14:42
amoraleji can take a more conservative approach and update only mariadb-libs until new package is provided14:43
pradkEmilienM, yep i'm hitting the same locally on centos 7.314:43
amoralejmariadb-libs issue pradk?14:43
amoralejusing tripleo.sh?14:43
pradkyea packaging file conflict14:43
pradkyep14:43
*** jeckersb_gone is now known as jeckersb14:43
EmilienMit sounds like we need amoralej's patch first to fix multinode jobs (voting)14:43
EmilienMthe other ones are for ovb14:44
EmilienM(non-voting)14:44
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: pin ansible-lint at v3.4.7 until we resolve issues with 3.4.8  https://review.openstack.org/41184314:44
EmilienMamoralej: I'm doing recheck. Can you address trown's comment by patching doc too (later when you can)14:45
amoraleji'm on it14:45
openstackgerritFlorian Fuchs proposed openstack/tripleo-ui: Poll nodes list while nodes are in progress state  https://review.openstack.org/39563714:45
*** tosky has quit IRC14:45
amoralejbut i think that's not the right bug...14:46
*** tremble has joined #tripleo14:46
*** tremble has quit IRC14:46
*** tremble has joined #tripleo14:46
EmilienMamoralej: you mean it won't help to fix the mariadb thing?14:46
trownamoralej: it is a different symptom of the same problem14:46
trownamoralej: or to put it differently, the solution to the mariadb thing also fixes that bug...14:47
amoraleji'm not sure, without more logs....14:47
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: pin ansible-lint at v3.4.7 until we resolve issues with 3.4.8  https://review.openstack.org/41184314:47
*** bogdando has joined #tripleo14:47
amoralejyes, i understand the point14:47
*** liverpooler has joined #tripleo14:47
amoralejbut i trust you trown :)14:47
trown:), the bug I pointed to is a differenent side effect of 7.3 that only effects trying to deploy with a 7.2 image14:48
trownbut it would also be solved by updating packages earlier14:48
amoralejtrown, i'll put it as related-bug ok?14:49
trownamoralej: sure that makes sense14:49
openstackgerritAlfredo Moralejo proposed openstack-infra/tripleo-ci: Update packages after configuring openstack repos  https://review.openstack.org/41172514:50
*** noslzzp has quit IRC14:51
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci: Implement major upgrade for Newton to Ocata  https://review.openstack.org/40483114:51
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci: Add basic pre/post upgrade sanity checks  https://review.openstack.org/41184614:51
shardymatbu: ^^ FYI I added some basic smoke tests since we can't yet run the pingtest14:51
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci: Add basic pre/post upgrade sanity checks  https://review.openstack.org/41184614:52
matbushardy: nice /me looks14:53
*** flepied has quit IRC14:53
openstackgerritFlorian Fuchs proposed openstack/tripleo-ui: Fix blank overcloud credentials  https://review.openstack.org/40813814:54
*** Vijayendra has quit IRC14:54
EmilienMshardy: your stuff is cool for smoke test but we already did something similar for undercloud smoke test. Could we maybe try to use the same things? (fwiw I prefer your code now)14:55
mariosmatbu: have you come across this before " "ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.\n" - it failed on step0 but not how i wanted :) (I put the cluster down again). at least this time the ansible is running, but seems to be a nit with one of the tasks i added in step014:56
EmilienMshardy: maybe we could have some bash functions for smoke test and re-use them for undercloud-only job and overcloud upgrade job14:57
matbumarios: yep, probablye because the module is not well configure14:57
*** paramite_ has quit IRC14:57
matbumarios: check your /etc/ansible/ansible.cfg (the line library=)14:57
*** paramite has quit IRC14:58
matbumarios: then make sure the modules are in the specified path14:58
*** jpena|lunch is now known as jpena14:58
mariosmatbu: ah you mean you suspect it might be the pacemaker_cluster.py/resource from the new module... k will check thanks14:58
jristjtomasek, florianf - so the ids should be what format? for automation?14:58
matbumarios: yes at 99% :)14:58
jristjtomasek, florianf - you're not happy with underscore namespaced?14:59
mariosmatbu: heh cool thanks man14:59
*** milan|afk has quit IRC14:59
*** coolsvap has quit IRC14:59
*** milan has joined #tripleo15:00
florianfjrist: I'd prefer if the component name would be part of the id, potentially separated by a dot. That would be similar to how the namespacing is done with i18n.15:00
jristah15:00
florianfjrist: AFAIK dots are valid characters in id attributes.15:01
jristok15:01
jristthis is new to me :)15:01
shardyEmilienM: definitely, I'll look at refactoring so they use the same code15:01
EmilienMshardy: nothing urgent, just fyi. I can take this task, as I wrote this undercloud check thing15:01
shardyEmilienM: Cool, yeah I just wanted to keep this separate initially, as I was thinking there's some chance we'll delete this code when we get the pingtest working before/after upgrades15:02
shardybut obviously we can keep it if it's still useful15:02
shardyEmilienM: I'll add sanity checks for the other services so we can prove the upgrade patches, then we can look at refactoring, if that's OK with you?15:03
mariosmatbu: so we have a chicken/egg situation i mean we need to install those ansible modules as the (new) upgrade init shardy moved it, so it happens before the steps run15:04
mariosmatbu: ie.. in upgrade init we can copy over those ansible modules from your review to the nodes15:05
mariosmatbu: (for now/testing )15:05
mariosshardy: i just noticed/saw /var/lib/heat-config/heat-config-ansible/3b0f9dc7-0a9f-4a8a-99bb-7c0e5ad957b8_playbook.yaml for first time pretty cool to see it composed :)15:05
matbumarios: hm right, but we would need also to install the heat ansible hook15:05
matbumarios: so i'm wondering how we can put that on the nodes15:06
mariosshardy: well actually, its not composed... its the steps which give the order and they are done via the dependencies15:06
matbumarios: it could also be pushed by heat itself15:07
mariosshardy: so we'd need to process that to get it intoan execution order right i mean to run it independently (I know that is low priority right now)15:07
*** trown is now known as trown|afk15:08
mariosmatbu: yeah i believe the idea of shardy moving the upgrade init was so we can do things like that (the heat-ansible-*)15:08
mariosmatbu: well actually it was for installing the new hiera hook15:08
mariosmatbu: but same applies so we could deliver those ansible modules there.15:09
matbumarios: yep15:09
*** flepied has joined #tripleo15:09
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974215:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165050315:10
*** ooolpbot has quit IRC15:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]15:10
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]15:10
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add metadata settings for needed kerberos principals  https://review.openstack.org/41134015:10
*** paramite_ has joined #tripleo15:12
shardymarios, matbu: Yeah I *had* to move it because of the hiera hook, but we can add other pre-upgrade dependencies in there too15:12
EmilienMshardy: yes15:13
shardyDeployArtifactURLs can also be used to deliver things like ansible modules if needed15:13
shardymarios: re the steps, yeah we'd have to generate another playbook which runs the one heat drives step by step15:13
shardymarios: that would be pretty easy, but I've not done it yet15:13
*** b00tcat has quit IRC15:13
shardyI have driven the steps manually via ansible-playbook on the CLI tho, works well15:14
shardycombined with the tripleo-validations dynamic inventory15:14
*** tremble has quit IRC15:15
shardyFor now I just see it as a useful-for-debugging thing tho, at least until we have the full heat-driven upgrade working15:15
*** jaosorior has quit IRC15:15
*** derekh has quit IRC15:15
*** tremble has joined #tripleo15:15
*** paramite has joined #tripleo15:16
*** b00tcat has joined #tripleo15:16
*** b00tcat has quit IRC15:16
*** b00tcat has joined #tripleo15:16
mariosshardy: yeah well we can use it to deliver matbu modules from https://review.openstack.org/#/c/403397/11/ansible/library/pacemaker_cluster.py and _resource.py as one immediate eexample... we can put those into the default module path whatever that is matbu is there such a location?15:16
mariosshardy: i rebased my step0 onto yours since the init is gone now... i made it part of the main loop and started at 0 https://review.openstack.org/#/c/408631/5/puppet/major_upgrade_steps.j2.yaml15:16
d0ugalthrash, toure, rbrady: I do think I agree with jtomasek's comment on https://review.openstack.org/#/c/404736/15:17
d0ugalbut I need to think about it a bit more.15:17
*** prateek has quit IRC15:17
mariosshardy: it got me thinking how we might be able to get extra steps in... something like ExtraSteps param e.g. [2.21, 3.11, 3.12, 4.1, 5.6] ... i was trying to come up with a way of dynamically including those in that loop15:17
shardymarios: the problem is we don't know the steps defined in the templates when j2 runs15:18
mariosshardy: e.g. make the 2.21 depends on the 2 etc ...15:18
shardywe'd have to use e.g ResourceChain instead15:18
mariosshardy: yeah the idea is that you add a new step15:18
*** zoli|brb is now known as zoli15:18
mariosshardy: liek i just added 1.1 and then you would add that into the extrasteps... so it would be hardwired15:18
shardywhich I did consider, but this seemed cleaner15:18
*** zoli is now known as zoliXXL15:18
rbradyd0ugal: I don't see jtomasek's comment on that review15:19
shardymarios: Yeah, possibly - I'll give some thought to how we might do that15:19
mariosshardy: but still i couldnt' come up with  a way of including that in the for loop (e.g. added 1.1 so it would then make it depend on 1 etc etc )15:19
shardymarios: FWIW I went with this approach because it's simple, and it should hopefully tend to discourage adding $many additional steps, as there's obviously cost associated with each one15:20
d0ugaljtomasek, thrash, toure, rbrady: sorry, I meant https://review.openstack.org/#/c/405531/15:20
mariosshardy: the use case was for things like migrations15:20
mariosshardy: which it may depend at which point we want to run it15:20
shardymarios: Yeah, you can't do it via a heat parameter, we'd need to add it as some input data to jinja2 with the current model15:20
shardyor, completely rework how the steps are run (which might be possible, but I'd rather not do that right now)15:21
*** chlong has joined #tripleo15:21
mariosshardy: heh no lets not do that15:22
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-heat-templates: add collectd composable service  https://review.openstack.org/41104815:22
shardymarios: if we wanted to get really fancy we'd do some sort of preview create that gave us the flattened list of tags without creating the stack, then feed those into j2 as the steps15:23
* shardy will think more about it15:23
mariosshardy: if we got that we could even go as far as getting the tasks out too right I mean for runinng stand-alone (well subject to processing)15:23
mariosshardy: but i have no idea how we would do that shardy :)15:24
mariosshardy: i mean the preview create15:24
shardyThe thing I like about the current implementation is it's simple and easy to understand (relatively ;)15:24
mariosshardy: could we add all the upgrade tasks to the output for the service? is that sane15:24
*** dtantsur is now known as dtantsur|afk15:24
shardymarios: They are already output from all the services, and combined via the ResourceChains - we'd just have to add an output for each role's playbook of tasks to overcloud.j2.yaml15:26
mariosshardy: sorry thats actually where their defined... i mean some way of extracting them without running them15:26
*** jeckersb is now known as jeckersb_gone15:26
shardyI already did exactly that for testing15:26
shardyso the missing part is seeing if stack-preview would actually give us that data15:26
shardyI'll try it15:26
*** abehl has quit IRC15:27
shardybut relative to all the other work we have to do, this is probably a lower priority?15:27
*** sanagikoki has joined #tripleo15:27
shardye.g if we have to finesse the steps a bit for ocata, it's still *way* more hackable than the bash script approach :)15:27
*** jeckersb_gone is now known as jeckersb15:30
shardymarios: also, for each service, isn't the task listing enough to serialize e.g a migration task with some other thing, even when they're the same step?15:30
shardyI thought ansible walked the talks in order, but maybe I'm mistaken15:30
shardys/talks/tasks/15:31
*** flepied has quit IRC15:32
*** brault has joined #tripleo15:37
openstackgerritLars Kellogg-Stedman proposed openstack/puppet-tripleo: add support for collectd  https://review.openstack.org/41104715:41
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Add Octavia API service definitions  https://review.openstack.org/41187215:42
*** hjensas has quit IRC15:44
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Introduce Octavia implememtation services  https://review.openstack.org/41187415:46
*** tobias-fiberdata has quit IRC15:46
*** dsneddon has joined #tripleo15:46
*** hjensas has joined #tripleo15:47
EmilienMamoralej: your patch is still failing, /me reading logs15:48
amoraleji was checking it also, something configuring ovs15:48
EmilienMhttp://logs.openstack.org/25/411725/4/check/gate-tripleo-ci-centos-7-nonha-multinode/6260e1c/console.html#_2016-12-16_15_34_26_35815115:49
*** flepied has joined #tripleo15:49
EmilienMit looks like undercloud can't ping overcloud15:49
EmilienMswitching on infra channel15:50
EmilienM#openstack-infra15:50
*** dsneddon has quit IRC15:51
EmilienMbnemec, derekh, have you seen that before? ^15:51
*** hjensas has quit IRC15:54
EmilienMI think all multinodes in OpenStack Infra are now broken15:55
EmilienMor at least in tripleo15:55
mariosshardy: sorry was on scrum, reading back15:55
openstackgerritMartin AndrĂ© proposed openstack/tripleo-docs: Fix OpenStack client invocation  https://review.openstack.org/41187815:56
mariosshardy: oh i see so yeah if we can explicitly define the order like that with same step number but defined in series that would be great15:56
*** arxcruz has quit IRC15:56
*** bana_k has joined #tripleo15:57
EmilienMamoralej: I'm doing recheck for now, since we fixed the " issue in JJB15:57
amoralejlast run was after fixing " , but go ahead15:58
shardymarios: ack - OK lets try that, and btw the preview approach does also work15:58
shardyhttp://paste.openstack.org/show/592653/15:58
shardyso we have that as an option if we need it15:58
pradkyum update does seem to resolve mariadb issue, thx for the tip15:58
pradkamoralej, ^^15:58
amoralejcool15:58
*** bana_k has quit IRC15:59
mariosshardy: awesome with a bit of login in tripleo-common we could parse that and put thm into step order too15:59
mariosshardy: s/login/logic15:59
shardymarios: Yep we probably could16:00
EmilienMamoralej: the patch is maybe merged but not applied on all zuul nodes16:00
EmilienMamoralej: puppet runs periodically16:00
amoralejit'd have given a different error EmilienM16:00
amoraleji'm sure it was merged16:00
*** yprokule has quit IRC16:01
EmilienMamoralej: the ovs issue is new now16:01
amoralejyep16:01
bnemecOf course it is.16:02
bnemecThis week.  Good grief.16:02
*** Guest31304 has quit IRC16:02
*** aufi has quit IRC16:02
yolandahi, i tried upgrading tripleoclient and i'm getting an error:16:03
yolandaDiscovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL.16:03
yolandaSSL exception connecting to https://192.168.24.2:13000/v2.0/tokens: ("bad handshake: Error([('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')],)",)16:03
yolandaanyone knows how to skip it?16:03
bnemecyolanda: Sounds like an ssl cert wasn't installed correctly for some reason.16:06
bnemecHow did you configure ssl on the undercloud?16:06
yolandabnemec, so what i did is to deploy an undercloud with ooq. But then, i upgraded tripleoclient to consume latest patchsets, and wanted to redeploy overcloud again16:06
*** saneax is now known as saneax-_-|AFK16:07
yolandathe initial deploy worked, it started to fail as soon as i upgraded tripleoclient16:07
openstackgerrityolanda.robla proposed openstack/tripleo-quickstart: Create directories with root  https://review.openstack.org/38489216:09
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974216:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165050316:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]16:10
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]16:10
EmilienMwe should add ovs now ^16:10
EmilienMI'm filling a bug16:10
*** mcornea has quit IRC16:10
yolandabnemec, i remember that this was something related with requests16:11
yolandabut i don't remember the way to disable that ssl check, i just want to do a test uploading images16:11
bnemecyolanda: I'm not sure.  You could look for a --insecure option or something in osc.16:12
yolandai exported OS_INSECURE=true, but no luck16:14
yolandai will try redeploying and instead of upgrading with pip, just add the bits in tripleoclient i need...16:14
yolandawith the tripleoclient upgrade , there were lots of dependencies upgrades as well, so there may be conflicts16:15
*** udesale has quit IRC16:16
EmilienMhttps://bugs.launchpad.net/tripleo/+bug/165061216:17
openstackLaunchpad bug 1650612 in tripleo "nodepool subnodes can't ping each others" [Critical,Triaged]16:17
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart-extras: Add doc templating to validate-tempest role  https://review.openstack.org/41188816:19
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart-extras: [WIP] - Add basic documentation for overcloud-upgrade role  https://review.openstack.org/40932816:21
openstackgerritPaul Belanger proposed openstack/diskimage-builder: Revert "Fix pip-and-virtualenv to work with python3"  https://review.openstack.org/41189216:22
amoralejSame error in recheck EmilienM16:23
*** rcernin has quit IRC16:23
EmilienMdamn16:23
*** pkovar has quit IRC16:24
EmilienMamoralej: we have no way to debug easily, I'm about to ask access to one node on #openstack-infra16:25
*** pcaruana has quit IRC16:25
amoraleji'm trying to understand how this multinode jobs work16:25
amoralejok16:25
openstackgerritPaul Belanger proposed openstack/diskimage-builder: Add ubuntu-precise support to dib-python  https://review.openstack.org/41189816:27
*** chlong has quit IRC16:29
openstackgerritBrent Eagles proposed openstack/tripleo-puppet-elements: Octavia integration  https://review.openstack.org/41190216:32
bnemecEmilienM: I wonder if something changed in the firewall on 7.3.  We seem to be having a lot of weird connectivity issues with stuff that used to work.16:32
EmilienMbnemec: see on infra channel, it looks like transient16:33
EmilienMbnemec: I see a lot of jobs passing now16:34
EmilienMI'll do recheck again on amoralej's patch16:34
amoralejhopefully16:34
openstackgerritLars Kellogg-Stedman proposed openstack/puppet-tripleo: add support for collectd  https://review.openstack.org/41104716:37
*** cwolferh has joined #tripleo16:37
bnemecEmilienM: slagle used to have a multinode dev environment on one of the rh clouds.  Maybe we could set one up with 7.3 and see if we can reproduce these problems?16:38
openstackgerritJiri Stransky proposed openstack/tripleo-common: Heat-agents container: set up trunk repos correctly  https://review.openstack.org/41190816:38
EmilienMbnemec: yeah we could, jeblair also proposed to give access to the infra resource16:39
EmilienMbnemec: let me see if I can get it first otherwise, let's use slagle's env16:40
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Update packages after configuring openstack repos  https://review.openstack.org/41172516:40
EmilienMamoralej: just kickoff a new CI job ^ by changing commit messagfe16:41
amoralejyes, i see, i have an alternative in which i only update mariadb-libs if we think that's the issue16:42
bnemecWhat mariadb-libs problem are we fixing here?  Jobs are passing without this workaround.16:43
openstackgerritPaul Belanger proposed openstack/diskimage-builder: Increase func testing for ubuntu-minimal element  https://review.openstack.org/41191016:43
amoralejbnemec, there is a conflict between the mariadb-libs package in CentOS 7.3 and the one in RDO repos16:44
amoraleji've seeing jobs failing because of that issue16:44
bnemecamoralej: How are jobs passing then?16:44
* amoralej checking logs16:44
amoralejit can pass if it's updated before trying to deploy mariadb-server or if mariadb-libs are not in the base image16:45
openstackgerritJiri Stransky proposed openstack/tripleo-common: Heat-agents container: set up trunk repos correctly  https://review.openstack.org/41190816:47
bnemecWhich should always be the case.  This kind of thing is exactly why we have a forced yum update early in the undercloud install.16:47
*** rhallisey has quit IRC16:47
EmilienMhttps://review.openstack.org/#/c/411340/ is a good example of multinode job working fine now16:48
*** akrivoka has quit IRC16:48
*** akrivoka has joined #tripleo16:48
EmilienMdo we really need this workaround?16:48
*** tremble has quit IRC16:50
*** chlong has joined #tripleo16:53
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Add Octavia API service definitions  https://review.openstack.org/41187216:53
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Introduce Octavia implememtation services  https://review.openstack.org/41187416:54
*** b00tcat has quit IRC16:55
EmilienMbnemec: 411514 is about to move in gate soon :D16:57
bnemecEmilienM: As soon as the stupid non-voting job finishes. :-)16:58
bnemecOf course, it's been there before so I'm not counting my chickens here.16:58
* EmilienM googles "counting chickens"16:59
*** lmiccini has quit IRC16:59
*** b00tcat has joined #tripleo16:59
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart-extras: [WIP] - Add basic documentation for overcloud-upgrade role  https://review.openstack.org/40932817:00
bnemecThis is less than promising: http://logs.openstack.org/33/411733/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/930da6b/console.html#_2016-12-16_16_40_27_44052117:01
EmilienMbnemec: social reported an issue with introspection this morning17:02
EmilienMsocial: have you found out?17:02
rlandydprince: https://bugs.launchpad.net/heat-templates/+bug/1650625 ... what are your thoughts? currently we have a failing bond-with-vlans job in the promotion pipeline and a rejected proposed review to fix it.17:02
openstackLaunchpad bug 1650625 in Heat Templates "Relative path to run-os-net-config will fail for custom nic-configs saved in a different directory" [Undecided,New]17:02
EmilienMbnemec: I would be surprises it comes from upstream since we didn't promote tripleo CI for 9 days17:03
amoralejEmilienM, bnemec, the reason why some jobs are passing is because they are still using 7.2 image17:03
* bnemec is concerned that we are stuck in quicksand and the more we fight it the the more it sucks us down.17:04
*** rcernin has joined #tripleo17:04
amoralejweird, we have different images in different jobs17:05
EmilienMlet's see what cloud providers have what17:06
amoralejosic is 7.217:07
EmilienMamoralej: where do you see the centos version?17:07
amoralejkernel version17:07
amoralejhttp://logs.openstack.org/40/411340/8/check/gate-tripleo-ci-centos-7-nonha-multinode/966cbe7/console.html#_2016-12-16_15_13_13_09967717:07
amoralej3.10.0-327 is 7.217:07
amoralejhttp://logs.openstack.org/09/409809/1/gate/gate-tripleo-ci-centos-7-nonha-multinode/43ed9c4/console.html#_2016-12-16_08_20_53_13255517:08
amoralej7.317:08
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974217:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165050317:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165061217:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]17:10
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]17:10
openstackLaunchpad bug 1650612 in tripleo "nodepool subnodes can't ping each others" [Critical,Triaged]17:10
*** jpena is now known as jpena|off17:13
*** panda is now known as panda|off17:14
*** zoliXXL is now known as zoli|gone17:15
*** yamahata has joined #tripleo17:17
*** saibarspeis has quit IRC17:19
*** arxcruz has joined #tripleo17:20
*** saibarspeis has joined #tripleo17:20
*** athomas has quit IRC17:20
*** bana_k has joined #tripleo17:25
EmilienMamoralej: sounds like it should be synced now17:27
openstackgerritMerged openstack/diskimage-builder: Add ubuntu-precise support to dib-python  https://review.openstack.org/41189817:27
amoralejanyway, i think we should merge the fix mariadb-libs to get ready for when it cames back EmilienM17:27
EmilienMamoralej: we'll see how CI works on this one17:28
amoralejif it fails, i'll send an update to update only mariadb-libs, i'm pretty sure it will work17:28
EmilienMok17:28
*** akrivoka has quit IRC17:29
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates: WIP: Add custom role for realtime compute  https://review.openstack.org/41192517:30
openstackgerritPaul Belanger proposed openstack/diskimage-builder: Start func testing on centos-minimal again  https://review.openstack.org/41192617:31
mwhahahado we have abug for the bariadb-libs thing yet?17:31
*** dsneddon has joined #tripleo17:35
*** ayoung has quit IRC17:35
*** hewbrocca is now known as hewbrocca_afk17:36
rascaguys my deploy on master just failed with "openstack overcloud deploy: error: unrecognized arguments: --neutron-bridge-mappings" with previous versions this always worked. When this was changed?17:36
bnemecrasca: That's been deprecated for multiple releases and was removed in https://github.com/openstack/python-tripleoclient/commit/3ab40d4ea4b320bcfe85bc1f3c1ded987c65e5d017:39
bnemecYou need to pass those parameters in an env file now.17:39
*** dsneddon has quit IRC17:40
rascabnemec, so it is sufficient to use NeutronBridgeMappings: "datacentre:br-ex,floating:br-floating" in my env?17:40
bnemecrasca: I believe so.17:40
EmilienMmwhahaha: not afik17:42
rascabnemec, ok, many thanks, I'll give it a try, but you think this will be retro compatible?17:42
rascabnemec, I mean if I don't declare the option in newton or mitaka will it work?17:42
rasca(I know it sounds like a silly question)17:42
bnemecrasca: You've been able to define these in an env file at least since they were deprecated in liberty, so it should work fine.17:43
rascabnemec, ok, I will test it and let you know, many thanks Ben17:43
bnemecThe cli params were a mistake in the kilo-based downstream release that tripleoclient originated in.17:43
rascathat's good to know.17:43
rascabnemec, and it also answer a question I made to shardy and EmilienM a week ago17:44
*** milan has quit IRC17:44
rascabut I need to test it first17:44
* EmilienM afk lunch break17:44
openstackgerritDan Prince proposed openstack/tripleo-common: Don't require mistralclient for password gen  https://review.openstack.org/41097017:45
openstackgerritPaul Belanger proposed openstack/diskimage-builder: Start func testing on centos-minimal again  https://review.openstack.org/41192617:49
*** [1]cdearborn has quit IRC17:50
*** rhallisey has joined #tripleo17:57
*** dsneddon has joined #tripleo17:58
*** dprince has quit IRC18:02
*** dsneddon has quit IRC18:02
*** ohamada_ has quit IRC18:07
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974218:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165050318:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165061218:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]18:10
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]18:10
openstackLaunchpad bug 1650612 in tripleo "nodepool subnodes can't ping each others" [Critical,Triaged]18:10
openstackgerritPaul Belanger proposed openstack/diskimage-builder: Start func testing on centos-minimal again  https://review.openstack.org/41192618:16
*** flepied has quit IRC18:18
EmilienMbnemec: I see quite a lot of multinode jobs passing :D http://tripleo.org/cistatus.html18:22
amoralejEmilienM, https://review.openstack.org/#/c/411725/ passed18:23
*** lucasagomes is now known as lucas-afk18:23
bnemecEmilienM: There's only one I really care about: telnet://146.20.105.108:19885 :-)18:23
EmilienMamoralej: do we actually need it?18:24
EmilienMbnemec: which one is that?18:24
amoralejwe'll need it if 7.3 images are pushed before we fix mariadb18:25
bnemecEmilienM: The ovb workaround18:25
EmilienMamoralej: ok +2 then, bnemec you ok too?18:25
bnemecIf that passes then it merges.  If not, we waste another half day.18:25
EmilienMbnemec: ssl?18:25
bnemecYeah18:25
amoralejif you prefer, i can test a less aggressive update to just fix mariadb-libs18:26
openstackgerritLucas Alvares Gomes proposed openstack/tripleo-quickstart: VirtualBMC support for tripleo-quickstart  https://review.openstack.org/39970418:27
EmilienMamoralej: it's ok for me as it is now18:27
EmilienMamoralej: I'll let bnemec approve it if he's fine18:27
amoralejok18:28
EmilienMamoralej: did we report a bug in launchpad for this problem?18:28
amoralejNot sure18:28
amoralejlemme check18:28
amoraleji didn't18:28
bnemecI'm not crazy about merging a fix for a problem that may or may not happen depending on the state of packages when we move to 7.3.18:31
*** trown|afk is now known as trown18:31
bnemecIt also needs to be documented if it ends up being required.18:31
EmilienMbnemec: it sounds that no matter 7.2 or 7.3, we need to get latest version of this package18:32
bnemecEmilienM: Not if jobs are passing right now.18:32
amoralejno EmilienM, it depends on the version of mariadb-libs in base image18:32
EmilienMbnemec: we could hold it for now, and use it if mariadb is not working anymore18:33
amoralejthat's why we don't need it with 7.2 images18:33
bnemecI'm also still not sure why the yum update in the undercloud install doesn't take care of this.18:33
EmilienMit probably happens too late18:33
EmilienMlet me check18:33
openstackgerritMerged openstack/tripleo-quickstart: pin ansible-lint at v3.4.7 until we resolve issues with 3.4.8  https://review.openstack.org/41184318:33
amoralejwe need to have the RDO repos enabled18:33
*** ctayal has joined #tripleo18:33
EmilienMamoralej: it should be the case when puppet run yum update18:34
EmilienMI think it's an orchestration thing in the catalog18:34
EmilienMamoralej: see https://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.pp#L2618:34
amoralejwe have a yum update in undercloud install?18:34
amoralejthen we need to move that to the very beginning of the undercloud install, IMO18:35
EmilienMamoralej: yes, https://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.pp#L39-L4418:35
amoralejwe are doint it at the end18:35
amoralejiiuc18:35
amoralejis this used for undercloud upgrades EmilienM?18:37
amoralejif so, it make sense to do it at the end18:37
EmilienMamoralej: no, everytime undercloud is deployed or updated18:37
EmilienMtechnically speaking, everytime puppet is run on the undercloud18:38
amoralejbut for updates too, right?, mmm doing a full update at the beginning is not a good idea  for update case18:38
bnemecSigh, I'm pretty sure the undercloud ssl patch has hit the multinode hang bug.18:39
trownamoralej: EmilienM bnemec that is exactly the bug I filed yesterday18:39
trownnamely that the undercloud install should happen before any service configuration18:40
trownerr. undercloud upgrade rather18:40
EmilienMtrown: yeah, I assigned it to myself I think, it's on my todo18:40
trownhttps://bugs.launchpad.net/tripleo/+bug/165037418:40
openstackLaunchpad bug 1650374 in tripleo "[instack-undercloud] package upgrade should happen before service configuration" [Critical,Triaged] - Assigned to Emilien Macchi (emilienm)18:40
bnemecUmm, it does, doesn't it?18:40
bnemecOh, this is a new install, not an upgrade.18:41
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: make quickstart-extras-requirements.txt a default requirements file  https://review.openstack.org/41075718:41
trownbnemec: it does in the file... but not when actually running puppet.,, there are no dependencies setup18:41
EmilienMbnemec: I'm wondering if we should run yum update before running puppet18:41
*** dprince has joined #tripleo18:41
EmilienMI've never been a fan of running yum in a exec from puppet :D18:41
EmilienMmaybe we could just run that command just *before* running puppet18:41
trownif we move it out, we just need to maintain the option to not do it at all (which currently is wired in via hiera)18:42
bnemecThat's what it used to do, although I think it was in an o-r-c script back then.18:42
amoralejbut running a "yum update -y" before running puppet it's good for first deployment18:42
amoraleji have doubts for the update18:42
amoralejfor update we may need orchestration to update service by service18:43
amoralejno?18:43
bnemecNo :-)18:43
EmilienMtrown: yes, definitly18:43
bnemecWe take all the services down as part of the update because otherwise things tend to go badly.18:43
amoralejoooook, i see18:43
bnemecSeveral of the services have issues when they get restarted in the post scripts of the package install.18:43
amoralejthen full upgrade at the beginning should be ok18:44
bnemecWhich doesn't happen if they're not running in the first place.18:44
EmilienMif you guys are ok, I can work on a patch that 1) run "yum update" befre running puppet 2) make it optional using the same "update_packages" parameter18:44
*** yamahata has quit IRC18:44
trownso we can make an update o-r-c script in https://github.com/openstack/instack-undercloud/tree/master/elements/puppet-stack-config/os-refresh-config/configure.d guess we would control it via ENV var?18:44
EmilienMtrown: yes, +118:44
trownEmilienM: ya I think that would be a good solution to https://bugs.launchpad.net/tripleo/+bug/165037418:45
openstackLaunchpad bug 1650374 in tripleo "[instack-undercloud] package upgrade should happen before service configuration" [Critical,Triaged] - Assigned to Emilien Macchi (emilienm)18:45
amoralejthen i could abandon my patch too18:45
trownEmilienM: solving it via puppet would be a lot of dependencies... I was looking at it yesterday and we would need a Service tag for each service18:45
EmilienMtrown: right, it sounds too complex18:46
amoralejare all repos enabled at that time?18:46
trownya18:46
amoralejthen, it should work fine18:47
*** fragatina has quit IRC18:47
*** gfidente is now known as gfidente|afk18:51
*** saibarspeis has quit IRC18:54
*** flepied has joined #tripleo18:54
*** egafford has joined #tripleo18:54
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: Run `yum update -y` before Puppet run  https://review.openstack.org/41195718:58
EmilienMtrown, amoralej: that's a quick PoC ^18:58
EmilienMtrown, amoralej: any feedback is welcome :)18:59
trownEmilienM: before we updated openstack packages first then the rest... I don't totally understand why we needed that, but we lose that ability with your patch19:01
trownalso not sure there is a good solution for that without puppet involved... but just wanted to point it out19:03
EmilienMtrown: we wanted to update the undercloud everytime puppet run19:05
EmilienMI think it's fine now19:05
EmilienMbut I'll let instack experts looking :D i'm sure it can be better19:05
trownya, I am not sure putting it in python there is good... for one `tox -e py27` is running yum update on my machine :P19:06
EmilienMtrown: how would you handle it?19:07
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974219:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165050319:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165061219:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]19:10
trownEmilienM: commented in the review, but I think we can make a script like https://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/os-refresh-config/configure.d/50-puppet-stack-config in the same directory like "40-yum-update-before-puppet"19:10
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]19:10
openstackLaunchpad bug 1650612 in tripleo "nodepool subnodes can't ping each others" [Critical,Triaged]19:10
mwhahahawell the yum update was after the openstack package updates which would capture at least the openstack service restarts. But it also assumed that the dependency versions would magically get updated as well which we saw not to be the case yesterday with the python2-cryptography thing and ironic19:10
*** rbrady is now known as rbrady-afk19:13
*** dsneddon has joined #tripleo19:16
*** florianf has quit IRC19:21
*** dsneddon_ has joined #tripleo19:21
*** yamahata has joined #tripleo19:21
*** dsneddon has quit IRC19:24
weshayjistr++19:32
*** florianf has joined #tripleo19:36
*** yamahata has quit IRC19:36
*** yamahata has joined #tripleo19:36
EmilienMtrown: ok looking19:37
*** owalsh has quit IRC19:40
*** rhallisey has quit IRC19:41
*** amoralej is now known as amoralej|off19:48
*** florianf has quit IRC19:54
*** fragatina has joined #tripleo19:55
*** rhallisey has joined #tripleo19:59
*** chem has quit IRC20:06
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974220:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165050320:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165061220:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]20:10
openstackLaunchpad bug 1650612 in tripleo "nodepool subnodes can't ping each others" [Critical,Triaged]20:10
*** dsariel has joined #tripleo20:20
*** paramite has quit IRC20:23
*** paramite_ has quit IRC20:24
*** ayoung has joined #tripleo20:29
*** rhallisey has quit IRC20:33
*** jtomasek has quit IRC20:33
*** b00tcat has quit IRC20:34
larsksDuring an update operation, is it normal for a stack to show state UPDATE_IN_PROGRESS when all the resources in that stack show either UPDATE_COMPLETE or CREATE_COMPLETE?20:40
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: Run `yum update -y` before Puppet run  https://review.openstack.org/41195720:44
*** chlong has quit IRC20:48
*** jcoufal has quit IRC20:48
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo: Include nova::compute::libvirt::qemu from the libvirt profile  https://review.openstack.org/41198420:49
*** leanderthal is now known as leanderthal|afk20:51
*** fragatina has quit IRC20:53
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Increase libvirt/qemu.conf max_files and max_processes  https://review.openstack.org/41198720:56
*** jcoufal has joined #tripleo21:01
*** jcoufal has quit IRC21:01
*** b00tcat has joined #tripleo21:01
*** dsneddon has joined #tripleo21:07
*** liverpooler has quit IRC21:07
*** dprince has quit IRC21:08
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974221:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165050321:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]21:10
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart-extras: Add modify-images role  https://review.openstack.org/41199521:20
*** rlandy has quit IRC21:21
*** ctayal has quit IRC21:22
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart: Fetch images in a standalone role  https://review.openstack.org/40876021:24
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: Run `yum update -y` before Puppet run  https://review.openstack.org/41195721:25
*** dsneddo__ has joined #tripleo21:25
*** dsnedd___ has joined #tripleo21:27
*** dsneddon_ has quit IRC21:28
openstackgerritMerged openstack/tripleo-quickstart: Fetch images in a standalone role  https://review.openstack.org/40876021:28
*** dsneddo__ has quit IRC21:30
*** jayg is now known as jayg|g0n321:31
*** trozet has quit IRC21:31
*** gfidente|afk has quit IRC21:31
*** trozet has joined #tripleo21:46
*** trozet has quit IRC21:49
*** fultonj has quit IRC21:58
*** scorcoran has joined #tripleo22:00
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: Run `yum update -y` before Puppet run  https://review.openstack.org/41195722:01
*** trown is now known as trown|outtypewww22:05
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974222:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/165050322:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1650503 in tripleo "nodepool slaves failing to boot" [Critical,Triaged]22:10
*** fzdarsky is now known as fzdarsky|afk22:10
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras: add the containers prep role to the quickstart-extras  https://review.openstack.org/40098322:11
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: config for containerized-compute  https://review.openstack.org/39334822:11
*** bana_k has quit IRC22:12
*** tzumainn has quit IRC22:12
*** ctayal has joined #tripleo22:14
*** lblanchard has quit IRC22:16
EmilienMbnemec: looks like introspection is broken or something22:18
EmilienMhttp://logs.openstack.org/14/411514/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/7030340/console.html#_2016-12-16_21_21_37_08976022:18
EmilienMbnemec: but gate-tripleo-ci-centos-7-ovb-ha passed22:19
*** bana_k has joined #tripleo22:19
* EmilienM checking centos version22:19
*** jeckersb is now known as jeckersb_gone22:19
*** scorcoran is now known as scorcoran_biab22:19
bnemecEmilienM: Yeah, I noticed a similar error in nonha this morning.22:20
EmilienMit might be centos 7.2 vs 7.3?22:20
*** rasca has quit IRC22:21
bnemecEmilienM: Could be.  I don't grok mistral errors very well though.22:21
EmilienMbnemec: looks like both are 7.322:23
bnemecOnly one error in the logs, and it's the same one in the console: http://logs.openstack.org/14/411514/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/7030340/logs/undercloud/var/log/mistral/engine.txt.gz#_2016-12-16_21_21_35_09322:23
EmilienMindeed22:23
EmilienMthrash: are you still around ? :D22:23
EmilienMI don't see anything in ironic-inspector22:24
EmilienMneither in ironic-conductor22:24
bnemecDitto22:25
bnemecI tried to debug a similar error once and basically got nowhere.22:25
EmilienMI'm filing a (new) bug with an alert22:25
bnemechttps://review.openstack.org/#/c/411902/1 passed nonha on newton, so it must be something to do with master.22:27
EmilienMif you look at logstash, it only happens to gate-tripleo-ci-centos-7-ovb-nonha22:27
bnemecThat's the only job where we run introspection22:27
EmilienMwhat is different between ha & nonha here22:27
EmilienMah :D22:27
*** jkilpatr has quit IRC22:27
bnemecIt takes a while, so we put it on the shortest job.22:27
* EmilienM looking latest patches in Ironic22:28
EmilienMbnemec: what is weird is that we haven't got promotion for a while so how a patch in ironic or ironic-inspector could break us22:29
EmilienMwe need to find the latest sucessful job and compare packages22:29
* EmilienM clicks22:29
bnemecCould be we merged something that accidentally broke this.  Anything that went in this week couldn't have had a recent introspection pass.22:30
EmilienMtonight at 1.17am it worked22:30
EmilienMon https://review.openstack.org/#/c/411541/122:31
* EmilienM runs diff now22:31
EmilienMI'm afraid it was centos 7.222:31
*** weshay is now known as weshay_lata22:32
EmilienMbnemec: https://www.diffchecker.com/t4XiBVfM22:33
EmilienMit sunds like DIB22:33
bnemecOr puppet-mistral22:34
*** toure has quit IRC22:34
bnemecThose are the only two that seem relevant though.22:34
EmilienMmwhahaha: https://github.com/openstack/puppet-mistral/commit/bf3625d5af5aabd5b2d3679f2ae61d63153cb2a422:35
EmilienMthat!22:35
EmilienMit *could* be related22:35
bnemecWeird.  Why did that package even change today though?  The patch merged three days ago.22:35
EmilienMbnemec: indeed22:36
*** toure has joined #tripleo22:36
bnemecMaybe dlrn was having 7.3 issues too. :-)22:36
EmilienMwe should maybe switch puppet-mistral to use ovb-nonha22:37
EmilienMnot sure22:37
mwhahahawhat22:38
bnemecMight be good.  The introspection workflow seems to be a common breakage.22:38
bnemecAnd the rest are pretty much the same between ha and nonha.22:38
EmilienMmwhahaha: we're investigating http://logs.openstack.org/14/411514/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/7030340/console.html#_2016-12-16_21_21_37_08976022:38
EmilienMbnemec: when things are stable again, i'll add it :D22:38
*** flepied has quit IRC22:40
mwhahahanot sure how switching out something that was backwards compatible would break the data input into a work flow22:42
* mwhahaha shrugs22:42
mwhahahathis week has proven anything is possible22:42
*** scorcoran_biab is now known as scorcoran22:42
EmilienMlol indeed, and it's time for a break :)22:42
EmilienMI want to do recheck on https://review.openstack.org/#/c/411541/ and see if it pass again :D22:45
EmilienMbnemec: looking at logstash it's somthing that happened today22:46
EmilienMi'll call it a week22:47
EmilienMsee you folks22:47
*** scorcoran is now known as scorcoran_afk22:48
*** jkilpatr has joined #tripleo22:50
*** fragatin_ has joined #tripleo22:51
*** fragatin_ has quit IRC22:53
bnemeco/22:54
*** fragatina has joined #tripleo22:56
*** fragatina has quit IRC23:00
openstackgerritMerged openstack/tripleo-common: Remove remaining vendor plugins from default image YAML  https://review.openstack.org/40980923:04
openstackgerritAlex Schultz proposed openstack/instack-undercloud: Add cell_v2 simple_cell_setup  https://review.openstack.org/41200623:07
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/160968823:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164974223:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1609688 in tripleo "CI: nonha jobs fails in introspection step because of mistral error" [Critical,Confirmed] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)23:10
openstackLaunchpad bug 1649742 in tripleo "postci timeouts on ovb-ha and ovb-updates" [Critical,Triaged]23:10
openstackgerritAlex Schultz proposed openstack/instack-undercloud: Add cell_v2 simple_cell_setup  https://review.openstack.org/41200623:12
*** saneax-_-|AFK is now known as saneax23:12
*** b00tcat has quit IRC23:12
*** b00tcat has joined #tripleo23:15
*** b00tcat has quit IRC23:15
*** b00tcat has joined #tripleo23:15
*** trozet has joined #tripleo23:15
*** shinobu__ has quit IRC23:20
*** mhenkel has quit IRC23:20
*** mhenkel has joined #tripleo23:21
*** shinobu__ has joined #tripleo23:23
openstackgerritAlex Schultz proposed openstack/puppet-tripleo: Add cell_v2 setup for nova  https://review.openstack.org/41200723:24
mwhahahahrm has anyone actually tried 'openstack overcloud deploy --templates' lately? The last few times i've just run that (and only that) it's timed out23:30
*** almondjoy has quit IRC23:32
openstackgerritBrent Eagles proposed openstack/tripleo-puppet-elements: Octavia integration  https://review.openstack.org/41201223:48
*** scorcoran_afk has quit IRC23:52
*** chatter has joined #tripleo23:54
chatterhey guys23:55
mwhahahahi23:55
chatterallah is doing23:55
chattersun is not doing allah is doing23:55
chatterto accept islam say that i bear witness that there is no deity worthy of worship except allah and muhammad peace be upon him is his slave and messenger23:55
chatterhey23:56
*** chatter has left #tripleo23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!