Thursday, 2017-08-03

*** http_GK1wmSU has joined #tripleo00:04
*** http_GK1wmSU has left #tripleo00:05
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814600:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]00:10
*** ooolpbot has quit IRC00:10
*** psahoo has joined #tripleo00:26
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert controller-role.yaml to role.role.j2.yaml  https://review.openstack.org/48027200:29
*** rwsu has quit IRC00:38
*** gbarros has joined #tripleo00:38
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/48814800:42
openstackgerritMerged openstack/tripleo-heat-templates master: Fix keystone, cinder, heat-api cron containers  https://review.openstack.org/48941900:44
openstackgerritMerged openstack/tripleo-quickstart-extras master: Enable tempest to run with whitelist  https://review.openstack.org/48525200:44
openstackgerritMerged openstack/os-net-config stable/newton: Use os-net-config mapping file when interface in bond or bridge  https://review.openstack.org/48970900:45
openstackgerritMerged openstack/tripleo-heat-templates master: Update TLS-everywhere docker environment  https://review.openstack.org/48947900:45
openstackgerritMerged openstack/tripleo-ui master: Implement Zaqar logger adapter  https://review.openstack.org/45929400:45
openstackgerritMerged openstack/tripleo-quickstart master: Enable tempest on scenario001-multinode job  https://review.openstack.org/48478800:45
*** limao has joined #tripleo00:46
openstackgerritMerged openstack/tripleo-quickstart master: Prepare pike.yaml config for TripleO CI  https://review.openstack.org/49006600:46
*** nyechiel has quit IRC00:50
*** chem has quit IRC00:53
*** psahoo has quit IRC00:53
EmilienMianw: tripleoclient job still failing :(00:55
*** dmacpher has quit IRC00:56
EmilienMsounds like diskimage-builder==2.8.000:56
EmilienMso we still have a problem :(00:56
ianwEmilienM: ok ... can you point me at the logs?00:56
EmilienMhttp://logs.openstack.org/88/487888/1/check/gate-tripleo-buildimage-overcloud-full-centos-7/389cfe8/console.html#_2017-08-02_23_54_20_78814600:57
EmilienMstevebaker: ^00:57
EmilienMI'm afk 1h, I'm back after00:57
stevebakerianw: what/when rebuilds the nodepool nodes?00:58
*** dixiaoli has joined #tripleo00:58
ianwstevebaker: oh, well yeah that will happen tomorrow00:58
stevebakerianw: I mean the images that nodepool nodes boot with00:59
ianwi could kick of an asynchronous build00:59
stevebakerianw: if that is a done thing, it would be useful01:00
EmilienMare we sure we don't already have the lastest version downloaeded?01:01
EmilienMI see in logs 2.8.001:01
openstackgerritMerged openstack/puppet-tripleo master: Use normal socket file permissions instead of polkit  https://review.openstack.org/47981601:02
*** mdnadeem has joined #tripleo01:02
openstackgerritMerged openstack/tripleo-heat-templates master: Cinder volume/backup containers shouldn't mount two paths at same point  https://review.openstack.org/48982501:02
openstackgerritMerged openstack/tripleo-heat-templates master: Set redis password hiera value in compute agent  https://review.openstack.org/48748001:02
stevebakerEmilienM: the fix is for the image built for nodepool to boot from, not the image built in the CI job itself01:03
ianwEmilienM: but the missing file is on the build host.  i mean ... perhaps tripleo should be installing it, which would also work ... but i do think it can be considered part of the base image01:03
EmilienMah, ok01:03
EmilienMI see, thanks01:03
*** psahoo has joined #tripleo01:03
ianwstevebaker: puppet hasn't rolled out the new version to the builders yet, but when it has, i can trigger a build01:04
stevebakerianw: ok, thanks01:05
stevebakerEmilienM: in the meantime we could just install policycoreutils-python here http://logs.openstack.org/88/487888/1/check/gate-tripleo-buildimage-overcloud-full-centos-7/389cfe8/console.html#_2017-08-02_23_38_54_34610601:05
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814601:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]01:10
openstackgerritSteve Baker proposed openstack/python-tripleoclient master: Install policycoreutils-python  https://review.openstack.org/49026501:14
openstackgerritSteve Baker proposed openstack/python-tripleoclient master: Add --set argument to prepare command  https://review.openstack.org/48908301:18
*** yamahata has quit IRC01:20
openstackgerritSteve Baker proposed openstack/python-tripleoclient master: Minor fixes for container image prepare command  https://review.openstack.org/48406101:21
*** tzumainn has quit IRC01:32
*** noslzzp has quit IRC01:34
*** gbarros has quit IRC01:38
openstackgerritMerged openstack/tripleo-heat-templates master: Make RoleParameters and key_name descriptions consistent  https://review.openstack.org/48353901:40
stevebakerianw, EmilienM: No need for an async image build I think, this fixes the job for now https://review.openstack.org/#/c/49026501:42
*** ansmith has quit IRC01:43
ianwstevebaker: i kicked off a build anyway, new version just rolled out -> http://nodepool.openstack.org/dib-image-list01:47
stevebakerianw: ok, ta01:47
*** ramishra has joined #tripleo01:51
*** gbarros has joined #tripleo01:57
*** ansmith has joined #tripleo02:02
*** itlinux has joined #tripleo02:02
*** dmacpher has joined #tripleo02:04
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/48814802:07
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778802:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814602:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]02:10
*** jrist has joined #tripleo02:11
*** jrist has joined #tripleo02:11
*** trozet has quit IRC02:16
*** itlinux has quit IRC02:50
*** kbyrne has quit IRC02:54
*** kbyrne has joined #tripleo02:58
*** dhill_ has quit IRC03:02
*** dhill_ has joined #tripleo03:06
*** ansmith has quit IRC03:06
*** ansmith has joined #tripleo03:07
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778803:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814603:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)03:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]03:10
*** mmethot_ has joined #tripleo03:18
*** mmethot_ has quit IRC03:19
*** mmethot_ has joined #tripleo03:20
*** _mmethot_ has quit IRC03:22
*** dixiaoli has quit IRC03:28
*** udesale has joined #tripleo03:34
*** gkadam-afk has joined #tripleo03:37
*** ykarel has joined #tripleo03:39
*** ykarel has quit IRC03:46
*** ykarel has joined #tripleo03:46
*** _mmethot_ has joined #tripleo03:49
*** mmethot_ has quit IRC03:53
*** links has joined #tripleo03:54
*** gbarros has quit IRC03:54
*** dbecker has quit IRC03:54
EmilienMmichapma: what's up with neutron/odl?04:03
EmilienMstevebaker: excellent fix, thank you04:03
EmilienMstevebaker: and thanks for self approving this one, no problem at all04:03
stevebakerEmilienM: its going to need a recheck anyway04:03
EmilienMah, why?04:03
EmilienMwe'll probably need to backport it04:04
stevebakerEmilienM: undercloud-containers job failure04:04
*** itlinux has joined #tripleo04:06
*** dbecker has joined #tripleo04:06
*** limao has quit IRC04:09
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778804:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814604:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]04:10
*** limao has joined #tripleo04:17
*** limao has quit IRC04:18
*** limao has joined #tripleo04:18
*** limao has quit IRC04:19
*** limao has joined #tripleo04:19
*** limao has quit IRC04:20
*** limao has joined #tripleo04:20
*** limao has quit IRC04:21
*** limao has joined #tripleo04:21
*** limao has quit IRC04:21
*** limao has joined #tripleo04:22
*** ramishra has quit IRC04:22
*** limao has quit IRC04:22
*** limao has joined #tripleo04:23
*** limao has quit IRC04:23
*** ramishra has joined #tripleo04:24
*** limao has joined #tripleo04:24
michapmaEmilienM: the CI job still has that dependency on websockets missing afaik04:24
*** limao has quit IRC04:24
openstackgerritMerged openstack/tripleo-heat-templates master: Render isolated network templates using jinja2  https://review.openstack.org/47561204:30
EmilienMmichapma: right, nevermind my question :)04:31
michapmaEmilienM: no worries :D do you know when the new packages will be promoted?04:31
EmilienMmichapma: I hope tonight04:32
*** yamahata has joined #tripleo04:32
EmilienMbut it seems like we have 2 bugs in alerts, have you seen them ^04:32
EmilienMthey block promotion I'm pretty sure04:32
*** janki has joined #tripleo04:34
*** shreshtha has joined #tripleo04:36
*** ecerquei has quit IRC04:36
michapmaEmilienM: looks like the first one is fixed but the heat one is ongoing. I'm a bit out of my depth on that so I don't think I can help.04:36
*** skramaja has joined #tripleo04:38
*** ansmith has quit IRC04:46
*** ansmith has joined #tripleo04:48
weshay_ptoEmilienM, you on04:52
weshay_pto?04:52
*** _mmethot_ has quit IRC04:56
*** _mmethot_ has joined #tripleo04:57
*** limao has joined #tripleo04:57
*** _mmethot_ has quit IRC04:58
*** mmethot has joined #tripleo05:03
*** mmethot has quit IRC05:03
*** mmethot has joined #tripleo05:04
*** udesale__ has joined #tripleo05:05
*** udesale has quit IRC05:08
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778805:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814605:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]05:10
*** ratailor has joined #tripleo05:11
*** ansmith has quit IRC05:12
*** pgadiya has joined #tripleo05:15
*** pdeore has joined #tripleo05:19
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Move subunit test results generation pieces in tempest-results  https://review.openstack.org/48696505:25
*** dpawar has joined #tripleo05:28
*** dpawar has quit IRC05:28
*** ansmith has joined #tripleo05:30
*** marios has joined #tripleo05:30
*** prateek has joined #tripleo05:31
jaosoriorbandini: hey dude, I'm a little bit confused on what's missing here https://review.openstack.org/#/c/489132/3/puppet/services/pacemaker.yaml05:33
*** dsariel has joined #tripleo05:36
*** itlinux has quit IRC05:37
*** mandre has joined #tripleo05:43
jaosoriormandre: hey, any idea what's missing from this patch? https://review.openstack.org/#/c/475366/05:49
*** dpawar has joined #tripleo05:50
*** psachin has joined #tripleo05:51
*** cshastri has joined #tripleo05:53
mandrejaosorior: hey! nothing strikes me as wrong or missing in that patch05:55
*** masco has joined #tripleo05:55
jaosoriormandre: morning dude!05:55
jaosoriormandre: alright. I'm trying it out locally now. Just wanted to know if something obvious is missing05:55
mandrejaosorior: but I just came back from PTO so I don't know about the status of the CI05:56
jaosoriorwelcome back05:56
jaosoriorhow did the PTO go?05:56
mandrejaosorior: very well, thank you!05:57
*** agurenko has joined #tripleo05:57
mandrejaosorior: the only thing to keep in mind when changing the from eventlet to apache is to change the user that runs inside the container to root, and I see you did it already05:58
jaosoriorright05:58
*** brault has joined #tripleo05:58
jaosoriormandre: so, this patch is on top of the one I just passed you https://review.openstack.org/#/c/489885/ and had better luck in CI (CI had been broken for a bit, but now it's fixed)05:59
jaosoriormandre: and when trying to access nova I get a 503. Meaning the vhost is not actually running05:59
jaosoriormandre: seems there are no httpd logs for the nova container :/ so yeah. something went wrong and I don't know what it is.05:59
jaosoriorI guess I'll know when this finishes deploying06:00
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260706:00
jaosorioris there a solution already for getting the logs from containers? I see there is the /var/log/containers folder which has the logs for the service it's running ( keystone.log, nova-compute.log, etc. ) but it doesn't seem to take into account the httpd logs06:00
mandrejaosorior: hmm, no httpd logs you say? has anything changed with log collection?06:01
*** marios has quit IRC06:01
openstackgerritDebo Zhang proposed openstack/python-tripleoclient master: Change tag link in README to the newest address  https://review.openstack.org/49033706:01
*** lmiccini has joined #tripleo06:01
*** marios has joined #tripleo06:02
jaosoriormandre: not sure.06:02
*** mwhahaha has quit IRC06:02
jaosoriormandre: at least I couldn't find them.06:02
mandrejaosorior: ack, I'll see if there is anything that needs to be done wrt http logs after I've done a first pass on my email inbox06:04
*** dsariel has quit IRC06:06
*** jaganathan has joined #tripleo06:09
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778806:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814606:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]06:10
marios /exit06:11
*** marios has quit IRC06:11
*** marios has joined #tripleo06:11
*** pgadiya has quit IRC06:18
*** rcernin has joined #tripleo06:19
*** dpawar has left #tripleo06:20
*** dsariel has joined #tripleo06:21
*** gkadam-afk is now known as gkadam06:22
openstackgerritDebo Zhang proposed openstack/puppet-tripleo master: Update link addresses in README.md  https://review.openstack.org/49033906:22
*** iranzo has joined #tripleo06:23
*** iranzo has joined #tripleo06:23
*** jfrancoa has joined #tripleo06:26
*** pgadiya has joined #tripleo06:34
*** rlandy has joined #tripleo06:37
*** rlandy is now known as rlandy|ci_mtgs06:37
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates stable/newton: Ensure yum cache is ready before update  https://review.openstack.org/49034706:39
*** iranzo has quit IRC06:42
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Add logging documentation  https://review.openstack.org/48913406:43
openstackgerritMerged openstack/tripleo-heat-templates master: Fix CA file bind mounting in containers  https://review.openstack.org/48950906:46
openstackgerritMerged openstack/tripleo-quickstart-extras master: Better long validation task management  https://review.openstack.org/49009806:48
*** pgadiya has quit IRC06:48
*** pcaruana has joined #tripleo06:48
*** jaganathan has quit IRC06:51
*** jaganathan has joined #tripleo06:52
*** coolsvap has joined #tripleo06:53
*** limao has quit IRC06:56
*** pgadiya has joined #tripleo07:00
*** jlinkes has joined #tripleo07:01
*** mcornea has joined #tripleo07:03
*** coolsvap has left #tripleo07:04
*** jtomasek has joined #tripleo07:09
openstackgerritMarius Cornea proposed openstack/tripleo-heat-templates master: Stop and disable openstack-nova-compute service on compute nodes  https://review.openstack.org/49022607:09
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778807:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814607:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]07:10
openstackgerritMartin AndrĂ© proposed openstack/puppet-tripleo master: Configure dockerd with --iptables=false  https://review.openstack.org/49020107:12
*** tesseract has joined #tripleo07:17
*** udesale__ has quit IRC07:18
*** udesale__ has joined #tripleo07:19
*** ebarrera has joined #tripleo07:19
*** ccamacho has joined #tripleo07:20
*** paramite has joined #tripleo07:20
*** anshulbehl has joined #tripleo07:22
*** limao has joined #tripleo07:25
*** yprokule has joined #tripleo07:25
*** amoralej|off is now known as amoralej07:28
*** snecklifter has quit IRC07:30
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Install openstack services tempest plugin  https://review.openstack.org/49036907:36
*** jpena|off is now known as jpena07:39
*** cylopez has joined #tripleo07:42
jaosoriordciabrin: ping07:43
jaosoriordciabrin: did you figure out the issue from yesterday? Or should we open a tmate?07:43
*** egonzalez has joined #tripleo07:43
*** ykarel has quit IRC07:45
arxcruzchandankumar: hello07:45
arxcruzchandankumar: do you know which tempest plugin package has the TelemetryAlarming tests ?07:45
*** oidgar has joined #tripleo07:47
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Install openstack services tempest plugin  https://review.openstack.org/49036907:48
*** ansmith has quit IRC07:49
*** ansmith has joined #tripleo07:50
chandankumararxcruz: hello07:52
arxcruzchandankumar: don't worry, I figured out :)07:52
arxcruzthanks07:52
chandankumararxcruz: python-aodh-tests and python-ceilometer-tests07:52
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Use haproxy::endpoint resource for horizon  https://review.openstack.org/48959207:52
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Enable TLS in the internal network for horizon  https://review.openstack.org/48959307:53
openstackgerritArx Cruz proposed openstack/tripleo-quickstart master: Configure the set of tests to execute in scenario001  https://review.openstack.org/49037607:57
chandankumararxcruz: https://review.openstack.org/#/c/490369/2/roles/validate-tempest/tasks/pre-tempest.yml@35 change state to latest07:59
chandankumarif package is already there, then it will update it07:59
chandankumarif not then it will install it07:59
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Install openstack services tempest plugin  https://review.openstack.org/49036908:00
chandankumararxcruz: you are so quick08:00
*** ramishra has quit IRC08:01
*** mmethot has quit IRC08:02
*** mwhahaha has joined #tripleo08:03
*** limao has quit IRC08:05
*** limao has joined #tripleo08:05
*** ramishra has joined #tripleo08:08
*** limao has quit IRC08:09
jtomasekhonza: have you seen this? I am getting 7 failed tests when I rebase my error handling patch on top of current master http://paste.openstack.org/show/617375/08:10
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778808:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814608:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]08:10
honzajtomasek: looks like a recursive import, this can usually be solved by reordering imports in tests08:11
*** limao has joined #tripleo08:11
honzajtomasek: try moving the store import to the top08:11
jtomasekcrazy08:11
chandankumararxcruz: do we have ping test for undercloud also?08:11
honzajtomasek: but it's weird that it passed the gate --- i'm checking locally now, too08:12
jtomasekhonza: I don't see it on master08:12
jtomasekhonza: maybe I've moved the store import back to the bottom when I did the rebase08:12
honzajtomasek: ah, sorry, didn't read that properly08:12
jtomasekchecking now08:12
*** hewbrocca_afk is now known as hewbrocca08:12
jaosoriormandre: where can I find nowadays the container configuration?08:14
jaosoriorfor isntance, for nova08:15
*** limao has quit IRC08:15
mandrejaosorior: that would be http://logs.openstack.org/85/489885/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq/06a957f/logs/overcloud-controller-0/var/log/config-data/nova/etc/08:15
honzayeah, master is fine for me08:15
*** panda has joined #tripleo08:16
jaosoriormandre: I see that keystone has the httpd configuration there (in the config-data directory) but nova doesn't08:17
jaosoriorseems I might be missing something :/08:17
*** derekh has joined #tripleo08:18
*** openstackgerrit has quit IRC08:18
*** udesale has joined #tripleo08:18
*** snecklifter has joined #tripleo08:19
mandrejaosorior: wait, are you using a container image that has apache to generate the configuration?08:19
jaosoriormandre: I'm using the image that was uploaded, as far as I can remember we had enabled httpd there.08:20
*** udesale__ has quit IRC08:20
jaosoriorunless we're removing it somewhere?08:20
*** ukalifon has joined #tripleo08:20
*** dmacpher has quit IRC08:21
jaosoriorwhich we're not as far as I can tell08:21
*** chem has joined #tripleo08:22
*** bogdando has joined #tripleo08:23
*** openstackgerrit has joined #tripleo08:24
openstackgerritHonza Pokorny proposed openstack/tripleo-common master: Add GUI logging workflows  https://review.openstack.org/46919608:24
mandrejaosorior: just had a quick look at the code, nova config is generated using DockerNovaConfigImage which defaults to centos-binary-nova-base08:24
arxcruzchandankumar: I believe yes, but we don't test08:25
jaosoriormandre:08:25
jaosoriordude08:25
jaosorioryou're right!08:25
mandrejaosorior: I don't think nova-base has http08:25
jaosoriorit doesn't08:25
jaosorioronly nova-api does08:25
jaosoriormandre: what do you recommend? should we do a DockerNovaApiConfigImage parameter and use that instead?08:26
jaosoriormandre: yeah, placement uses it's own08:26
jaosorior*its08:26
mariossoc_off: o/ hey man was just wondering if this one was forgotten or we didn't want it for some reason https://review.openstack.org/49034708:28
mandrejaosorior: probably it's better to change the default image for the nova config to be nova-api, I don't see why we would have a different config-volume just for nova-api08:28
mariossoc_off: while we still have a stable/newton may as well , but thought to check08:28
*** lucas-afk is now known as lucasagomes08:28
*** paken has joined #tripleo08:28
soc_offmarios: you are right08:28
mandrejaosorior: not sure if it's all wired up correctly now, but normally you should just have to update https://github.com/openstack/tripleo-common/blob/master/container-images/overcloud_containers.yaml.j208:29
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Enable TLS for nova api and placement containers  https://review.openstack.org/48988508:29
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Make containerized nova-api run with httpd  https://review.openstack.org/47536608:29
soc_offmarios: +1 there, that review should go in08:29
jaosoriormandre:  lets try that08:29
mariosack thanks for checking soc_off08:30
mandrejaosorior: just deleting lines 235-237 should do it I think08:31
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-common master: Use nova-api image for configuration as well  https://review.openstack.org/49038508:32
honzajtomasek: any luck?08:32
jaosoriormandre: ^^08:32
*** milan has joined #tripleo08:32
jaosoriorwhat about that?08:32
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Enable TLS for nova api and placement containers  https://review.openstack.org/48988508:33
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Make containerized nova-api run with httpd  https://review.openstack.org/47536608:33
*** jaganathan has quit IRC08:33
*** jaganathan has joined #tripleo08:34
jtomasekhonza: not so much, but I bet it is related to the fact that we import store in different places in UI and we call the logger.setReduxDispatch there08:34
jtomasekhonza: I have no idea why this does not occur in master08:35
honzajtomasek: is it broken for you only in tests or also in the browser?08:35
jtomasekhonza: my patch is not related to any of this08:35
* jtomasek checks08:35
*** bogdando has quit IRC08:35
jtomasekhonza: just tests08:36
openstackgerritDebo Zhang proposed openstack/tripleo-heat-templates master: Update documentation links in README  https://review.openstack.org/49038808:37
jtomasekhonza: app works fine08:37
honzajtomasek: yeah, i have definitely seen this before08:37
*** bogdando has joined #tripleo08:39
*** zoli is now known as zli08:42
*** zli is now known as zoli08:42
*** suuuper has joined #tripleo08:46
openstackgerritMerged openstack/tripleo-puppet-elements master: Adds puppet-odl module to overcloud image  https://review.openstack.org/48143308:48
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates master: Add validation task in docker services [Ceilometer]  https://review.openstack.org/49039208:53
*** sri_ has joined #tripleo08:53
mandrejaosorior: commented on the review, your solution won't work08:54
rdopieradid anybody manage to install an older version of osp (like osp10) with tripleo-quickstart? I'm getting 404 errors on trunk.rdoproject.org/centos708:57
*** apetrich has quit IRC08:58
adarazsrdopiera: for OSP you need the downstream documentation.08:59
rdopieraadarazs: I'm using it08:59
rdopieraadarazs: but there is no downstream tripleo channel to ask at09:00
adarazsrdopiera: there's #rhos-dev / #rhos-mgt :)09:00
*** nyechiel has joined #tripleo09:00
*** ykarel has joined #tripleo09:00
rdopieraadarazs: yeah, asked there, got ignored09:01
adarazsthat's the downstream channel though :)09:01
rdopierawell, it's not for tripleo specifically09:01
*** isq_ has joined #tripleo09:02
*** isq has quit IRC09:03
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778809:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814609:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]09:10
*** ykarel has quit IRC09:12
*** akrivoka has joined #tripleo09:14
openstackgerritNuman Siddique proposed openstack/tripleo-heat-templates master: Support deploying OVN as container services  https://review.openstack.org/46886009:16
*** ffiore has joined #tripleo09:18
*** ansmith has quit IRC09:25
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-ui master: Imported Translations from Zanata  https://review.openstack.org/48835709:25
*** ansmith has joined #tripleo09:26
*** iranzo has joined #tripleo09:28
*** bogdando has quit IRC09:29
*** salmankhan has joined #tripleo09:29
*** psahoo has quit IRC09:31
*** ffiore has quit IRC09:31
openstackgerritMerged openstack/python-tripleoclient master: Skip empty resource_registry sections  https://review.openstack.org/48925809:33
*** iranzo has quit IRC09:33
*** fzdarsky has joined #tripleo09:36
*** stendulker has joined #tripleo09:36
*** apetrich has joined #tripleo09:44
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs  https://review.openstack.org/35921509:45
openstackgerritSanjay Upadhyay proposed openstack/tripleo-quickstart master: This adds a variable to add stack user to the undercloud image  https://review.openstack.org/49040709:46
*** jaganathan has quit IRC09:46
*** jaganathan has joined #tripleo09:47
*** tosky has joined #tripleo09:47
*** strigazi_OFF has quit IRC09:49
openstackgerritSanjay Upadhyay proposed openstack/tripleo-quickstart master: This adds a variable to add stack user to the undercloud image  https://review.openstack.org/49040709:50
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: [WIP]Enable additional params for scenario/tempest ping test.  https://review.openstack.org/48712709:50
*** strigazi_OFF has joined #tripleo09:50
*** ramishra has quit IRC09:50
openstackgerritJose Luis Franco proposed openstack/tripleo-heat-templates master: Add validation task in docker services [Ceilometer]  https://review.openstack.org/49039209:51
*** ramishra has joined #tripleo09:53
*** yamahata has quit IRC09:55
openstackgerritLee Yarwood proposed openstack/instack-undercloud master: WIP zaqar: restorecon zaqar.log to ensure httpd has access  https://review.openstack.org/49040909:56
*** ramishra has quit IRC09:57
jtomasekhonza: ok, so this is example of a failing test: http://paste.openstack.org/show/617393/09:59
*** ansmith has quit IRC09:59
jtomasekhonza: the stack trace goes by imports09:59
jtomasekhonza: if I add import store to RolesActions.tests.js, the test passes10:00
honzajtomasek: yep10:00
*** dsariel has quit IRC10:00
jtomasekhonza: but I cant understand why10:00
*** ramishra has joined #tripleo10:00
honzajtomasek: because the store.js file has to be the first one to import the logging service so that the dispatch function is available10:02
honzajtomasek: i think it's caused by the way that imports work/don't work in webpack (or whatever)10:02
honzajtomasek: i.e. it can't detect circular imports accurately10:02
honzajtomasek: but yeah, it's not ideal in tests; we don't want to have to set up any unnecessary state10:05
*** katkapilatova has joined #tripleo10:09
*** dtantsur|afk is now known as dtantsur10:10
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778810:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814610:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]10:10
honzajtomasek: one idea to solve this: make the logging service part of the store10:10
honzajtomasek: => you can't have a logger without the store and vice versa10:10
honzajtomasek: am i making any sense? :)10:11
jtomasekhonza: I am having hard time to understand this, but am I correct that if we import LoggerService, we get a logger instance10:12
*** ansmith has joined #tripleo10:12
jtomasekhonza: but in case when we do this anywhere else than the store, the adapters won't get loaded10:12
*** dsariel has joined #tripleo10:13
honzajtomasek: it's a singleton so you get the same instance everywhere... except the store needs to import it first so that we can set the dispatch function10:13
honzaso it's a race condition10:13
*** ramishra has quit IRC10:13
jtomasekhonza: right, and that is what tests reveal10:14
honzajtomasek: yes, and it works fine in the browser because we're loading the whole application in order; in tests you only load some tiny pieces that you're testing10:15
*** ramishra has joined #tripleo10:15
honzajtomasek: in tests, we could add some beforeEach where the store is set up properly10:15
honzajtomasek: to mimick the real-world app10:16
jtomasekhonza: yeah10:16
*** dsariel has quit IRC10:19
jtomasekhonza: I think that if we defined a new middleware for logging it would be better as we could pass the dispatch to the LoggingService10:19
*** dciabrin_ has joined #tripleo10:20
jtomasekhonza: rather than reusing redux-logger which does not allow loggers to access dispatch10:20
honzajtomasek: but how?  you need the store to create middleware which can access the dispatch function, and you need to create the store with the middleware...10:22
honzajtomasek: that's why i was thinking of making our own custom store instance10:22
honzajtomasek: but to be honest, i haven't looked at the machinery in createStore10:23
jtomasekhonza: why you need to create store with the middleware?10:23
honzajtomasek: the applyMiddleware call happens when calling createStore or before, right?10:24
honzajtomasek: can you apply middleware after the store is created?10:24
* honza confused10:25
* honza reads about redux middleware10:26
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Install openstack services tempest plugin  https://review.openstack.org/49036910:28
*** hewbrocca is now known as hewbrocca_afk10:32
*** dsariel has joined #tripleo10:32
honzajtomasek: interesting, i think i know what you mean now; middleware is exactly what we want10:33
*** bkopilov_ has quit IRC10:35
jtomasekhonza: then we'd just need to dispatch an agreed type of action when an error is supposed to be logged which the middleware catches10:36
*** dsariel has quit IRC10:38
owalshjistr: I think we can rebase https://review.openstack.org/469116 now10:39
owalshjistr: I can take on this if you're busy10:40
*** bkopilov_ has joined #tripleo10:41
*** shreshtha has quit IRC10:43
*** suuuper has quit IRC10:43
*** jkilpatr has quit IRC10:44
*** mdnadeem has quit IRC10:47
*** bogdando has joined #tripleo10:47
*** ckyriakidou has joined #tripleo10:50
*** dsariel has joined #tripleo10:50
*** udesale__ has joined #tripleo10:52
*** udesale has quit IRC10:53
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Adjust for new pcs output.  https://review.openstack.org/49042210:54
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Adjust for new pcs output.  https://review.openstack.org/49042210:54
*** dciabrin has quit IRC10:54
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: MistralApiService, SwiftApiService error handling  https://review.openstack.org/48438210:55
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: IronicInspectorApiService error handling  https://review.openstack.org/48697710:55
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: IronicApiService error handling  https://review.openstack.org/48697610:55
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: KeystoneApiService error handling  https://review.openstack.org/48697910:55
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: HeatApiService error handling  https://review.openstack.org/48697810:55
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Adjust for new pcs output.  https://review.openstack.org/49042210:56
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Fix plan export URL  https://review.openstack.org/48816410:58
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Update updatePlan action  https://review.openstack.org/48919610:59
*** dixiaoli has joined #tripleo10:59
*** dixiaoli has quit IRC11:04
*** amoralej is now known as amoralej|lunch11:05
*** hewbrocca_afk is now known as hewbrocca11:06
*** udesale__ has quit IRC11:08
*** agurenko has quit IRC11:08
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778811:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814611:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]11:10
*** ansmith has quit IRC11:14
*** stendulker_ has joined #tripleo11:15
*** jaganathan has quit IRC11:16
*** jkilpatr has joined #tripleo11:17
*** stendulker has quit IRC11:18
*** stendulker_ has quit IRC11:19
openstackgerritLee Yarwood proposed openstack/instack-undercloud master: WIP zaqar: restorecon zaqar.log to ensure httpd has access  https://review.openstack.org/49040911:20
adarazstherve, d0ugal, zaneb: SOS, I just reran the periodic promotion job and it still failed in the timeframe (~7 minutes after the start of the overcloud deploy) that looks like the issue with https://bugs.launchpad.net/tripleo/+bug/170814611:27
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]11:27
adarazswe really need a solution for this ASAP, it blocks everything down the line :S11:28
jistrowalsh: ok if you could that would be great. I'm looking into creating ssh user for Ansible on split stack envs11:29
therveadarazs, :/11:29
therveadarazs, Got the logs11:30
therve?11:30
adarazstherve: not yet, the job is finishing.11:30
openstackgerritLee Yarwood proposed openstack/instack-undercloud master: WIP zaqar: restorecon zaqar.log to ensure httpd has access  https://review.openstack.org/49040911:32
oidgarHi guys, I installed new undercloud then tried to deploy overcloud with containers but it failed at the beginning11:34
oidgarhttp://paste.openstack.org/show/617408/11:34
adarazstherve: same thing again: http://logs.openstack.org/15/359215/122/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq/4ea32d2/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz#_2017-08-03_11_18_4511:34
oidgarhas someone else encountered the same problem?11:34
dtantsurhi folks! may I get 2nd +2 on 3 quite simple patches please: https://review.openstack.org/#/c/489228/ https://review.openstack.org/#/c/487166/ https://review.openstack.org/#/c/487175/11:35
* hewbrocca -2 dtantsur 11:36
hewbroccasorry dtantsur general principles11:36
dtantsurhewbrocca: if you don't behave, I'll tell Piklz!11:36
hewbroccaO NOES /me runs11:36
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Adjust for new pcs output.  https://review.openstack.org/49042211:36
dtantsur:D11:36
therveadarazs, That doesn't make sense :/11:37
*** dsariel has quit IRC11:37
adarazstherve: so you're saying that everything master (including heat) should not have this problem anymore?11:37
*** lmiccini has quit IRC11:38
therveadarazs, Yeah see the last logs of https://review.openstack.org/#/c/477047/11:38
therveWell we can't trust what it runs anymore though11:39
*** atoth has joined #tripleo11:40
adarazsoh wow, these are the old, non-oooq jobs :S11:40
*** apetrich has quit IRC11:41
*** lmiccini has joined #tripleo11:41
adarazstherve: looking at the logs, it looks to me that heat-engine is from the latest upstream here instead of built on the job:11:42
adarazsopenstack-heat-engine   noarch 1:9.0.0-0.20170803031044.1cdd02c.el7.centos11:42
therveadarazs, What's the latest upstream?11:42
therveThat id doesn't match anything11:43
adarazsbut maybe I don't understand something about how the old jobs work. but we should have the new oooq jobs in the "check experimental" queue. /me goes and fixes that.11:43
therveadarazs, Well there is http://logs.openstack.org/47/477047/20/experimental/gate-tripleo-ci-centos-7-nonha-multinode-oooq/ee7438d/11:43
adarazstherve: oh wait, never mind. the job did run around 3am UTC, so this package was generated for this job.11:43
therveBut it doesn't seem really trust worthy either11:43
adarazsI'll check that out.11:44
hewbroccaadarazs: did we launch the wrong job?11:44
adarazshewbrocca: no, I checked that the job that failed was with this hash: https://trunk.rdoproject.org/centos7/b3/b7/b3b7d520396888f05dc497da7a509c58541cef6f_d767b7fe/?C=M;O=D -- which has the latest packages from 10:01 AM UTC, including fresh heat packages from 9:40 AM UTC11:46
*** agurenko has joined #tripleo11:46
adarazsso the periodic jobs did run with the latest consistent hash from master.11:46
*** fzdarsky has quit IRC11:47
*** mcornea has quit IRC11:47
*** lucasagomes is now known as lucas-hungry11:48
*** ukalifon has quit IRC11:48
therveadarazs, So https://review.openstack.org/#/c/490438/11:49
therveWe'd need at leas 4h before it lands though :/11:49
therveadarazs, We used to be able to remove commits from tripleo CI? can we still do that?11:49
*** mcornea has joined #tripleo11:50
adarazstherve: those gates on the heat change run with the latest *promoted* hash, some packages unpinned + that heat change. so if they pass my only idea is that the problem is not with heat but something else.11:50
adarazshttps://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/master.yml#L47-L63 -- these are the packages that are unpinned to latest master on the gates.11:50
adarazs*unpinned from the latest promoted.11:50
fultonjjistr: Regarding https://review.openstack.org/#/c/489613, is it easier for you to tell me the extra parameters I need to use it or do you prefer to just make another patch?11:51
jistrfultonj: we'll need probably 3 more patches11:51
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Add logrotate with crond service  https://review.openstack.org/49005011:51
jistrfultonj: it's nothing we can write into the workflow directly11:51
therveadarazs, So. The revert https://review.openstack.org/490075 had CI passing11:51
therveadarazs, master had CI failing: https://review.openstack.org/49019411:51
adarazstherve: I can't selectively include that in a promotion :/11:52
therveadarazs, And the supposed fix had CI passing too11:52
therveSo the outlier is the promotion job here11:52
jistrfultonj: it has to be connected into the CI. I'm working on a bash script that will be generic enough both for users and for CI to enable ssh admin for deployed server environments. Then we'll plug that into TOCI and OOOQ.11:52
adarazshmmm.11:52
therveadarazs, Except if non-ooo jobs don't make sense?11:52
therveoooq I mean11:52
*** dsariel has joined #tripleo11:52
* adarazs looks at how those jobs failed11:54
adarazsoh, you're right. they exhibit the problem.11:54
therveadarazs, Can you double check the promo job used the proper heat version?11:54
therveIt seemd okay to me, but... I don't trust anything anymore :)11:54
openstackgerritBogdan Dobrelya proposed openstack/tripleo-common master: Add logrotate-crond container images params  https://review.openstack.org/49006811:55
adarazstherve: so do you have a change where these experimental gates passed and it's already merged?11:55
therveadarazs, No. I run again on the test commit.11:56
therveBut in theory that's what the check experimental in the patch did11:56
adarazstherve: I'm confused. so what made you think that running on the latest master will work without any revert?11:57
fultonjjistr: So openstack-infra/tripleo-ci, openstack/tripleo-quickstart, and ... THT?11:58
therveadarazs, Which run are you talking about? https://review.openstack.org/490194 without the fix?11:58
jistrfultonj: yes. Working on tht, that should be the first one.11:58
adarazstherve: FWIW the period job did run with the recent heat from master: http://logs.openstack.org/15/359215/122/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq/4ea32d2/logs/undercloud/var/log/extra/yum-list-installed.txt.gz11:58
adarazsopenstack-heat-engine.noarch  1:9.0.0-0.20170803093435.4e56332.el7.centos  @delorean11:58
fultonjthank you jistr11:58
jistrsure thing :) it's a pity that we hit this bug. It has nothing to do with Ceph or even containers...11:59
*** jchhatbar has joined #tripleo11:59
adarazstherve: I'm taling about the latest promotion run from master. I understood that the issue is already solved in the latest master without a revert (because a fix got merged during the night), so I can just rerun it and it should work. did I misunderstand you?11:59
*** akrivoka has quit IRC12:00
*** jchhatbar has quit IRC12:00
*** jchhatbar has joined #tripleo12:00
therveadarazs, Ah, because we made related fix in the mean time: https://review.openstack.org/#/c/477047/12:01
therveWhich had CI green12:01
jkilpatrso in recent tripleo builds / quickstart images the perms on /tmp have changed. (a playbook I use had to be updated) was this intentional?12:01
*** sbrzozow has joined #tripleo12:01
*** morazi has joined #tripleo12:02
*** pdeore has quit IRC12:02
*** janki has quit IRC12:02
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Add logrotate with crond service  https://review.openstack.org/49005012:03
adarazstherve: okay, so I didn't misunderstand you, and I understand your confusion... let me check the packages on that. but yeah, this doesn't make much sense. unless that ovb-ha-tempest-oooq gate is somehow runs different things than the other two promotion job.12:03
adarazstherve: which they might.12:03
therveadarazs, Yeah, there is something going on at least in CI...12:04
therveadarazs, Can we find the actual package bits in delorean?12:04
adarazstherve: for which run?12:04
therveadarazs, The latest promotion test12:04
adarazstherve: it was running with https://trunk.rdoproject.org/centos7/b3/b7/b3b7d520396888f05dc497da7a509c58541cef6f_d767b7fe/?C=M;O=D12:05
adarazsthis is what you mean?12:06
*** agurenko has quit IRC12:06
*** agurenko has joined #tripleo12:07
therveadarazs, Oh wow. The code is not there12:08
adarazshow is that possible?12:08
therveadarazs, So the hash is 4e56332 right? Which should be the merge hash?12:09
*** amoralej|lunch is now known as amoralej12:09
adarazstherve: yes12:09
adarazsas far as I know DLRN :)12:10
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778812:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814612:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]12:10
therveadarazs, Maybe the forced build didn't operate as you thought?12:11
*** dprince has joined #tripleo12:12
hewbroccaadarazs: can dmsimard help with this?12:12
adarazshewbrocca: yeah, we need jpena or dmsimard for telling us why things are as they are :)12:12
therveadarazs, Can you double check https://trunk.rdoproject.org/centos7/b3/b7/b3b7d520396888f05dc497da7a509c58541cef6f_d767b7fe/openstack-heat-common-9.0.0-0.20170803093435.4e56332.el7.centos.noarch.rpm ?12:12
jpenaadarazs: what do you need?12:13
* jpena reads back12:13
*** akrivoka has joined #tripleo12:13
adarazstherve: I don't have to, you can see that hash is just before the fix: https://github.com/openstack/heat/commits/master12:13
adarazsjpena: let me summarize it.12:13
*** pchavva has joined #tripleo12:13
therveadarazs, I mean, check the content :)12:13
therveOh snap12:13
therveadarazs, Sorry :/12:13
therveOk that was just a stupid mistake on me then12:14
adarazsjpena: so we have this change that merged this morning: https://review.openstack.org/47704712:14
therveI don't know why it didn't take the latest code though12:14
adarazsjpena: but somehow the latest consistent DLRN build that has the timestamp for way later does not contain that commit.12:15
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: WIP: Adds pacemaker update_tasks for Pike minor update workflow  https://review.openstack.org/48656712:16
adarazsjpena: https://trunk.rdoproject.org/centos7/consistent/?C=M;O=D -> openstack-heat-engine-9.0.0-0.20170803093435.4e56332.el7.centos.noarch.rpm12:16
therveThat's weird but way less than I feared it could be12:16
adarazsso the package is generated from 4e56332 which merged quite a while ago.12:16
adarazsbut the timestamp says that the package is built at 09343512:17
therveadarazs, While we sort that out, you should relaunch, it should pick it up now12:17
adarazstherve: nope, /consistent/ still points to the same package with the commit just right before the fix. it won't work again.12:18
therveHum, ok12:18
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: WIP: Adds pacemaker update_tasks for Pike minor update workflow  https://review.openstack.org/48656712:18
*** dhill_ has quit IRC12:19
jpenaadarazs: it's still in the build queue: https://trunk.rdoproject.org/centos7-master/queue.html12:19
jpenanow I'll check why we have this lag12:19
adarazsphew, okay, we're not going crazy at least :)12:19
adarazsjpena: yeah, I was assuming that the change will be in the DLRN build after a few hours, didn't even think about checking it. let us know what's up.12:20
adarazshewbrocca: so it seems we ran the promotion with a hash that didn't yet contain the fix because somehow DLRN is having a long build queue.12:20
hewbroccaweeeeeeee12:21
hewbroccaadarazs: OK put that on your "things to check next time" list :)12:21
adarazsyeah, I rarely chase the tail of master this much. :)12:22
hewbroccaLOL chase the tail of master12:22
hewbrocca"there it goes, get it!"12:22
hewbroccalike that12:22
*** akrivoka has quit IRC12:22
adarazsyeah, feels like a dog chasing its own tail sometimes :)12:23
jpenaadarazs: so we have a combination of factors: 1 is cinder builds taking 30 minutes each (I'll check that, it's way too much and builds up the queue)12:23
*** akrivoka has joined #tripleo12:24
jpenaand 2 is that we have some DB disconnections from the DLRN builder. I've just proposed a review to fix that12:24
adarazsjpena: cool, thanks for looking into it. our heat change is next in line at least \o/ :)12:24
therveNow we still need to fix the experimental jobs on Heat master12:27
*** tzumainn has joined #tripleo12:27
d0ugaladarazs, therve: sorry, I was on lunch - any luck?12:27
therved0ugal, Yeah we *should* be okay12:28
adarazsd0ugal: we solved some mysteries regarding DLRN, probably we're on track.12:28
d0ugallol12:28
d0ugalI feel very confident about this12:28
therveHeh heh12:28
therveI'm glad you noticed our skepticism12:28
d0ugal:)12:28
hewbroccaI love this plan! I'm excited to be a part of it! Let's do it!12:28
d0ugalYes!12:29
d0ugalWait, what is the plan?12:29
therveExactly12:29
*** eck` is now known as eck`gone12:29
hewbroccaI think it's12:30
hewbrocca* Let DLRN finish building the change we need12:30
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Add script to create tripleo-admin on deployed servers  https://review.openstack.org/49045812:30
hewbrocca* re-kick the promote job12:30
jistrfultonj: ^ tried, tested, trusted :)))12:30
jistrnow just to plug it into CI...12:30
fultonjthanks jistr12:30
* fultonj looks12:30
hewbrocca* meantime fix the Heat experimental jobs12:30
hewbrocca(not sure about that last one)12:30
adarazshewbrocca: yep, and then meanwhile try a parallel job on RDO Phase1 to get more info :)12:31
adarazsI liked panda's summary at our last retrospective: "Every time I have an answer, the question has changed"12:31
d0ugalGreat12:31
panda:)12:31
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Add script to create tripleo-admin on deployed servers  https://review.openstack.org/49045812:32
adarazstherve: so I you mean we need to fix the experimental jobs because these gates should not have failed on this blank change, right? https://review.openstack.org/49019412:32
therveadarazs, Well, gate-tripleo-ci-centos-7-nonha-multinode-oooq should have failed12:33
therveadarazs, And apparently the other ones should use oooq ?12:33
therveNot it's expected that they failed, it was before the fix12:33
jistrbogdando: for now i'll go with a way that plugs best into what we already have in TOCI and OOOQ, e.g. https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-deploy/tasks/deployed-server.yml12:34
adarazstherve: I think sshnaidm wanted to keep those jobs there. I thought we don't have any oooq jobs there, so the type of jobs ran there should be OK actually.12:35
jistrbogdando: but we can make it use the overcloud_user and overcloud_key variables at some point12:35
adarazstherve: if that change is before the fix, then it should have failed (even the non-oooq ones)12:35
jistrbogdando: it just wouldn't make sense to mix those two right now12:35
bogdandojistr: looking into the patches open12:35
therveadarazs, Yep exactly. It's the oooq one the outlier12:35
adarazstherve: now the thing is that they should pass if you rebase it on top of the fix. so probably the expeirmental gates are working as expected.12:35
*** catintheroof has joined #tripleo12:36
jistrbogdando: we need it also for non-quickstart environments, that's why i did the bash script12:36
jistrdeployed server users (not just CIs etc.) might want to configure the overcloud ssh admin user too12:36
bogdandojistr: do you mean the heat-admin user requirement will stay with us?12:36
bogdandono alts?12:36
openstackgerritNuman Siddique proposed openstack/tripleo-heat-templates master: [WIP] Support OVN DBs docker HA using pacemaker bundle  https://review.openstack.org/48437612:37
jistrbogdando: not heat admin. see the tables in the commit message https://review.openstack.org/#/c/489613/6//COMMIT_MSG12:37
*** thrash is now known as thrash|biab12:38
jistrso split stack envs never had heat-admin, and they won't have it (unless user explicitly wants to create it)12:38
numansbogdando, hi, can you please have a look at this patch - https://review.openstack.org/#/c/468860/ : "Support deploying OVN as container services"12:38
*** pgadiya has quit IRC12:38
EmilienMweshay_pto: yes12:40
amoralejadarazs, the heat fix is now in consistent how could we test it with periodic job?12:40
adarazsyes, I just looked.12:40
amoralejadarazs, how do you do it?, with https://review.openstack.org/#/c/359215/?12:41
adarazsamoralej: yes, with that change. I just rechecked.12:42
dmsimardjpena: cinder build durations are because of the docs (see my email about ftbfs storm)12:42
jpenadmsimard: yep, I'm trying to find some workaround12:42
dmsimardjpena: number80 was open to the idea to only build man docs instead of man+html12:43
dmsimardBut I don't know the ramifications12:43
jpenadmsimard: it would just mean the -doc subpackage would not be created12:44
*** jpena is now known as jpena|lunch12:44
number80well, shipping man in a package is required, doc is recommended, that's the difference :)12:44
dmsimardLet's discuss it on #rdo after lunch12:45
adarazstherve: so now openstack-heat-engine-9.0.0-0.20170803122112.c80011c.el7.centos.noarch.rpm -> that commit hash looks good. :)12:45
adarazsthat's the one with the fix.12:45
*** dmacpher has joined #tripleo12:46
bogdandonumans: looking12:47
chandankumarEmilienM: mwhahaha https://review.openstack.org/#/c/486965/12:48
number80dmsimard: spanish time lunch? :o)12:48
chandankumarplease have a look when you are free.12:48
dmsimardnumber80: ... Yeah12:48
*** ecerquei has joined #tripleo12:49
*** gkadam has quit IRC12:50
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/48814812:50
*** apetrich has joined #tripleo12:52
*** hewbrocca is now known as hewbrocca_afk12:52
*** hewbrocca_afk is now known as hewbrocca12:53
*** cdearborn has joined #tripleo12:53
*** udesale has joined #tripleo12:53
number803pm and I have yet to get solid food, I've been jumping from calls to calls12:53
*** liverpooler has joined #tripleo12:53
*** Goneri has joined #tripleo12:54
*** lucas-hungry is now known as lucasagomes12:54
*** skramaja has quit IRC12:55
adarazshewbrocca, dmsimard: further good news: when I used a fresh (not-yet-promoted) hash the weirdo jobs passed, so if we manage to promote upstream it will likely pass RDO phase1 as well: https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo_trunk-promote-master-current-tripleo/179/12:55
*** Vijayendra has quit IRC12:56
openstackgerritArx Cruz proposed openstack/tripleo-quickstart master: Configure the set of tests to execute in scenario001  https://review.openstack.org/49037612:56
*** gbarros has joined #tripleo12:58
hewbroccaadarazs: ahh that was your time travel experiment from earlier?12:58
adarazsyes.12:58
*** ykarel has joined #tripleo12:58
*** milan has quit IRC12:58
*** _milan_ has joined #tripleo12:58
hewbroccaAll right, let's get DLRN to update12:58
hewbroccago DLRN go!12:58
adarazshewbrocca: that's done. the periodic promotion jobs are already running.12:59
*** noslzzp has joined #tripleo13:00
hewbroccaoh even better13:00
* adarazs gets a coffee.13:01
hewbroccagood idea13:01
*** eck`gone is now known as eck`13:01
*** ansmith has joined #tripleo13:03
*** jchhatbar has quit IRC13:04
moraziadarazs, that's excellent, thanks!13:05
*** Goneri has quit IRC13:05
*** links has quit IRC13:06
openstackgerritJiri Stransky proposed openstack-infra/tripleo-ci master: Set OVERCLOUD_HOSTS for deployed server ssh admin creation  https://review.openstack.org/49046813:06
openstackgerritJiri Stransky proposed openstack/tripleo-quickstart-extras master: Create tripleo-admin user on deployed servers  https://review.openstack.org/49047013:09
jistrfultonj: i think that should do it ^^13:10
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778813:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814613:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)13:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]13:10
*** ratailor has quit IRC13:10
fultonjjistr: thanks! so in https://review.openstack.org/#/c/479288, i should add a depends-on https://review.openstack.org/#/c/490470 ?13:11
*** Goneri has joined #tripleo13:12
*** pdeore has joined #tripleo13:12
jistrfultonj: exactly, i was just typing it :) we also can remove the other depends-ons, as all 4 necessary patches should be pulled in recursively. We actually have to remove dependency on https://review.openstack.org/#/c/483867/  as it would conflict with the other TOCI patch.13:13
fultonjjistr: ok, great. thanks i will update it13:13
*** pdeore has quit IRC13:13
*** ramishra has quit IRC13:13
jistrthanks :)13:13
*** psachin has quit IRC13:13
openstackgerritAlfredo Moralejo proposed openstack/tripleo-quickstart master: Add consistent-pike.ynml configuration  https://review.openstack.org/49047213:14
openstackgerritAlfredo Moralejo proposed openstack/tripleo-quickstart master: Add consistent-pike.yml configuration  https://review.openstack.org/49047213:15
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Convert scenario001-multinode-containers job to ceph-ansible  https://review.openstack.org/47928813:15
*** ramishra has joined #tripleo13:15
* jistr crosses fingers for CI on 47928813:17
* fultonj crosses toes13:17
openstackgerritBogdan Dobrelya proposed openstack/tripleo-common master: Add logrotate-crond container images params  https://review.openstack.org/49006813:17
*** gbarros has quit IRC13:19
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Add bits to support stable/pike  https://review.openstack.org/49007713:20
*** pblaho has quit IRC13:22
*** udesale has quit IRC13:22
*** jcoufal has joined #tripleo13:23
*** udesale has joined #tripleo13:23
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Add bits to support stable/pike  https://review.openstack.org/49007713:23
*** psachin has joined #tripleo13:24
fultonjjistr: in the dependency tree tht depends on quickstart-extras depends on tht ; is that ok? i.e. i can depend on the same repos as long as its not a direct depends on?13:25
jistrfultonj: nope that wouldn't be ok13:25
* jistr looks13:25
fultonj479288 --> 490470 --> 49045813:25
*** lmiccini has quit IRC13:26
jistrfultonj: we'll need to rebase 479288 on top of 490458. nice catch :)13:27
lyarwoodmwhahaha: hey, owalsh suggested that I ask for your take on https://review.openstack.org/#/c/490409/ if you have any review bandwidth today13:28
jistrfultonj: should i or will you?13:28
*** udesale has quit IRC13:28
mwhahahalyarwood: k13:28
fultonjjistr: let me try...13:28
jistrok thanks13:28
lyarwoodmwhahaha: thanks13:29
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Convert scenario001-multinode-containers job to ceph-ansible  https://review.openstack.org/47928813:29
fultonjjistr: done13:29
*** marios has quit IRC13:29
*** marios has joined #tripleo13:30
fultonjjistr: now that we have the rebase, the chain still exists; will CI just ignore the dep that 490470 because 479288 provides the file ?13:30
openstackgerritAdriano Petrich proposed openstack/tripleo-quickstart-extras master: WIP workaround is not needed anymore  https://review.openstack.org/48565313:31
fultonji.e. if you follow the reviews depends-on down one path you still end up taking 479288 --> 490470 --> 49045813:31
jistrfultonj: yea. I'm not 100% sure but i do think it should ignore that depends-on. If we removed the depends-on from 490470, it would likely fail CI and become unmergeable. So hopefully we now have it ok.13:33
openstackgerritKarthik S proposed openstack/tripleo-heat-templates master: NetworkDeploymentActions shall be made role specific  https://review.openstack.org/49047413:33
fultonjjistr: seems about right, let's see what happens.13:34
lyarwoodowalsh: https://review.openstack.org/#/c/490226/ btw, not sure if you've seen this.13:34
*** lblanchard has joined #tripleo13:35
hewbroccaadarazs: how soon should we know if that promote will succeed?13:36
*** pblaho1 has joined #tripleo13:36
owalshlyarwood: ack, LGTM13:36
adarazshewbrocca: at around 14:15 our time (CEST)13:38
*** lmiccini has joined #tripleo13:38
adarazsso ~half an hour13:38
adarazsif I calculated correctly from the previous run :)13:38
*** pblaho1 is now known as pblaho13:41
*** lblanchard has quit IRC13:41
*** jmelvin has joined #tripleo13:41
*** lblanchard has joined #tripleo13:41
hewbroccaadarazs: coool13:41
hewbroccaLMK, all right?13:42
openstackgerritLee Yarwood proposed openstack/instack-undercloud master: WIP zaqar: restorecon zaqar.log to ensure httpd has access  https://review.openstack.org/49040913:44
adarazshewbrocca: you'll be the first to know, except if somebody else is looking at the telnet job output like me. :)13:44
*** Vijayendra has joined #tripleo13:45
*** jpena|lunch is now known as jpena13:46
*** masco has quit IRC13:46
numansbogdando, thanks13:49
*** thrash|biab is now known as thrash13:52
jistrfultonj: it already failed on unrelated issue13:54
jistrOne of the configured repositories failed (Unknown),", "2017-08-03 13:45:58 |  and yum doesn't have enough cached data to continue. At this point the only13:54
jistrhttp://logs.openstack.org/88/479288/20/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq/b295195/console.html#_2017-08-03_13_45_58_27353713:54
jistri discovered i probably need to amend the TOCI patch anyway, so i'll do that and we can recheck13:54
jistr(i forgot to export the variable i added)13:55
fultonjjistr: ok, thanks for catching that13:55
openstackgerritJiri Stransky proposed openstack-infra/tripleo-ci master: Set OVERCLOUD_HOSTS for deployed server ssh admin creation  https://review.openstack.org/49046813:56
*** agopi|away has joined #tripleo13:57
fultonjjistr: as per ^ i just issued a recheck on https://review.openstack.org/#/c/47928813:58
*** mkovacik__ has joined #tripleo13:58
*** _milan_ has quit IRC13:59
jistrfultonj: thanks. Not sure if Zuul caught it tough.. when i amended the toci patch, it should have stopped the CI jobs that depended on it, but it didn't. http://status.openstack.org/zuul/13:59
fultonjyeah, i was just going to ask about that13:59
fultonje.g. can we tell zuul not to waste it's time and start at the start?13:59
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260714:00
adarazsEmilienM, weshay_pto, panda, rlandy|ci_mtgs, arxcruz: TripleO CI Squad meeting in 30 minutes: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting \o/14:00
*** marios has quit IRC14:01
*** cylopez has quit IRC14:01
rlandy|ci_mtgsadarazs: won't be able to make it - clashing session at CI meetings14:02
adarazshewbrocca: 2017-08-03 14:01:58.215102 | TASK [overcloud-deploy : Deploy the overcloud] *********************************14:02
adarazsif it doesn't fail in the next 8-10 minutes then it should be probably fixed (at least the heat issues)14:02
*** pdeore has joined #tripleo14:03
*** marios has joined #tripleo14:03
jistrfultonj: now it noticed, -1'd the patch, and i rechecked again14:03
fultonjjistr: so it noticed the failure you pointed out and voted a -114:05
hewbroccawoop woop14:05
EmilienMadarazs: I'm pretty sure we'll be 2 or 3  :D14:05
*** pblaho has quit IRC14:05
jistrfultonj: it noticed that i pushed the new TOCI patch, which invalidated its dependency chain, so it stopped everything (also on the lower patches)14:06
adarazsEmilienM: yeah. it's more important for talking with panda about handing over his tasks, he'll be out for 4 weeks, starting next week.14:06
fultonjjistr: ok, so that had to percolate up and then it noticed. so then we rechecked14:07
*** mrunge has quit IRC14:07
jistryea14:07
fultonjthanks for explaining it to me14:07
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Drop step_config as top level docker requirement  https://review.openstack.org/44271614:08
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: docker: Add unit tests on service_name  https://review.openstack.org/44275514:08
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Add docker templates to configure Ironic inspector  https://review.openstack.org/45782214:08
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci master: Add adarazs to tripleo-cd-admins  https://review.openstack.org/49049314:09
adarazsEmilienM, panda: ^ please vote on it.14:09
pandaadarazs: -214:09
adarazspanda: that means I get to call you on your cell phone when I need access to those things :)14:09
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170778814:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814614:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1707788 in tripleo "gate-tripleo-buildimage-overcloud-full-centos-7 is broken on missing semanage" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)14:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]14:10
*** pchavva has quit IRC14:10
pandaadarazs: oh yeah ? -4 !14:11
pandabrb14:13
*** panda has quit IRC14:13
adarazshewbrocca: still at the "deploy the overcloud" step, previously it failed under 10 minutes. I'd say that's promising.14:14
*** panda has joined #tripleo14:14
*** mrunge has joined #tripleo14:15
hewbroccanice!14:15
jaosoriormandre: did you see my message on the review?14:16
mandrejaosorior: nope sorry, looking14:16
jaosoriorno biggie14:16
jaosoriorI'll do it :D14:17
adarazsbnemec: can you do some magic, if it's necessary, on the upstream images server to let me get access when this merges? https://review.openstack.org/49049314:17
openstackgerritNuman Siddique proposed openstack/tripleo-common master: Add neutron-server-ovn image to the overcloud containers  https://review.openstack.org/48987514:17
jaosoriorso... your suggestion was to use the nova-api image for almost all nova containers?14:17
jaosoriorI mean, their configuration14:17
jaosoriormandre: ^^14:17
mandrejaosorior: oh the one where you're having lunch? I figured it was just momentary :)14:18
jaosoriormandre: it was a long lunch :P14:18
jaosoriorI wouldn't have asked if it didn't take long14:18
mandrejaosorior: yeah, use nova-api image for the nova config_volume14:19
jaosoriormandre: but yeah. I'm not sure if doing that is the way to go though. Maybe just having nova-api use the nova-api image for the config is the way to go. That's what nova-placement does anyway14:19
*** udesale has joined #tripleo14:19
*** mmethot has joined #tripleo14:19
jaosoriornova-placement just uses the nova-placement iamge14:19
openstackgerritMerged openstack/tripleo-heat-templates master: Make many networking parameters consistent  https://review.openstack.org/48354014:19
*** ykarel has quit IRC14:20
mandrejaosorior: haha, a french sized lunch14:20
openstackgerritMerged openstack/python-tripleoclient master: Install policycoreutils-python  https://review.openstack.org/49026514:20
openstackgerritNuman Siddique proposed openstack/tripleo-common master: PLEASE DON'T REVIEW: FOR TESTING ONLY  https://review.openstack.org/49049514:21
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Remove baremetal cron jobs on docker upgrade  https://review.openstack.org/49049614:21
*** thrash is now known as thrah|biab14:21
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Enable TLS for nova api and placement containers  https://review.openstack.org/48988514:21
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Make containerized nova-api run with httpd  https://review.openstack.org/47536614:21
jaosoriormandre: like that ^^14:21
jaosoriormandre: and this being the tripleo-common change https://review.openstack.org/#/c/490385/14:22
*** athomas has quit IRC14:23
jaosoriorbandini, marios: could you check this out https://review.openstack.org/#/c/489891/ ?14:23
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Add support for installing Ceph MDS via ceph-ansible  https://review.openstack.org/47942614:23
*** jfrancoa has quit IRC14:24
mariosack jaosorior added to reviews14:24
openstackgerritNuman Siddique proposed openstack/tripleo-heat-templates master: DO NOT REVIEW: TESTING ONLY TO TEST OVN CONTAINER SERVICES  https://review.openstack.org/48909714:25
mandrejaosorior: it's not going to work because the nova config_volume is probably still going to use nova-base in the end14:25
mandrejaosorior: for example https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/nova-consoleauth.yaml#L7514:25
*** athomas has joined #tripleo14:26
jaosoriormandre: why? We only need it for nova-api14:26
mandrejaosorior: I'd really prefer we don't introduce a new nova-api config volume, and just change the DockerNovaConfigImage to default to nova-api image instead of nova-base14:26
mandrejaosorior: yeah but it's the same 'nova' config_volume, meaning it's a single puppet run and we need to know which image it will use14:28
jaosoriormandre: sorry, I don't mean to be dense but I really don't understand the issue. Can you explain a bit more?14:28
mandrejaosorior: you can't use nova-api in the nova-api template and nova-base somewhere else14:28
bogdandofolks, PTAL https://review.openstack.org/#/q/topic:bug/170091214:28
bogdandotesting it locally14:29
jaosoriormandre: but, as far as I can see that's what nova-placement does14:29
bogdandohave issues to make it tested with upstream CI14:29
openstackgerritRabi Mishra proposed openstack/tripleo-image-elements master: Create symlink for dib-run-parts  https://review.openstack.org/49050014:29
mandrejaosorior: nova-placement is using a different config_volume, https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/nova-placement.yaml#L7514:29
*** oidgar has quit IRC14:29
bogdandojenkins yells on me "This change depends on a change that failed to merge"14:29
bogdandosigh14:30
jaosoriormandre: oooh14:30
jaosoriormandre: now I get it14:30
jaosoriormandre: thanks. Sorry it took too long14:30
jaosoriorbogdando: need to rebase the change it depends on14:30
EmilienMtripleo ci squad meeting starting: https://bluejeans.com/u/whayutin/14:30
openstackgerritBogdan Dobrelya proposed openstack/puppet-tripleo master: Add logrotate-crond configuration  https://review.openstack.org/49004814:31
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Add logrotate with crond service  https://review.openstack.org/49005014:31
mandrejaosorior: no worries at all, always happy to help14:32
*** apetrich has quit IRC14:34
*** links has joined #tripleo14:34
*** links has quit IRC14:34
*** nyechiel has quit IRC14:38
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-common master: Use nova-api image for configuration as well  https://review.openstack.org/49038514:40
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Enable TLS for nova api and placement containers  https://review.openstack.org/48988514:41
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Make containerized nova-api run with httpd  https://review.openstack.org/47536614:41
jaosoriormandre: what about that? ^^14:41
*** pdeore has quit IRC14:41
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Change the directory for haproxy certs/keys to be service-specific  https://review.openstack.org/48989614:43
*** agurenko has quit IRC14:44
*** pchavva has joined #tripleo14:45
jaosoriormwhahaha: hey, the pinning of stdlib landed. But there still appear to be some issues :/ http://logs.openstack.org/34/489534/5/check/gate-puppet-pacemaker-puppet-unit-4.5-centos-7/3cdd921/console.html14:45
mwhahahajaosorior: i'll take a look at it today14:46
openstackgerritMerged openstack/python-tripleoclient master: Minor fixes for container image prepare command  https://review.openstack.org/48406114:46
openstackgerritMerged openstack/tripleo-heat-templates stable/newton: Ensure yum cache is ready before update  https://review.openstack.org/49034714:46
*** jfrancoa has joined #tripleo14:46
*** MVenesio has joined #tripleo14:47
hewbroccaadarazs: ?14:48
*** slagle has quit IRC14:48
adarazshewbrocca: still deploying the overcloud :)14:49
adarazshewbrocca: I think the typical job run is around 210 minutes, so it might take another ~40-60 minutes for it to finish.14:49
MVenesioHi guys, i'm trying to map the networks and ips for my ocata deploy using already deployed servers, so i've created the deployed-ctrl-plane but when i use something like compute01-tenant: to fix a tenant ip for a specific compute, it doesn't work14:49
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: MistralApiService, SwiftApiService error handling  https://review.openstack.org/48438214:49
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: IronicInspectorApiService error handling  https://review.openstack.org/48697714:49
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: IronicApiService error handling  https://review.openstack.org/48697614:49
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: KeystoneApiService error handling  https://review.openstack.org/48697914:49
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: HeatApiService error handling  https://review.openstack.org/48697814:49
*** slagle has joined #tripleo14:50
hewbroccaadarazs: OK if it's still rolling that is all good news14:51
hewbroccamburns said something about the upload failing?14:51
adarazshewbrocca: that was me blocking the promotion on RDO CI phase1 on that not-really-promoted hash.14:52
mwhahahajaosorior: oh i know, puppet-pacemaker doesn't use p-o-i pins. let me propose a pin just for puppet-pacemaker14:53
adarazshewbrocca: it looked good, so we should expect the run that I will start on RDO CI to succeed with this hash.14:53
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient stable/pike: Install policycoreutils-python  https://review.openstack.org/49050414:53
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient stable/pike: Update .gitreview for stable/pike  https://review.openstack.org/48788614:54
jaosoriormwhahaha: it doesn't? where is that done usually? in project-config?14:54
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient stable/pike: Update .gitreview for stable/pike  https://review.openstack.org/48788614:54
mwhahahajaosorior: .fixtures.yaml in the repo (classic puppet module style)14:54
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient stable/pike: Update UPPER_CONSTRAINTS_FILE for stable/pike  https://review.openstack.org/48788714:54
mwhahahajaosorior: sec i've got it14:54
openstackgerritAlex Schultz proposed openstack/puppet-pacemaker master: Pin puppet-stdlib to 4.16.0  https://review.openstack.org/49050714:57
EmilienMmwhahaha: shouldn't we rather change puppet-pacemaker to use p-o-i stuffs to clone modules?14:58
EmilienMimho we should remove fixtures14:58
mwhahahanot really14:58
mwhahahacause it just needs stdlib14:58
mwhahahait's way overkill for it14:58
EmilienMmhh ok14:58
amoralejadarazs, a ovb job has passed overcloud deploy phase with heat patch :)15:00
*** gbarros has joined #tripleo15:01
hewbroccaamoralej: woowoo15:02
openstackgerritDerek Higgins proposed openstack/tripleo-heat-templates master: Add IronicPxe to the default controller  https://review.openstack.org/49051215:04
catintheroofguys, hi.im having a conceptual issue, let's suppose that i dont know how to configure the my_ip key on neutron's ml2_conf.ini, i change some things and want to re-run the deploy updating the stack, what i see is that updating the stack does nothing updating new values, just seems to do things like deploying new things, is updating the stack on the undercloud what i need to do to try new things ? do i need to delete the stack en re15:06
catintheroof-run it ? or im i missing something conceptually ?15:06
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Fix plan export URL  https://review.openstack.org/48816415:08
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/170814615:10
openstackLaunchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged]15:10
*** ooolpbot has quit IRC15:10
*** gbarros has quit IRC15:12
*** paken has quit IRC15:12
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates stable/ocata: Remove docker references from stable/ocata scenarios  https://review.openstack.org/48987415:15
*** cshastri has quit IRC15:17
*** jlinkes has quit IRC15:17
*** yprokule has quit IRC15:20
openstackgerritAlex Schultz proposed openstack/puppet-pacemaker master: Switch requires for wait-for-settle into resource collectors  https://review.openstack.org/48953415:21
*** panda has quit IRC15:23
*** dmacpher has quit IRC15:24
*** lmiccini has quit IRC15:24
jaosoriormwhahaha: thanks for rebasing15:26
*** chem has quit IRC15:26
mwhahahawe'll get that merged (eventually)15:26
mwhahaha:D15:26
*** chem has joined #tripleo15:27
*** dprince has quit IRC15:27
*** mkovacik__ is now known as milan15:28
*** nyechiel has joined #tripleo15:29
EmilienMbnemec: when you have time https://review.openstack.org/#/c/490077/15:29
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: Containerize virtlogd  https://review.openstack.org/46911615:32
*** hewbrocca is now known as hewbrocca_afk15:34
*** katkapilatova has left #tripleo15:34
*** tzumainn has quit IRC15:34
*** ckyriakidou has quit IRC15:37
adarazsbnemec: I used https://review.openstack.org/359215 to start a manual promotion job. now I think everything succeeded but I don't see https://ci.centos.org/view/rdo/view/promotion-pipeline/job/tripleo-dlrn-promote/ triggered by anything.15:39
adarazsbnemec: do you know how that should happen?15:39
*** dprince has joined #tripleo15:40
*** jfrancoa has quit IRC15:41
adarazsanybody else knows?15:43
amoralejadarazs, not all jobs used the same hash :(15:45
amoraleji think that's why15:45
adarazsnooooo15:45
adarazsargh :(15:45
amoralejlook at http://logs.openstack.org/15/359215/122/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/4cbfa33/console.html15:45
amoralejd0be7d8b26763196e0d5c35c7255339c7ab9ea4c_31fde87115:46
amoralejwhile others have15:46
EmilienMadarazs, amoralej: do we want to promote manually?15:46
amoralejc80011cd91cbbd71c66064dbfdd69fed0fb6d705_65fed6215:46
amoralejEmilienM, +215:46
amoraleji can promote c80011cd91cbbd71c66064dbfdd69fed0fb6d705_65fed6215:46
adarazsEmilienM: I think so. I feel quite confident in this c80011cd91cbbd71c66064dbfdd69fed0fb6d705_65fed62 hash15:46
amoralejthat has the patch from heat15:46
EmilienMok15:46
amoralejand ovb passed15:46
*** tzumainn has joined #tripleo15:47
*** nyechiel has quit IRC15:47
*** ccamacho has quit IRC15:47
adarazsamoralej: let me know when you did the promote.15:47
adarazsI want to start the rdo job15:47
amoralejadarazs, EmilienM https://ci.centos.org/view/rdo/view/promotion-pipeline/job/tripleo-dlrn-promote/197/15:47
EmilienMadarazs, amoralej : done15:47
adarazsthanks15:47
amoralejadarazs, did you re-enable build containers job?15:48
amoraleji think it was disabled15:48
amoraleji just enabled15:49
adarazsI didn't disable or enable it :)15:49
amoralejsomeone disabled, dunno who, but i've enabled, anyway15:50
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Add consistent-pike.yml  https://review.openstack.org/49057415:50
EmilienMadarazs, amoralej ^ need your review15:50
amoralejEmilienM, https://review.openstack.org/#/c/490472/15:50
*** anshulbehl has quit IRC15:51
EmilienMamoralej: +2, I'll let adarazs +A15:51
adarazslooking.15:51
jistrfultonj: whohooo \o/15:51
jistr192.168.24.16              : ok=32   changed=6    unreachable=0    failed=115:51
jistrthat is to say, it still failed http://logs.openstack.org/88/479288/20/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/667c903/logs/undercloud/var/log/mistral/executor.log.txt.gz15:52
amoralejadarazs, promotion is done15:52
jistrfultonj: but we've moved past the point of overcloud being unreachable15:52
fultonjjistr++15:52
*** ckyriakidou has joined #tripleo15:52
d0ugaljistr: Lots of red in that file15:53
adarazsamoralej: cool, I see. I started https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo_trunk-promote-master-current-tripleo/180/15:53
adarazsokay, it got the right hash.15:53
* adarazs goes to play some squash and later checks on the job.15:53
d0ugalfultonj, jistr: is there any way to have ansible return structured data that we can format?15:54
fultonjjistr: yes, looks like the playbook itself is failing so that means i have something to debug. thank you!15:54
*** mcornea has quit IRC15:55
jistrfultonj: sure thing :) I gotta run, ttyt15:55
*** udesale has quit IRC15:55
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Add logrotate with crond service  https://review.openstack.org/49005015:55
jistrd0ugal: i'm not sure about that but we may follow up on that issue15:56
*** thrah|biab is now known as thrash16:00
*** psachin has quit IRC16:01
bogdandoPTAL https://review.openstack.org/#/q/topic:bug/1700912 . Works for my local env16:01
*** dbecker has quit IRC16:01
EmilienMbogdando: do we have tests coverage in CI?16:02
*** dbecker has joined #tripleo16:02
openstackgerritMichael Bayer proposed openstack/puppet-tripleo master: Enable innodb_buffer_pool_size configuration  https://review.openstack.org/49004616:03
*** egonzalez has quit IRC16:03
openstackgerritMichael Bayer proposed openstack/puppet-tripleo stable/ocata: Enable innodb_buffer_pool_size configuration  https://review.openstack.org/49057716:03
EmilienMfultonj: do you have a moment to debug scenario001 failing to upload an image to glance, when deploying with containers16:04
catintheroofguys, hi.im having a conceptual issue, let's suppose that i dont know how to configure the my_ip key on neutron's ml2_conf.ini, i change some things and want to re-run the deploy updating the stack, what i see is that updating the stack does nothing updating new values, just seems to do things like deploying new things, is updating the stack on the undercloud what i need to do to try new things ? do i need to delete the stack en re16:04
catintheroof-run it ? or im i missing something conceptually ?16:04
fultonjEmilienM: yes16:04
EmilienMfultonj: so glance is containerized but not ceph16:04
EmilienMfultonj: http://logs.openstack.org/10/489710/2/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/fff73d0/logs/undercloud/home/jenkins/overcloud_validate.log.txt.gz#_2017-08-02_20_11_5816:04
EmilienMfultonj: I don't see any trace in glance api http://logs.openstack.org/10/489710/2/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/fff73d0/logs/subnode-2/var/log/containers/glance/api.log.txt.gz16:05
EmilienMfultonj: sounds like http://logs.openstack.org/10/489710/2/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/fff73d0/logs/subnode-2/var/log/containers/glance/api.log.txt.gz#_2017-08-02_20_09_58_89716:05
EmilienMfultonj: I've seen that before16:05
bogdandoEmilienM: I've modified https://review.openstack.org/#/c/490050/8/ci/environments/multinode-containers.yaml I hope that works as I expect16:05
fultonjEmilienM: is this job using split stack?16:05
EmilienMfultonj: ceph logs are here: http://logs.openstack.org/10/489710/2/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/fff73d0/logs/subnode-2/var/log/ceph/16:06
EmilienMfultonj: yes16:06
EmilienMfultonj: but not ceph-ansible16:06
*** jchhatbar has joined #tripleo16:06
openstackgerritMichael Bayer proposed openstack/puppet-tripleo stable/newton: Enable innodb_buffer_pool_size configuration  https://review.openstack.org/49057816:07
slagleyea, there's some misinformation being spread about ceph and split-stack, that it doesn't work16:07
*** Vijayendra has quit IRC16:07
slaglewe've been deploying ceph with splitstack for a while, and it works16:07
slagleit's the new ceph-ansible changes that don't work16:07
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: Containerize virtlogd  https://review.openstack.org/46911616:07
EmilienMslagle: I'm testing the puppet ceph now16:08
fultonjslagle: true, they need a user for ansible to run as e.g. https://review.openstack.org/#/c/49047016:08
EmilienMslagle: context: https://review.openstack.org/#/c/48971016:08
*** marios has quit IRC16:08
EmilienMfultonj, slagle: I just enabled pingtest on scenario001-containers and it fail16:08
catintherooffultonj: EmilienM slagle, any help ?16:09
EmilienMlet's not talk about ceph-ansible now, this is out of context of my problem16:09
EmilienMfultonj: any idea why glance isn't available when deploying in containers (again except ceph)16:09
slagleEmilienM: i'm not talking about your problem16:10
EmilienMcatintheroof: as you can see we're debugging something at this time16:10
EmilienMslagle: :D thanks!16:10
slagleEmilienM: no problem16:10
*** apetrich has joined #tripleo16:10
fultonjEmilienM: i suspect it's because the ceph client configuration for glance, as a container, didn't happen. it should have as per https://review.openstack.org/#/c/48250016:11
fultonjor didn't complete with success16:11
* fultonj will try to figure out why from those logs16:11
catintheroofEmilienM: ok, please let me know when you can have a min16:11
*** rcernin has quit IRC16:12
EmilienMfultonj: thanks for the help - I'm debugging scenario003 and 004 now16:12
EmilienM(they also have failures in pingtest)16:12
*** rlandy|ci_mtgs has quit IRC16:12
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Upgrade testing  https://review.openstack.org/48971016:13
EmilienMfultonj: in the meantime, I'm trying again with the patches that merged recently ^16:13
fultonjack16:14
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Test containers scenarios with pingtest - do not merge  https://review.openstack.org/48971016:14
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI testing: run tempest in container jobs  https://review.openstack.org/49017216:15
openstackgerritBogdan Dobrelya proposed openstack/puppet-tripleo master: Add logrotate-crond configuration  https://review.openstack.org/49004816:17
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Test containers scenario001 with tempest - do not merge  https://review.openstack.org/49017216:18
akrivokajtomasek: you mentioned some patch regarding the introspection workflow, could you link me to it?16:19
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Support for Dell EMC Unity Manila Driver  https://review.openstack.org/49058116:20
jtomasekakrivoka: sorry, I forgot. it is already merged, check 'introspect' workflow in tripleo-common16:21
akrivokajtomasek: ack, thanks!16:21
*** snecklifter has quit IRC16:21
jtomasekakrivoka: it calls _introspect workflow for each node separately and sends a message16:21
*** nyechiel has joined #tripleo16:23
fultonjEmilienM: the container host has a key and ceph.conf like it should http://logs.openstack.org/10/489710/2/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/fff73d0/logs/subnode-2/etc/ceph/16:23
fultonjmy next question is if it made it into the container16:24
EmilienMfultonj: it must be something else, i'm trying to reproduce now16:24
EmilienMfultonj: how is going the work with ceph-ansible now?16:24
EmilienMfultonj: what are blockers?16:24
*** ccamacho has joined #tripleo16:24
fultonjEmilienM: jistr just got the playbook running and it failed and i don't know why yet16:24
*** jtomasek has quit IRC16:25
fultonjit was blocked on the tripleo-admin user, but he got me past that ~20 minutes ago16:25
EmilienMfultonj: ok16:25
*** ccamacho has quit IRC16:26
EmilienMcatintheroof: puppet will override the value that it has in catalog16:26
EmilienMcatintheroof: that's the whole purpose of using tools like puppet or ansible: they ensure resources exist16:26
catintheroofEmilienM: what i mean is, doing a stack upgrade is the way to go right ?16:28
catintheroofEmilienM: if i want to try if something i changed will change the config files ?16:28
openstackgerritJames Slagle proposed openstack/tripleo-common stable/newton: Add skip_deploy_identifier  https://review.openstack.org/48981316:31
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Enable pingtest on scenarios container jobs  https://review.openstack.org/49012916:31
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Test containers scenarios with pingtest - do not merge!  https://review.openstack.org/48971016:31
EmilienMcatintheroof: if you run stack update, puppet will run again and your custom parameters will be overidden16:33
*** Goneri has quit IRC16:33
EmilienMcatintheroof: does it help?16:33
openstackgerritBogdan Dobrelya proposed openstack/puppet-tripleo master: Add logrotate-crond configuration  https://review.openstack.org/49004816:33
*** dsariel has quit IRC16:34
openstackgerritIan Main proposed openstack/tripleo-docs master: Fix some grammar and add a few notes to docs.  https://review.openstack.org/49058516:35
*** nyechiel has quit IRC16:36
*** gkadam has joined #tripleo16:37
*** lucasagomes is now known as lucas-afk16:42
fultonjhow do you debug a container failure by looking at the logs of the subnode from multinode-oooq-container ? any output from http://tripleo.org/containers_deployment/tips_tricks.html#debugging-container-failures or similar available ?16:43
fultonjEmilienM: ^16:43
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Persist containerized services httpd logs  https://review.openstack.org/48995516:44
openstackgerritJames Slagle proposed openstack/python-tripleoclient stable/newton: Add --skip-deploy-identifier  https://review.openstack.org/48981216:44
openstackgerritJohn Eckersberg proposed openstack/tripleo-quickstart-extras master: Use shell module when deploying freeipa  https://review.openstack.org/49059016:45
EmilienMfultonj: in a mtg, bbl16:47
fultonjack16:47
catintheroofEmilienM: yes, it does, but i dont see that happen16:48
openstackgerritIan Main proposed openstack/tripleo-heat-templates master: Make our json output prettier.  https://review.openstack.org/49059116:48
catintheroofEmilienM: i've created the deployed-ctrl-plane but when i use something like compute01-tenant: to fix a tenant ip for a specific compute, it doesn't work when i run the stack update16:49
fultonjhe's in a meeting16:49
bogdandoPTAL this one also please https://review.openstack.org/#/c/489955/16:50
catintherooffultonj: oka! can you help ? or is it better to wait for him ?16:50
fultonjcatintheroof: network settings are created but not updated16:51
fultonjyes, 'openstack overcloud update' will re-run puppet (or similar idempotent config tools)16:51
fultonjbut network settings are exlucded to prevent errors from knocking you off line (that's my understanding of it)16:52
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Add logrotate with crond service  https://review.openstack.org/49005016:52
catintherooffultonj: so, if something was not set right from the deployment stage, and you want to "update" it, how can i do that the right way ?16:53
catintherooffultonj: by hand ?16:53
catintherooffultonj: server by server ?16:53
fultonjdepends on what somehting is16:53
catintherooffultonj: eg. my_ip value on ml2_conf.ini on every compute node16:54
catintherooffultonj: used to close the vxlan tunnels between network nodes and computes16:54
*** pcaruana has quit IRC16:54
catintherooffultonj: sorry, local_ip16:55
fultonjthere may be a way to do it via update, but if the box is experimental, redeployment would be easier16:55
fultonji know the basics of tripleo network config for my purposes, but it's not my area16:56
*** stee_3_ has joined #tripleo16:56
catintherooffultonj: is not experimental, what would be the way to do that via update16:56
*** Goneri has joined #tripleo16:56
fultonjsince you know tripleo won't update your network (badly) config outside of tripleo shouldn't get broken by updates16:57
*** bugzy_ has joined #tripleo16:57
*** dtantsur is now known as dtantsur|afk16:57
fultonjby badly i mean, in the case of user erro or by differeing the config tools definition to undo your "outside of tripleo change"16:58
*** derekh has quit IRC16:58
catintherooffultonj: so i should touch it by hand, ok16:58
fultonjerrr16:58
fultonjyou could16:58
fultonji don't want to misrepresent what better advice others may have16:58
catintherooffultonj: last one, for "update" i run this command : openstack overcloud deploy (and the same parameters as the first time)16:59
*** bogdando has quit IRC16:59
fultonjyes16:59
catintherooffultonj: is that the same as openstack overcloud update ?16:59
fultonji don't use 'openstack overcloud update'17:00
fultonji use deploy again and it 'makes it so'17:00
*** stee_3 has quit IRC17:00
*** bugzy has quit IRC17:00
fultonjif i were to make a change like scale a node for example17:00
*** bugzy_ has quit IRC17:00
*** tosky has quit IRC17:01
*** bugzy has joined #tripleo17:02
fultonj'openstack overcloud update' is for updating versions of the packages17:02
fultonjsee https://docs.openstack.org/tripleo-docs/latest/install/post_deployment/package_update.html17:02
*** rcernin has joined #tripleo17:03
*** nyechiel has joined #tripleo17:03
catintherooffultonj: we are using pre-deployed servers, if i configure on deployed-ctrl-plane eg compute01-tenant would that set the right loca_ip on ml2_conf.ini for the tunnel/tenant networks ?17:05
fultonji do not know anything about network configuration with pre-deployed servers17:05
*** ebarrera has quit IRC17:05
catintherooffultonj: no problem, thanks !17:06
fultonji don't know if it leaves their network settings unchanged or if it updates them.17:06
*** ckyriakidou has quit IRC17:07
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Add OPNFV scenario environment  https://review.openstack.org/48690517:12
EmilienMmichapma: ^ we got promotion17:12
fultonjhow do you debug a container failure by looking at the logs of the subnode from multinode-oooq-container ? any output from http://tripleo.org/containers_deployment/tips_tricks.html#debugging-container-failures or similar available ?17:16
fultonjEmilienM: ^ ?17:17
fultonje.g. for https://review.openstack.org/#/c/489710 i want to see in http://logs.openstack.org/10/489710/2/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/fff73d0/ if https://review.openstack.org/#/c/482500/21/docker/services/glance-api.yaml happened17:18
EmilienMfultonj: I'm not sure we export /etc from containers17:20
EmilienMlet me look17:20
*** atoth has quit IRC17:20
EmilienMwe do in http://logs.openstack.org/10/489710/2/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/fff73d0/logs/subnode-2/var/log/config-data/17:21
EmilienMfultonj: http://logs.openstack.org/10/489710/2/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/fff73d0/logs/subnode-2/var/log/config-data/glance_api/etc/glance/17:21
* fultonj looks17:21
*** atoth has joined #tripleo17:22
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Deploy Mistral with Keystone v3 options (authtoken)  https://review.openstack.org/46104017:23
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Deploy Mistral with Keystone v3 options (authtoken)  https://review.openstack.org/46104017:23
EmilienMthrash: ^ let's see how it works now17:23
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates stable/ocata: Remove docker references from stable/ocata scenarios.  https://review.openstack.org/48987417:24
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient master: Update reno for stable/pike  https://review.openstack.org/48788817:24
*** jpena is now known as jpena|off17:25
*** yamahata has joined #tripleo17:25
michapmaEmilienM: ok thanks.17:28
*** milan has quit IRC17:33
openstackgerritVictoria Martinez de la Cruz proposed openstack/tripleo-heat-templates master: Containerize Manila Share for HA  https://review.openstack.org/48268017:36
*** gkadam has quit IRC17:39
*** gkadam has joined #tripleo17:39
*** paramite has quit IRC17:41
*** ebarrera has joined #tripleo17:48
*** tesseract has quit IRC17:49
*** jchhatbar has quit IRC17:50
*** tosky has joined #tripleo17:50
thrashEmilienM: ack17:53
thrashEmilienM: it occurred to me that poi should be updated as well... But I'm having some issues with it...17:54
thrashEmilienM: https://review.openstack.org/#/c/490494/17:54
EmilienMthrash: ok I'll look17:55
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert controller-role.yaml to role.role.j2.yaml  https://review.openstack.org/48027217:56
openstackgerritMerged openstack/tripleo-heat-templates master: Update capabilities map to match latest environments  https://review.openstack.org/48387317:59
openstackgerritMerged openstack/python-tripleoclient master: Change tag link in README to the newest address  https://review.openstack.org/49033717:59
*** amoralej is now known as amoralej|off18:04
openstackgerritNuman Siddique proposed openstack/tripleo-heat-templates master: DO NOT REVIEW: TESTING ONLY TO TEST OVN CONTAINER SERVICES  https://review.openstack.org/48909718:12
*** nyechiel has quit IRC18:16
*** akrivoka has quit IRC18:18
EmilienMnumans: instead of re-using an existing job for testing OVN in containers, you should rather push for scenario007-containers, isn't?18:23
numansEmilienM, ack. I just noticed that neutron-server-ovn image is available in tripleoupstream docker hub(which I thought isn't available until now). So I will work on the patches for scenario007-containers. Thanks to dprince for pushing the image18:24
EmilienMnumans: cool - you also need to patch project-config to add the experimental job probably18:25
numansEmilienM, right. I worked on it some time back. I need to push the patches :)18:25
EmilienMcool18:25
EmilienMnumans: let me know if any help18:25
numansEmilienM, sure. thanks18:25
*** gkadam has quit IRC18:29
*** salmankhan has quit IRC18:31
catintheroofEmilienM: hi! quick question, we have a networking nooped on a pre-deployed servers overcloud, the value of local_ip on the ml2_conf.ini only is getting completed with the ctlplane ip of each node, can you please advise me where to complete the servers ips to be completed the right way for the tenant networks ?18:31
openstackgerritJustin Kilpatrick proposed openstack/tripleo-quickstart-extras master: WIP: Support for TripleO centralized logging  https://review.openstack.org/49062318:43
*** MVenesio has quit IRC18:47
openstackgerritNuman Siddique proposed openstack/tripleo-quickstart master: Add fset030 for OVN with containerized oc  https://review.openstack.org/49062418:49
*** prateek has quit IRC18:51
*** nyechiel has joined #tripleo18:52
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: WIP - use infra mirror for buildlogs repo  https://review.openstack.org/49062918:57
EmilienMpabelanger: ^^doing tripleo18:57
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: WIP - use infra mirror for buildlogs repo  https://review.openstack.org/49062919:03
EmilienMthrash: http://logs.openstack.org/94/490494/4/check/gate-puppet-openstack-integration-4-scenario003-tempest-centos-7/61ef58b/logs/puppet.txt.gz#_2017-08-03_16_39_3319:04
EmilienMPlease set password for mistral service user19:04
thrashEmilienM: that's what I don't get... It is set...19:04
EmilienMweird19:04
EmilienMyeah let me loolk19:04
EmilienMthrash: problem in the logic :)19:06
EmilienMthrash: I think it should help https://review.openstack.org/49063219:08
*** rhallisey has quit IRC19:10
thrashEmilienM: I swear that looked correct to me. haha19:10
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert controller-role.yaml to role.role.j2.yaml  https://review.openstack.org/48027219:10
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert compute-role.yaml to role.role.j2.yaml  https://review.openstack.org/48262319:11
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert controller-role.yaml to role.role.j2.yaml  https://review.openstack.org/48027219:11
EmilienMthrash: we'll see19:11
dsneddonI need some help. I have two patches, A and B. A's parent branch is Master, B's parent branch is A. Whenever I update patch B, it reverts patch A to a much older version. At this point, both A and B have the same version of role.role.j2.yaml, so why does committing changes to B overwrite A with older content?19:14
dsneddonA: https://review.openstack.org/48027219:14
*** jcoufal has quit IRC19:15
dsneddonB: https://review.openstack.org/48262319:15
dsneddonI just made these changes to A: https://review.openstack.org/#/c/480272/23..24/puppet/role.role.j2.yaml19:15
dsneddonCopied the same version of the file (role.role.j2.yaml) into change B, did a git review, and this happened to change A: https://review.openstack.org/#/c/480272/23..24/puppet/role.role.j2.yaml19:16
dsneddonEmilienM, thrash, mwhahaha: Any ideas? ^^^19:18
*** pchavva has quit IRC19:18
vkmcdciabrin_,  I'm trying to deploy containerized manila with netapp as the backend and I'm hitting this http://paste.openstack.org/show/617467/19:20
openstackgerritSwaminathan Vasudevan proposed openstack/diskimage-builder master: Failed to open parameter YAML error while trying to unmount imagedir  https://review.openstack.org/49063719:20
vkmcdciabrin_, is this because this one is not yet merged? https://review.openstack.org/#/c/482680/19:20
EmilienMdsneddon: first, you need to rebase on top of master19:21
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Convert controller-role.yaml to role.role.j2.yaml  https://review.openstack.org/48027219:21
EmilienMdone ^19:21
EmilienMthen let's the the other one19:21
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Convert compute-role.yaml to role.role.j2.yaml  https://review.openstack.org/48262319:21
EmilienMdsneddon: can you tell if it's ok?19:22
EmilienMdsneddon: i'm not familiar with the code but please say if it's good now19:23
dsneddonEmilienM, That didn't make a difference. Patch A still has an old version of role.role.j2.yaml.19:23
EmilienMok brb lunch and I'll look closer19:24
dsneddonEmilienM, Thanks19:24
*** fzdarsky has joined #tripleo19:24
mwhahahamight be how we clone stuff19:25
mwhahahai have a feeling we don't handle dependencies well19:25
* mwhahaha starts looking19:25
bnemecAre both patches on the same local branch?19:28
dsneddonbnemec, No, A is a branch off of master, B is a branch off of A19:28
dsneddonbnemec, At least I think so, based on the "Parent(s)" branch (Steven Hardy did these changes originally)19:29
dsneddonWhat I'm trying to do is end up with Patch A having all the files from patchset 23, but with a new role.role.j2.yaml that is the same as patch B patchset 2119:29
mwhahahais bad in CI or are we just confusing gerrit19:29
dsneddonmwhahaha, Yeah, I suspect that Gerrit is confused. This is why I have started doing depends-on between changes based on Master, rather than sub-branches. I've run into this before.19:30
bnemecdsneddon: If the patches are being pushed from two different local branches that may well be the problem.19:30
bnemecWhen working with patch series you need to keep all the changes in one branch or it gets confusing.19:31
dsneddonbnemec, I guess I just don't know how to do that with a series of patches that were posted by someone else.19:33
mwhahahavery painfully19:33
mwhahahawe should merge these and stop adding more :D19:33
bnemecIf you have branch foo with patch A, then branch bar with patches A and B, you make changes to A on foo, then changes to B on bar, push foo and bar (in that order), you'll end up with A not having the changes you made because the bar version will overwrite it.19:33
openstackgerritBob Fournier proposed openstack/tripleo-heat-templates master: Render VIPs dynamically based on network_data.yaml  https://review.openstack.org/48626019:33
*** nyechiel has quit IRC19:34
dsneddonbnemec, OK, I think I'm getting it. So any changes to A (like those releasenotes I added) need to be duplicated into B, or it will overwrite?19:34
bnemecdsneddon: Go to the last patch in the series, click the Download menu and run the Checkout command.19:34
bnemecdsneddon: Yeah, I _suspect_ that's the problem here.19:35
dsneddonbnemec, OK, so submit the changes in reverse order? That's worth a try, thanks.19:35
mwhahahadsneddon: well you should rebase them all together and only git review the last one of the change19:35
openstackgerritJames Slagle proposed openstack/python-tripleoclient stable/newton: Add --skip-deploy-identifier  https://review.openstack.org/48981219:35
bnemecYeah, don't do multiple git reviews.19:35
dsneddonmwhahaha, OK, and that will trigger updating all the other patches, that makes sense.19:36
bnemecOnly the last one counts anyway.19:36
mwhahahayou can fix it by pulling down the first then just cherrypicking them all on top of eachother19:36
dsneddonbnemec, Of course, this becomes difficult when you find that one patch at the beginning of the series needs a one-line change.19:36
bnemecdsneddon: git rebase -i HEAD~6 (or however many commits are in the series).19:37
mwhahahayea that's why when we have these we need to merge them soon and not just let them sit because it's a giant pain (and time suck) to manage this over time19:37
bnemecIt sucks because it forces a re-run of ci on everything, but actually making the change shouldn't be that difficult.19:38
*** artom_ has quit IRC19:38
*** artom_ has joined #tripleo19:38
dsneddonbnemec, mwhahaha: Yeah, I'm mostly complaining because this means that I can't split the workload (me editing patches 1-3, Bob editing paches 4-7)19:38
dsneddonBut it's not the end of the world. Thanks for your help!19:39
mwhahahawell you can19:39
bnemecAh, yeah.   That's one thing gerrit is terrible at.19:39
mwhahahabut it's just complicated19:39
dsneddonAlright, I'm just going to bang out all these changes19:39
mwhahahait's harder for the 2nd guy who has to constantly rebase on your patches19:39
dsneddonmwhahaha, That's the situation I'm in while shardy is out on PTO. ;-)19:41
mwhahaha:D19:41
mwhahahalet's figure out controller and get that merged at least19:41
mwhahahathat seems to be the sticking point19:41
bnemecSolution: Don't let shardy go on PTO anymore. :-)19:41
mwhahahai thought that was a given19:41
dsneddonmwhahaha, That's what I'm trying to do, but then when I thought I was close to getting controller done, I did the next patch in the series, and it wiped out the controller changes.19:41
mwhahahadoh19:42
*** jmelvin has quit IRC19:42
mwhahahawell you can pull down the old one that was close and redo tha tone19:42
dsneddonmwhahaha, That's my next step, I want to see if Controller passes CI before I clone the changes to the rest of the branches19:42
* bnemec is concerned that there are still multiple branches in play here19:44
bnemecDo these patches actually need to be in a series?19:44
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert controller-role.yaml to role.role.j2.yaml  https://review.openstack.org/48027219:45
mwhahahayea it's a rewrite of roles.role.j2.yaml19:45
mwhahahathat's the sticking point19:45
mwhahahato handle the inconsistencies we have with those19:45
mwhahahaso there's a bunch of controller/compute/etc specific stuff that we still need to handle19:45
bnemecToo bad.19:46
mwhahahathis is why consistency is important and we shouldn't one off things :D19:46
dsneddonWe could do it as a single patch, but it wouldn't be clear which changes to role.role.j2.yaml were necessary for each role.19:46
mwhahahaand it's be so hard to review19:46
bnemecThat^19:46
dsneddonmwhahaha, I couldn't agree more. If we had been consistent between Compute and NovaCompute in the first place, SO MANY headaches could have been avoided.19:47
* mwhahaha is currently trying to figure out why docker deployments keep coming up no host found :/19:47
bnemecHeh, that downstream-only change continues to bite us in the ass.19:47
bnemecWhat, two and a half years later?19:47
mwhahahadat techdebt19:48
* EmilienM back19:52
*** ebarrera_ has joined #tripleo19:56
EmilienMthrash: I think I was wrong19:58
*** fzdarsky has quit IRC19:58
mwhahahashould we quote you on that one19:58
* mwhahaha documents this time for history19:58
EmilienMlol19:58
*** milan has joined #tripleo20:00
EmilienMthrash: i'm trying some code ordering20:00
mwhahahaanyone know anything about the pre deployment validations (and why it would think i don't have any nodes available when i do or where that stuff even lives)20:02
fultonjmwhahaha: i know that if you use deployed server you can't use validations20:03
mwhahahathis is an ovb setup20:03
mwhahahait's erroring say i have 0 nodes but it's saying 5 are available in the json response20:04
mwhahahaso i have no idea wtf it's doing20:04
bnemecLooks like it's a mistral workflow now: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/workflows/validations.py#L2020:04
mwhahahayea20:05
mwhahahai'm trying to figure out where this stuff lives :/20:05
*** sbrzozow has quit IRC20:05
mwhahahai found the workbook at least20:05
* mwhahaha keeps digging20:05
*** salmankhan has joined #tripleo20:05
bnemecTripleO is like an onion, the more layers you peel away the more you cry.20:06
fultonjlol20:06
mwhahahaalso true20:06
mwhahahait's a work flow to run /usr/bin/run-validation20:06
mwhahahahttps://github.com/openstack/tripleo-common/blob/master/tripleo_common/utils/validations.py#L8020:06
* bnemec thinks that needs to be a t-shirt20:06
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: DNM - run tempest in scenario001-container  https://review.openstack.org/49016720:07
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Test containers scenario001 with tempest - do not merge  https://review.openstack.org/49017220:07
EmilienMbnemec: nice one20:07
mwhahahasoooo the question is what provides run-validation20:08
*** liverpooler has quit IRC20:08
mwhahahatripleo-validations20:08
mwhahahathat's a new one20:08
mwhahahahrm can't find a reference to this error there20:10
openstackgerritNuman Siddique proposed openstack/tripleo-common master: Add OVN images to the overcloud containers  https://review.openstack.org/48987520:10
* mwhahaha sighs20:10
*** salmankhan has quit IRC20:10
mwhahahaah https://github.com/openstack/tripleo-common/blob/3a9301739650b511774fc9f6a5cde9ace243715e/tripleo_common/actions/validations.py#L42420:10
*** jmelvin has joined #tripleo20:11
openstackgerritNuman Siddique proposed openstack/tripleo-common master: Add OVN images to the overcloud containers  https://review.openstack.org/48987520:13
*** eck` is now known as eck`gone20:14
*** mmethot_ has joined #tripleo20:14
*** mmethot has quit IRC20:17
*** morazi has quit IRC20:17
mwhahahawhy do we keep changing the node import command20:26
* mwhahaha throws things20:26
*** morazi has joined #tripleo20:26
*** dhill_ has joined #tripleo20:28
*** nyechiel has joined #tripleo20:31
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Support for Dell EMC VNX Manila Driver  https://review.openstack.org/49064220:36
*** nyechiel has quit IRC20:36
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Support for Dell EMC VNX Manila Driver  https://review.openstack.org/49064220:37
*** ansmith has quit IRC20:37
*** hamzy has quit IRC20:37
stevebakermorning20:42
*** ebarrera has quit IRC20:43
EmilienMafternoon20:46
EmilienMstevebaker: nice fix for oooclient, it works, we're backporting it to stable/ocata now20:46
EmilienMerr, stable/pike20:46
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient stable/ocata: Install policycoreutils-python  https://review.openstack.org/49064520:47
EmilienMocata now20:47
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient stable/newton: Install policycoreutils-python  https://review.openstack.org/49064620:47
EmilienMeven newton20:47
openstackgerritMerged openstack/tripleo-common master: [refactoring] stop using pxe_ssh as the default drivers in tests  https://review.openstack.org/48716620:48
*** eck`gone is now known as eck`20:48
*** jkilpatr has quit IRC20:48
EmilienMtherve: sounds like zaqar fails to work when containerized20:49
EmilienMtherve: http://logs.openstack.org/10/489710/5/check/gate-tripleo-ci-centos-7-scenario002-multinode-oooq-container/019039b/logs/undercloud/home/jenkins/overcloud_validate.log.txt.gz#_2017-08-03_18_02_1920:50
*** rhallisey has joined #tripleo20:51
EmilienMdprince: is zaqar suppose to work in containers?20:52
openstackgerritSteve Baker proposed openstack/python-tripleoclient master: Add --set argument to prepare command  https://review.openstack.org/48908320:52
EmilienMdprince: pingtest doesn't seem to pass20:52
EmilienMand no logs:20:52
EmilienMhttp://logs.openstack.org/10/489710/5/check/gate-tripleo-ci-centos-7-scenario002-multinode-oooq-container/019039b/logs/subnode-2/var/log/containers/20:52
EmilienMprobably because we need https://review.openstack.org/#/c/490460/1 for logs20:53
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Test containers scenarios with pingtest - do not merge!  https://review.openstack.org/48971020:53
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Test containers scenarios with pingtest - do not merge!  https://review.openstack.org/48971020:53
EmilienMlet's try again with logs20:53
*** nyechiel has joined #tripleo20:55
*** _mmethot_ has joined #tripleo20:57
*** milan has quit IRC20:58
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: WIP - use infra mirror for buildlogs repo  https://review.openstack.org/49062920:59
*** mmethot_ has quit IRC21:00
openstackgerritDavid Hill proposed openstack/diskimage-builder master: Clear /etc/machine-id to avoid duplicate machine-ids  https://review.openstack.org/48901321:02
*** itlinux has joined #tripleo21:03
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Configure the set of tests to execute in scenario001  https://review.openstack.org/49037621:03
EmilienMmwhahaha: with autoscaling test ^21:03
EmilienMbrb 20 min21:03
*** agopi|away has quit IRC21:04
*** dparkes has joined #tripleo21:05
stevebakerEmilienM: cool21:08
mwhahahablah bmc is giving me fits trying to restart my ovb nodes it looks like21:10
*** lblanchard has quit IRC21:13
*** mmethot_ has joined #tripleo21:14
openstackgerritLokesh Jain proposed openstack/tripleo-heat-templates master: Addition of Nuage as mechanism driver for ML2  https://review.openstack.org/47478821:15
*** _mmethot_ has quit IRC21:16
*** agopi has joined #tripleo21:17
openstackgerritSteve Baker proposed openstack/tripleo-common master: Upload remove defaults for pull_source, push_destination  https://review.openstack.org/48983721:19
mwhahahabnemec: have you seen introspection fail before when the node is already off? it seems that ipmitool is error when trying to request the node off21:20
openstackgerritJulia Kreger proposed openstack/diskimage-builder master: Add netbase to ensure /etc/protocols is placed for debian  https://review.openstack.org/49065621:20
bnemecmwhahaha: RDO cloud?  I know dtrainor was having issues with his bmc in that cloud recently.21:21
mwhahahabnemec: rh221:21
mwhahahai'm using my original ovb setup from like 6 months ago21:21
mwhahahaand it used to work21:21
mwhahahabut now introspection is just hanging and all i see in the logs is trying to turn the nodes off21:22
openstackgerritMerged openstack/tripleo-common master: Update the documentation link for doc migration  https://review.openstack.org/48691421:22
openstackgerritMerged openstack/tripleo-common master: Update URL home-page in documents according to document migration  https://review.openstack.org/48375021:22
bnemecmwhahaha: Hmm, anything in the bmc console-log?21:22
openstackgerritMerged openstack/os-net-config master: OvS 2.7 support - dpdk-devargs needs to provided for DPDK devices  https://review.openstack.org/48054021:22
mwhahahabnemec: just Aug  3 21:22:21 bmc openstackbmc: Getting power state for e04690d1-ddc6-4b17-bf4d-4a8126945ba021:22
openstackgerritDavid Hill proposed openstack/diskimage-builder master: Clear /etc/machine-id to avoid duplicate machine-ids  https://review.openstack.org/48901321:22
*** ansmith has joined #tripleo21:22
mwhahahabnemec: ironic-conductor shows http://paste.openstack.org/show/617470/21:23
*** agopi has quit IRC21:23
mwhahahabnemec: so it seems that if the nodes aren't getting powered on or something21:24
bnemecmwhahaha: Hmm, I wonder if that's from back when bmc caching was enabled by default.  Have you tried rebooting the bmc instance?21:24
mwhahahaif i manually start them introspection fiinsihes21:24
openstackgerritDavid Hill proposed openstack/diskimage-builder master: Clear /etc/machine-id to avoid duplicate machine-ids  https://review.openstack.org/48901321:24
*** morazi has quit IRC21:25
openstackgerritDavid Hill proposed openstack/diskimage-builder master: Clear /etc/machine-id to avoid duplicate machine-ids  https://review.openstack.org/48901321:26
bnemecmwhahaha: Also, is there any chance this is old enough to not have https://github.com/cybertron/openstack-virtual-baremetal/commit/0b73548248a33baf3167fe59c00e8d520c4f021d ?21:26
mwhahahabnemec: probably21:26
bnemecIt looks like that may be related.21:26
mwhahahai don't want to upgrade cause it was working :D and everytime i touch this stuff it just falls apart21:27
mwhahahabut i guess i'll do that tomorrow21:27
dtrainoryea, i had issues with it21:27
bnemecmwhahaha: You could hack the bmc code.  It's just two lines that need to be removed.21:27
mwhahahai'm sure i'm missing a bunch of stuff21:27
openstackgerritDavid Hill proposed openstack/diskimage-builder master: Clear /etc/machine-id to avoid duplicate machine-ids  https://review.openstack.org/48901321:27
mwhahahayay my introspection finally finished21:28
dtrainorit has good days, and it has bad days21:28
* mwhahaha is too familar with this concept and would like more good than bad21:28
dtrainorthey're talking of applying that nova boot patch, i was supposed to test it today but didn't hear back about it21:29
dtrainorthat would alleviate many of those boot related problems during introspection21:29
mwhahahawell this was my rh2 setup which has that patch21:30
dtrainorwe'd be less dependent on what the bmc reports because we'd know that the power and boot operation on the node actually worked because the node would start to report introspection data21:30
dtrainoroh, my bad, missed that part21:30
mwhahahai'm trying to reproduce the ovb-containers (without quickstart)21:30
dtrainorgot ya21:30
mwhahahaand when i go to deploy containers it's like you have no nodes21:30
dtrainorgo back to ignoring me :)21:30
dtrainoryeah21:30
mwhahahaso my introspection was bad21:30
mwhahahajust fixed tha tbut i think it's still going to fail21:30
dtrainorohnodes21:30
mwhahahaso now i want to know wtf we're doing by default21:30
mwhahahaOnly 0 nodes are exposed to Nova of 5 requests. Check that enough nodes are in "available" state with maintenance mode off.21:31
* mwhahaha sighs21:31
fultonjEmilienM: FYI: it looks to me like containerized glance as a ceph-client in https://review.openstack.org/#/c/489710 is configured cortrectly ; details in http://paste.openstack.org/show/61747221:31
* mwhahaha flips tables21:40
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci master: Don't do pre-pingtest cleanup with --skip-pingtest-cleanup  https://review.openstack.org/49066121:40
mwhahahawhy is the ComputeFilter excluding all my hosts21:43
*** rhallisey has quit IRC21:50
openstackgerritMerged openstack/tripleo-quickstart master: Add consistent-pike.yml configuration  https://review.openstack.org/49047221:50
openstackgerritMerged openstack-infra/tripleo-ci master: Add bits to support stable/pike  https://review.openstack.org/49007721:51
openstackgerritMerged openstack/tripleo-quickstart master: Move release notes to the right directory  https://review.openstack.org/49012721:51
openstackgerritMerged openstack/puppet-tripleo stable/ocata: Remove unnecessary references to neutron core plugin hiera  https://review.openstack.org/48221121:52
openstackgerritMerged openstack/python-tripleoclient stable/pike: Install policycoreutils-python  https://review.openstack.org/49050421:52
openstackgerritMerged openstack/python-tripleoclient master: Fail early in case of error in env directory path  https://review.openstack.org/47544221:52
openstackgerritMerged openstack/tripleo-heat-templates master: Add environment for setting a custom domain name  https://review.openstack.org/47621921:52
openstackgerritMerged openstack/python-tripleoclient stable/pike: Update .gitreview for stable/pike  https://review.openstack.org/48788621:52
openstackgerritMerged openstack/python-tripleoclient stable/pike: Update UPPER_CONSTRAINTS_FILE for stable/pike  https://review.openstack.org/48788721:52
openstackgerritMerged openstack/tripleo-quickstart-extras master: Install openstack services tempest plugin  https://review.openstack.org/49036921:52
openstackgerritMerged openstack/python-tripleoclient master: Update reno for stable/pike  https://review.openstack.org/48788821:53
openstackgerritMerged openstack/tripleo-heat-templates master: Make UpgradeLevelNovaCompute parameters consistent  https://review.openstack.org/48404121:53
dprinceEmilienM: zaqar in containers works for me21:53
dprinceEmilienM: on the undercloud it does at least21:53
* dprince has to sign off now though21:54
*** dprince has quit IRC21:54
*** cdearborn has quit IRC21:54
*** jmelvin has quit IRC21:57
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260722:00
*** jlabarre has quit IRC22:12
*** lblanchard has joined #tripleo22:15
*** ebarrera_ has quit IRC22:17
*** lblanchard has quit IRC22:17
mwhahahawe test containers with a baremetal (vm) undercloud right?22:17
mwhahahait seems as though my nova-compute service was disabled on my undercloud (wtf)22:23
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates master: Update EventPipelinePublisher param description to include zaqar  https://review.openstack.org/48786122:38
dtrainormwhahaha, disabled, or died?22:57
*** rcernin has quit IRC22:58
*** nyechiel has quit IRC23:01
EmilienMfultonj: but do we have the keys in the container?23:05
EmilienMfultonj: I mean, /etc/ceph/ceph.conf in the glance-api container?23:05
stevebakerEmilienM: hey, I've just sent an FFE email to the list23:06
*** gbarros has joined #tripleo23:06
EmilienMstevebaker: ok, my gmail doesn't have it yet but I'll look23:07
mwhahahadtrainor: disabled23:07
*** tosky has quit IRC23:07
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Convert controller-role.yaml to role.role.j2.yaml  https://review.openstack.org/48027223:07
EmilienMdamn I think ovb jobs are broken again23:09
EmilienMmwhahaha: I think https://review.openstack.org/#/c/481151/ broke us again23:09
EmilienMI'm confused though23:09
EmilienMit passed CI23:09
EmilienMoh maybe we don't have https://review.openstack.org/#/c/481947/ in the image23:10
mwhahahaBlah23:11
EmilienMthe new image should have it23:14
EmilienMhttp://66.187.229.139/builds-master/current-tripleo/metadata.txt23:14
EmilienMhash is c8/00/c80011cd91cbbd71c66064dbfdd69fed0fb6d705_65fed62423:14
EmilienMhttps://trunk.rdoproject.org/centos7-master/c8/00/c80011cd91cbbd71c66064dbfdd69fed0fb6d705_65fed624/23:14
EmilienMhttps://trunk.rdoproject.org/centos7-master/c8/00/c80011cd91cbbd71c66064dbfdd69fed0fb6d705_65fed624/openstack-mistral-5.0.0-0.20170803070025.c7fe4e8.el7.centos.src.rpm23:14
EmilienMc7fe4e8 is the commit we need, so we should be good23:14
EmilienMbut it sounds like it doesn't work, failing jobs have this commit:23:15
EmilienMhttp://logs.openstack.org/72/480272/26/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq/08a0163/logs/rpm-qa.txt.gz23:15
EmilienMthrash: ^23:18
EmilienMany idea on this trace? http://logs.openstack.org/72/480272/26/check-tripleo/gate-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024/4ce0ea6/logs/undercloud/home/jenkins/overcloud_prep_images.log.txt.gz#_2017-08-03_20_53_3923:18
*** Goneri has quit IRC23:20
openstackgerritMerged openstack/tripleo-common master: Pass UserKnownHostsFile and deploy ssh key  https://review.openstack.org/48765023:21
*** mmethot has joined #tripleo23:22
*** _mmethot_ has joined #tripleo23:24
*** itlinux has quit IRC23:25
*** mmethot_ has quit IRC23:25
*** Goneri has joined #tripleo23:27
*** mmethot has quit IRC23:28
EmilienMarxcruz: the tempest pushed by quickstart requires cinder, we need to stop that23:29
EmilienMarxcruz: scenario003 doesn't have cinder: http://logs.openstack.org/72/490172/4/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-container/a3efcae/logs/undercloud/home/jenkins/tempest_output.log.txt.gz#_2017-08-03_22_27_2823:29
*** tosky has joined #tripleo23:31
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Deploy Mistral with Keystone v3 options (authtoken)  https://review.openstack.org/46104023:32
*** Goneri has quit IRC23:33
*** hamzy has joined #tripleo23:33
*** _mmethot_ has quit IRC23:34
*** _mmethot_ has joined #tripleo23:34
openstackgerritEmilien Macchi proposed openstack/instack-undercloud master: Configure Mistral with authoken parameters  https://review.openstack.org/49068023:35
fultonjEmilienM: regarding your question, do we know if the conf and keys made it into the container...23:37
fultonjI don't see in the subnode-2 logs a way to see how 'docker run' was used or the output of 'docker inspect'23:37
fultonjhttp://logs.openstack.org/10/489710/5/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/e2c09a1/logs/subnode-2/23:37
*** tosky has quit IRC23:38
*** nyechiel has joined #tripleo23:38
fultonjthe code to put it there merged on the 25th, so I'm assuming it is present but i agree we should be able to find evidence that it ran ; i am not sure how ; ci collect logs change?23:39
EmilienMre: mistral/introspection, I filed a new alert https://bugs.launchpad.net/tripleo/+bug/170855423:39
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,Triaged]23:39
EmilienMfultonj: I don't know23:39
EmilienMfultonj: I'm pretty sure there is no code that takes care of pushing ceph.conf file in glance container, when ceph-ansible isn't used, right?23:40
fultonjno, i will show you23:40
fultonjthe client change which pushes it in doesn't care which tool made the ceph.conf or keys23:40
fultonjshould work with puppet-ceph or ceph-ansible23:41
* fultonj gets review23:41
fultonjhttps://review.openstack.org/#/c/482500/23:41
fultonjhttps://review.openstack.org/#/c/482500/21/docker/services/glance-api.yaml23:41
EmilienMlooking23:42
EmilienMfultonj: oh indeed23:42
openstackgerritMerged openstack/python-tripleoclient stable/ocata: Install policycoreutils-python  https://review.openstack.org/49064523:42
EmilienMsounds like it doesn't work23:43
EmilienMor not like we want23:43
fultonjand since i see the right config in glance-api.conf and the keys and config on the container23:43
fultonjwe need to know what's going on though23:43
EmilienMyeah?23:43
EmilienMwhere do you see the ceph.conf?23:43
fultonjhttp://logs.openstack.org/10/489710/5/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/e2c09a1/logs/subnode-2/etc/ceph/23:44
fultonjclarification23:44
fultonj... and the keys and config on the container HOST23:44
EmilienMfultonj: ok23:45
fultonjsorry, yes it would be gerat to see it on teh container23:45
EmilienMfultonj: so it should work :D23:46
EmilienMI now understand why you wanted to see docker outputs23:46
EmilienMto see if /etc/ceph is actually mounted to the glance-api container23:46
EmilienMright?23:46
EmilienMstevebaker: nice one :D23:46
EmilienMI don't know if we can23:46
EmilienMtbh23:46
fultonjexactly23:46
EmilienMfultonj: anyway, let's make progress on the ceph-ansible thing23:47
EmilienMfultonj: and hope we can make it work that way23:48
EmilienMfultonj: are we going to support both ways in pike?23:48
EmilienMor just ceph-ansible?23:48
fultonjboth ways23:48
EmilienMmhh ok23:48
EmilienMso we have to make this thing work23:48
EmilienMfultonj: can you help me and get some eyes on my problem23:48
EmilienM?23:48
fultonjyes23:48
EmilienMsee why glance doesn't work23:48
EmilienMfultonj: the patch is https://review.openstack.org/#/c/489710/23:49
EmilienMsee all -container jobs failing when pingtest is enabled23:49
EmilienMexcept for scenario003! wooo23:49
fultonjprogress23:49
EmilienMyes :)23:50
fultonji will ask someone in storage-dfg regarding glance23:50
EmilienMI have to go now but I'm back later after dinner23:50
thrashEmilienM: Looks like a bug that's rearing its ugly head... But I'm not sure why.23:50
fultonji'll be here23:50
fultonjquestion: ...23:50
EmilienMthrash: do you think it's related to puppet-mistral patch or the mistral patch? or none?23:50
fultonjwhy not add a patch that 489710 can depends-on which pulls in those logs from the container?23:50
thrashEmilienM: something keeps introspection from starting... And if that doesn't start, we can't get the actions from the client.23:51
EmilienMthrash: I file https://bugs.launchpad.net/tripleo/+bug/1708554 - feel free to add more debug / thoughts23:51
openstackLaunchpad bug 1708554 in tripleo "Introspection is failing with latest Mistral & puppet-mistral" [Critical,Triaged]23:51
thrashEmilienM: not sure yet. Let me dig deeper.23:51
EmilienMthrash: thanks23:51
fultonjperhaps easier said then done, but should we take that approach23:51
fultonjEmilienM: ^?23:51
EmilienMthrash: I need to do some errand, I'm back in a few23:51
thrashEmilienM: ack23:51
EmilienMfultonj: yeah23:52
fultonjok23:52
EmilienMfultonj: we can do that, feel free to do so if you know how to get these logs23:52
EmilienMfultonj: keep in mind the existing depends-on23:52
fultonjyeah23:52
EmilienMif you have to add one to an existing repo, put it on top of the patch23:52
EmilienMdon't adds multiple Depends-on from the same repo23:52
EmilienMit doesn't work23:52
fultonjright23:52
EmilienMuse git rebase + 1 depends-on23:52
EmilienMjust in case23:52
* EmilienM afk now23:52
fultonjgeneral question for anyone: was larsks looking at this kind of thing?23:52
fultonj"this kind of thing" --> getting info about the running containers into the logs available for someone to examine in the ci results23:53
*** rbowen has quit IRC23:54
larsksfultonj: I was looking into setting up container healthchecks, and I've been mumbling about container logging.  I'm not sure that either of those things is what you're asking about, though.23:58
fultonjlarsks: ok, yes i saw you eamil about that but wasn't sure if what i was looking for was included23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!