| openstackgerrit | Merged openstack/tripleo-common master: Add log capturing support workflows https://review.openstack.org/405002 | 00:01 |
|---|---|---|
| *** links has joined #tripleo | 00:06 | |
| *** seseseok has joined #tripleo | 00:06 | |
| seseseok | test | 00:06 |
| *** seseseok has quit IRC | 00:07 | |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 00:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 00:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 00:10 |
| *** liverpooler has quit IRC | 00:14 | |
| *** hjensas has quit IRC | 00:14 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Server blacklist support https://review.openstack.org/464733 | 00:24 |
| *** yamahata has quit IRC | 00:35 | |
| *** mdnadeem has joined #tripleo | 00:36 | |
| *** rwsu has quit IRC | 00:41 | |
| *** artom has quit IRC | 00:45 | |
| *** limao has joined #tripleo | 00:46 | |
| *** artom has joined #tripleo | 01:01 | |
| *** fragatina has quit IRC | 01:03 | |
| *** fragatina has joined #tripleo | 01:04 | |
| *** liverpooler has joined #tripleo | 01:04 | |
| *** guoguo has joined #tripleo | 01:08 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 01:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 01:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 01:10 |
| *** links has quit IRC | 01:14 | |
| *** artom has quit IRC | 01:21 | |
| *** mdnadeem has quit IRC | 01:27 | |
| *** artom has joined #tripleo | 01:30 | |
| *** kbyrne has quit IRC | 01:39 | |
| *** kbyrne has joined #tripleo | 01:42 | |
| *** rbrady has quit IRC | 01:46 | |
| *** tbonds has joined #tripleo | 01:50 | |
| *** dmacpher_ has quit IRC | 01:57 | |
| *** rbrady has joined #tripleo | 01:59 | |
| *** rbrady has joined #tripleo | 01:59 | |
| *** atheurer has quit IRC | 02:08 | |
| *** atheurer has joined #tripleo | 02:08 | |
| *** mdnadeem has joined #tripleo | 02:09 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 02:10 |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 02:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 02:10 |
| *** flepied has quit IRC | 02:11 | |
| *** tbonds has quit IRC | 02:13 | |
| *** rwsu has joined #tripleo | 02:16 | |
| *** oneswig has joined #tripleo | 02:18 | |
| *** oneswig has quit IRC | 02:23 | |
| *** owalsh_ has joined #tripleo | 02:44 | |
| dmsimard | FYI: Widespread CI failures due to broken python setuptools: https://www.redhat.com/archives/rdo-list/2017-June/msg00000.html | 02:47 |
| *** owalsh has quit IRC | 02:47 | |
| *** links has joined #tripleo | 02:52 | |
| *** udesale has joined #tripleo | 03:02 | |
| *** rbrady has quit IRC | 03:02 | |
| *** eck` is now known as eck`gone | 03:06 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 03:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 03:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 03:10 |
| *** ykarel__ has joined #tripleo | 03:12 | |
| *** dbecker has quit IRC | 03:19 | |
| *** ramishra has joined #tripleo | 03:20 | |
| *** janki has joined #tripleo | 03:26 | |
| *** prateek has joined #tripleo | 03:26 | |
| *** mdnadeem has quit IRC | 03:27 | |
| *** ratailor has joined #tripleo | 03:28 | |
| *** dmacpher has joined #tripleo | 03:33 | |
| *** prateek has quit IRC | 03:33 | |
| *** dbecker has joined #tripleo | 03:34 | |
| *** prateek has joined #tripleo | 03:34 | |
| *** prateek has quit IRC | 03:34 | |
| *** ykarel_ has joined #tripleo | 03:35 | |
| *** prateek has joined #tripleo | 03:36 | |
| *** ykarel__ has quit IRC | 03:36 | |
| *** prateek_ has joined #tripleo | 03:43 | |
| *** mdnadeem has joined #tripleo | 03:43 | |
| *** prateek has quit IRC | 03:46 | |
| *** psahoo has joined #tripleo | 03:50 | |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-heat-templates master: WIP: Support deploying OVN as container services https://review.openstack.org/468860 | 03:51 |
| *** rbrady has joined #tripleo | 03:58 | |
| *** rwsu has quit IRC | 04:07 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 04:10 |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 04:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 04:10 |
| *** Tyrantelf has quit IRC | 04:17 | |
| *** Tyrantelf has joined #tripleo | 04:18 | |
| *** ykarel_ is now known as ykarel|bootcamp | 04:23 | |
| *** fultonj has quit IRC | 04:23 | |
| *** fultonj has joined #tripleo | 04:25 | |
| *** ekoorjr has quit IRC | 04:32 | |
| *** tzumainn has quit IRC | 04:33 | |
| *** pcaruana has joined #tripleo | 04:43 | |
| *** dparkes has quit IRC | 04:48 | |
| *** masco has quit IRC | 04:55 | |
| *** ramishra has quit IRC | 04:58 | |
| *** limao has quit IRC | 05:04 | |
| *** limao has joined #tripleo | 05:04 | |
| *** skramaja has joined #tripleo | 05:08 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 05:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 05:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 05:10 |
| *** yamahata has joined #tripleo | 05:11 | |
| *** agurenko has joined #tripleo | 05:13 | |
| *** itzikb has joined #tripleo | 05:16 | |
| *** itzikb has quit IRC | 05:18 | |
| *** jaosorior_away is now known as jaosorior | 05:24 | |
| *** noslzzp has quit IRC | 05:26 | |
| *** noslzzp has joined #tripleo | 05:27 | |
| *** brault has joined #tripleo | 05:29 | |
| *** ykarel_ has joined #tripleo | 05:37 | |
| *** rwsu has joined #tripleo | 05:38 | |
| *** ykarel|bootcamp has quit IRC | 05:39 | |
| *** lmiccini has joined #tripleo | 05:41 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: Providing parameters specific to a workflow via plan-environment https://review.openstack.org/457874 | 05:45 |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: Providing parameters specific to a workflow via plan-environment https://review.openstack.org/457874 | 05:46 |
| *** florianf has joined #tripleo | 05:47 | |
| *** pcaruana has quit IRC | 05:49 | |
| openstackgerrit | Numan Siddique proposed openstack/puppet-tripleo master: Pacemaker support for OVN DB servers https://review.openstack.org/372274 | 05:51 |
| *** dparkes has joined #tripleo | 05:52 | |
| *** dsneddon_ has quit IRC | 05:54 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use picked nodes for later cmd_* calls https://review.openstack.org/469330 | 05:55 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add state to NodeBase class https://review.openstack.org/469749 | 05:55 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Remove 'state' argument from later cmd_* calls https://review.openstack.org/469750 | 05:55 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use global state to check for duplicate fs labels https://review.openstack.org/469751 | 05:55 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move global mount tracking into state https://review.openstack.org/469752 | 05:55 |
| *** rcernin has joined #tripleo | 05:58 | |
| *** pcaruana has joined #tripleo | 06:02 | |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-quickstart-extras master: This patch does the following when overdeploy fails: https://review.openstack.org/469754 | 06:03 |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-quickstart-extras master: This patch does the following when overcloud deploy fails: https://review.openstack.org/469754 | 06:03 |
| *** flepied has joined #tripleo | 06:05 | |
| *** yprokule has joined #tripleo | 06:08 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 06:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 06:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 06:10 |
| *** ebarrera has joined #tripleo | 06:14 | |
| *** ebarrera has quit IRC | 06:15 | |
| *** ebarrera has joined #tripleo | 06:15 | |
| *** gbarros has joined #tripleo | 06:16 | |
| *** jprovazn has joined #tripleo | 06:18 | |
| *** oneswig has joined #tripleo | 06:20 | |
| *** gbarros has quit IRC | 06:23 | |
| *** ahrechan has joined #tripleo | 06:23 | |
| *** oneswig has quit IRC | 06:25 | |
| openstackgerrit | Numan Siddique proposed openstack/puppet-tripleo master: Pacemaker support for OVN DB servers https://review.openstack.org/372274 | 06:25 |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-heat-templates master: Pacemaker HA suport for OVN DB servers https://review.openstack.org/436767 | 06:25 |
| ahrechan | Hi! Today I got "quickstart.sh: line 483: ansible-playbook: command not found" when tried to install ocata | 06:26 |
| janki | ahrechan, run ./quckstart --install-deps first | 06:27 |
| *** ramishra has joined #tripleo | 06:27 | |
| ahrechan | janki: 06:25:00 + su '--session-command=/bin/bash /tmp/quickstart/quickstart.sh --install-deps' jenkins. And yesterday all worked. | 06:29 |
| janki | ahrechan, try running ./quickstart.sh --bootstrap virshost | 06:30 |
| janki | the error says ansible is not installed in your system. quickstart installs it | 06:31 |
| janki | or rerun .quickstart.sh --install-deps and everything should go fine | 06:31 |
| *** jaganathan has joined #tripleo | 06:34 | |
| *** gbarros has joined #tripleo | 06:36 | |
| yolanda | hi, good morning. Can i get an extra +2 on https://review.openstack.org/#/c/469528/ ? for security hardened images, i was missing some content due to a rebase | 06:39 |
| *** gbarros has quit IRC | 06:44 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-quickstart master: Install six to workaround setuptools issue https://review.openstack.org/469770 | 06:47 |
| jaosorior | chandankumar: ^^ | 06:47 |
| *** gbarros has joined #tripleo | 06:47 | |
| chandankumar | jaosorior: Thanks ! | 06:49 |
| *** tesseract has joined #tripleo | 06:51 | |
| *** gbarros has quit IRC | 06:53 | |
| *** fzdarsky has joined #tripleo | 06:53 | |
| *** fzdarsky has quit IRC | 06:53 | |
| *** iranzo has joined #tripleo | 06:56 | |
| *** masco has joined #tripleo | 06:56 | |
| *** jaganathan has quit IRC | 06:57 | |
| *** jaganathan has joined #tripleo | 06:58 | |
| *** jlinkes has joined #tripleo | 06:58 | |
| *** zoli|gone is now known as zoli | 07:00 | |
| *** jpich has joined #tripleo | 07:00 | |
| *** bogdando has joined #tripleo | 07:02 | |
| *** fragatina has quit IRC | 07:04 | |
| *** fragatina has joined #tripleo | 07:05 | |
| *** cschwede_ has quit IRC | 07:05 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-quickstart master: Install six to workaround setuptools issue https://review.openstack.org/469770 | 07:06 |
| *** lhinds has quit IRC | 07:06 | |
| *** pliu has quit IRC | 07:07 | |
| *** faceman has quit IRC | 07:08 | |
| *** rnoriega has quit IRC | 07:08 | |
| *** cschwede has joined #tripleo | 07:09 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 07:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 07:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 07:10 |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-quickstart-extras master: Gracefully exit when overcloud deploy doesnot create overcloud stack https://review.openstack.org/469754 | 07:10 |
| skramaja | jaosorior: if possible, can you have look at https://review.openstack.org/#/c/462171/ ? its the default directory change in the environment file for dpdk. | 07:11 |
| *** dmacpher has quit IRC | 07:14 | |
| *** slagle has quit IRC | 07:16 | |
| *** shardy has joined #tripleo | 07:16 | |
| *** stevebaker has quit IRC | 07:16 | |
| *** mcornea has joined #tripleo | 07:18 | |
| *** jpena|off is now known as jpena | 07:22 | |
| *** flepied has quit IRC | 07:22 | |
| *** dsariel has joined #tripleo | 07:26 | |
| *** aufi has joined #tripleo | 07:28 | |
| ahrechan | janki: problem is global - http://pastebin.test.redhat.com/489838 | 07:29 |
| *** janki has left #tripleo | 07:29 | |
| *** guoguo has quit IRC | 07:29 | |
| *** janki has joined #tripleo | 07:30 | |
| *** suuuper has joined #tripleo | 07:31 | |
| *** karimb has joined #tripleo | 07:34 | |
| janki | ahrechan, that seems to be bug. Check if a bug is already file or go ahead and file one | 07:37 |
| ahrechan | janki: https://github.com/pypa/setuptools/issues/1042, and workaround - VIRTUALENV_NO_DOWNLOAD=1 | 07:38 |
| *** cylopez has joined #tripleo | 07:40 | |
| janki | ahrechan, manually installing six is not encouraged. there might be version mismatch. quickstart must install "six" if it is required later in the process | 07:40 |
| *** oidgar has joined #tripleo | 07:40 | |
| yolanda | hi shardy , question about the security hardened images. I created those on the yaml files of tripleo-common. Is that enough for the security hardened image to be generated and packaged? is there any extra step i need to do? | 07:42 |
| jaosorior | Hey guys, I'm seeing these errors while trying out a newton install | 07:42 |
| jaosorior | 2017-06-01 07:23:34 | 2017-06-01 07:23:34 - ESC[1;31mError: Execution of '/bin/yum -d 0 -e 0 -y install openstack-mistral-engine' returned 1: Error downloading packages: | 07:42 |
| jaosorior | 2017-06-01 07:23:34 | 2017-06-01 07:23:34 - openstack-mistral-engine-3.0.3-0.20170307090022.12527ab.el7.centos.noarch: [Errno 256] No more mirrors to try.ESC[0m | 07:42 |
| jaosorior | any clues? | 07:42 |
| *** iranzo has quit IRC | 07:43 | |
| *** yamahata has quit IRC | 07:44 | |
| michapma | is images.rdoproject.org down? | 07:47 |
| cschwede | michapma: seems so - doesn't work for me neither | 07:51 |
| *** hewbrocc` is now known as hewbrocca | 07:53 | |
| *** mdnadeem has quit IRC | 07:53 | |
| *** flepied has joined #tripleo | 07:53 | |
| michapma | cschwede: thanks! I guess I'll check back later. | 07:54 |
| *** amoralej|off is now known as amoralej | 07:54 | |
| *** tosky has joined #tripleo | 07:55 | |
| jaosorior | michapma: sure seems it is | 07:56 |
| *** pgadiya has joined #tripleo | 07:57 | |
| *** tosky has quit IRC | 07:57 | |
| *** zzzeek_ has quit IRC | 08:00 | |
| *** zzzeek has joined #tripleo | 08:00 | |
| b00tcat | Hi, I defined my own network in network/service_net_map.j2.yaml which maps to the internal_api network - Is it possible to obtain the CIDR of such network? | 08:00 |
| b00tcat | I used InternalApiNetCidr directly but it gave me a weird value | 08:01 |
| *** tvignaud has quit IRC | 08:03 | |
| *** gbarros has joined #tripleo | 08:03 | |
| b00tcat | is perhaps InternalApiIpSubnet what I'm looking for? | 08:03 |
| *** suuuper has quit IRC | 08:04 | |
| *** mdnadeem has joined #tripleo | 08:08 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 08:10 |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 08:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 08:10 |
| openstackgerrit | Matthias Runge proposed openstack/tripleo-heat-templates master: Containerized collectd https://review.openstack.org/469465 | 08:10 |
| *** gbarros has quit IRC | 08:12 | |
| jaosorior | shardy: hey, is there a hiera key for getting the internal API VIP ? | 08:14 |
| mandre | hello, I'm looking at ways to increase the disk size of the undercloud in CI because we're dangerously close to the limit | 08:14 |
| mandre | it looks like it's something we can control with the 'undercloud' flavor of tripleo-test-cloud-rh1 | 08:14 |
| mandre | https://github.com/openstack-infra/project-config/blob/master/nodepool/nodepool.yaml#L823 | 08:14 |
| *** sid1 has joined #tripleo | 08:16 | |
| *** egonzalez has joined #tripleo | 08:16 | |
| *** tvignaud has joined #tripleo | 08:16 | |
| shardy | jaosorior: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/all-nodes-config.yaml#L216 | 08:16 |
| mandre | shardy: hi! do you know who I should talk to regarding flavor config for tripleo-test-cloud-rh1? | 08:17 |
| *** akrivoka has joined #tripleo | 08:17 | |
| shardy | yolanda: I don't think we package images upstream, so you may need a bz to get the new image packaged in downstream builds | 08:17 |
| yolanda | shardy, so what's the workflow with those images? once the image is reflect on the yaml, is automatically created? | 08:18 |
| *** gbarros has joined #tripleo | 08:18 | |
| jaosorior | shardy: awesome, thanks | 08:19 |
| shardy | mandre: I'd chat with bnemec when he comes online, or perhaps sshnaidm|off can help (/me sees he's off) | 08:19 |
| *** hjensas has joined #tripleo | 08:19 | |
| shardy | https://github.com/openstack/tripleo-incubator/blob/master/tripleo-cloud/tripleo-cd-admins | 08:19 |
| shardy | mandre: as a workaround could we disable the docker registry on the undercloud and just pull from dockerhub (is that cached in CI now?) | 08:20 |
| shardy | or are we already doing that? | 08:20 |
| *** suuuper has joined #tripleo | 08:20 | |
| ansiwen | scenario002-multinode-upgrades on stable/ocata is failing for me since two weeks (it succeeded before with the same change) and with every rebase the error seems to change. really frustrating. >_< | 08:21 |
| *** hjensas has quit IRC | 08:21 | |
| shardy | yolanda: the overcloud-full image is built as part of CI and uploaded to the RDO images location, but I'm not sure on the workflow for downstream packaged images, I'd check with mburns later | 08:21 |
| shardy | ansiwen: it was broken on a cinder error, IIRC we asked some cinder experts to help resolve the error | 08:22 |
| yolanda | shardy , so as part of my work, i created that overcloud-security-hardened image. That will be also built and uploaded to RDO as part of CI? | 08:22 |
| ansiwen | shardy: right, before it was zaqar, now it's cinder... :-/ | 08:22 |
| shardy | yolanda: Probably not, since it's not tested in CI? | 08:23 |
| mandre | shardy: great, thanks for the pointer. you're right it may be worth trying to bypass the registry on the undercloud and pull from the dockerhub on the overcloud nodes as a workaround | 08:23 |
| ansiwen | shardy: that sounds like, solution is still in process? | 08:23 |
| yolanda | shardy so first step will be to create a job that does it? | 08:23 |
| mandre | shardy: I believe infra runs a proxy for the dockerhub, we should be fine | 08:24 |
| shardy | mandre: ack, I guess that should save us quite a bit of space then | 08:24 |
| ansiwen | mandre: that sounds like a good thing to do anyway, right? (a single caching proxy in the infrastructure) | 08:25 |
| *** yamahata has joined #tripleo | 08:25 | |
| shardy | yolanda: yes I guess either an upstream or RDO CI job to build and validate the image would be the first step | 08:26 |
| yolanda | ok i will look | 08:26 |
| yolanda | shardy, also can you look at https://review.openstack.org/469528 ? i needed minor fixes, due to rebase | 08:27 |
| *** lucas-afk is now known as lucasagomes | 08:27 | |
| ansiwen | wow, now ALL Ci jobs are read! http://logs.openstack.org/42/462542/3/check/gate-tripleo-ci-centos-7-nonha-multinode-oooq/0e63fc1/console.html | 08:28 |
| *** karthiks has joined #tripleo | 08:29 | |
| *** athomas has joined #tripleo | 08:30 | |
| *** hjensas has joined #tripleo | 08:31 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Refactor setup_docker_host.sh as host_prep_tasks https://review.openstack.org/469163 | 08:37 |
| openstackgerrit | Martin André proposed openstack/tripleo-quickstart-extras master: WIP Testing infra provided docker registry https://review.openstack.org/469807 | 08:37 |
| jistr | ansiwen: apparently not only our jobs. Zuul status it quite red for all projects | 08:41 |
| mandre | ansiwen: right, this could be a shortcut we take in CI because infra now caches docker images, but it means we wouldn't be testing how you would normally deploy a containerized overcloud | 08:41 |
| *** salmankhan has joined #tripleo | 08:41 | |
| mandre | ansiwen, jistr: https://bugs.launchpad.net/devstack/+bug/1694904 | 08:42 |
| openstack | Launchpad bug 1694904 in devstack "pip_install failed with 'ImportError: No module named six'" [Undecided,New] | 08:42 |
| *** ckyriakidou has joined #tripleo | 08:43 | |
| *** ahrechan has quit IRC | 08:44 | |
| *** derekh has joined #tripleo | 08:44 | |
| *** salmankhan1 has joined #tripleo | 08:44 | |
| *** salmankhan has quit IRC | 08:45 | |
| *** salmankhan1 is now known as salmankhan | 08:45 | |
| mrunge | any ideas how to delete a stack in "DELETE_FAILED" ? | 08:48 |
| jaosorior | mrunge: depends on the reason it failed | 08:49 |
| jaosorior | what does openstack server list report? is there any node in error state? | 08:49 |
| EmilienM | mrunge: for ex, it can be a neutron port that failed to be removed if attached to an instance, etc | 08:50 |
| EmilienM | mrunge: usually, it's not good when it happens (e.g. probably a bug) | 08:50 |
| mrunge | openstack server list returns two active overcloud nodes | 08:51 |
| *** iranzo has joined #tripleo | 08:51 | |
| *** iranzo has joined #tripleo | 08:51 | |
| mrunge | that's what I expected before | 08:51 |
| mrunge | err, that's ok, it was as before, just a minimal install | 08:51 |
| mrunge | jaosorior, independent of the failure reason, it doesn't seem to delete the stack anyways | 08:53 |
| *** mdnadeem has quit IRC | 08:54 | |
| EmilienM | jaosorior: maybe doing a "stack show" can also help to see what failed to be removed | 08:54 |
| mrunge | Resource DELETE failed: resources.AllNodesDeploySteps: Engine went down during stack DELETE | 08:55 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Custom containerized undercloud deployments https://review.openstack.org/450792 | 08:56 |
| shardy | mrunge: did the heat-engine get OOM killed during the delete perhaps? | 08:56 |
| yolanda | shardy, so something like that will be a good start? https://review.openstack.org/#/c/469809/ | 08:57 |
| mrunge | shardy, uhm, heat took > 100% cpu over 5 mins, so I restarted it via systemctl | 08:58 |
| mrunge | heat-engine, to be specific | 08:58 |
| jaosorior | uhm... yeah, I've had that issue a lot lately as well :/ ended up "fixing it" by just deploying an undercloud with more RAM (12G currently) | 08:59 |
| mrunge | the machine running is neither underpowered nor out of RAM or out of disk | 08:59 |
| EmilienM | mrunge: can you paste the output of stack show please? | 08:59 |
| mrunge | I'm using 16 gigs for the undercloud machine | 08:59 |
| shardy | mrunge: OK, well that's why it failed, restarting heat-engine will cause any IN_PROGRESS stacks to fail when the engine comes back up, unless you enable convergence which we don't by default for TripleO atm | 08:59 |
| mrunge | shardy, is there a way to recover? | 09:00 |
| shardy | mrunge: you can try doing the delete again, but without restarting any services | 09:00 |
| *** atheurer has quit IRC | 09:00 | |
| mrunge | shardy, already did twice, nothing happened | 09:00 |
| mrunge | openstack stack show overcloud still running, I'm waiting for it to return | 09:00 |
| *** flepied has quit IRC | 09:01 | |
| mrunge | still no answer | 09:02 |
| mrunge | fantastic | 09:02 |
| mrunge | might be quicker to do a complete redeploy, and container rebuild | 09:02 |
| *** iranzo has quit IRC | 09:03 | |
| shardy | mrunge: You have a few options, but basically restarting heat-engine when things are in progress is a bad idea | 09:04 |
| shardy | you can do one of: | 09:04 |
| shardy | 1. Clean up the servers etc via nova directly, then re-try the delete, something must be in an inconsistent state in the heat DB because you restarted things | 09:04 |
| shardy | 2. Drop the heat DB and re-create it via heat-manage | 09:04 |
| shardy | 3. Force the state of the stack to a non-failed state via heat-manage | 09:05 |
| mrunge | ok, thank you, I'll try to set the value via heat manage | 09:05 |
| shardy | mrunge: it's hard to be certain of the best steps without seeting all the logs - basically it *should* be possible to always delete from any state, but you've encounteterd an edge case due to the corruption caused by your service restart | 09:06 |
| shardy | if you collect heat-engine logs showing why the delete isn't progressing, we can help figure it out | 09:06 |
| mrunge | I'm now waiting openstack stack show overcloud for over 5 minutes now. I suspect something terribly broken | 09:06 |
| mrunge | not sure if it's worth the effort to look at that | 09:07 |
| shardy | mrunge: if the API calls are hanging, the most likely reason is the heat-engine is still down | 09:09 |
| shardy | it'll hang until the RPC times out | 09:09 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 09:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 09:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 09:10 |
| mrunge | heat-engine takes 100% cpu | 09:10 |
| *** pgadiya has quit IRC | 09:10 | |
| bogdando | hi folks | 09:10 |
| *** atheurer has joined #tripleo | 09:11 | |
| bogdando | does openstack overcloud deploy re-generate tht from jinja templates? | 09:11 |
| shardy | mrunge: Ok, check the /var/log/heat/heat-engine.log, something is probably broken and spinning I guess | 09:11 |
| *** mdnadeem has joined #tripleo | 09:11 | |
| shardy | bogdando: yes | 09:11 |
| bogdando | I faced a bug yesterday when custom t-h-t quickstart tasks erases everything rendered by undercloud deploy time, but overcloud deploy creates nothing... this ends up being deployed w/o a network temlpate and fails | 09:12 |
| mrunge | shardy, I get 1000s of those messages: 2017-06-01 09:11:15.070 17757 DEBUG heat.engine.service [req-de19d3e5-07c8-457a-85ee-ac1a47d2c10f - - - - -] Service f9cad333-822f-40c8-90ea-6eeeeb9b2cfb is updated service_manage_report /usr/lib/python2.7/site-packages/heat/engine/service.py:2382 | 09:12 |
| bogdando | shardy: I do overcloud redeploy with deleting the stack then just applying ansible | 09:12 |
| bogdando | do I miss a step which makes that overcloud deploy jinja magic to happen again? | 09:12 |
| shardy | bogdando: overcloud deploy first does an overcloud plan create, then the heat stack create | 09:13 |
| janki | panda|pnq, this one is not ignored | 09:13 |
| janki | TASK [provision/local : Ensure local working dir exists] ********************************************************************************************************************************************************** | 09:13 |
| janki | task path: /home/janki/.quickstart/tripleo-quickstart/roles/provision/local/tasks/main.yml:18 | 09:13 |
| janki | Thursday 01 June 2017 05:50:52 +0530 (0:00:00.199) 0:00:12.252 ********* | 09:13 |
| janki | fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "module_stderr": "sudo: a password is required\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1} | 09:13 |
| shardy | it's the plan create which does the j2 rendering, it uploads the --templates files to swift, then a mistral action j2 renders them | 09:13 |
| bogdando | shardy: yes, the plan is created/uploaded... just find / -name deployed-server-environment.yaml finds nothing | 09:14 |
| bogdando | the last time it existed was after undercloud deploy | 09:14 |
| bogdando | then, overcloud-deploy task calls custom tht that does rm -rf and ... | 09:14 |
| bogdando | does not create new files | 09:14 |
| shardy | bogdando: make sure the paths to --templates /foo -e /foo/environments/x.yaml match | 09:14 |
| bogdando | shardy: I noted that undrcloud_deploy.log contains records for rendered jinja2 templates, but not the overcloud_deploy.log | 09:15 |
| *** gbarros has quit IRC | 09:16 | |
| bogdando | so I'm asking... why my templates vanish? :) | 09:16 |
| shardy | bogdando: deployed-server-environment.j2.yaml is rendered to deployed-server-environment.yaml, but tripleoclient relies on the paths matching | 09:16 |
| shardy | bogdando: are you doing --templates /foo -e /notfoo/environments/deployed-server-environment.yaml ? | 09:16 |
| bogdando | shardy: nope | 09:17 |
| bogdando | the path is the same for undercloud/overcloud templates | 09:17 |
| *** flepied has joined #tripleo | 09:17 | |
| bogdando | I'm used to deploy with those pointint to /usr/share/openstack-tripleo-heat-templates | 09:18 |
| shardy | mrunge: Ok that sounds wrong, so some debugging is needed to figure out why heat is spinning like that | 09:18 |
| mrunge | shardy, I've found a message in heat-api.log, followed by a stack trace | 09:19 |
| mrunge | the message is: 2017-06-01 09:11:15.070 17757 DEBUG heat.engine.service [req-de19d3e5-07c8-457a-85ee-ac1a47d2c10f - - - - -] Service f9cad333-822f-40c8-90ea-6eeeeb9b2cfb is updated service_manage_report /usr/lib/python2.7/site-packages/heat/engine/service.py:2382 | 09:19 |
| mrunge | err | 09:19 |
| mrunge | 2017-06-01 09:09:41.430 31171 ERROR heat.common.wsgi [req-3e559ebd-d30f-453b-996b-1034407d1fb6 - admin - - -] Unexpected error occurred serving API: Timed out waiting for a reply to message ID 894814b218114a71b7d81fa92642bdbe | 09:19 |
| shardy | mrunge: yeah it sounds like heat-engine isn't starting correctly, so it's not responding to RPC calls | 09:19 |
| mrunge | that would make sense | 09:20 |
| mrunge | shardy, should I give a restart another try? | 09:20 |
| shardy | mrunge: yeah you could, or stop heat-engine and run it on the CLI, looking for clues early in the log output re why it's not starting | 09:21 |
| *** yamahata has quit IRC | 09:22 | |
| *** owalsh_ is now known as owalsh | 09:30 | |
| *** openstackstatus has quit IRC | 09:36 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-common master: Pass network_data into per-role j2 templating https://review.openstack.org/469822 | 09:36 |
| *** flepied has quit IRC | 09:37 | |
| bogdando | shardy: here are the commands from the uc/oc logs https://pastebin.com/vWT121pE | 09:38 |
| bogdando | I marked with *** the missing template | 09:38 |
| *** gbarros has joined #tripleo | 09:38 | |
| *** ahrechan has joined #tripleo | 09:39 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Custom containerized undercloud deployments https://review.openstack.org/450792 | 09:39 |
| *** openstackstatus has joined #tripleo | 09:39 | |
| *** ChanServ sets mode: +v openstackstatus | 09:39 | |
| ahrechan | is there connectivity problems with http://images.rdoproject.org/ocata/rdo_trunk/current-tripleo ? | 09:39 |
| adarazs | ahrechan: I'll check the server. | 09:43 |
| mrunge | shardy, I guess one lesson learned is, heat stack-delete could trigger deleting the stack failed, where openstack stack delete could not | 09:44 |
| *** salmankhan has quit IRC | 09:44 | |
| -openstackstatus- NOTICE: There is a known issue with setuptools 36.0.0 and errors about the "six" package. For current details see https://github.com/pypa/setuptools/issues/1042 and monitor #openstack-infra | 09:46 | |
| *** salmankhan has joined #tripleo | 09:46 | |
| shardy | mrunge: no, those use the same heat API, the failure was caused by restarting heat-engine | 09:47 |
| adarazs | ahrechan: looks like the host is down and I don't have access to reboot it. :/ should be back in about 2 hours when the owner comes back. | 09:47 |
| *** shardy is now known as shardy_afk | 09:47 | |
| adarazs | ahrechan: thanks for reporting it. | 09:47 |
| *** zoli is now known as zoli|lunch | 09:47 | |
| *** paramite has joined #tripleo | 09:48 | |
| owalsh | jistr: hi, have you seen anything like this when running virtlogd in a container: overcloud-novacompute-0 libvirtd[18523]: End of file while reading data: Input/output error | 09:48 |
| *** milan has joined #tripleo | 09:50 | |
| owalsh | jistr: I suspect it's something else I've changed, but can't figure out what... | 09:51 |
| *** flepied has joined #tripleo | 09:53 | |
| ccamacho | bogdando quick question, when you run swift download overcloud you get the deployed-server-environment.yaml file? | 09:56 |
| *** sid1 has quit IRC | 09:57 | |
| bogdando | ccamacho: I'll check once I have it redeployed back | 09:57 |
| bogdando | thanks! | 09:57 |
| openstackgerrit | Lee Yarwood proposed openstack/tripleo-quickstart-extras master: WIP ovb: Add missing ansible_user var https://review.openstack.org/469824 | 09:57 |
| bogdando | but I'm still wondering how the overcloud-deploy.sh is supposed to catch the file from the path | 09:57 |
| *** limao has quit IRC | 09:58 | |
| bogdando | am I missing some mistral workflow that downloads and extracts templates from swift container? | 09:58 |
| bogdando | it looks odd. overcloud deploy -e foo.yaml expects that foo.yaml has yet been created?.. | 09:59 |
| bogdando | quickstart overcloud-deploy just creates the script from template and invokes it | 09:59 |
| bogdando | no special workflows to prepend it with, AFAIK... | 10:00 |
| *** dtantsur|afk is now known as dtantsur | 10:00 | |
| bogdando | may be this is not in quickstart, but in ci scripts? | 10:00 |
| * bogdando confused | 10:00 | |
| ccamacho | bogdando, you should get there all the rendered templates. they are stored in a swift container and IIRC mmm extracted from the deployment plan, the tripleo client process the templates and store all in the swift container | 10:03 |
| bogdando | ccamacho: right, so I'm nut sure *what* prepares the missing deployed-server template by the moment of the deploy command invocation from the script | 10:04 |
| *** sid1 has joined #tripleo | 10:04 | |
| bogdando | it is clear for undercloud deploy, all things get stated in the logs | 10:04 |
| bogdando | it is a mistery for oc deploy ( | 10:04 |
| bogdando | s/nut/not :D | 10:05 |
| shardy_afk | bogdando: during plan create the "missing" file is rendered via j2 via a mistral action: | 10:05 |
| bogdando | I'm not nut yet! :) | 10:05 |
| *** shardy_afk is now known as shardy | 10:05 | |
| shardy | https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/templates.py#L106 | 10:06 |
| ccamacho | https://github.com/openstack/tripleo-common/blob/fbfe481ee4fe68add4b615b5fb92745a1e69b6b8/tripleo_common/actions/templates.py#L106 | 10:06 |
| ccamacho | aa | 10:06 |
| ccamacho | jaja | 10:06 |
| shardy | bogdando: as ccamacho says, if you download the files from swift, it should be there | 10:06 |
| ccamacho | same paste | 10:06 |
| ccamacho | : | 10:06 |
| ccamacho | .P | 10:06 |
| shardy | hehe | 10:06 |
| bogdando | shardy: hm. So the flow is: | 10:06 |
| bogdando | - script invokes cmd -e foo.yaml (and foo.yaml is missing by that point) | 10:06 |
| bogdando | - the cmd invokes mistral to create foo.yml | 10:06 |
| bogdando | - it is catched up by the internal logic and deployment proceeds | 10:06 |
| bogdando | ? | 10:06 |
| bogdando | this looks really odd | 10:07 |
| shardy | bogdando: yeah, it's because previously we didn't suport j2 rendered environment files | 10:07 |
| bogdando | like chicken and egg | 10:07 |
| bogdando | problem | 10:07 |
| shardy | but now we do, so yeah chicken/egg | 10:07 |
| bogdando | oh my got | 10:07 |
| bogdando | that was a hard one to get it right... | 10:07 |
| shardy | the solution we have is to render the template, put it in swift, then tripleoclient downloads it to a tmpdir | 10:08 |
| shardy | as you sometimes have references to e.g local paths in the rendered template | 10:08 |
| shardy | it's not ideal, but we had to do it like that because of interfaces in already released versions | 10:08 |
| bogdando | ok, that I got is totally normal if invoked arguments of overcloud deploy lead to non existant files, correct? | 10:08 |
| shardy | bogdando: yes, provided there's a corresponding .j2.yaml file | 10:09 |
| bogdando | got it | 10:09 |
| shardy | we should probably just add support to tripleoclient for referencing the j2.yaml directly | 10:09 |
| shardy | maybe that would make it clearer | 10:09 |
| bogdando | ok, I'll check if deployed-server networking is ok in a swift container | 10:09 |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 10:10 |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 10:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 10:10 |
| owalsh | is there an easy way to run a copy of a container (i.e with the same mounts etc...)? I need to step inside with -ti /bin/bash to debug | 10:11 |
| owalsh | I could figure out all of the volumes to mount but that's not fun :-( | 10:11 |
| shardy | owalsh: yes there's some ways to do that documented in https://docs.openstack.org/developer/tripleo-docs/containers_deployment/tips_tricks.html | 10:12 |
| owalsh | shardy: sweet thanks! the inspect trick is what I was looking for | 10:15 |
| *** sai has quit IRC | 10:16 | |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Fix inventory without existing stack https://review.openstack.org/469554 | 10:16 |
| *** lucasagomes has quit IRC | 10:18 | |
| *** rook has quit IRC | 10:18 | |
| *** florianf has quit IRC | 10:19 | |
| *** iranzo has joined #tripleo | 10:23 | |
| *** lucasagomes has joined #tripleo | 10:25 | |
| *** florianf has joined #tripleo | 10:25 | |
| *** gbarros has quit IRC | 10:28 | |
| *** rook has joined #tripleo | 10:33 | |
| *** rook is now known as Guest86464 | 10:33 | |
| cschwede | how can i ensure a container config is updated once there is a change in the cluster, for example when adding another node? I noticed that puppet won't be triggered for existing nodes in this case | 10:34 |
| *** sai has joined #tripleo | 10:34 | |
| *** egafford has joined #tripleo | 10:42 | |
| *** jpena is now known as jpena|lunch | 10:43 | |
| *** jkilpatr has quit IRC | 10:45 | |
| openstackgerrit | Lee Yarwood proposed openstack/tripleo-quickstart master: libvirt: Remove use of deprecated state=running https://review.openstack.org/469331 | 10:52 |
| *** pblaho has joined #tripleo | 10:58 | |
| shardy | cschwede: I've been working on some patches to address that: | 10:59 |
| shardy | https://review.openstack.org/#/c/467581/ | 10:59 |
| shardy | https://review.openstack.org/#/c/462211/ | 10:59 |
| shardy | cschwede: the idea is we have docker-puppet run every time, and restart the container if the CONFIG_HASH changes | 10:59 |
| shardy | we need one more patch after those I think to make docker-puppet re-run cleanly when generating the config | 11:00 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/docker/docker-puppet.py#L173 | 11:01 |
| jistr | owalsh: hi, i haven't seen that yet, does it cause trouble for the deployment? i saw the multinode container job passed on the virtlogd containerization patch, including a pingtest | 11:01 |
| shardy | that line is a problem because the mount is in use when we re-run docker puppet, it may be enough to just make the rm -Rf failure non-fatal e.g || echo "failed rm ..." | 11:01 |
| owalsh | jistr: no, pingtest works for me... well it did until I git devicemapper errors (I thought we switched to overlayfs2????) | 11:02 |
| owalsh | s/git/got/ | 11:02 |
| *** jkilpatr has joined #tripleo | 11:02 | |
| *** morazi has joined #tripleo | 11:02 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Custom containerized undercloud deployments https://review.openstack.org/450792 | 11:03 |
| cschwede | shardy: thx, that's interesting! so right now docker-puppet doesn't run every time? | 11:04 |
| cschwede | ie only on new nodes? | 11:04 |
| jistr | owalsh: yea we did but there's https://bugs.launchpad.net/tripleo/+bug/1692502 | 11:04 |
| openstack | Launchpad bug 1692502 in tripleo "Docker graph driver setting not honored" [High,In progress] - Assigned to Steve Baker (steve-stevebaker) | 11:04 |
| owalsh | jistr: ah, that explains it... | 11:07 |
| *** ansmith has joined #tripleo | 11:07 | |
| *** maeca1 has joined #tripleo | 11:07 | |
| *** maeca1 has quit IRC | 11:08 | |
| *** amoralej is now known as amoralej|lunch | 11:09 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 11:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 11:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 11:10 |
| *** faceman has joined #tripleo | 11:11 | |
| *** faceman has quit IRC | 11:12 | |
| *** faceman has joined #tripleo | 11:18 | |
| *** faceman has quit IRC | 11:18 | |
| *** lhinds has joined #tripleo | 11:19 | |
| *** zoli|lunch is now known as zoli | 11:19 | |
| *** lmiccini has quit IRC | 11:20 | |
| *** pliu has joined #tripleo | 11:20 | |
| *** rnoriega has joined #tripleo | 11:20 | |
| *** faceman has joined #tripleo | 11:23 | |
| shardy | cschwede: yes that's correct, GenerateConfigDeployment in docker-steps.j2 isn't currently triggered every update, my patches aim to correct that but there may be additional fixes needed to make it fully idempotent | 11:27 |
| saneax | bfournie, can you please review https://review.openstack.org/#/c/466693 | 11:28 |
| *** pgadiya has joined #tripleo | 11:28 | |
| cschwede | shardy: alright, i have some fixes for swift and i'll apply your patches on top of my dev env and will test them | 11:29 |
| shardy | cschwede: ack, thanks, any feedback and help validating the approach would be great :) | 11:29 |
| *** ratailor has quit IRC | 11:30 | |
| dtantsur | d0ugal: hi! sorry, another mistral question: how can I easily see a workflow error details without diving into logs? | 11:30 |
| dtantsur | d0ugal: 'workflow execution show' only displays State=ERROR, but no details | 11:30 |
| *** mdnadeem has quit IRC | 11:31 | |
| *** rhallisey has joined #tripleo | 11:31 | |
| ansiwen | mwhahaha: which releasenostes category would you suggest for https://review.openstack.org/#/c/465100/ ? | 11:32 |
| *** pkovar has joined #tripleo | 11:32 | |
| *** lmiccini has joined #tripleo | 11:32 | |
| *** skramaja_ has joined #tripleo | 11:33 | |
| *** skramaja has quit IRC | 11:33 | |
| *** udesale has quit IRC | 11:33 | |
| *** pgadiya has quit IRC | 11:34 | |
| *** psahoo has quit IRC | 11:35 | |
| *** prateek_ has quit IRC | 11:35 | |
| *** janki has quit IRC | 11:35 | |
| *** shardy has quit IRC | 11:36 | |
| *** links has quit IRC | 11:37 | |
| *** colonwq has quit IRC | 11:38 | |
| openstackgerrit | Sven Anderson proposed openstack/tripleo-heat-templates master: Increase default for NovaReservedHostMemory to 4096 https://review.openstack.org/465100 | 11:39 |
| ansiwen | mwhahaha: nevermind, I took "other" | 11:39 |
| Guest86464 | guess CI is really pissed off this Thursday. | 11:41 |
| Guest86464 | Oh, and I am a guest, what a day. | 11:41 |
| *** Guest86464 is now known as rook | 11:41 | |
| *** gbarros has joined #tripleo | 11:41 | |
| *** lucasagomes is now known as lucas-hungry | 11:41 | |
| *** maeca1 has joined #tripleo | 11:41 | |
| *** abishop has joined #tripleo | 11:44 | |
| *** colonwq has joined #tripleo | 11:44 | |
| *** adarazs is now known as adarazs_lunch | 11:44 | |
| hjensas | scale-down. With deployed-server there is no nova uuid for overcloud nodes. Doc's says run openstack overcloud node delete giving UUID of nodes to remove. How can we do it with split-stack? | 11:44 |
| *** pradk has joined #tripleo | 11:45 | |
| hjensas | there is a uuid is included in the swift container used to store hiera data. Would that be the uuid to use? | 11:45 |
| *** psahoo has joined #tripleo | 11:48 | |
| *** atoth has joined #tripleo | 11:49 | |
| *** jpena|lunch is now known as jpena | 11:50 | |
| *** liverpooler has quit IRC | 11:55 | |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates master: Containerized Sensu client https://review.openstack.org/464085 | 11:56 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Custom containerized undercloud deployments https://review.openstack.org/450792 | 11:58 |
| *** shardy has joined #tripleo | 11:58 | |
| *** skramaja_ has quit IRC | 11:59 | |
| *** skramaja has joined #tripleo | 11:59 | |
| *** zoli is now known as zoli|brb | 12:02 | |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-common master: Run also sensu-client containers https://review.openstack.org/469848 | 12:03 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates master: Containerized Sensu client https://review.openstack.org/464085 | 12:05 |
| bfournie | saneax: will review https://review.openstack.org/#/c/466693 this morning | 12:06 |
| saneax | thanks bfournie | 12:06 |
| *** bfournie has quit IRC | 12:07 | |
| *** slagle has joined #tripleo | 12:09 | |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 12:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 12:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 12:10 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Update metric processing delay default https://review.openstack.org/469215 | 12:12 |
| *** sid1 has quit IRC | 12:13 | |
| *** amoralej|lunch is now known as amoralej | 12:17 | |
| *** pchavva has joined #tripleo | 12:21 | |
| *** lucas-hungry is now known as lucasagomes | 12:22 | |
| *** ahrechan has quit IRC | 12:23 | |
| *** ykarel_ has quit IRC | 12:23 | |
| *** remixtj is now known as remix_tj | 12:25 | |
| *** sid1 has joined #tripleo | 12:26 | |
| *** pgadiya has joined #tripleo | 12:27 | |
| *** pgadiya has quit IRC | 12:27 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: [WIP] Add workflow for IPMI nodes discovery https://review.openstack.org/462577 | 12:28 |
| *** dsariel has quit IRC | 12:29 | |
| *** artom has quit IRC | 12:35 | |
| *** zoli|brb is now known as zoli | 12:35 | |
| *** tbonds has joined #tripleo | 12:36 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Make network-isolation environment rendered for all roles https://review.openstack.org/409920 | 12:37 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Convert role templates to consume roles_data map https://review.openstack.org/469853 | 12:37 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Modify generic role template to support custom networks https://review.openstack.org/469854 | 12:37 |
| weshay | EmilienM, is there a bug open on the pip install errors? | 12:37 |
| weshay | sec | 12:38 |
| shardy | Can someone check https://review.openstack.org/#/c/465625/ please? Simple docs patch | 12:39 |
| EmilienM | weshay: which one? | 12:40 |
| weshay | ImportError: No module named six | 12:40 |
| weshay | http://logs.openstack.org/31/469331/2/check/gate-tripleo-ci-centos-7-nonha-multinode-oooq/7f7946f/console.html | 12:41 |
| *** hewbrocca is now known as hewbrocca_afk | 12:41 | |
| jistr | that's a global bug, half a world is broken | 12:41 |
| trown | seems like openstack wide issue with setuptools | 12:41 |
| trown | ya | 12:41 |
| jistr | https://github.com/pypa/setuptools/issues/1042 | 12:41 |
| *** dprince has joined #tripleo | 12:41 | |
| trown | is it friday yet? | 12:41 |
| jistr | :D | 12:42 |
| jistr | just looking at the PRs that reference the issue -- airbnb, mozilla-services, ... | 12:42 |
| jistr | and that's just the ones that are on github && public | 12:42 |
| *** bfournie has joined #tripleo | 12:43 | |
| openstackgerrit | Merged openstack/tripleo-docs master: Reference SoftwareDeploymentGroup resource https://review.openstack.org/465625 | 12:44 |
| *** hewbrocca_afk is now known as hewbrocca | 12:45 | |
| *** jmelvin has joined #tripleo | 12:47 | |
| *** eck`gone is now known as eck` | 12:48 | |
| *** janki has joined #tripleo | 12:49 | |
| *** panda|pnq is now known as panda | 12:51 | |
| *** panda is now known as panda|travel | 12:51 | |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart master: Exclude broken setuptools release https://review.openstack.org/469857 | 12:51 |
| *** jmelvin has quit IRC | 12:51 | |
| *** jmelvin has joined #tripleo | 12:51 | |
| trown | weshay: it does not reproduce locally, because setuptools removed the busted release from pypi, maybe ^ will fix our gates? | 12:52 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart master: Exclude broken setuptools release https://review.openstack.org/469857 | 12:53 |
| *** rbowen has joined #tripleo | 12:53 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-docs master: Add documentation for role-specific parameters support https://review.openstack.org/465028 | 12:57 |
| *** zaneb has quit IRC | 12:58 | |
| *** ansiwen is now known as ansiwen|lunch | 12:59 | |
| weshay | mwhahaha, I have the tripleo-ui working w/ ssl now btw :) | 12:59 |
| mwhahaha | weshay: nice with your replaceline thing or did you figure it out? | 13:00 |
| weshay | it was just a self-signed cert issue | 13:00 |
| mwhahaha | ah | 13:00 |
| *** gbarros has quit IRC | 13:00 | |
| weshay | I have instructions in the doc on how to use both | 13:00 |
| *** liverpooler has joined #tripleo | 13:03 | |
| *** psahoo has quit IRC | 13:03 | |
| *** Goneri has joined #tripleo | 13:09 | |
| chandankumar | trown: you need to check .tox/py27/log/py27-1 logs there you will find the error | 13:09 |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 13:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 13:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 13:10 |
| chandankumar | trown: in RDO we have added a workaround to overcome it | 13:10 |
| chandankumar | trown: https://softwarefactory-project.io/r/#/c/8323/ | 13:10 |
| *** adarazs_lunch is now known as adarazs | 13:12 | |
| *** zaneb has joined #tripleo | 13:13 | |
| *** skramaja has quit IRC | 13:17 | |
| *** dsariel has joined #tripleo | 13:19 | |
| *** karthiks has quit IRC | 13:20 | |
| *** masco has quit IRC | 13:22 | |
| *** gbarros has joined #tripleo | 13:23 | |
| adarazs | the quickstart pip install is working again \o/ | 13:24 |
| *** tzumainn has joined #tripleo | 13:24 | |
| *** agopi has joined #tripleo | 13:24 | |
| *** agopi has quit IRC | 13:25 | |
| *** agopi has joined #tripleo | 13:26 | |
| trozet | dprince: hi | 13:27 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: Explicitly use bash for pipefail shell blocks https://review.openstack.org/466316 | 13:27 |
| *** rhallisey has quit IRC | 13:30 | |
| *** rhallisey has joined #tripleo | 13:30 | |
| dprince | trozet: hi | 13:31 |
| trozet | dprince: so made some progress, and looks like i got up to now where the container config files are generated | 13:32 |
| trozet | dprince: it failed for opendaylight immediately: | 13:32 |
| trozet | 2017-05-31 16:47:26,875 INFO: Removing container: docker-puppet-neutron | 13:33 |
| trozet | 2017-05-31 16:47:27,240 DEBUG: docker-puppet-neutron | 13:33 |
| trozet | 13:33 | |
| trozet | 2017-05-31 16:47:27,241 ERROR: ERROR configuring opendaylight | 13:33 |
| trozet | deploy_stderr: | | 13:33 |
| trozet | dprince: it doesnt show any output, and it looks like the container instance is gone on the controller | 13:33 |
| trozet | dprince: how do you normally go about debugging something like this? | 13:33 |
| dprince | trozet: I would run docker-puppet.py manually for this | 13:34 |
| dprince | trozet: on your overcloud there is a file called /var/lib/docker-puppet/docker-puppet.json | 13:34 |
| dprince | trozet: it has in it all the data to generate config files. Heat creates this via Ansible (and some yaql) | 13:35 |
| jaosorior | mandre: still around? | 13:35 |
| dprince | trozet: if you download this file to your overcloud node http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/docker/docker-puppet.py | 13:36 |
| dprince | trozet: you can just run it with the opendailight subsection | 13:36 |
| *** limao has joined #tripleo | 13:36 | |
| trozet | dprince: ok thanks I will try that. There is one more thing I noticed looking at my heat template | 13:36 |
| dprince | trozet: take /var/lib/docker-puppet/docker-puppet.json, slice out the opendaylight bits. Write a new json file: x.json (or whatever) | 13:36 |
| dprince | trozet: then you can run it with CONFIG=x.json docker-puppet.py | 13:37 |
| dprince | trozet: okay, go ahead | 13:37 |
| trozet | dprince: for neutron, I see service_config_settings: {get_attr: [NeutronBase, role_data, service_config_settings]} | 13:37 |
| *** dsariel has quit IRC | 13:37 | |
| shardy | dprince: Hey that reminds me, I have a question about re-running docker-puppet to refresh config | 13:38 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/docker/docker-puppet.py#L173 | 13:38 |
| *** limao_ has joined #tripleo | 13:38 | |
| *** limao_ has quit IRC | 13:38 | |
| shardy | in that case, it fails because the /var/lib/config-data/foo is in-use, e.g the container is still running | 13:38 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: remove outdated playbook, tripleo-roles.yml https://review.openstack.org/469864 | 13:38 |
| *** artom has joined #tripleo | 13:38 | |
| shardy | dprince: I planned to just make that rm tolerate failure, does that sound reasonable? | 13:38 |
| *** limao_ has joined #tripleo | 13:38 | |
| shardy | dprince: then I checksum the config-data/foo directory and use the hash as a salt to trigger restarting the container via paunch | 13:39 |
| trozet | dprince: i just added that to my opendaylight one, but now looking back at the regular opendaylight api heat template it doesnt have that variable | 13:39 |
| shardy | https://review.openstack.org/#/c/467581/ | 13:39 |
| trozet | dprince: so is that required? or should I just remove it | 13:39 |
| dprince | shardy: I think that sounds okay, but we might also consider using unique directories for config data too so that (should the original container persist, and get restarted somehow it doesn't fail) | 13:40 |
| shardy | dprince: yeah I wondered about that, but overwriting the existing one seemed a little simpler | 13:40 |
| dprince | trozet: I don't think you need NeutronBase | 13:40 |
| shardy | and we then immediately restart the container | 13:40 |
| trozet | dprince: no i dont mean neutron base | 13:40 |
| shardy | but only when the config has changed | 13:40 |
| *** limao has quit IRC | 13:40 | |
| *** trown is now known as trown|brb | 13:41 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Add workflow for IPMI nodes discovery https://review.openstack.org/462577 | 13:41 |
| trozet | dprince: i mean the 'service_config_settings' variable itself, does that need to be set in the container template | 13:41 |
| dprince | trozet: this was meant to allow the docker/service templates to "extend" the puppet/templates | 13:41 |
| dprince | trozet: so if OpenDaylight has service_config_settings. Just add that to your docker/service/ templates (with the ODL param versions) | 13:41 |
| dprince | trozet: it won't hurt if it is empty either | 13:42 |
| dprince | trozet: in puppet/services if it is empty, it won't hurt to extend it I mean | 13:42 |
| dprince | shardy: looking at your review | 13:42 |
| trozet | dprince: ok, perhaps that could be the reason my stuff just errored out. Let me try to fix that and follow what you said for debugging. thanks! | 13:43 |
| shardy | dprince: ack, thanks, open to other suggestions! | 13:43 |
| dprince | trozet: fwiw, I now do all of my first pass t-h-t work via the new 'undercloud deploy' (experimental) installer | 13:44 |
| *** artom has quit IRC | 13:44 | |
| dprince | trozet: it is much faster to get the basic config stuff right in a single node ENV like that | 13:44 |
| *** artom has joined #tripleo | 13:45 | |
| dprince | trozet: have you seen this yet? | 13:45 |
| trozet | dprince: even if the service you are using wouldn't normally run on undercloud? | 13:45 |
| dprince | trozet: the idea is "anything is an undercloud" sort of | 13:45 |
| dprince | trozet: perhaps we should stop calling it undercloud, and just have that be a config file this tool can consume for clarity | 13:45 |
| trozet | dprince: im using the container minimal oooqs now, ive got a good setup I think...hopefully it doesnt go away :) | 13:45 |
| dprince | trozet: totally fine to stay in your comfort zone :) | 13:46 |
| shardy | trozet: I heard you were working on making oooqs work with root libvirt, so the VMs can persist over reboot? | 13:46 |
| shardy | that's a major oooqs annoyance for me atm | 13:46 |
| *** ykarel has joined #tripleo | 13:47 | |
| shardy | I don't care about the unpriveleged virt stuff at all, but I do want to shut down the host from time to time and be able to pick up where I left my work | 13:47 |
| trozet | shardy: yeah and use libvirt system | 13:47 |
| shardy | trozet: cool, got any patch I can try? | 13:47 |
| trozet | shardy: i got that working on this setup, so i can connect to the vms from my laptop | 13:47 |
| trozet | shardy: i can tell you what to change, i dont have any patch posted yet | 13:47 |
| shardy | trozet: ack, any info appreciated | 13:48 |
| *** gbarros has quit IRC | 13:50 | |
| *** lblanchard has joined #tripleo | 13:51 | |
| dprince | shardy: +2 on your config-data checksum patch. I can't help much w/ the ansible foo you commented on though | 13:51 |
| *** lblanchard has quit IRC | 13:51 | |
| shardy | dprince: ack thanks - we can always revisit if versioned directories seems better later | 13:52 |
| shardy | dprince: yeah, I'll find some way to clean that up, but it works for now | 13:52 |
| trozet | shardy: actually my setup is still using stack user, i just changed the directories where he stores stuff cause my /home partition is really small, and changed the VMs to run under system, and you can access those as root | 13:52 |
| *** lblanchard has joined #tripleo | 13:52 | |
| trozet | shardy: michapma had done the work to run oooqs as root i think | 13:52 |
| trozet | shardy: but i can provide the changes to get the VMs to run under system, which might be what you need | 13:53 |
| *** yamahata has joined #tripleo | 13:53 | |
| shardy | trozet: yeah I tried to run it as root and it doesn't work, but I'm possibly missing something: | 13:53 |
| shardy | https://bugs.launchpad.net/tripleo/+bug/1692987 | 13:53 |
| openstack | Launchpad bug 1692987 in tripleo "tripleo-quickstart deploying VMs as root user doesn't work" [High,Triaged] | 13:53 |
| shardy | trozet: yes, please, I'll certainly try it | 13:53 |
| *** jkilpatr has quit IRC | 13:53 | |
| shardy | I just want the VMs to be visible under virt-manager as the user I log in as, and still be there after a reboot of the host | 13:54 |
| rbrady | fultonj: reviewed the patch | 13:54 |
| *** janki has quit IRC | 13:54 | |
| shardy | I don't really understand how folks are developing without that capability tbh, but I guess everyone works differently | 13:54 |
| *** atheurer has quit IRC | 13:54 | |
| *** janki has joined #tripleo | 13:54 | |
| fultonj | rbrady: thanks, i will respond. | 13:55 |
| *** atheurer has joined #tripleo | 13:55 | |
| *** jkilpatr has joined #tripleo | 13:55 | |
| *** jaganathan has quit IRC | 13:55 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: collect-logs: add README for log files https://review.openstack.org/456317 | 13:55 |
| michapma | shardy: set ssh_user to root and libvirt_uri to qemu:///system | 13:57 |
| openstackgerrit | Sanjay Upadhyay proposed openstack/tripleo-heat-templates master: Role Specific parameters for neutron-sriov-agent service https://review.openstack.org/469464 | 13:58 |
| adarazs | EmilienM, rbrady, trown|brb, weshay, hrybacki, bkero, arxcruz, bogdando: CI Squad meeting in half an hour! add/remove/edit agenda items at https://etherpad.openstack.org/p/tripleo-ci-squad-meeting :) | 13:59 |
| trozet | shardy: https://gist.githubusercontent.com/trozet/41b50516153cb27acb4c43b15095b3f2/raw/c53aff084e4d6f57ef5132c57b63bf3b0d6205d2/oooqs_changes.txt | 13:59 |
| adarazs | rbrady, hrybacki: should I remove you from the squad/pinglist? I guess it's not necessary to bother you weekly if you're not attending. :) | 14:00 |
| michapma | shardy: the idea was to have the user that runs the commands be root, but still use the stack user's home directory to store images etc. since I wanted to avoid cleanup doing anything to the root user or the root user's files | 14:00 |
| rbrady | adarazs: I did just remove myself. Its something I wanted to contribute to, but not something I'm finding time for | 14:00 |
| trozet | shardy: i think there might have been a couple more changes...but i cant remember them. Let me know what you run into and im sure it will spark my memory | 14:01 |
| *** limao_ has quit IRC | 14:01 | |
| adarazs | rbrady: sure thing, I understand. :) | 14:01 |
| hrybacki | adarazs: I was actually going to attend today :P | 14:01 |
| openstackgerrit | John Fulton proposed openstack/tripleo-common master: Add Mistral workbook to call ceph-ansible to install Ceph https://review.openstack.org/469644 | 14:02 |
| adarazs | hrybacki: good to hear that :) | 14:02 |
| *** janki has quit IRC | 14:02 | |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart master: Adding featureset for periodic tempest jobs https://review.openstack.org/464598 | 14:02 |
| hrybacki | adarazs: do you know when sshnaidm get's back? | 14:02 |
| adarazs | hrybacki: Sunday I think. | 14:02 |
| mandre | bnemec: hi! we've started to see disk space issue in the containers CI, do you think it's possible to bump the undercloud disk space in tripleo cloud? | 14:03 |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart master: Adding featureset for periodic tempest jobs https://review.openstack.org/464598 | 14:04 |
| mandre | bnemec: maybe add another flavor with larger disk that we can use in containers job? | 14:04 |
| hrybacki | ack, thanks | 14:04 |
| trozet | dprince: does this look OK to you? Is that the right format to have multiple step configs? https://paste.fedoraproject.org/paste/SJyeyowSnl8RoQ4LKR94kF5M1UNdIGYhyRLivL9gydE=/raw | 14:05 |
| adarazs | hrybacki: despite what the company calendar says it's still a holiday in Israel today. /o\ | 14:06 |
| *** trown|brb is now known as trown | 14:07 | |
| dprince | trozet: the whole blob needs to be in a [] I think | 14:07 |
| dprince | trozet: also remove the last , | 14:07 |
| dprince | trozet: then its valid JSON and you can just run it w/ CONFIG=<your json filename> docker-puppet.py | 14:07 |
| trozet | dprince: hmm ok, let me try manipulating it and running docker-puppet.py | 14:07 |
| trozet | dprince: thanks! | 14:08 |
| bnemec | mandre: I'm not sure and I can't check right now since there are basically no jobs running, but I can say that we are already heavily overcommitted on disk in rh1. | 14:08 |
| dprince | bnemec: we could just bump the disk space on the existing UC flavor | 14:08 |
| bnemec | Even if jobs are going to start using the full amount of space that they've been allocated we may run into issues. | 14:08 |
| dprince | bnemec: we aren't super constrained on disk are we? | 14:08 |
| bnemec | dprince: Yes, very. | 14:09 |
| dprince | bnemec: I thought it was mostly memory | 14:09 |
| bnemec | Each compute node has a 200 GB SSD for instance storage. | 14:09 |
| bnemec | We were running somewhere north of 60% on most of them even before containers. | 14:09 |
| dprince | bnemec: right, I guess I forget the SSD cost us a lot of extra space there | 14:09 |
| bnemec | We've had to set a 4x overcommit just to be able to deploy the testenvs. | 14:09 |
| bnemec | Yeah, at the time it wasn't a problem. | 14:09 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 14:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 14:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 14:10 |
| shardy | trozet, michapma: ack thanks for the info, will give it a go | 14:10 |
| bnemec | Given how much we overcommit right now, I'd be concerned that once all the jobs are using containers we might run into problems even at the current flavor size. | 14:10 |
| bnemec | Our existing test setup assumes most jobs will only use a fraction of their allocated disk. | 14:10 |
| bnemec | But I need to look again once we're actually running jobs. | 14:11 |
| *** pkovar has quit IRC | 14:11 | |
| bnemec | I haven't checked into our disk usage in quite a while. | 14:11 |
| * bnemec notes that this would be an excellent time to reboot all the things again | 14:11 | |
| trown | bnemec: dprince mandre... undercloud runs on the nodepool node though | 14:12 |
| owalsh | prad: hi, AFAICT the containerized compute ceilometer agent needs to map /var/run/libvirt (ro) | 14:14 |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Provide commands for example roles https://review.openstack.org/466900 | 14:14 |
| *** limao has joined #tripleo | 14:15 | |
| trown | adarazs: can we merge https://review.openstack.org/456225 ? I get at least 1 "Why are packages broken?" question per day | 14:15 |
| bnemec | trown: Which runs in rh1. | 14:16 |
| mandre | bnemec: I see, I didn't know we were so much constrained on rh1 | 14:16 |
| owalsh | pradk: ^^^ are you prad too? | 14:16 |
| bnemec | It's just the vm that nodepool manages instead of our te-broker. | 14:16 |
| bnemec | mandre: I'm not saying a definite no. I just can't answer at this point since the environment is completely unloaded. | 14:17 |
| trown | bnemec: oh... I never got that part... that makes more sense why nested virt was working fine for us until infra disabled it globally | 14:17 |
| adarazs | trown: sure. I thought we merged this a while ago o_o | 14:17 |
| pradk | owalsh, probably since we talk to libvirt api | 14:17 |
| trown | adarazs: ya.. every time I get the question I think the same :P | 14:17 |
| bnemec | And I haven't been keeping track of the disk usage in rh1 closely enough to know off the top of my head. | 14:17 |
| pradk | owalsh, seeing any issues? | 14:17 |
| dprince | bnemec: I think we can bump the UC flavor by 1 Gig disk and see if it helps us | 14:17 |
| dprince | bnemec: would you be opposed to trying that? | 14:18 |
| owalsh | pradk: yea, libvirtd logging "End of file while reading data: Input/Output Error" | 14:18 |
| *** dsariel has joined #tripleo | 14:18 | |
| owalsh | pradk: appears to have stopped when I stopped the ceilometer agent container | 14:18 |
| pradk | hmm i hvae no idea who other prad is | 14:18 |
| dprince | pradk: ha! you've got a stalker | 14:19 |
| pradk | owalsh, i see, ok i'll take a look today | 14:19 |
| mandre | bnemec, dprince: we can even restrict it to just the containers jobs if we create a new undercloud-containers flavor | 14:19 |
| pradk | dprince, lol | 14:19 |
| dprince | mandre: good point, more work that way though I think | 14:19 |
| bnemec | dprince: If you've blown through 40 gig, is 1 more going to help that much? | 14:19 |
| bnemec | Won't we just be having this discussion again in a week? | 14:19 |
| dprince | bnemec: they images should all get layered on top of the Centos base | 14:20 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-common master: Start using generic dracut-regenerate element https://review.openstack.org/469874 | 14:20 |
| dprince | bnemec: my docker registry (hosting all the containers) is < 5 Gigs | 14:21 |
| mandre | dprince, bnemec: uh I don't know about that, I think we may have one image for ceph that doesn't share any of the layers | 14:21 |
| dprince | bnemec: so something else is clearly taking up some of that space | 14:21 |
| dprince | mandre: Ceph containers isn't integrated into CI yet is it? | 14:21 |
| dprince | mandre: I thought it was still on BM | 14:21 |
| mandre | dprince: there is a pending patch I think | 14:22 |
| bnemec | dprince: If you think 1 GB is going to make a difference then I'm fine with that. I doubt it's going to make or break our disk usage. | 14:22 |
| dprince | mandre: yes, so your point will soon be valid but not yet | 14:22 |
| dprince | mandre: 1 Gig? I think it should be plenty for now | 14:22 |
| mandre | bnemec: 1GB will definitely help | 14:23 |
| trozet | dprince: + /usr/bin/puppet apply --verbose --tags file,file_line,concat,augeas,odl_user,tripleo::profile::base::neutron::opendaylight::configure_cluster /etc/config.pp | 14:24 |
| trozet | /var/lib/docker-puppet/docker-puppet.sh: line 11: /usr/bin/puppet: No such file or directory | 14:24 |
| *** pkovar has joined #tripleo | 14:25 | |
| dprince | mandre: I'm looking at the flavor now | 14:25 |
| *** ykarel_ has joined #tripleo | 14:26 | |
| dprince | mandre: per your patch I commented here: https://review.openstack.org/#/c/469807/1 | 14:26 |
| dprince | mandre: two things (unless I misunderstand) | 14:26 |
| trozet | dprince: does that expect puppet to be installed in the opendaylight container? | 14:26 |
| dprince | mandre: the infra "registry" is a caching mirror I think. Not a registry | 14:27 |
| dprince | mandre: if we want to use that, puppet-tripleo supports it. But we'd have to wire it into t-h-t parameters I think to set the hiera settings | 14:27 |
| atoth | shardy, EmilienM, beagles, still looking for reviewers on https://review.openstack.org/#/c/314225 to move LBaaSV2 forward | 14:28 |
| *** ykarel has quit IRC | 14:28 | |
| hrybacki | adarazs: are we using weshay's bluejeans? | 14:28 |
| adarazs | hrybacki: yep. | 14:28 |
| weshay | oh man.. don't let that guy join :) | 14:28 |
| *** jprovazn has quit IRC | 14:29 | |
| beagles | atoth: ack will get to it asap | 14:29 |
| atoth | beagles, thanks | 14:29 |
| trozet | dprince: i think i built the container locally without the template override so I am missing puppet int eh container... | 14:29 |
| shardy | atoth: ack thanks for the reminder, lgtm | 14:30 |
| shardy | atoth: if you do another rebase you might like to add yourself as Co-Authored-By in the commit message :) | 14:30 |
| adarazs | bkero: here? | 14:31 |
| dprince | trozet: the puppet modules are bind mounted in from the host | 14:31 |
| dprince | trozet: so long as they are on the overcloud they would get mounted in | 14:31 |
| trozet | dprince: yeah it doesnt have the puppet package though | 14:31 |
| dprince | trozet: you could check it out into /etc/puppet/modules in your host | 14:31 |
| dprince | trozet: and then it would show up | 14:31 |
| dprince | mandre: bumping the flavor for the UC now | 14:31 |
| trozet | dprince: no i mean like the puppet rpm itself i think is missing | 14:32 |
| *** limao has quit IRC | 14:32 | |
| atoth | shardy, thanks, didn't know Co-Authored-By was a thing :-) | 14:32 |
| *** limao has joined #tripleo | 14:32 | |
| *** flepied has quit IRC | 14:34 | |
| *** openstackgerrit has quit IRC | 14:34 | |
| *** oidgar has quit IRC | 14:34 | |
| -openstackstatus- NOTICE: python-setuptools 36.0.1 has been released and now making its way into jobs. Feel free to 'recheck' your failures. If you have any problems, please join #openstack-infra | 14:35 | |
| *** openstackgerrit has joined #tripleo | 14:35 | |
| openstackgerrit | Jacob Liberman proposed openstack/puppet-tripleo master: Add conditional for setting authlogin_nsswitch_use_ldap selboolean if selinux is enabled https://review.openstack.org/469877 | 14:35 |
| beagles | atoth: what's the deal with the pacemaker profile for lbaas? Is there a discussion somewhere on why this isn't handled via systemd like the other services? | 14:36 |
| *** jaosorior has quit IRC | 14:36 | |
| atoth | beagles, forgot about that comment on your last review, thanks for bringing it up. I didn't originate that file so I'm not sure why it's there. Again my tag line "new to all this" looks relevant ;-) | 14:38 |
| beagles | :) | 14:38 |
| atoth | beagles, so what is the "right" thing to do? | 14:38 |
| atoth | beagles, is there a different file needed for it? | 14:38 |
| beagles | atoth: pacemaker is only used for a small subset of services | 14:39 |
| beagles | atoth: everything else is just a systemd kind-of-thing | 14:39 |
| beagles | atoth: I'm trying to think of a reason why lbaas would be different | 14:39 |
| atoth | beagles, and no other file necessary for systemd then? | 14:39 |
| beagles | atoth: mmm.. nope shouldn't be - as long as puppet-neutron has the right bits and the packaging installs a service file you should be good | 14:41 |
| *** ramishra has quit IRC | 14:41 | |
| *** karthiks has joined #tripleo | 14:43 | |
| atoth | beagles, thanks for the lesson :-) | 14:43 |
| atoth | shardy, any reaction to taking out the pacemaker file as recommended by beagles? | 14:45 |
| *** hjensas has quit IRC | 14:45 | |
| *** ramishra has joined #tripleo | 14:47 | |
| dprince | bnemec, mandre: http://paste.openstack.org/show/611215/ | 14:47 |
| dprince | I think a flavor update isn't going to work due to our policy. Are we okay to (quickly) delete and re-create the same named flavor? | 14:48 |
| bnemec | dprince: Yeah, that's how we've done it in the past. The flavors are referenced by name, so it shouldn't be a problem. | 14:49 |
| dprince | bnemec: ack | 14:49 |
| dprince | bnemec: I just quickly looked at the nodepool provider_manager.py code and that seems to follow | 14:49 |
| *** dmacpher has joined #tripleo | 14:50 | |
| *** hjensas has joined #tripleo | 14:50 | |
| *** hjensas has quit IRC | 14:50 | |
| *** hjensas has joined #tripleo | 14:50 | |
| *** lmiccini has quit IRC | 14:51 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: WIP add docker template for octavia services https://review.openstack.org/467254 | 14:51 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: DO NOT MERGE: testing octavia containers https://review.openstack.org/469392 | 14:51 |
| *** lmiccini has joined #tripleo | 14:51 | |
| *** tbonds has quit IRC | 14:51 | |
| *** flepied has joined #tripleo | 14:52 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Custom containerized undercloud deployments https://review.openstack.org/450792 | 14:53 |
| *** tbonds has joined #tripleo | 14:53 | |
| atoth | Alex Schultz on this channel? | 14:55 |
| shardy | atoth: he's mwhahaha - will lbaas be managed by pacemaker? | 14:56 |
| shardy | if not, the pacemaker profile can be removed | 14:56 |
| shardy | in general we've moved away from pacemaker manging most active-active capable services, instead having systemd restart them | 14:56 |
| atoth | ah that's right, want to make sure I know everyones handle :-) | 14:56 |
| * mwhahaha waves | 14:57 | |
| atoth | shardy, thanks. I'm not sure how one determines if pacemaker should control a service or not. | 14:57 |
| * atoth waves back to mwhahaha | 14:58 | |
| mwhahaha | i think it depends on the service, pacemaker should only be used when needing some sort of HA | 14:58 |
| shardy | atoth: if it can run active/active vs needing a managed failover, you probably don't need/want pacemaker to manage it | 14:58 |
| mwhahaha | if the service can just be managed by systemd all the better, but if the operator needs it to always be running somewhere then pacemaker would be that | 14:58 |
| *** jkilpatr has quit IRC | 14:59 | |
| atoth | so generic LBaaS may not need it but Partner implementations may | 14:59 |
| *** yamahata has quit IRC | 15:00 | |
| shardy | atoth: Ok, I'd focus on the non-pacemaker case, and perhaps follow-up with a patch removing the pacemaker profile if it becomes clear it won't be needed | 15:00 |
| dtrainor | Anyone ever notice > 900 haproxy processes in a zombie state after a while? I've seen that a few times now. | 15:01 |
| atoth | shardy, so keep the file for now? | 15:03 |
| bogdando | mwhahaha, atoth: let me rephrase as pacemaker should only be used when needing some sort of inter-node coordination | 15:03 |
| dtrainor | I see no logs to indicate anything happened pertaining to the haproxy service that would cause this. I know in my past experience with haproxy that sometimes the old process never goes away when haproxy is restarted | 15:03 |
| shardy | atoth: well it doesn't hurt anything, but please remove it if your t-h-t patch doesn't end up using it (which I suspect it won't) | 15:03 |
| atoth | bogdando, great tidbit, thanks | 15:04 |
| bogdando | which goes beyond of the inner-node systemd responsibilities | 15:04 |
| atoth | shardy, ok, and partner patches will be totally different so they could have one if needed, correct? | 15:04 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Custom containerized undercloud deployments https://review.openstack.org/450792 | 15:06 |
| *** jmelvin_ has joined #tripleo | 15:06 | |
| dprince | mandre, bnemec: undercloud flavor is bumped | 15:06 |
| bogdando | ok, I simlified the custom undercloud patch as much as possible | 15:07 |
| bogdando | simplyfied* | 15:07 |
| bogdando | aargh... simplified** | 15:08 |
| *** hewbrocca is now known as hewbrocca_afk | 15:08 | |
| bogdando | so PTAL :) | 15:08 |
| bnemec | dprince: Can you send a patch to http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/prepare-ovb-cloud.sh#n13 to reflect that? | 15:08 |
| dprince | bnemec: sure. np | 15:08 |
| *** jmelvin has quit IRC | 15:09 | |
| *** jkilpatr has joined #tripleo | 15:09 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Add docker template for octavia-api service https://review.openstack.org/467254 | 15:09 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: DO NOT MERGE: testing octavia containers https://review.openstack.org/469392 | 15:09 |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 15:10 |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 15:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 15:10 |
| openstackgerrit | Dan Prince proposed openstack-infra/tripleo-ci master: Bump undercloud flavor to disk 41 https://review.openstack.org/469917 | 15:11 |
| *** iranzo has quit IRC | 15:12 | |
| dprince | bnemec, mandre: patch is there to reflect the disk bump | 15:12 |
| dprince | ^^^ | 15:12 |
| bnemec | dprince: Thanks | 15:12 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: When enrolling Dell nodes, use drac_address instead of deprecated drac_host https://review.openstack.org/469918 | 15:13 |
| mandre | dprince: ack, thanks | 15:13 |
| shardy | atoth: yes, but I'd lean towards removing it, as I suspect vendor backends won't influence the HA failover of the service itself | 15:13 |
| *** limao has quit IRC | 15:13 | |
| shardy | but we can always discuss that when there's a vendor integration under review | 15:14 |
| dprince | mandre: pending other CI fixes I think we should make it a bit further now with containers | 15:14 |
| *** limao has joined #tripleo | 15:14 | |
| *** aufi has quit IRC | 15:15 | |
| *** salmankhan has quit IRC | 15:15 | |
| owalsh | dprince: hola, got a second to help me out of permissions hell? | 15:17 |
| dprince | owalsh: hola, whats up? | 15:18 |
| shardy | chem: Hey I noticed a problem: | 15:18 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/neutron-l3.yaml | 15:18 |
| shardy | the docker l3 agent template doesn't have any upgrade_tasks | 15:18 |
| shardy | should look like: | 15:18 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/neutron-api.yaml#L164 | 15:19 |
| shardy | chem: you'll probably need to add that or we won't stop the process on the host | 15:19 |
| chem | shardy: ack, thanks, looking | 15:19 |
| owalsh | dprince: so I have ssh host keys on the host -rw-r-----. 1 root ssh_keys 1675 Jun 1 12:33 /etc/ssh/ssh_host_rsa_key | 15:19 |
| *** mcornea has quit IRC | 15:19 | |
| owalsh | dprince: I have /etc/ssh mapped ro to a container to run sshd | 15:20 |
| *** ebarrera has quit IRC | 15:20 | |
| owalsh | dprince: but as the uids/gids are different: -rw-r-----. 1 root input 1675 Jun 1 12:33 /etc/ssh/ssh_host_rsa_key | 15:21 |
| *** florianf has quit IRC | 15:21 | |
| owalsh | dprince: and sshd ignores it as a result | 15:21 |
| atoth | shardy, perfect, thanks | 15:21 |
| *** sid2 has joined #tripleo | 15:22 | |
| owalsh | dprince: instead of mapping these files as a docker volume can I tell kolla to copy and chown? | 15:22 |
| dprince | owalsh: okay, to deal with different uuid's on host/container we've been using a combination of the kolla_config sections 'permissions' and an ad-hoc docker_cmd container to create/and or copy data from one place to another | 15:22 |
| *** sid1 has quit IRC | 15:22 | |
| dprince | owalsh: you could bind mound /etc/ssh into the container as /etc/ssh_tmp. | 15:22 |
| dprince | owalsh: then have a docker-cmd copy it to /etc/ssh and chown it as root:root accordingly | 15:23 |
| dprince | owalsh: thus avoiding permissions issues I think. I think this would be fine for selinux as well eventually | 15:23 |
| dprince | owalsh: see the 'mkdir' and permissions stuff in the existing services and I think you'll have examples | 15:24 |
| dprince | owalsh: you could probably 'copy && chown' in the same command too to avoid the extra complexity around permissions w/ kolla_config if you wnated to | 15:24 |
| dprince | owalsh: either way works I think | 15:24 |
| dprince | owalsh: make sense? | 15:25 |
| owalsh | dprince: cool, yea, should work for this one | 15:25 |
| owalsh | dprince: the other issue is trickier... | 15:25 |
| owalsh | dprince: polkitd rule on the host stops the container nova user accessing the libvirtd socket.... again because the uids do not match | 15:26 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Add workflow for IPMI nodes discovery https://review.openstack.org/462577 | 15:26 |
| chem | shardy: oki, I've captured that there https://bugs.launchpad.net/tripleo/+bug/1695017, thanks for the heads up. | 15:26 |
| openstack | Launchpad bug 1695017 in tripleo "docker l3 agent template doesn't have any upgrade_tasks." [Critical,Triaged] | 15:26 |
| dprince | owalsh: oh. that is tricky | 15:27 |
| owalsh | dprince: makes me wonder why don't we override all of the default kolla uid/gids? | 15:27 |
| dprince | mandre: per owalsh's questions above. Are we overriding the uuid's in the RDO (and downstream) containers too? | 15:28 |
| dprince | owalsh: yeah, this is a problem. lets see what mandre has to say | 15:28 |
| * mandre reads the question | 15:29 | |
| *** ansiwen|lunch is now known as ansiwen | 15:29 | |
| owalsh | dprince: the bigger issue is that this means nova-compute depends on the host config, which is horrible... | 15:29 |
| dprince | owalsh: I think we could look into disable Kolla's uuid generation. The reason they did that though was to allow "package" and "source" containers to have the same UUIds, which is a useful thing if you wanted to swap them out for some reason | 15:29 |
| dprince | owalsh: which package installs the polkit rule? nova-compute? | 15:30 |
| owalsh | dprince: yes | 15:30 |
| dprince | owalsh: and would our upgrades (eventually) remove that package on the host? | 15:30 |
| *** cylopez has quit IRC | 15:30 | |
| owalsh | dprince: yup | 15:30 |
| dprince | owalsh: if so, I think we might be okay here | 15:30 |
| owalsh | dprince: don't think so, unless we run nova as root... | 15:31 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Stop/disable l3 agent in docker service upgrade_tasks https://review.openstack.org/469924 | 15:31 |
| ansiwen | dprince: another option is to create the user as you need it when starting the container. I did that when I didn't know yet which user owns a bind-mount-directory. | 15:31 |
| shardy | chem: ^^ that should fix it I think | 15:31 |
| ansiwen | dprince: not sure if this is feasible in this case though | 15:32 |
| chem | shardy: ack in a mtg ... | 15:32 |
| owalsh | ansiwen: changing the user is easy, more a question of should we.... and the bigger issue of depending on the host | 15:32 |
| *** openstackgerrit has quit IRC | 15:33 | |
| ansiwen | owalsh: I just jumped onto the conversion without back-reading, probably I didn't get the problem | 15:33 |
| *** yprokule has quit IRC | 15:33 | |
| *** ykarel_ has quit IRC | 15:33 | |
| dprince | owalsh: I might start a thread on the polkit question here | 15:33 |
| mandre | owalsh: for the first question, i think it's cleaner to use the kolla set_configs script to copy the files to the right place with the right permission (if it's a container where we're using kolla_start) | 15:34 |
| dprince | owalsh: perhaps targetted to [TripleO][Kolla] | 15:34 |
| *** sid2 has quit IRC | 15:34 | |
| *** dbecker has quit IRC | 15:34 | |
| *** limao has quit IRC | 15:35 | |
| mandre | owalsh: problem with kolla uid/gids is that it's set at build time | 15:35 |
| *** thrash is now known as thrash|biab | 15:35 | |
| owalsh | mandre: yea, but we could use a custom kolla config file (that's what I'm doing) | 15:35 |
| dprince | mandre: in this case we are mounting /etc/ssh as ro, and we don't want to chown it differently on the host | 15:35 |
| *** limao has joined #tripleo | 15:35 | |
| dprince | mandre: that is why I suggested the copy approach | 15:35 |
| dprince | mandre: I suppose set_configs also does a copy so it would work though too | 15:36 |
| dprince | mandre: we've moved way from using it in general as it was too verbose in most cases. But this is an example where it is useful | 15:36 |
| *** florianf has joined #tripleo | 15:36 | |
| dprince | owalsh: I think I like mandre's idea for set_configs (for your original /etc/ssh/ question) | 15:36 |
| Slower | dprince: so I'm confused, is that meeting with HA folks in a half hour or so? | 15:36 |
| dprince | Slower: yes :) | 15:37 |
| mandre | dprince: right, we don't want to bind mount /etc/ssh:/etc/ssh, but to a tmp directory in the container and set_configs will take care of copying the file to the right place | 15:37 |
| Slower | hrrm, have to take kids to the dentist.. | 15:37 |
| Slower | hrrm hrrm hrrm | 15:37 |
| EmilienM | anyone can review a release note and maybe merge it? https://review.openstack.org/#/c/469388/ thanks | 15:37 |
| Slower | I can cell some of it | 15:38 |
| dprince | Slower: okay, we'll take notes and you can catch up later perhaps | 15:38 |
| Slower | cell service is hit and miss but I'll try to do as much as I can | 15:38 |
| *** openstackgerrit has joined #tripleo | 15:39 | |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Stop using CentOS buildlogs CDN https://review.openstack.org/456225 | 15:39 |
| *** agurenko has quit IRC | 15:39 | |
| mandre | owalsh: I'm not sure I understand what you want to do with the uid/gid, you want to map them to the ones from the host so they match? | 15:39 |
| EmilienM | shardy: have you seen this kind of error before? http://logs.openstack.org/98/467698/6/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/71ace21/console.html#_2017-05-31_15_49_49_528896 - context: https://review.openstack.org/#/c/467698 | 15:40 |
| owalsh | mandre: for the ssh host keys? I need them to be owned by root:ssh_keys inside the container | 15:40 |
| owalsh | mandre: sshd is picky about this | 15:41 |
| owalsh | mandre: it's currenty root:input because the gids don't math | 15:41 |
| owalsh | match | 15:41 |
| mandre | owalsh: will the ssh_keys group always have the same gid? | 15:41 |
| owalsh | mandre: no, it's dynamic | 15:41 |
| owalsh | mandre: so I need a copy that I can chown | 15:43 |
| owalsh | mandre: how do I do this with set_configs? | 15:43 |
| owalsh | NB I can't chown the host files | 15:43 |
| *** jmelvin_ has quit IRC | 15:43 | |
| mandre | owalsh: yeah, I think the copy will solve your first issue | 15:43 |
| *** jmelvin has joined #tripleo | 15:44 | |
| ansiwen | owalsh: or bind mount /etc/groups as well? ;-) ok, there might be no ssh_keys group. | 15:44 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Add workflow for IPMI nodes discovery https://review.openstack.org/462577 | 15:44 |
| ansiwen | owalsh: what is creating a dynamic gid? is that a security feature? and it's created ar image build time or container start time? | 15:44 |
| owalsh | ansiwen: openssh-server rpm creates the group | 15:45 |
| ansiwen | owalsh: ok, so image build time... hmm... | 15:45 |
| owalsh | ansiwen: it's *wherever* the next gid happens to be at the time | 15:45 |
| owalsh | *whatever* | 15:45 |
| ansiwen | owalsh: yes, got it | 15:46 |
| ansiwen | that's a general problem of different containers images that wanna share files... | 15:46 |
| ansiwen | if the gid are not deterministic | 15:47 |
| *** dbecker has joined #tripleo | 15:48 | |
| *** oidgar has joined #tripleo | 15:48 | |
| bkero | adarazs: Sorry about that, just read the meeting notes. | 15:49 |
| *** sid1 has joined #tripleo | 15:49 | |
| ansiwen | owalsh: but what do you mean with host dependency. doesn't a bind mount alsways imply a host dependency (and that's exactly why they are considered harmful)? | 15:49 |
| dweeks | Is there a proper way to configure the overcloud for an external Ceph cluster? I follow RHEL's documentation but the routes estableshed by the compute nodes don't make sense for such heavy traffic. I am asking again today because I still am unclear how Ceph is accessed via the compute node. It seems odd that an external Ceph cluster is accessed via a default route through a NAT on the undercloud. | 15:49 |
| adarazs | bkero: no worries :) | 15:49 |
| owalsh | ansiwen: that's the other issue with polkit | 15:50 |
| mandre | owalsh: moment, I'll show you an example | 15:50 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Add workflow for IPMI nodes discovery https://review.openstack.org/462577 | 15:53 |
| owalsh | dprince: BTW I don't think ubuntu enables polkit when building libvirt, so it's probably just a RHEL/Centos issue | 15:53 |
| *** limao has quit IRC | 15:56 | |
| *** sid1 has quit IRC | 15:56 | |
| *** jlinkes has quit IRC | 15:56 | |
| *** limao has joined #tripleo | 15:56 | |
| owalsh | ah, mandre, can I just drop another kolla config file into /var/lib/kolla/config_files/? | 15:57 |
| mandre | owalsh: something like that should work, http://paste.openstack.org/show/611220/ | 15:57 |
| dprince | owalsh: ack, good point | 15:58 |
| *** trown is now known as trown|lunch | 15:59 | |
| *** sid1 has joined #tripleo | 15:59 | |
| owalsh | mandre: perfect, thanks | 16:00 |
| owalsh | dprince: for now I think we continue running nova as root to avoid polkit | 16:01 |
| *** limao has quit IRC | 16:01 | |
| owalsh | dprince: unless I hit an ownership issues with /var/lib/nova/instances... | 16:02 |
| *** dparkes has quit IRC | 16:02 | |
| owalsh | dprince: damn, actually, I think that's why I had to change it - for live migration | 16:02 |
| *** rcernin has quit IRC | 16:04 | |
| dprince | owalsh: we could add a NOTE about why it needs to run as root | 16:04 |
| ansiwen | does containers-multinode-nv does deploy the overcloud with containers, right? where can I find the container configs in the CI artefacts? I can't find them, I'm lost >_< | 16:04 |
| *** dsneddon has joined #tripleo | 16:04 | |
| openstackgerrit | Merged openstack/python-tripleoclient master: Add overcloud support report collect command https://review.openstack.org/405001 | 16:05 |
| owalsh | dprince: it breaks migration I think, we ssh as nova to write to /var/lib/nova/instances, so ownership is significant | 16:05 |
| dprince | owalsh: so we have to fix this to have all of it working | 16:06 |
| owalsh | dprince: yea... changing the uid to match the host works... but it's a hack | 16:06 |
| *** maeca1 has quit IRC | 16:07 | |
| *** lmiccini has quit IRC | 16:07 | |
| owalsh | dprince: or I could hack the polkit conf on the host | 16:07 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Mount /var/run/libvirt on ceilo agent compute https://review.openstack.org/469935 | 16:07 |
| dprince | owalsh: we could do that via puppet | 16:08 |
| dprince | owalsh: seems a bit dirty as well, but it is a way forward perhaps | 16:08 |
| owalsh | dprince: yea, I'll see what I can do... | 16:09 |
| *** egonzalez has quit IRC | 16:09 | |
| *** flepied has quit IRC | 16:09 | |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 16:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 16:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 16:10 |
| owalsh | pradk: I get log errors after mapping this, before mapping this I think the agent was logging errors about reading the libvirtd socket | 16:10 |
| owalsh | dprince: thanks... I'll get back in my cave to hack away at this for a while :-) | 16:11 |
| *** jpich has quit IRC | 16:12 | |
| pradk | owalsh, did the errors go away after mounting /var/run/libvirt? | 16:12 |
| ansiwen | I think changing the UID in the container is safer than messing with the host. imagine another container wants something else? you easily end in a catch22 | 16:12 |
| ansiwen | owalsh: ^ | 16:12 |
| pradk | owalsh, libvirtd should have been pulled in as a dep but i need to double check | 16:12 |
| dprince | owalsh: cool. sorry I couldn't be more helpful. Reach out on the list. Slower, mandre, and others might have other ideas too on the polkit thing | 16:12 |
| owalsh | pradk: yea, no errors from the agent after mapping it but libvirtd complaing about EOFs | 16:13 |
| karimb | hello, which component is in charge of providing dhcp for the overcloud nodes when deploying ? | 16:13 |
| *** ykarel_ has joined #tripleo | 16:14 | |
| *** jaganathan has joined #tripleo | 16:15 | |
| owalsh | ansiwen: not sure I get it... you mean what if another container relies on the nova polkit rule? | 16:15 |
| ansiwen | owalsh: it's more a general remark. if you can fix it in the container or the host, I would go for container, else two containers (or their deployment framework) try to change mutually exclusive things. it's just my general precausion how to design things. | 16:17 |
| *** sid1 has quit IRC | 16:17 | |
| ansiwen | *might try | 16:18 |
| owalsh | ansiwen: ultimately this boils down to unix sockets and docker not playing very nicely, so whatever we do I think it's going to be a hack... | 16:18 |
| owalsh | ansiwen: and I think relaxing the nova polkit rule is the least hacky | 16:18 |
| ansiwen | right, the idea to run libvirt in a container is a hack already :-) | 16:19 |
| ansiwen | so, polkit expects what exactly? | 16:19 |
| ansiwen | that the libvirt socket is only accessed by whom? | 16:19 |
| owalsh | libvirtd asks polkit to authorized the connection (using dbus IIRC), so it doesn't matter if libvirtd is in a container or not. What matters is that polkitd is on the host | 16:20 |
| owalsh | so polkit rule to allow the nova user write access is denied.. in fact I think it fails earlier becaues the uid isn't local | 16:21 |
| *** dsariel has quit IRC | 16:21 | |
| ansiwen | yeah, sorry, I meant accessing libvirt from within a container is a hack. | 16:22 |
| *** yamahata has joined #tripleo | 16:23 | |
| *** zoli is now known as zoli|gone | 16:23 | |
| owalsh | yea, connecting over TLS would be much better, but then we would need TLS by default | 16:24 |
| *** flepied has joined #tripleo | 16:25 | |
| *** liverpooler has quit IRC | 16:28 | |
| *** karthiks has quit IRC | 16:29 | |
| *** liverpooler has joined #tripleo | 16:30 | |
| *** salmankhan has joined #tripleo | 16:34 | |
| *** rbowen has quit IRC | 16:36 | |
| *** ykarel_ has quit IRC | 16:37 | |
| *** salmankhan has quit IRC | 16:38 | |
| *** derekh has quit IRC | 16:39 | |
| *** fragatina has quit IRC | 16:43 | |
| *** fragatina has joined #tripleo | 16:44 | |
| *** rbowen has joined #tripleo | 16:45 | |
| *** dtantsur is now known as dtantsur|afk | 16:45 | |
| *** akrivoka has quit IRC | 16:45 | |
| *** oidgar has quit IRC | 16:50 | |
| *** ckyriakidou has quit IRC | 16:50 | |
| karimb | hello buddies, when deploying overcloud, i get this: DHCPDISCOVER(tapde7ec6fc-44) f2:08:60:e0:00:20 no address available | 16:50 |
| karimb | and indeed, nodes dont get their boot image, discovery went fine previously though | 16:51 |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo master: DNM: Test gnocchi api issue https://review.openstack.org/469506 | 16:51 |
| *** pkovar has quit IRC | 16:52 | |
| *** shardy has quit IRC | 16:53 | |
| bnemec | karimb: Do you have enough addresses in the DHCP range? | 16:53 |
| karimb | bnemec i d dare to say yes | 16:54 |
| bnemec | It's separate from the introspection range. | 16:54 |
| *** tesseract has quit IRC | 16:54 | |
| karimb | dhcp_start = 10.147.11.101, dhcp_end = 10.147.11.112 | 16:54 |
| karimb | not much bnemec , but those are only 3 controllers and 1 compute i m currently trying to deploy | 16:55 |
| bnemec | karimb: Yeah, that should be fine then. | 16:55 |
| karimb | and neutron port-list only shows one port | 16:55 |
| karimb | (the dhcp one) | 16:55 |
| bnemec | That sounds problematic. By the time the nodes are booting for deployment the provisioning port should have been created in neutron. | 16:56 |
| karimb | bnemec oh you mean that s actually where their ip reservation lives.... | 16:56 |
| karimb | bnemec so what could cause this ? | 16:56 |
| bnemec | karimb: Yeah, neutron does our IPAM. | 16:56 |
| openstackgerrit | Doug Hellmann proposed openstack/puppet-tripleo master: make release note a list of strings https://review.openstack.org/469943 | 16:57 |
| openstackgerrit | Doug Hellmann proposed openstack/puppet-tripleo master: do not include remote name in branch spec for release notes https://review.openstack.org/469944 | 16:57 |
| * karimb to test infoblox as ipam for openstack tomorrow actually | 16:57 | |
| openstackgerrit | Doug Hellmann proposed openstack/puppet-tripleo stable/ocata: make release note a list of strings https://review.openstack.org/469945 | 16:57 |
| karimb | bnemec my bad, ports show up now | 16:58 |
| karimb | ( launching stack from start ) | 16:58 |
| bnemec | karimb: Okay, maybe neutron and dnsmasq got out of sync then. Rebooting the undercloud is probably the easiest thing to try. | 16:59 |
| karimb | yeah, that s what we are trying now | 16:59 |
| *** pbandark has joined #tripleo | 16:59 | |
| karimb | let s see if the rebooting did good | 16:59 |
| *** jpena is now known as jpena|off | 16:59 | |
| pbandark | Hi all, with tripleo UI, is it possible to deploy entire overcloud with customization like, isolating the network, bonding etc. without making any changes from undercloud CLI ? | 17:02 |
| * karimb believes so pbandark bu never tried... | 17:02 | |
| pbandark | i.e. manage entire deployment from tripleo UI without going to undercloud CLI. | 17:03 |
| pbandark | ok karimb | 17:03 |
| *** ebarrera has joined #tripleo | 17:03 | |
| pbandark | lets wait for confirmation from anyone who has tried ^ | 17:03 |
| mwhahaha | bnemec: can you +A https://review.openstack.org/#/c/469945 i think that'll unblock the master releasenotes job | 17:04 |
| bnemec | mwhahaha: Done. I missed the backport. | 17:05 |
| *** thrash|biab is now known as thrash | 17:07 | |
| *** trown|lunch is now known as trown | 17:08 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 17:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1695039 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 17:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 17:10 |
| openstack | Launchpad bug 1695039 in tripleo "puppet-tripleo releasenotes job is failing" [Critical,Triaged] | 17:10 |
| karimb | bnemec weird enough, ports are now there, i set DHCPOFFER in the messages file of the director, but nodes dont get any ip | 17:13 |
| *** milan has quit IRC | 17:15 | |
| *** dmsimard is now known as dmsimard|off | 17:16 | |
| *** cylopez has joined #tripleo | 17:16 | |
| dmsimard|off | dprince, mandre, Slower: fyi http://rdo-container-registry.readthedocs.io/en/latest/using.html | 17:21 |
| *** Slower has quit IRC | 17:23 | |
| *** florianf has quit IRC | 17:24 | |
| *** flepied has quit IRC | 17:26 | |
| openstackgerrit | Merged openstack/puppet-tripleo stable/ocata: make release note a list of strings https://review.openstack.org/469945 | 17:31 |
| *** Goneri has quit IRC | 17:31 | |
| *** Goneri has joined #tripleo | 17:33 | |
| *** bogdando has quit IRC | 17:37 | |
| *** Slower has joined #tripleo | 17:38 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: make release note a list of strings https://review.openstack.org/469943 | 17:39 |
| *** amoralej is now known as amoralej|off | 17:44 | |
| weshay | dprince, mandre Slower I have some questions about feeding containers to tripleo-ci | 17:49 |
| *** suuuper has quit IRC | 17:50 | |
| weshay | dprince, mandre, Slower I'll have to send this to openstack-dev, but the question is whether or not the tripleo community would be comforable w/ rdo freeding tripleo container images | 17:52 |
| weshay | or if we have to duplicate the infra in tripleo itself. | 17:53 |
| weshay | the implications of rdo feeding tripleo containers is that basically it changes the promotion topology where atm tripleo is at the top of the pyramid, if tripleo uses rdo containers rdo would be on top and feeding delorean builds and containers to tripleo | 17:54 |
| *** fragatin_ has joined #tripleo | 17:54 | |
| weshay | EmilienM, ^ | 17:54 |
| *** rbowen has quit IRC | 17:54 | |
| *** fragatina has quit IRC | 17:58 | |
| *** pbandark has quit IRC | 18:00 | |
| dprince | dmsimard|off: thansk for the docs link. | 18:04 |
| openstackgerrit | Matt Young proposed openstack/tripleo-quickstart master: Migrate quickstart-extras.yml --> TQE https://review.openstack.org/469611 | 18:05 |
| *** cYberDe has joined #tripleo | 18:06 | |
| dprince | weshay: I think we may need to duplicate some of it in TripleO | 18:06 |
| cYberDe | Hi guys, I'm trying to install openstack with tripleo (Redhat OSP) but I'm running against failures during the deployment where it seems that there is a bug regarding the deployment of Mysql. | 18:07 |
| weshay | dprince, ok we thought that would be the case, but wanted to check | 18:07 |
| *** fragatin_ has quit IRC | 18:08 | |
| *** fragatina has joined #tripleo | 18:08 | |
| cYberDe | During deploying it seems that I have to set CloudDomain to localdomain because if I set it to anything else, MySQL still gets deployed with the localdomain as domain but all the localdomains are pointing to localhost and therefore, MySQL is unable to transfer the state between eachother. | 18:08 |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 18:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 18:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 18:10 |
| *** rbowen has joined #tripleo | 18:10 | |
| *** rcernin has joined #tripleo | 18:14 | |
| *** lucasagomes is now known as lucas-afk | 18:14 | |
| *** gvrangan has joined #tripleo | 18:14 | |
| *** jkilpatr_ has joined #tripleo | 18:15 | |
| *** sri_ has quit IRC | 18:16 | |
| *** ramishra has quit IRC | 18:16 | |
| *** jkilpatr has quit IRC | 18:18 | |
| *** fragatina has quit IRC | 18:19 | |
| *** thrash is now known as thrash|bbl | 18:24 | |
| bnemec | weshay: Is it really that different? Today RDO builds packages, we consume the packages and tell RDO which set of packages works together. | 18:25 |
| bnemec | Wouldn't containers be essentially the same thing? | 18:26 |
| weshay | bnemec, it would require a job that builds the containers and validates the containers work | 18:27 |
| weshay | so let's walk it | 18:27 |
| weshay | tripleo periodic jobs find a good build.. | 18:28 |
| weshay | then we can kick a container build in rdo and test them in tripleo | 18:28 |
| weshay | bnemec, the container builds could either be compared to images or rpms | 18:29 |
| weshay | bnemec, so either way.. tripleo promotions are dependent on rdo promotions/jobs to some degree | 18:30 |
| weshay | unless the containers are built in tripleo | 18:30 |
| bnemec | Mmm, I suppose delorean for containers would have insane storage requirements, so this can't work quite the same as packages. | 18:30 |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Provide commands for example roles https://review.openstack.org/466900 | 18:31 |
| weshay | david's thought is that some percentage of delorean builds would come w/ container images | 18:31 |
| bnemec | I don't know that that flow works long-term either though. Eventually all the jobs will be using containers so there won't be any way to test without containers. | 18:31 |
| weshay | we thought there were two possibilities.. basically | 18:32 |
| bnemec | That could work. We'd lose some granularity in promotions, but realistically we're not running promotion jobs on every delorean build anyway. | 18:32 |
| weshay | either build containers in tripleo.. or put rdo on top of the promotion pyramid and have rdo feed tripleo builds | 18:32 |
| weshay | bnemec, ya.. re: only some builds having containers.. it just lowers the percentage that we'll find a working build | 18:33 |
| openstackgerrit | Alex Schultz proposed openstack/puppet-tripleo stable/ocata: Restrict nova migration ssh tunnel https://review.openstack.org/469423 | 18:33 |
| bnemec | As long as we get one every couple of hours (or however long the promotion jobs take) I suspect it won't make much practical difference. | 18:34 |
| weshay | this is really all assuming we *need* containers built w/ the same exact delorean repo that is configured on the env.. | 18:34 |
| weshay | which sounds like a very good idea, but I'm not sure if it's 100% required | 18:34 |
| weshay | bnemec, ya | 18:34 |
| weshay | bnemec, to keep the high level structure of builds and promotions that we have today, it seems like we need to move the container builds to tripleo | 18:35 |
| bnemec | We need that anyway, don't we? Otherwise how do you test a proposed change that affects what goes in the container? | 18:35 |
| weshay | bnemec, atm we're planning on doing a local delorean build of the change.. and running yum update on all the containers | 18:36 |
| weshay | so it won't be rebuilt as that may take a very long time depending on how many containers the change impacts | 18:36 |
| weshay | Slower, and panda|travel are working on a solution for that problem atm | 18:37 |
| weshay | we're going to add a delorean yum repo container that hosts the repo locally so it will be fast and reliable | 18:37 |
| weshay | so I think we're ok w/ pulling in a change.. at least in the design | 18:38 |
| weshay | bnemec, wdyt? | 18:39 |
| bnemec | weshay: Sounds not very container-y, but we CI with cached images in a similar way for basically the same reason. | 18:39 |
| AzazeL_ | Anyone try TripleO on an Oracle Linux stack? ;) Plausable? | 18:40 |
| *** liverpooler has quit IRC | 18:48 | |
| dsneddon | AzazeL_, That sounds like a pretty masochistic exercise. | 18:48 |
| dsneddon | AzazeL_, Let us know how it goes. ;-) | 18:48 |
| dsneddon | AzazeL_, In all seriousness, I think you'd find a lot of missing dependencies. There are some packages that are not carried in the base OS, but are carried in the OpenStack repos. Whether the dependencies match up in the Oracle system distribution to allow those RPMs to be installed is anyone's guess. | 18:49 |
| *** liverpooler has joined #tripleo | 18:50 | |
| dsneddon | AzazeL_, Open vSwitch in particular, but also many Python packages. | 18:50 |
| dsneddon | AzazeL_, Even if you ran the Undercloud on CentOS/RHEL, you'd still need to satisfy a lot of RPM dependencies when building the overcloud images. | 18:52 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: Make the containers prep role more generic https://review.openstack.org/469665 | 18:53 |
| openstackgerrit | James Slagle proposed openstack/puppet-tripleo master: Install rsync package for galera https://review.openstack.org/467362 | 18:55 |
| dciabrin | ping trown, suppose i'd like to use oooq run CI jobs locally with some packages from http://cbs.centos.org/repos/cloud7-openstack-common-pending/x86_64/os/. What flags would i need to use/override in oooq to achieve that? | 18:57 |
| openstackgerrit | Merged openstack/python-tripleoclient stable/ocata: Fix 'introspection bulk status' when some nodes were never introspected https://review.openstack.org/467935 | 19:00 |
| openstackgerrit | Merged openstack/puppet-tripleo master: do not include remote name in branch spec for release notes https://review.openstack.org/469944 | 19:00 |
| trown | dciabrin: you would need to overwrite or copy the release file https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/master.yml | 19:02 |
| dciabrin | trown, awesome i'll have a look thx | 19:02 |
| cYberDe | tripleo has a bug when setting a domain name, found the future is 2 years old (https://bugzilla.redhat.com/show_bug.cgi?id=1259003) but when I change it, deployment fails | 19:04 |
| openstack | bugzilla.redhat.com bug 1259003 in openstack-tripleo-heat-templates "RFE: Provide mechanism to set domain name for overcloud" [Medium,Closed: errata] - Assigned to mandreou | 19:04 |
| trown | dciabrin: https://github.com/openstack/tripleo-quickstart/tree/master/roles/repo-setup has examples of different ways to add repos to the config | 19:04 |
| *** jaganathan has quit IRC | 19:05 | |
| *** oneswig has joined #tripleo | 19:06 | |
| dciabrin | trown, great thx :) | 19:06 |
| bnemec | cYberDe: That bug was fixed over a year ago. What version of tripleo are you deploying? | 19:07 |
| cYberDe | Deploying newton currently | 19:08 |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 19:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 19:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 19:10 |
| cYberDe | bnemec: Deploying Newton | 19:11 |
| bnemec | cYberDe: Okay, I believe it should be fixed there. Did you set the domain in both places - the CloudDomain parameter and the undercloud DHCP configuration? | 19:12 |
| cYberDe | Not sure, is there also one in the undercloud config? | 19:13 |
| bnemec | cYberDe: Yes, it's mentioned in the description for CloudDomain: https://github.com/openstack/tripleo-heat-templates/blob/7996b2e817b8f53f6e3dc579023e20ef61751c31/overcloud.j2.yaml#L103 | 19:13 |
| bnemec | I don't recall which neutron conf file has dhcp_domain, but you should be able to grep for it. | 19:14 |
| bnemec | That description should probably say "This _must_ match the dhcp_domain..." | 19:14 |
| cYberDe | bnemec: Indeed, seems like that might be the issue | 19:16 |
| openstackgerrit | Jacob Liberman proposed openstack/puppet-tripleo master: Add conditional for setting authlogin_nsswitch_use_ldap selboolean if selinux is enabled https://review.openstack.org/469877 | 19:16 |
| cYberDe | So basically means, reinstall undercloud? | 19:16 |
| bnemec | cYberDe: No, just change the value and restart the neutron services (probably just neutron-dhcp-agent, but doing them all is safest). | 19:17 |
| bnemec | That's one we should probably expose in undercloud.conf though. :-/ | 19:18 |
| *** sid1 has joined #tripleo | 19:18 | |
| *** sid1 has quit IRC | 19:19 | |
| cYberDe | bnemec: indeed :) Do you think I will need to reinstall the undercloud or just remove the overcloud and reboot the undercloud? | 19:19 |
| cYberDe | Ow you already answered that, nevermind | 19:19 |
| bnemec | cYberDe: Yeah, reinstalling the undercloud won't help since this isn't an exposed configuration value. | 19:20 |
| bnemec | Which is problematic because it means the value will be overwritten when you update your undercloud. | 19:20 |
| bnemec | Now that I think about this some more I'm surprised we haven't heard more complaints about this. | 19:21 |
| bnemec | I would think a lot of real world deployments are setting custom domains. | 19:21 |
| * bnemec adds to the todo list | 19:21 | |
| jidar | I personally haven't found a reason for it | 19:23 |
| *** dtrainor has quit IRC | 19:25 | |
| cYberDe | bnemec: I guess all of them just find it out after a few hours/days of troubleshooting and than they fix it without someone receiving that it was that hard to find out | 19:25 |
| cYberDe | Anyway, if you don't update that file, it fails on the mysql startup because there it still uses the localdomain | 19:26 |
| *** gvrangan has quit IRC | 19:30 | |
| cYberDe | bnemec dhcp_domain is deprecated for dns_domain? | 19:32 |
| sai | bnemec: post,deploy if i changed a value in service config file (for example changed rbd_pool in cinder.conf) should pass this new value in parameter_defaults when i try to scale up the deploymentt? | 19:32 |
| *** gvrangan has joined #tripleo | 19:32 | |
| *** gvrangan has quit IRC | 19:37 | |
| bnemec | cYberDe: Could be. If that's what neutron's config says we should update the param description. | 19:37 |
| bnemec | sai: Yes, otherwise your changes will be overwritten on all the nodes. | 19:38 |
| cYberDe | bnemec: indeed, deprecated for a setting in the neutron config , so would indeed be valid to update it | 19:38 |
| sai | bnemec: roger, thank u | 19:38 |
| openstackgerrit | John Fulton proposed openstack/tripleo-common master: Add Mistral workbook to call ceph-ansible to install Ceph https://review.openstack.org/469644 | 19:41 |
| *** yamahata has quit IRC | 19:45 | |
| trozet | dprince: hey, farther along now, hitting a new problem | 19:45 |
| trozet | dprince: it looks like my volume that i mount for the config files is empty, how does tripleo determine which fils to place in /var/lib/config-data/ ? | 19:45 |
| dprince | trozet: whats up? | 19:45 |
| trozet | s/fils/files | 19:45 |
| AzazeL_ | dsneddon, Thanks for the reply.. I figured it was that way. Generally my team and I just rebuild upstream products for our oracle box's, openvswitch, xen, qemu, etc.. Would it just behoove me to go through the regular openstack install? I don't want to use Oracle's Mitaka version, would like to follow a stream that would keep me more up to date. I enjoy the openstack on openstack thought for updating/maintaining (especially for | 19:49 |
| AzazeL_ | dsneddon, Thanks for the reply. I might try it just to see what snags we get from the get-go. I know my clients love Redhat (damn gov't) and so it still might be the route I have to go but I wanted to save some money on the licensing costs over-all with oracle, plus the public repo is nice. But you're right, the packaging and versions will definitely be different. | 19:50 |
| AzazeL_ | dsneddon, I'll let you know how it goes.. probably will fail quickly. Didn't want to go the kolla route! | 19:51 |
| *** rasca_ is now known as rasca | 19:53 | |
| dsneddon | AzazeL_, I think the kolla route will eventually be easier, but probably not yet. | 19:53 |
| *** limao has joined #tripleo | 19:53 | |
| dsneddon | AzazeL_, If you do decide to go the kolla route, Ansible OpenStack might be helpful | 19:54 |
| dsneddon | AzazeL_, But if I were rolling it myself for a different distribution, I would probably try to build disk images based on a hand-built config. | 19:54 |
| dsneddon | AzazeL_, With whatever configuration management tool of choice you prefer (Puppet, Chef, Ansible, Salt, etc.) | 19:55 |
| AzazeL_ | dsneddon, you're suggesting golden images and deploy to specific box's in a different automation manner and just skip the tripleo portion? I suppose I didn't want to build the infrastructure around it to keep updating it | 19:55 |
| dsneddon | AzazeL_, Yeah, I gotcha. I had also thought about Kolla for your use case, just because it makes upgrades so much easier (at least it makes rollbacks easier). | 19:56 |
| *** lblanchard has quit IRC | 19:56 | |
| AzazeL_ | dsnesddon, yea.. oracle supports a heavily modified kolla version that is included in their repo... but damnit, didn't want to be completely stuck with oracle on that.. since they are still on mitaka (over a year!) | 19:58 |
| dsneddon | AzazeL_, For development, if you have a beefy host to use as a virtual server, you could use tripleo-quickstart to create a virtual baremetal environment with networks and VMs. Then, replace the OS on the VMs so you have a dev environment. | 19:58 |
| dsneddon | AzazeL_, That would even allow you to build a CentOS VM to use as a baseline to compare to your hand-built Oracle VM. | 19:59 |
| trozet | dprince: i would assume its just done by the volumes, but they are listed as read only in the templates | 19:59 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: WIP: formalize the notion of an environment configuration https://review.openstack.org/469992 | 19:59 |
| trozet | dprince: and i thought those were just for running the service, not generating config | 19:59 |
| AzazeL_ | dsneddon, yeah, hardware isn't my problem. Got a few machines with 72-120 cores, also a blade center with 20 nodes, 72 threads each, some 30+TB raids with high end SSD (soon to have me a nvme version). I see what you're saying, but that would only help me with the overcloud portion right? The undercloud baremetal I would like to be able to run my oracle xen versions (IA requirements for future use). | 20:01 |
| trozet | dprince: oh now i see there is config_volume under puppet_config | 20:02 |
| *** cylopez has quit IRC | 20:02 | |
| AzazeL_ | dsneddon, I see what you're saying.. sounds like a massive undertaking.. might just have to hand jam most of it and use ansible/puppet to automate that part of it. | 20:02 |
| dsneddon | AzazeL_, Yeah, if you have that much hardware at your disposal, there wouldn't be any advantage to building a virtual baremetal env, unless you wanted faster reboots. ;-) | 20:02 |
| AzazeL_ | dsnesddon, haha yeah, hardware aint much of a problem.. it's dollars for labor. I might be able to use some of my time while i'm making a stig'ed oracle xen version for another team. So really it sounds like i'll just be going from upstream openstack and possibly using their ansible version.. givess me the most flexibility to work around package problems. | 20:05 |
| dprince | trozet: yes, correct. | 20:05 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: baseos-undercloud release configuration for all supported releases. https://review.openstack.org/464033 | 20:05 |
| trozet | dprince: yeah mine is set to 'opendaylight' so i assume that just mounts the /var/lib/config-data/opendaylight dir? | 20:05 |
| trozet | dprince: how does it figure out waht should go in there? I see: | 20:05 |
| dprince | trozet: yes | 20:05 |
| trozet | [root@overcloud-controller-0 ~]# ls /var/lib/config-data/opendaylight/ | 20:06 |
| trozet | etc opt root | 20:06 |
| dprince | trozet: docker-puppet.py just copies a few directories | 20:06 |
| dprince | trozet: correct, just those | 20:06 |
| trozet | dprince: well thats cool, because odl stores its config in /opt/opendaylight | 20:06 |
| AzazeL_ | dsneddon, and i'm assuming tripleo is only going to stay in the redhat realm (centos included)? any ideas of expansion to say Scientific Linux, Oracle, or other redhat derivatives? | 20:06 |
| dprince | trozet: ha! | 20:06 |
| trozet | dprince: so then i see this path exists: | 20:06 |
| trozet | [root@overcloud-controller-0 ~]# ls /var/lib/config-data/opendaylight/opt/opendaylight/ | 20:07 |
| trozet | configuration etc | 20:07 |
| dsneddon | AzazeL_, As long as you are doing it with upstream, you might want to also utilize something like a PXE/kickstart in order to build systems up to baseline. This is similar to what the Undercloud does, but it uses a self-contained OpenStack with Ironic instead of a simple PXE/kickstart setup (although the old Foreman system used kickstart and Puppet). | 20:07 |
| trozet | dprince: but there is nothing in the configuration or etc directories | 20:07 |
| dprince | trozet: so, I think you'll need to patch docker-puppet.py to "archive" this directory as well | 20:07 |
| trozet | dprince: oh...now its all making sense starting to see that code in docker-puppet.py | 20:07 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Update CloudDomain description https://review.openstack.org/469996 | 20:08 |
| bnemec | cYberDe: ^ | 20:09 |
| dprince | trozet: so I did a short blog on docker-puppet.py a while back if it helps https://dprince.github.io/docker-puppet.html | 20:09 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Standardize example role definitions https://review.openstack.org/445687 | 20:09 |
| dprince | trozet: I think you've mostly got it now though | 20:09 |
| *** eck` is now known as eck`gone | 20:09 | |
| trozet | dprince: yeah, let me try to locally change it and see if that fixes it. thanks | 20:09 |
| dsneddon | AzazeL_, I suspect that other Fedora variants will be supported once the transition from CentOS/RHEL 7.x to version 8. Most of the packages will be included in base at that poing. | 20:09 |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 20:10 |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 20:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 20:10 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/newton: SSHD Service extensions https://review.openstack.org/469410 | 20:11 |
| AzazeL_ | dsneddon, ahh, yes, that is similar to the route I was going.. for the pxe/kickstart approach I was going to utilize building the base & vm... since I will be using it for other builds as well. Would I then be able to put those into an ironic and deploy from there to other machines? that was my thought. | 20:11 |
| dsneddon | AzazeL_, Yeah, once you've got the image, Ironic can deliver it to any node. | 20:12 |
| AzazeL_ | dsneddon, Nice.. yea, too bad my field of gov't is pretty delayed on updating to newer versions. Still going to be good though, might be able to convince them to move forward since I have a window of opportunity in the next 1.5 years. | 20:12 |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-quickstart master: DO NOT REVIEW validate new pacemaker builds for containers in HA deployments https://review.openstack.org/469998 | 20:12 |
| AzazeL_ | dsneddon, Nice, okay... looks like I have a path forward, just can't use tripleO out the box but I can at least follow the methodology of it. I appreciate your attention to my questions and continued conversation... man I miss IRC, good old days. | 20:13 |
| dsneddon | AzazeL_, Not sure if this matters, but Pike is likely to get a lot of backports in the year after it is released. Newton and Ocata, probably not as much. | 20:13 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/newton: Stop SSHD profile clobbering SSH client config https://review.openstack.org/469411 | 20:14 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Add "make clean" https://review.openstack.org/469999 | 20:14 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/newton: Refactor SSHD config to allow both SSHD options and banner/motd to be set https://review.openstack.org/469412 | 20:14 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/newton: Configure migration SSH tunnel https://review.openstack.org/469413 | 20:14 |
| *** ebarrera has quit IRC | 20:14 | |
| *** limao has quit IRC | 20:14 | |
| AzazeL_ | dsneddon, oh? you suggesting starting with Pike (that is the newest, right?) to possibly help my troubles in the future? It does matter because if I'm the maintainer I do care about that. | 20:14 |
| dsneddon | AzazeL_, Actually, wait, that's Queens I'm thinking of that's probably going to be getting extended backports. So actually, that won't help. | 20:14 |
| openstackgerrit | Jacob Liberman proposed openstack/puppet-tripleo master: Add conditional for setting authlogin_nsswitch_use_ldap selboolean if selinux is enabled https://review.openstack.org/469877 | 20:15 |
| AzazeL_ | dsneddon, lol, it's okay. | 20:15 |
| *** limao has joined #tripleo | 20:15 | |
| trozet | dprince: yeah that fixes it, will update my patch | 20:15 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Standardize example role definitions https://review.openstack.org/445687 | 20:16 |
| dprince | trozet: cool | 20:16 |
| dsneddon | AzazeL_, Newton also falls into that category. Every 3 releases, Red Hat puts out a long term support release based on upstream. Starting with Newton, then Queens, etc. | 20:16 |
| dsneddon | AzazeL_, But there isn't that much difference. It only matters if you plan to do minor updates, not if you upgrade to each new release. | 20:17 |
| trozet | dprince: last question, does /var/lib/docker-puppet/docker-puppet.py get copied to the overcloud during deploy time? | 20:17 |
| trozet | dprince: i see it on the overcloud, and its not owned by any package | 20:17 |
| AzazeL_ | dsneddon, yea, damn.. was really trying to get away from redhat to provide an alternative to other projects.. really license costs is the biggest. | 20:18 |
| dprince | trozet: it is copied in has heat agent metadata | 20:18 |
| trozet | dprince: great, thanks | 20:18 |
| AzazeL_ | dsneddon, Yea, minor updates for local dev.. but once it is deployed it would be long term management of it with possibly every 1-2 years of a major upgrade. | 20:18 |
| dprince | trozet: if you go and grep /var/lib/heat-config/heat-config-script you'll find that python script there | 20:19 |
| dprince | trozet: on your overcloud nodes | 20:19 |
| dprince | trozet: I think we may want to move this to an RPM package though | 20:20 |
| dprince | trozet: it is a useful file to have around for some things | 20:20 |
| trozet | dprince: yeah | 20:20 |
| openstackgerrit | Tim Rozet proposed openstack/tripleo-heat-templates master: Adds docker OpenDaylight https://review.openstack.org/466920 | 20:22 |
| *** liverpooler has quit IRC | 20:22 | |
| dsneddon | AzazeL_, Another option is to use RDO on CentOS, but you'd be without support. I don't know whether I'd want to use TripleO unsupported, but it might not be worse than rolling your own. | 20:22 |
| dsneddon | AzazeL_, And RDO doesn't necessarily do as well with upgrades as the commercial version, just due to testing and polish. | 20:23 |
| dsneddon | AzazeL_, But basic version-to-version upgrades do get tested in CI. | 20:26 |
| *** dprince has quit IRC | 20:30 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: WIP: formalize the notion of an environment configuration https://review.openstack.org/469992 | 20:31 |
| AzazeL_ | dsneddon, yea, centos is a no go in my environment.. well, it might be alright for the undercloud / hypervisor portion but it won't be satisfactory to deploy. | 20:33 |
| *** dtrainor has joined #tripleo | 20:35 | |
| *** limao has quit IRC | 20:36 | |
| *** limao has joined #tripleo | 20:36 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Standardize example role definitions https://review.openstack.org/445687 | 20:39 |
| *** pchavva has quit IRC | 20:39 | |
| *** Goneri has quit IRC | 20:40 | |
| dsneddon | AzazeL_, Yeah, I don't know what the deal would be if you ran RDO on RHEL. I mean, it should work, it's just not a tested combination. I don't think it would affect your OS support, though. | 20:40 |
| dsneddon | AzazeL_, But I'm no contract lawyer. ;-) | 20:41 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Restart docker after changing storage driver https://review.openstack.org/468201 | 20:41 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Standardize example role definitions https://review.openstack.org/445687 | 20:41 |
| *** jrist has quit IRC | 20:45 | |
| *** jkilpatr_ has quit IRC | 20:47 | |
| *** eck`gone is now known as eck` | 20:47 | |
| *** yamahata has joined #tripleo | 20:49 | |
| openstackgerrit | Ben Nemec proposed openstack/instack-undercloud master: Allow setting the DNS domain name in undercloud.conf https://review.openstack.org/470007 | 20:49 |
| *** jrist has joined #tripleo | 20:49 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: formalize the notion of an environment configuration https://review.openstack.org/469992 | 20:51 |
| openstackgerrit | Merged openstack/os-net-config master: MTU setting for OVSDpdkBond https://review.openstack.org/469056 | 20:56 |
| *** limao has quit IRC | 20:56 | |
| *** bfournie has quit IRC | 20:59 | |
| *** rbowen has quit IRC | 21:00 | |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart-extras master: Remove unneeded setting of ansible_user https://review.openstack.org/470011 | 21:01 |
| pradk | has anyone seen this error http://paste.openstack.org/show/611250/ | 21:02 |
| *** fragatina has joined #tripleo | 21:04 | |
| *** trown is now known as trown|outtypewww | 21:05 | |
| *** jkilpatr has joined #tripleo | 21:06 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-image-elements master: Use /etc/os-net-config/element_config.json https://review.openstack.org/470014 | 21:08 |
| *** stevebaker has joined #tripleo | 21:10 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 21:10 |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 21:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 21:10 |
| openstackgerrit | Merged openstack/diskimage-builder master: Drop support for Ubuntu precise https://review.openstack.org/468186 | 21:10 |
| openstackgerrit | Jacob Liberman proposed openstack/puppet-tripleo master: Add conditional for setting authlogin_nsswitch_use_ldap selboolean https://review.openstack.org/469877 | 21:10 |
| *** rbowen has joined #tripleo | 21:12 | |
| *** karimb has quit IRC | 21:14 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-common master: Install os-net-config as a package, not an element https://review.openstack.org/470017 | 21:15 |
| *** karimb has joined #tripleo | 21:15 | |
| *** arxcruz has quit IRC | 21:20 | |
| *** abishop has quit IRC | 21:23 | |
| *** arxcruz has joined #tripleo | 21:24 | |
| stevebaker | EmilienM: hey, I think this might be ready to land https://review.rdoproject.org/r/#/c/6896/ | 21:24 |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common master: [WIP] Adds ansible actions https://review.openstack.org/470021 | 21:25 |
| *** yamahata has quit IRC | 21:30 | |
| *** myoung is now known as myoung|gone | 21:36 | |
| *** paramite has quit IRC | 21:42 | |
| *** yamahata has joined #tripleo | 21:46 | |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common master: [WIP] Adds ansible actions https://review.openstack.org/470021 | 21:48 |
| weshay | bnemec, did http://zuul-status.tripleo.org/ go down? | 21:51 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use picked nodes for later cmd_* calls https://review.openstack.org/469330 | 21:54 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move global mount tracking into state https://review.openstack.org/469752 | 21:54 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use global state to check for duplicate fs labels https://review.openstack.org/469751 | 21:54 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Remove 'state' argument from later cmd_* calls https://review.openstack.org/469750 | 21:54 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add state to NodeBase class https://review.openstack.org/469749 | 21:54 |
| *** ipsecguy_ has joined #tripleo | 21:56 | |
| *** ipsecguy has quit IRC | 21:59 | |
| *** brault has quit IRC | 22:09 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 22:10 |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 22:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 22:10 |
| *** rcernin has quit IRC | 22:10 | |
| openstackgerrit | Merged openstack/puppet-tripleo stable/newton: Update gitignore not to exclude fixture hieradata https://review.openstack.org/466372 | 22:18 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Composable Role for Neutron LBaaS https://review.openstack.org/314225 | 22:20 |
| *** jmelvin has quit IRC | 22:22 | |
| *** agopi has quit IRC | 22:40 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: formalize the notion of an environment configuration https://review.openstack.org/469992 | 22:43 |
| mwhahaha | ovb is timingout again | 22:53 |
| *** oneswig has quit IRC | 22:53 | |
| *** flepied has joined #tripleo | 22:55 | |
| *** rhallisey has quit IRC | 23:00 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: update ci-scripts for environment parameter https://review.openstack.org/470055 | 23:06 |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1680195 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694524 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1694847 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1680195 in tripleo "Random ovb-ha ping test failures" [Critical,In progress] - Assigned to John Trowbridge (trown) | 23:10 |
| openstack | Launchpad bug 1694524 in tripleo "Neutron OVS agent fails to start when neutron-server is not available" [Critical,Triaged] | 23:10 |
| openstack | Launchpad bug 1694847 in tripleo "pingtest fails on ipv6/ssl: glanceclient fails to upload pingtest_initramfs" [Critical,Triaged] | 23:10 |
| *** karimb has quit IRC | 23:11 | |
| openstackgerrit | Dan Trainor proposed openstack/puppet-tripleo master: Support for proxying ironic-inspector via Apache https://review.openstack.org/470057 | 23:11 |
| openstackgerrit | Dan Trainor proposed openstack/instack-undercloud master: Add proxy and config endpoints of ironic-inspector https://review.openstack.org/470058 | 23:11 |
| *** atheurer has quit IRC | 23:13 | |
| *** atheurer has joined #tripleo | 23:14 | |
| *** yamahata has quit IRC | 23:27 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move global mount tracking into state https://review.openstack.org/469752 | 23:30 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use global state to check for duplicate fs labels https://review.openstack.org/469751 | 23:30 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Remove 'state' argument from later cmd_* calls https://review.openstack.org/469750 | 23:30 |
| *** flepied has quit IRC | 23:32 | |
| *** athomas has quit IRC | 23:38 | |
| *** limao has joined #tripleo | 23:42 | |
| *** tzumainn has quit IRC | 23:44 | |
| *** bfournie has joined #tripleo | 23:47 | |
| *** bfournie has quit IRC | 23:48 | |
| *** bfournie has joined #tripleo | 23:50 | |
| dtrainor | tox is giving me fits on some release notes. is this a symptom of my environment or is it fair to say that the releasenotes file it's complaining about is bad? http://paste.openstack.org/show/611261/ | 23:54 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!