Friday, 2017-06-16

openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260700:00
*** sid1 has joined #tripleo00:05
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813600:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817200:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817500:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,Triaged]00:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)00:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]00:10
*** jkilpatr has quit IRC00:12
openstackgerritSteve Baker proposed openstack-infra/tripleo-ci master: Don't capture quickstart.tar.gz  https://review.openstack.org/47481500:29
*** agopi has quit IRC00:31
*** ecerquei_ has joined #tripleo00:39
*** limao has joined #tripleo00:48
*** sid2 has joined #tripleo00:50
*** sid1 has quit IRC00:52
*** dmacpher_ has quit IRC00:54
*** sid2 has quit IRC00:56
*** gvrangan has joined #tripleo01:00
*** Goneri has quit IRC01:00
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813601:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817201:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817501:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,Triaged]01:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)01:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]01:10
stevebakerbnemec: hey, still about?01:12
*** ecerquei__ has joined #tripleo01:19
*** gvrangan has quit IRC01:19
*** fzdarsky_ has joined #tripleo01:19
*** ecerquei_ has quit IRC01:21
*** fzdarsky|afk has quit IRC01:23
*** dmacpher has joined #tripleo01:59
*** nyechiel has joined #tripleo02:09
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813602:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817202:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817502:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,Triaged]02:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)02:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]02:10
*** ramishra has joined #tripleo02:14
*** lblanchard has quit IRC02:17
*** lblanchard has joined #tripleo02:17
*** lblanchard has quit IRC02:18
openstackgerritMerged openstack/tripleo-heat-templates stable/newton: Add ignore_projects to filter gnocchi events  https://review.openstack.org/47472502:30
openstackgerritMerged openstack/tripleo-heat-templates master: Input enabled_services in all-nodes-config as comma_delimited_list  https://review.openstack.org/47448902:30
*** nyechiel has quit IRC02:35
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Add timestamp output filter  https://review.openstack.org/47483002:41
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] testing individual log files  https://review.openstack.org/47483102:41
*** ecerquei__ has quit IRC02:42
*** agopi has joined #tripleo02:44
openstackgerritMerged openstack/tripleo-heat-templates master: Remove reference too puppet/services/ironic-pxe.yaml  https://review.openstack.org/47417702:58
*** jaganathan has joined #tripleo02:59
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] testing individual log files  https://review.openstack.org/47483102:59
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Add timestamp output filter  https://review.openstack.org/47483002:59
*** yamahata_ has quit IRC03:02
*** ykarel has joined #tripleo03:03
*** jaganathan has quit IRC03:09
openstackgerritSteve Baker proposed openstack-infra/tripleo-ci master: Use undeprecated heat commands in get_host_info.sh  https://review.openstack.org/47483403:09
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813603:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,Triaged]03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817203:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817503:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)03:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]03:10
*** ratailor has joined #tripleo03:10
*** chlong has quit IRC03:12
openstackgerritSaravanan KR proposed openstack/tripleo-docs master: Update the OVS-DPDK document in line with the role-specific parameters  https://review.openstack.org/47483603:14
stevebakerEmilienM: if you're still conscious could you share your thoughts on this? https://bugs.launchpad.net/tripleo/+bug/169817203:16
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)03:16
EmilienMI'm still conscious and alive03:18
EmilienMI can't figure how after my evening though03:18
EmilienMbut I'm here03:18
* EmilienM clicks03:18
*** dbecker has quit IRC03:20
EmilienMstevebaker: https://review.openstack.org/#/c/474815/1 makes sense to me, +203:20
*** gbarros has quit IRC03:20
EmilienMstevebaker: --nested-depth 2 means less debug, right?03:21
stevebakeroh crap, that was supposed to replace line03:22
stevebakerheat event-list overcloud03:22
stevebakerEmilienM: it is the "heat stack-show" which is insanely large and unreadable03:22
EmilienMyeah03:22
stevebakerEmilienM: is bnemec referring to the logstash storage of these jobs? if so it would be the uncompressed size, which would be o_O03:23
EmilienMstevebaker: he's refering to the logs.openstack.org volumes03:24
EmilienMAFIK the logs are stored on huuuge cinder volume03:24
*** sasha2 has quit IRC03:25
openstackgerritSteve Baker proposed openstack-infra/tripleo-ci master: Use undeprecated heat commands in get_host_info.sh  https://review.openstack.org/47483403:25
stevebakerEmilienM: I'm struggling to mirror the contents of a logs dir locally without automatically decompressing .xz files, which is confounding my effort to find out what is large even when compressed03:26
EmilienMstevebaker: AFIK we are dupplicating the etc directories03:28
EmilienMlet me show03:28
EmilienMactually I don't see it because jobs are now failing03:28
EmilienMstevebaker: sorry but it's a bit late here... I need to go03:29
stevebakerEmilienM: no prob, thanks03:29
EmilienMstevebaker: maybe you can ping panda in your end of afternoon03:29
EmilienMstevebaker: or adarazs03:29
EmilienMthey'll help for sure03:29
stevebakerEmilienM: ok03:29
EmilienMhave a great day /me off03:30
*** limao has quit IRC03:34
*** limao has joined #tripleo03:35
*** dbecker has joined #tripleo03:35
*** limao has quit IRC03:39
*** jaganathan has joined #tripleo03:41
*** ramishra has quit IRC03:41
*** psahoo has joined #tripleo03:41
*** ramishra has joined #tripleo03:42
*** yamahata_ has joined #tripleo03:44
*** agopi has quit IRC03:51
*** yamahata_ has quit IRC03:51
*** yamahata_ has joined #tripleo03:51
*** akuznetsov has joined #tripleo03:53
*** jaosorior_away is now known as jaosorior03:53
*** gkadam has joined #tripleo03:55
*** ykarel has quit IRC03:58
*** akuznetsov has quit IRC04:04
*** prateek has joined #tripleo04:09
*** atheurer has quit IRC04:09
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813604:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,Triaged]04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817204:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817504:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)04:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]04:10
*** jaganathan has quit IRC04:14
*** jaganathan has joined #tripleo04:14
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Enable nova-api to run over httpd again  https://review.openstack.org/47413504:15
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] testing individual log files  https://review.openstack.org/47483104:16
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Add timestamp output filter  https://review.openstack.org/47483004:16
*** prateek has quit IRC04:22
*** prateek has joined #tripleo04:23
*** udesale has joined #tripleo04:32
*** hamzy has joined #tripleo04:36
*** itlinux_ has joined #tripleo04:40
*** gkadam is now known as gkadam-brb04:46
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813605:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,Triaged]05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817205:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817505:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)05:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]05:10
*** limao has joined #tripleo05:12
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] testing individual log files  https://review.openstack.org/47483105:17
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Add timestamp output filter  https://review.openstack.org/47483005:17
*** jbadiapa has quit IRC05:18
*** iranzo has joined #tripleo05:19
*** gkadam-brb is now known as gkadam05:20
*** itlinux_ has quit IRC05:21
*** itlinux_ has joined #tripleo05:25
*** ykarel has joined #tripleo05:31
openstackgerritJaganathan Palanisamy proposed openstack/python-tripleoclient master: Invoke the workflows specified in plan environment file.  https://review.openstack.org/46959605:35
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/47461405:35
*** janki has joined #tripleo05:41
*** pgadiya has joined #tripleo05:41
*** lmiccini has joined #tripleo05:44
*** prateek has quit IRC05:56
*** prateek has joined #tripleo05:56
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] testing individual log files  https://review.openstack.org/47483106:02
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Add timestamp output filter  https://review.openstack.org/47483006:02
*** dmacpher has quit IRC06:04
*** dmacpher has joined #tripleo06:06
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813606:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,Triaged]06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817206:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817506:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)06:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]06:10
*** milan has joined #tripleo06:11
*** yprokule has joined #tripleo06:15
*** nyechiel has joined #tripleo06:19
*** rcernin has joined #tripleo06:19
*** jaganathan has quit IRC06:22
*** jaganathan has joined #tripleo06:23
*** rcernin has quit IRC06:24
*** nyechiel has quit IRC06:28
*** jprovazn has joined #tripleo06:29
*** itlinux_ has quit IRC06:30
*** bogdando has joined #tripleo06:31
*** sid1 has joined #tripleo06:31
*** sid1 has quit IRC06:34
*** jbadiapa has joined #tripleo06:34
*** rcernin has joined #tripleo06:40
*** ebarrera has joined #tripleo06:41
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Enable nova-api to run over httpd again  https://review.openstack.org/47413506:41
*** pcaruana has joined #tripleo06:43
*** florianf has joined #tripleo06:46
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Only pass distinct services to enabled_services list  https://review.openstack.org/47486206:49
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Configure keystone haproxy endpoints through dynamic resource  https://review.openstack.org/47410706:49
openstackgerritHarald Jensås proposed openstack/instack-undercloud master: Tripleo routed networks ironic inspector, and Undercloud  https://review.openstack.org/43754406:49
*** ebarrera has quit IRC06:50
*** ebarrera has joined #tripleo06:51
*** nyechiel has joined #tripleo06:55
*** anshulbehl has joined #tripleo06:57
*** suuuper has joined #tripleo06:57
*** mcornea has joined #tripleo07:00
*** amoralej|off is now known as amoralej07:01
hjensasI have a problem with jinja rendering. I add a new property to all roles in roles_data.yaml, then in puppet/role.role.j2.yaml I add key: {{role.newproperty}}. When it renders the value is blank. Do I need to update the code doing the rendering to be aware of the property, or what?07:05
*** skramaja has joined #tripleo07:06
*** jlinkes has joined #tripleo07:06
*** tesseract has joined #tripleo07:09
*** jtomasek has joined #tripleo07:10
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813607:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,Triaged]07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817207:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817507:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)07:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]07:10
*** nyechiel has quit IRC07:10
*** ccamacho has quit IRC07:14
*** fragatina has joined #tripleo07:20
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Make fernet max active keys configurable  https://review.openstack.org/47453207:26
jaosoriormarios: could you check this out? ^^ It was green but I fixed a typo in the release note.07:26
*** brault has joined #tripleo07:27
*** shardy has joined #tripleo07:32
*** limao has quit IRC07:33
*** limao_ has joined #tripleo07:33
openstackgerritMerged openstack/tripleo-validations master: Adds pre-upgrade/update to the overcloud endpoint validation  https://review.openstack.org/47418707:33
*** stendulker has joined #tripleo07:35
mariosack jaosorior07:38
jaosoriorthanks marios07:38
*** karimb has joined #tripleo07:43
*** dmacpher has quit IRC07:46
*** egonzalez has joined #tripleo07:48
*** dtantsur|afk is now known as dtantsur07:49
*** jpich has joined #tripleo07:50
*** jaganathan has quit IRC07:52
*** paramite has joined #tripleo07:55
*** zzzeek has quit IRC08:00
*** zzzeek has joined #tripleo08:02
*** jbadiapa has quit IRC08:03
shardydciabrin: Hey, did you manage to get your environment up yesterday?08:05
dciabrinshardy, so fun fact, rasca did all the magic for setting it up, it seems baremetal works for us, but not vm08:06
*** jaganathan has joined #tripleo08:06
shardydciabrin: Ok, I rebuilt my VM based environment and it still works well for initial deploy08:06
*** cylopez has joined #tripleo08:06
shardydciabrin: but if you re-run the deploy command again, with no changes, there are errors, I think related to haproxy config08:06
shardyI just wondered if you'd seen that before I start investigating08:07
*** fzdarsky_ is now known as fzdarsky08:09
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813608:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,Triaged]08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817208:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817508:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)08:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]08:10
*** ccamacho has joined #tripleo08:17
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Make a copy of files touched by puppet in container  https://review.openstack.org/47489108:18
*** jbadiapa has joined #tripleo08:19
*** milan is now known as milan|bbl08:19
*** milan|bbl has quit IRC08:20
openstackgerritJaganathan Palanisamy proposed openstack/python-tripleoclient master: Invoke the workflows specified in plan environment file.  https://review.openstack.org/46959608:20
*** rcernin has quit IRC08:21
*** rcernin has joined #tripleo08:22
*** dsariel has joined #tripleo08:24
openstackgerritMartin André proposed openstack/tripleo-quickstart-extras master: Limit collection of config-data to puppet-generated files  https://review.openstack.org/47489608:26
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Improve logs from ansible, puppet, docker-puppet.py  https://review.openstack.org/47426908:27
bogdandofolks, I checked that locally, https://review.openstack.org/#/c/474269/, it seems look better than it is now08:28
bogdandoshardy, jistr, mandre: ^^ PTAL08:28
*** jbadiapa has quit IRC08:29
openstackgerritJaganathan Palanisamy proposed openstack/python-tripleoclient master: Invoke the workflows specified in plan environment file.  https://review.openstack.org/46959608:30
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Improve logs from ansible, puppet, docker-puppet.py  https://review.openstack.org/47426908:31
*** social has joined #tripleo08:33
shardybogdando: thanks, I'll pull it and give it a try08:34
*** florianf has quit IRC08:34
*** zoli|gone is now known as zoliXXL08:35
*** lucas-afk is now known as lucasagomes08:39
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Make a copy of files touched by puppet in container  https://review.openstack.org/47489108:39
openstackgerritMartin André proposed openstack/tripleo-quickstart-extras master: Limit collection of config-data to puppet-generated files  https://review.openstack.org/47489608:39
openstackgerritDerek Higgins proposed openstack/tripleo-quickstart master: [WIP] Test Ironic in Overcloud  https://review.openstack.org/47275508:39
shardydciabrin: FYI the problem seems to be that we get device or resource busy errors despite the move to rsync:08:41
shardyhttp://paste.openstack.org/show/612826/08:41
shardyI think we may have to move to creating a unique directory for every docker-puppet run08:41
*** japestinho has joined #tripleo08:42
*** jbadiapa has joined #tripleo08:44
dciabrinshardy, sorry I'm lacking context here :) what's the pb?08:45
shardydciabrin: any stack updates fail08:46
dciabrinshardy, oh ok saw you last comment only now...08:46
shardydciabrin: I didn't see this previously in my single node testing, but it seems we'll need to rework the docker-puppet workflow to create a new directory every run08:46
dciabrinshardy, oh ok, maybe that's because we're bind mounting /var/lib/config-data/haproxy/etc directly, rather than letting kolla copy the files here08:46
dciabrinshardy, checking08:46
shardydciabrin: ack, happy to hear any other ideas, the unique directory thing is doable but will take some work08:47
shardyI was hoping we could mount the config-data read only then just rsync each docker-puppet run08:47
openstackgerritMerged openstack-infra/tripleo-ci master: Don't capture quickstart.tar.gz  https://review.openstack.org/47481508:49
openstackgerritMerged openstack/tripleo-heat-templates stable/newton: Reconfigure interfaces before updating openvswitch  https://review.openstack.org/47138108:49
*** florianf has joined #tripleo08:51
*** iranzo has quit IRC08:53
*** tvignaud has quit IRC08:55
shardydciabrin: I raised https://bugs.launchpad.net/tripleo/+bug/1698323 to track discussion on the fix08:55
openstackLaunchpad bug 1698323 in tripleo "Stack updates fail during docker-puppet rsync for haproxy" [High,Triaged]08:55
dciabrinshardy ok thx08:55
*** athomas has quit IRC08:56
*** tosky has joined #tripleo08:57
*** iranzo has joined #tripleo09:00
*** iranzo has joined #tripleo09:00
openstackgerritMerged openstack/tripleo-heat-templates master: Conditional LVM storage setup for cinder-volume  https://review.openstack.org/47377509:07
openstackgerritMerged openstack/tripleo-heat-templates master: Ignore case for bootstrap node checks  https://review.openstack.org/47476209:07
*** tvignaud has joined #tripleo09:08
*** athomas has joined #tripleo09:09
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Fall back to non-containerized cinder-volume to unblock CI  https://review.openstack.org/47491509:10
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813609:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817209:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817509:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,In progress] - Assigned to Jiří Stránský (jistr)09:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)09:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]09:10
openstackgerritChristopher Brown proposed openstack/tripleo-docs master: Updates image location  https://review.openstack.org/47378509:11
bogdandofolks, my custom UC deploy is hit by "CREATE_FAILED  StackValidationFailed: resources[9]: Property error: resources.KeystoneBase.properties: Property KeystoneFernetKeys not assigned", but looks like continues anyway. I did yum update from the current-tripleo, shall I fallback to the "current" or is it something non critical?09:15
bogdandojaosorior: ^^ by a chance? :)09:16
jistrbogdando: could it be that you updated t-h-t but missed tripleo-common or tripleoclient?09:17
shardybogdando: did you re-run openstack undercloud install or just update the packages?09:17
bogdandoshardy, jistr: re-running after yum update09:18
shardybogdando: it sounds like you need https://github.com/openstack/tripleo-common/commit/2fb512160cdb8d3806196732036a5b7bb8a1362309:18
shardythat will require restarting the mistral services at least09:19
bogdandohm, yum log doesn't mention updates installed for those two09:19
bogdandoperhaps I need current dlrn09:19
*** akrivoka has joined #tripleo09:20
amoralejping sshnaidm|off, question about current-tripleo promotion09:21
openstackgerritChristopher Brown proposed openstack/tripleo-docs master: Updates image location  https://review.openstack.org/47378509:22
*** limao_ has quit IRC09:24
*** akrivoka has quit IRC09:24
*** limao has joined #tripleo09:25
*** akrivoka has joined #tripleo09:26
*** limao has quit IRC09:29
mariosmatbu: is there something waiting for https://review.openstack.org/#/c/470445/ ?09:33
mariosmatbu: or shall we land it?09:33
*** sudipto has joined #tripleo09:33
*** sudipto_ has joined #tripleo09:34
*** snecklifter_ has joined #tripleo09:35
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-ui master: Imported Translations from Zanata  https://review.openstack.org/47492609:35
shardydoes anyone know if it's possible to inspect all container bind mounted directories from the host?09:39
shardyneither mount nor findmnt seem to contain the data09:39
openstackgerritMerged openstack/python-tripleoclient master: Updated from global requirements  https://review.openstack.org/47055509:41
*** ebarrera has quit IRC09:42
*** ebarrera has joined #tripleo09:43
*** yamahata_ has quit IRC09:47
openstackgerritMerged openstack-infra/tripleo-ci master: Use undeprecated heat commands in get_host_info.sh  https://review.openstack.org/47483409:47
jaosoriorbogdando: you need to update your tripleo-common09:47
*** dmacpher has joined #tripleo09:47
bogdandojaosorior: yupp, thanks. Already did that from the current dlrn09:47
jaosoriorshardy: got a chance to check this out https://review.openstack.org/#/c/474862/ ?09:48
*** zoliXXL is now known as zoli|lunch09:48
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Configure keystone haproxy endpoints through dynamic resource  https://review.openstack.org/47410709:49
*** limao has joined #tripleo09:50
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Enable nova-api to run over httpd again  https://review.openstack.org/47413509:51
*** limao_ has joined #tripleo09:53
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo master: For http service endpoints always redirect to https  https://review.openstack.org/47410809:54
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Add resource to create haproxy endpoints dynamically  https://review.openstack.org/47410909:54
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Rely on dynamic haproxy endpoints to configure keystone  https://review.openstack.org/47411009:54
jpichhonza: You might have an idea about what is causing this, when you're around? https://bugs.launchpad.net/tripleo/+bug/169819409:55
openstackLaunchpad bug 1698194 in tripleo "tripleo_ui language settings not displayed correctly" [Medium,Triaged]09:55
*** limao has quit IRC09:55
*** limao_ has quit IRC10:01
*** limao has joined #tripleo10:02
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo master: Create the vhost root folder before creating the virtual host  https://review.openstack.org/47452610:06
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813610:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,In progress] - Assigned to Jiří Stránský (jistr)10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817210:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817510:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)10:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]10:10
jistrshardy: `docker inspect $CONTAINER`10:11
jistrshardy: if i understood the question right10:11
openstackgerritNuman Siddique proposed openstack/tripleo-heat-templates master: CI: Enable OVN services for scenario 3  https://review.openstack.org/41283110:11
shardyjistr: yeah, I was hoping to get the host mount information, as I'm not sure docker inspect will give the info I need if the bind mount is a symlink10:12
shardyjistr: to avoid the foo in user errors I mentioned earlier, I think we need docker-puppet to create a new directory every run10:12
shardythen switch a symlink and update the docker environment with the readlink information about the target10:12
shardybut the question then is how to decide which old dirs can be cleaned up10:13
*** limao has quit IRC10:13
snecklifter_Morning #tripleo, would it be possible to get https://review.openstack.org/#/c/473785/ merged please?10:13
shardyI was hoping we could inspect the host mount/lsof information but it doesn't seem possible10:13
snecklifter_dtantsur: thanks for your comments on the above10:13
*** limao has joined #tripleo10:13
*** ratailor has quit IRC10:13
snecklifter_I never thought getting a URL updated on a doc would be quite so involved but I think we have got there :)10:13
*** nenad has joined #tripleo10:14
dtantsurnp :) yeah, this happens quite often with documentation changes10:14
shardyjistr: maybe we'll have to read the environment of all running containers via docker inspect, and always have the new and n-1'th config-data10:14
shardysnecklifter_: done10:15
snecklifter_shardy: many thanks10:16
shardythanks for the fix10:16
openstackgerritMerged openstack/tripleo-docs master: Updates image location  https://review.openstack.org/47378510:18
*** dsariel has quit IRC10:19
*** ratailor has joined #tripleo10:21
jistrdciabrin: re your comment at https://bugs.launchpad.net/tripleo/+bug/1698323 -- we do the direct /etc mounts in other containers too, rather than having Kolla copy files. So this may not be specific to HAProxy i think...10:22
openstackLaunchpad bug 1698323 in tripleo "Stack updates fail during docker-puppet rsync for haproxy" [High,Triaged]10:22
*** limao has quit IRC10:24
dciabrinjistr, oh damned :( I have the patches ready anyway, so I'll push reviews so hopefully shardy can check whether it makes a difference or not :/10:24
shardyyeah I think we'll have to decide if we can avoid holding files open in /var/lib/config-data - if not we'll have to create a unique directory each docker-puppet run, and find a way to compare the new directory with the existing one10:26
jistrdciabrin: right, well it may make a difference. We do the direct mount everywhere, but it may actually cause real problem only for HAProxy perhaps...10:27
*** panda|ko is now known as panda10:28
openstackgerritMerged openstack/instack-undercloud master: Updated from global requirements  https://review.openstack.org/47464110:29
openstackgerritMerged openstack/instack-undercloud master: Add numa-topology colector to the list of default collectors  https://review.openstack.org/47412010:29
*** jprovazn has quit IRC10:29
jistrbut the generic issue "lurks" over all containers then. It only depends on the service's particular behavior.10:30
openstackgerritMerged openstack/tripleo-heat-templates master: Pacemaker HA suport for OVN DB servers  https://review.openstack.org/43676710:30
*** rhallisey has joined #tripleo10:32
*** fpan_ has joined #tripleo10:38
*** fpan has quit IRC10:38
*** mrchao has joined #tripleo10:39
*** gkadam has quit IRC10:40
jistrshardy, dciabrin: interesting, on my system i can't find anything holding those files (not sure if i'm doing it right). lsof and lslocks doesn't mention anything re haproxy10:41
dciabrinjistr, hmm10:41
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Improve logs from ansible, puppet, docker-puppet.py  https://review.openstack.org/47426910:42
jistrthe reason i tried this is that i wanted to guess for how many services this could be an issue10:42
jistri think puppet does a md5 check before it attempts to change a file, so i thought it could be possible that many more services actually have this problem but we'd only hit it if the config actually changes during the update10:43
jistrbut not sure how to check for the other services though, as i'm not able to see this problem with HAProxy either (via lsof/lslocks)10:44
*** stendulker has quit IRC10:45
bogdandojaosorior: and folks, oh, I have mysql init tasks error back again :)10:46
bogdandoJust "Error: Failed to apply catalog: Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)"10:47
*** sid1 has joined #tripleo10:47
bogdandoI have rebased on top of the jistr's patch fixing container races10:48
bogdandobut ... sigh10:48
jaosoriorbogdando: shouldn't it be the case that we can't log in using localhost?10:48
jaosorioryou should be using the IP or network-bound hostname for the node instead10:48
bogdandojaosorior: it can't login cuz of the file is not in place, I can't track down why. Puppet logs reported it was bucketed10:48
bogdando.my.cnf10:49
jaosoriorah10:49
jaosoriorfunky10:49
bogdandoindeed10:49
*** sid2 has joined #tripleo10:49
*** katkapilatova has joined #tripleo10:50
*** sid1 has quit IRC10:51
*** pgadiya has quit IRC10:51
*** nyechiel has joined #tripleo10:54
bogdandoso is it a bug? https://pastebin.com/N60zbB0p10:55
bogdandowhy it runs w/o mounting the .my.cnf?10:55
bogdandoI wondering why it works for CI w/o issues...10:55
*** dmanchad has quit IRC10:56
*** dmanchad has joined #tripleo10:56
bogdandoit's hard to say how to repro it, but looks like a bug in tht10:57
bogdandoshall we always mount .my.cnf for mysql init steps?10:57
bogdandojistr, shardy, mandre: ^^ thoughts please?10:58
bogdandoif I bind-mount it, all works10:59
*** jkilpatr has joined #tripleo10:59
*** zoli|lunch is now known as zoliXXL11:00
EmilienMhellow11:00
bogdandohttps://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/docker/services/database/mysql.yaml#n146 do not always provides .my.cnf! :D11:01
bogdandoEmilienM: o/11:01
openstackgerritDamien Ciabrini proposed openstack/tripleo-heat-templates master: Use HAproxy config files copied by Kolla bootstrap  https://review.openstack.org/47494311:02
*** bfournie has quit IRC11:02
*** bfournie has joined #tripleo11:03
*** bfournie has quit IRC11:04
*** morazi has quit IRC11:05
openstackgerritNuman Siddique proposed openstack/tripleo-common master: Add OVN container images  https://review.openstack.org/47494411:06
bogdandohttps://bugs.launchpad.net/tripleo/+bug/169834911:07
openstackLaunchpad bug 1698349 in tripleo "mysql_init_tasks does not always provide .my.cnf" [Medium,Triaged]11:07
*** agopi has joined #tripleo11:07
*** salmankhan has joined #tripleo11:07
*** nyechiel has quit IRC11:08
*** fultonj has quit IRC11:09
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Make sure .my.cnf bind-mounted  https://review.openstack.org/47494511:09
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813611:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,In progress] - Assigned to Jiří Stránský (jistr)11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817211:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817511:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)11:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]11:10
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs  https://review.openstack.org/35921511:11
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud master: Make neutron's dns_domain parameter configurable  https://review.openstack.org/47494611:14
sshnaidm|offpanda, can you respond please on https://review.openstack.org/#/c/474504 ? we need to fix it asap11:14
openstackgerritDamien Ciabrini proposed openstack/puppet-tripleo master: Use HAproxy config files copied by Kolla bootstrap  https://review.openstack.org/47494711:16
* jistr lunch, biag11:17
jistr*biab11:17
openstackgerritMerged openstack/tripleo-common master: Define default Image and Volume API versions in overcloudrc  https://review.openstack.org/47354011:18
*** bfournie has joined #tripleo11:21
*** lucasagomes is now known as lucas-hungry11:22
dciabrinshardy https://review.openstack.org/#/c/474943 and https://review.openstack.org/#/c/474947 <-- for the issue with haproxy container during stack update.11:22
dciabrinjistr, ^^11:22
*** sudipto_ has quit IRC11:24
*** sudipto has quit IRC11:24
EmilienMjaosorior: hey, could you please have a look at https://review.rdoproject.org/r/#/c/7098/ when you have some time, maybe today11:26
jaosoriorEmilienM: oh man, that's tricky11:28
jpichjaosorior: Sorry for the basic question - basically the fernet patches are necessary because it's not possible to distribute the keys over all the controllers at the moment, do I understand correctly? I was scratching my head a bit as to why we seem to be recreating the keystone-manage command11:28
EmilienMnevermind, just approve it11:28
EmilienM:D11:28
jaosoriorEmilienM: I don't have +2 rights there :/11:28
EmilienMjaosorior: I was joking.11:28
EmilienMjaosorior: your -1 / +1 is enough11:28
EmilienMjaosorior: I can re-explain the problem that I'm trying to solve if oyu need11:28
EmilienMbut the commit message explains quite well I think11:29
jaosoriorjpich: you are correct. It recreates the keystone-manage command, which can't be ran in a cluster :/11:29
jaosoriorEmilienM: right, I read the commit message. With my limited understanding of the issue it looks good (not sure what carrying the policy file in code actually entails)11:29
jaosoriorEmilienM: I'm actually asking the same questions for barbican though; since the same is being proposed over there.11:29
EmilienMjaosorior: where?11:30
jaosoriorlet me try to dig out the patch.11:30
jaosoriorbut yeah, all I know is that it's what projects are aiming for nowadays11:30
jaosoriorEmilienM: https://review.openstack.org/#/c/431524/11:31
EmilienMjaosorior: yes, there is a community goal, which is imho awesome11:31
*** japestinho has quit IRC11:32
EmilienMjaosorior: https://review.openstack.org/#/c/469954/11:32
jaosoriorEmilienM: I have my doubts about it to be honest. A lot of operators tend to modify the default policy... having the policy in one file makes it easier for deployers to have an overall view of it.11:32
jaosoriorgetting it embedded in the code makes that harder...11:32
jaosorioraaand doing further modifications of such a policy will be harder too11:32
cschwedemandre: Hi! I stumbled upon this commit: https://github.com/openstack/tripleo-validations/commit/2e1a198c and was wondering about this - do you remember this issue? was there a bug or something?11:33
EmilienMjaosorior: that's why we provide the option to manage it via the file, in tripleo11:33
EmilienMjaosorior: tripleo.org/advanced_deployment/api_policies.html11:33
*** jprovazn has joined #tripleo11:33
jaosoriorEmilienM: which I think it's a great option11:33
EmilienMbrb 20 minb11:33
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud master: Make neutron's dns_domain parameter configurable  https://review.openstack.org/47494611:37
*** tbonds has quit IRC11:38
*** ansmith has quit IRC11:38
jpichjaosorior: Ok, thank you! Do we expect the keys to be updated on stack update, then? I'm not familiar with when/how hiera data gets updated11:39
jaosoriorjpich: so, right now the only way to do it (once this work merges) is via a stack update. But, the next step is to create a workflow that actually does the rotation and persists it to the nodes hosting keystone on demand.11:40
jaosoriorjpich: the way it's gonna work is via ansible. And now that the ansible actions in tripleo-common merged, it's possible11:40
*** tbonds has joined #tripleo11:40
jaosoriorso that's the next step :)11:40
jpichjaosorior: Great, thank you for the explanations, I appreciate it!!11:41
jaosoriorjpich: no biggie, thanks a lot for checking it out11:43
*** salmankhan has quit IRC11:47
*** tbonds has quit IRC11:49
*** agopi has quit IRC11:50
*** tbonds has joined #tripleo11:51
*** pkovar has joined #tripleo11:51
*** udesale has quit IRC11:51
*** abishop has joined #tripleo11:52
*** morazi has joined #tripleo11:58
*** ykarel has quit IRC11:58
*** ykarel has joined #tripleo11:58
jaosoriorcan someone take a look at this t-h-t patch https://review.openstack.org/#/c/474862/ ?11:59
*** sbrzozow has joined #tripleo12:00
*** sid2 has quit IRC12:02
openstackgerritMerged openstack/instack-undercloud master: Fix the port for Panko API  https://review.openstack.org/47157712:05
openstackgerritMerged openstack/python-tripleoclient master: Replace assertRaisesRegexp with assertRaisesRegex  https://review.openstack.org/47447012:05
openstackgerritMerged openstack/tripleo-ui master: Imported Translations from Zanata  https://review.openstack.org/47492612:05
jaosoriormwhahaha, EmilienM: If I need to get another hiera value that returns a list from hieradata set in t-h-t. How would I do it? I tried this: server_names: "%{hiera('keystone_admin_api_node_names')}"    in here https://review.openstack.org/#/c/474107/6/puppet/services/keystone.yaml   and it seems to be failing cause it interprets it as a string instead of a list: parsing12:05
jaosorior[/etc/haproxy/haproxy.cfg20170616-13165-mo2g94:141] : character '[' is not permitted in server name '["overcloud-controller-0.internalapi.localdomain",'.12:05
*** dpivonka has quit IRC12:07
openstackgerritAlan Bishop proposed openstack/tripleo-heat-templates stable/ocata: Add support for Cinder "NAS secure" driver params  https://review.openstack.org/47495512:08
jaosoriorshardy: what could be causing os-collect-config to run again?12:09
*** fultonj has joined #tripleo12:09
jaosoriorshardy: I triggered an undercloud deployment and it seems that os-collect-config starts running again and again12:09
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813612:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,In progress] - Assigned to Jiří Stránský (jistr)12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817212:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817512:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)12:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]12:10
jaosoriorshardy: keeps running puppet and refreshing services for some reason :/12:11
jaosoriorshardy: http://paste.openstack.org/show/612844/ :/12:12
*** adarazs is now known as adarazs_brb12:14
*** dprince has joined #tripleo12:20
shardyjaosorior: it's enabled on my undercloud too, which might be a mistake, but it shouldn't re-run anything unless the metadata changed12:20
*** sid1 has joined #tripleo12:21
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/47469712:24
*** dpivonka has joined #tripleo12:24
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements  https://review.openstack.org/47461412:24
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-validations master: Updated from global requirements  https://review.openstack.org/47292912:24
EmilienMjaosorior: let me look12:26
EmilienMjaosorior: it reminds me an old issue12:27
EmilienMjaosorior: we we have to backslash something12:27
EmilienMjaosorior: gimme some time12:29
*** pkovar has quit IRC12:29
*** arxcruz is now known as arxcruz|off12:32
*** sid1 has quit IRC12:33
*** sid1 has joined #tripleo12:33
*** adarazs_brb is now known as adarazs12:34
*** pkovar has joined #tripleo12:34
shardyjistr, dciabrin: So, I'm trying to figure out the next steps re bug #1698323 - do you think the kolla bootstrap approach could be used everywhere, or should I go ahead and prototype the versioned directories solution?12:36
openstackbug 1698323 in tripleo "Stack updates fail during docker-puppet rsync for haproxy" [High,In progress] https://launchpad.net/bugs/169832312:36
shardydprince: ^^ If you're around interested in your views on this also12:36
*** ratailor has quit IRC12:36
*** trown|outtypewww is now known as trown12:37
dciabrinshardy, have you got a chance to test the reviews for haproxy?12:38
*** morazi has quit IRC12:38
*** sid1 has quit IRC12:38
dprinceshardy: looking...12:39
*** rlandy has joined #tripleo12:40
*** morazi has joined #tripleo12:40
dprinceshardy: commented, if we revert f600d459f051288042ce531bab029953563a11b3 would it fix it?12:41
dprinceshardy: while I like the cleanup aspects of stevebaker's rsync patch it was a cleanup. No net new functionality there that would cause a regression if we reverted it12:42
shardydprince: No, because previously the rm -fr failed12:42
*** prateek has quit IRC12:42
shardydprince: it's the same issue, I previously assumed the rsync would fix it because we're no longer removing the directory12:42
shardydprince: so I think we have two options - make the containers take a copy instead of bind mounting and holding files open on the host12:43
shardyor version the /var/lib/config-data directories, e.g append a timestamp every time docker-puppet runs or something12:43
shardythe tricky part of that will be working out when we can clean them up12:43
shardybut we can probably add logic to docker-puppet to introspect all the containers via docker inspect and ensure there's data about the absolute path that's in use contained there (e.g in the kolla environment like we do for the hash atm)12:44
*** lucas-hungry is now known as lucasagomes12:44
shardyif we could just copy the data, it'd be much simpler12:44
dprinceshardy: the timestamp complexity makes debugging a bit more painful too12:44
shardyyeah12:44
shardywe'd need to regenerate the config, diff with the old one, and switch a symlink if the new config differs12:45
dprinceshardy: more painful debugging is sort of a feature of using containers I think ;)12:45
shardythen restart containers, and later remove the old config dir12:45
shardyhehe, true12:45
shardydprince: if we do https://review.openstack.org/#/c/474943/1/docker/services/pacemaker/haproxy.yaml everywhere, would that fix this?12:46
shardydciabrin: sorry no not yet - I'm about to try it now12:47
jistris updating configs broken for everything, or just for haproxy?12:47
dprinceshardy: perhaps12:47
dprinceshardy: that interface is super verbose though12:47
shardyjistr: it'll be broken for anything where a file is held open and we try to update it12:47
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates master: Add missing Swift object reconstructor for containerized deployments  https://review.openstack.org/44589112:47
dprinceshardy: like for all the Apache API's it ends up being like 4-6 files per service in some cases12:47
shardyurgh12:48
dprinceshardy: we moved away from using 'kolla_config' as it duplicates logic in our config tooling12:48
dprinceshardy: we can do it though12:48
*** ratailor has joined #tripleo12:48
jistrshardy: yea... just wondering how many services it causes an actual breakage12:48
shardydprince: ack, understood12:48
jistri'd expect most services just read their /etc and close it12:48
shardyjistr: yeah I can test and find out, but it seems like a fragile pattern12:48
jistryea it is...12:49
dprinceother than duplicating the files that are actually used kolla_config is totally valid12:49
dprinceshardy: use it just for this service?12:49
jistrthough the solutions to make it less fragile aren't very appealing12:49
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates master: Add missing Swift containersync service for containerized deployments  https://review.openstack.org/47496412:49
dprincejistr: I think the various policy.json files are immediately effective too12:49
shardydprince: yeah, I think there are others though, e.g mysql was failing the rm -fr so I assume it'll fail in the same way12:50
dprincejistr: most files are read at startup, but some could be read periodically, or each time they are changed12:50
shardyyeah that could mean this is racy12:50
jistrhm ack :/12:50
* shardy sighs12:50
dprinceshardy: if kolla_config fixes this we can do it12:50
shardydprince: Ok let me test the patch from dciabrin and see where we get to12:51
shardythanks for the feedback12:51
dprinceshardy: I think the question is, do we want to keep the /var/lib/config-data tree more easy to understand on the host12:51
dprinceshardy: or do we use timestamps12:51
*** liverpooler has joined #tripleo12:51
shardydprince: if possible, I would prefer not to have the timestamp and lifecycle/cleanup complexity on the host12:51
*** limao has joined #tripleo12:51
shardyit'd be logically simpler if the containers just took a snapshot on boot12:52
shardyIMHO12:52
dprinceshardy: right, I think given the timing we shouldn't introduce a new complexity12:52
dprinceshardy: kolla_config already exists. We've used it before12:52
*** psahoo has quit IRC12:52
dprinceshardy: if it solves this, lets just go back and use it where needed, or everywhere for that matter12:52
dprincejistr: agree?12:52
shardyack, well perhaps its only a few places, I'll test and find out12:53
jistrdprince, shardy: yes. Copying sounds much simpler/safer than the symlink moving + old dirs cleanup.12:53
*** milan has joined #tripleo12:54
dprinceshardy: the timestamp thing to me sounds like what some people were asking for around logging (unique directories for each service). I consider that a new feature of sorts12:54
shardydprince: yeah, it could be done, but it's adding a bunch more complexity to docker-puppet.py12:54
dprinceshardy: which we could do, but this isn't the time in the release to do it12:55
shardy++12:55
dprinceshardy: unless we absolutely have to of course12:55
*** akuznetsov has joined #tripleo12:55
dprinceshardy: we could even blueprint it if it is of interest for Queens, lets see where we end up after this and get a feel for it12:56
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] toci_gate_test: handle tripleo-ci specific feature configuration  https://review.openstack.org/47496612:56
*** jaganathan has quit IRC12:56
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Keep floating ip reachability during pacemaker migration.  https://review.openstack.org/47496712:56
dprincemandre: hey, bit of backlog ^^^ you might be interested in there.12:56
dprincemandre: who knew 'kolla_config' would save the day!12:57
mandredprince: where do I start reading?12:57
*** akuznetsov has quit IRC12:57
*** ecerquei__ has joined #tripleo12:57
*** limao has quit IRC12:58
dprincemandre: about 30 minutes ago12:58
*** limao has joined #tripleo13:00
*** sasha2 has joined #tripleo13:00
*** limao has quit IRC13:02
*** limao_ has joined #tripleo13:02
jistrdprince, shardy: FWIW it supports copying directories too, so we can just copy full /etc/<service>, we don't have to go file by file.13:02
jistrhttps://github.com/openstack/kolla/blob/7b27262216d0bef65dc4c6ed24b96285bbba11db/docker/base/set_configs.py#L138-L14113:02
*** jcoufal has joined #tripleo13:03
shardyaha, that sounds much easier13:03
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] toci_gate_test: handle tripleo-ci specific feature configuration  https://review.openstack.org/47496613:04
openstackgerritFlavio Percoco proposed openstack/tripleo-common master: Use tempdir instead of files for ansible-action  https://review.openstack.org/47358613:05
openstackgerritFlavio Percoco proposed openstack/tripleo-common master: Change cwd and set HOME for ansible actions  https://review.openstack.org/47358713:06
dprincejistr: yes, that would help13:07
mdboothAfternoon. Trying to install OSP8, compute node is idle with this in the logs:13:08
mdboothJun 16 09:07:39 overcloud-compute-0.localdomain os-collect-config[2532]: 2017-06-16 09:07:39.524 2532 WARNING os_collect_config.cfn [-] 403 Client Error: AccessDenied13:08
mdboothLots of this in the logs13:08
*** limao has joined #tripleo13:08
*** cdearborn has joined #tripleo13:09
mdboothSlightly more: http://paste.openstack.org/show/612855/13:09
mdboothAny idea what is being denied?13:09
*** fpan_ has quit IRC13:10
shardymdbooth: it's failing trying to reach heat-api-cfn on the undercloud13:10
shardyI'd check it's running and any log clues there13:10
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813613:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,In progress] - Assigned to Jiří Stránský (jistr)13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817213:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817513:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)13:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]13:10
*** limao_ has quit IRC13:10
shardymdbooth: also, check /etc/os-collect-config.conf - there will be a URL there which you can test by curling it (quote the URL to avoid the & backgrounding the curl)13:11
shardythat will hopefully reproduce the error and you can work back from there13:11
mdboothshardy: Thanks13:11
bogdandoshardy: "if we could just copy the data, it'd be much simpler" +1, so we're back to using kolla's kolla_config it seems13:12
*** gbarros has joined #tripleo13:12
mdboothJun 16 09:12:48 undercloud.redhat.local heat-api-cfn[20881]: 2017-06-16 09:12:48.373 20881 INFO heat.api.aws.ec2token [-] AWS authentication failure.13:13
*** f13o has joined #tripleo13:13
mandredprince: ok, let's give kolla_config another try, this time maybe a little less verbose thanks to the new copy dir feature13:13
bogdandodprince: "more painful debugging is sort of a feature of using containers" haha, indeed! and I'm trying to ease pain here https://review.openstack.org/#/c/474269/, a little bit13:13
*** zoliXXL is now known as zoli|brb13:13
*** dpivonka has quit IRC13:13
openstackgerritDerek Higgins proposed openstack/tripleo-heat-templates master: [WIP] Implement scenario005 with Ironic in overcloud  https://review.openstack.org/47480213:13
shardymdbooth: Ok so next stop is keystone logs - the cfn api uses the keystone ec2tokens extension to validate the signature13:13
*** ratailor has quit IRC13:14
*** jmelvin has joined #tripleo13:14
bogdandoshardy, dprince: what if store config files with self expire attribute, versioned, under etcd keys? :)13:15
bogdandolet's start leveraging etcd right now, eh? :)13:15
shardybogdando: that sounds like something to consider for a future release perhaps?13:15
bogdandowe could store foo's configs versioned under some /foo/v<sha>, in base64'ed tarball13:16
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates master: Add Telemetry services to scenario002  https://review.openstack.org/47496913:16
bogdandofuture releases should be using a nice looking configs view13:16
bogdandowith tools from olso.config for etcd13:16
bogdandobut for now we could think of that shortcut, but then we need to deploy etcd early13:17
*** sid1 has joined #tripleo13:17
mdboothshardy: Do you happen to know much about keystone?13:17
mdboothLogs look like a bit of a failboat13:17
bogdandobut maybe just using kolla_config would work for now13:17
*** jcoufal_ has joined #tripleo13:17
shardymdbooth: not really tbh, other than having fixed a few bugs over the years13:18
bogdandobut then all configs will be ephemeral for container run-time hm13:18
honzarlandy: unfortunately, the DNS server patch doesn't fix things for me --- should i open a new bug with details?13:18
rlandyhonza: when did you try it?13:18
*** atoth has quit IRC13:18
bogdandoum, given a specific version though13:18
honzarlandy: about 16h ago13:18
*** limao has quit IRC13:18
dprincebogdando: ack, will give your review to improve logging a try13:18
mdboothhttp://paste.openstack.org/show/612856/13:19
rlandyhonza: the caching server was shut down due to security concerns13:19
mandredprince: the cinder-scheduler container is failing on a permission issue, http://logs.openstack.org/15/474915/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq-nv/5f98a83/logs/oooq/overcloud-controller-0/var/log/containers/cinder/cinder-scheduler.log.txt.gz#_2017-06-16_11_20_32_00413:19
*** pkovar has quit IRC13:19
*** atoth has joined #tripleo13:19
mdboothAnything stand out in there as obvious misconfigured?13:19
bogdandojistr, makes sense, let's just copy dirs with kolla config and remove host-path dances13:19
rlandyhonza: I am uploading a new patch that defaults to opendns for the moment13:19
rlandyI will ping you to try again13:19
honzarlandy: excellent!13:19
*** limao has joined #tripleo13:19
mdboothThis is pretty much a stock OSP8 install with no customisations13:19
mandredprince: do you know if /var/lib/cinder/conversion needs to be bind-mounted or just the ownership adjusted?13:19
*** jcoufal has quit IRC13:19
rlandyhonza: apologies for the run around - this is a developing issue for us.13:20
honzarlandy: no worries at al13:20
honzaal13:20
honzaall13:20
bogdandonot sure how to mitigate configs versioning then13:20
honzaugh13:20
mdboothWas ooo supposed to configure certificates?13:20
dprincemandre: I think conversion is a directory for ephemeral things. So maybe the permissions, but we should confirm w/ Cinder13:20
*** limao has quit IRC13:20
shardy"There is either no auth token in the request or the certificate issuer is not trusted"13:20
shardymdbooth: sounds suspicious13:21
shardyjaosorior: ^^ any ideas?13:21
jistrbtw here is first passing multinode-containers job after 2 days... I was hoping for OVB too but there's something more to be done apparently...13:21
jistrhttps://review.openstack.org/#/c/474915/13:21
shardyjistr: \o/13:21
*** fragatin_ has joined #tripleo13:21
*** fragatina has quit IRC13:21
jistressentially just a revert though...13:21
dprincemandre: I would just fix it, and then we can copy them on the fix13:22
jaosoriorshardy: where did you see that?13:22
dprincemandre: jbernard, eharney, and gorka13:22
mandrejistr: just commented on the ovb job, it's a permission issue in cinder-scheduler13:22
shardyjaosorior: in the logs mdbooth just pasted - http://paste.openstack.org/show/612856/13:22
mdboothHmm, ooo is rebuilding the computes13:22
*** limao has joined #tripleo13:23
jaosoriormdbooth: logs from where? or what was the action that triggered that?13:23
mandrejistr: it's a lot easier to debug with logs :) thanks for the fix13:23
mdboothjaosorior: Action was installation (OSP8)13:23
jistrmandre: LOL yes13:23
mdboothjaosorior: Logs are from keystone on the undercloud13:23
jaosoriorwow13:24
jaosoriorwell, that's pretty strange13:24
shardyjaosorior: os-collect-config failed to poll the heat-api-cfn, which failed because keystone nacked the ec2tokens signature13:24
shardywhich may just mean keystone is broken vs there actually being a bad signature13:24
jaosoriormdbooth: is heat running over httpd?13:24
jistrmandre: are you going after the cinder-scheduler issue with a patch or should i try? i'm hoping it would be an additional chown13:25
mdboothjaosorior: pass :) Heat is dark magic to me.13:25
jaosoriormdbooth: do you know what endpoint was used for keystone? was it the public one? and if it is, is it using tls?13:25
*** pchavva has joined #tripleo13:25
*** dtantsur is now known as dtantsur|brb13:25
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Fix ownership issue for cinder-scheduler  https://review.openstack.org/47497113:25
mdbooth[cfn]13:26
mdboothmetadata_url = http://172.16.0.1:8000/v1/13:26
mdboothstack_name = overcloud-Compute-br7vyhiy5nha-0-6auwgm72rokw13:26
mdboothsecret_access_key = 9b3bae445a884f0095fdfb2bab423a7413:26
mdboothaccess_key_id = 46664c2f508e4bd68739d3663d10f23513:26
mdboothpath = NovaCompute.Metadata13:26
jaosoriorshardy: there were similar issues when we first migrated heat to run over httpd. It required an extra setting in the apache configuration. But that shouldn't be the case in osp813:26
jistrmandre: actually maybe not, the permission issue is not within a volume mount. Strange...13:26
mdboothThat's on a compute13:26
*** dpivonka has joined #tripleo13:26
mdboothWait... it's heat on the undercloud hitting keystone13:27
mandrejistr: yeah, that's local to the container13:27
mdboothSec..13:27
*** swright has joined #tripleo13:27
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Change default DNS name server  https://review.openstack.org/47433413:27
jaosoriormdbooth: are openstack-heat-cfn and openstack-heat-api running?13:29
mdboothThis is what heat is doing: http://paste.openstack.org/show/612858/13:29
mdbooththat's heat-cfn13:30
*** amoralej is now known as amoralej|lunch13:30
*** ffiore has joined #tripleo13:30
mdboothThey're both running13:30
*** akuznetsov has joined #tripleo13:30
jaosoriormdbooth: right, just wanted to verify if they're running over httpd or not.13:31
jaosoriormdbooth: was there anything after 2017-06-16 09:17:45.137 21121 DEBUG keystone.middleware.core [req-46781308-5323-484c-abc3-386828c0c733 - - - - -] There is either no auth token in the request or the certificate issuer is not trusted. No auth co:    ??13:32
jaosoriorthat seemed truncated13:32
mdboothjaosorior: More of the same13:32
*** limao_ has joined #tripleo13:32
*** swright has quit IRC13:32
*** pkovar has joined #tripleo13:33
*** b00tcat has quit IRC13:33
jistrmandre: maybe i see it... i think the error is actually coming from cinder-volume service, and the problem is that we have containerized cinder-backup, which chowns /var/lib/cinder as the kolla's cinder user. I'll no-op cinder-backup too.13:33
mandrejistr: oh, it cinder-scheduler fails to create the /var/lib/cinder/conversion directory... that's odd, because the permission in the image seems right, http://paste.openstack.org/show/612860/13:33
*** b00tcat has joined #tripleo13:33
mandrejistr: that would make more sense13:34
jistrmandre: i'll try another patchset then. Or do you think we should merge this one as-is first?13:35
*** limao has quit IRC13:35
jaosoriormdbooth: so there's nothing after the "No auth co:" bit?13:37
mandrejistr: we can merge your first one already as it makes the containers-multinode CI happy again13:37
jistrmandre: ok so i'll post a follow up patch right on top of the current one13:38
mandrejistr: or maybe disable all of the conterinerized services and we'll fix everything in another patch series13:38
mdboothjaosorior: Yeah, there's more logs lines exactly like that13:38
*** zoli|brb is now known as zoliXXL13:38
jistrmandre: you mean all cinder related?13:38
* jistr checks scheduler and api manifests13:39
mdboothjaosorior: I can send you the whole log if you like13:39
jaosoriormdbooth: sure13:39
mdboothI really don't know what I'm looking at13:39
mdboothjaosorior: What's your email?13:40
openstackgerritFlavio Percoco proposed openstack/tripleo-common master: Set perms to 0600 for the ssh key file  https://review.openstack.org/47497613:40
*** limao has joined #tripleo13:41
*** limao_ has quit IRC13:41
jistrmandre: i think api/scheduler look ok... i'm hoping they wouldn't go touch /var/lib/cinder directly...13:41
mandrejistr: these 2 should be fine I think13:42
mandrejistr: we have multipathd that also bind mounts /var/lib/cinder rw :-/13:43
mandreso much for the hybrid model13:44
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Keep floating ip reachability during pacemaker migration.  https://review.openstack.org/47496713:47
*** ffiore has quit IRC13:47
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Fall back to non-containerized cinder-backup to unblock OVB CI  https://review.openstack.org/47497813:47
jaosoriormdbooth: I'm not sure what the issue is, sorry :/13:48
jaosoriorshardy ^^13:48
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Fall back to non-containerized cinder-backup to unblock OVB CI  https://review.openstack.org/47497813:48
openstackgerritChristopher Brown proposed openstack/tripleo-docs master: Remove K, L & M release docs  https://review.openstack.org/47497913:49
*** limao has quit IRC13:49
shardyjistr, dciabrin: so it seems https://review.openstack.org/#/c/474943/1/docker/services/pacemaker/haproxy.yaml doesn't work becuse resolv.conf is open, I wonder if we need to remove the bind mount and only use kolla_config?13:49
*** limao has joined #tripleo13:49
*** agopi has joined #tripleo13:49
*** shardy is now known as shardy_afk13:51
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates master: Add missing Swift containersync service for containerized deployments  https://review.openstack.org/47496413:51
openstackgerritAlan Bishop proposed openstack/tripleo-quickstart master: Don't fail if VM extradisks are already attached  https://review.openstack.org/47498113:51
jistrmandre: so the mystery why multinode was fixed while OVB wasn't is explained i think, see latest comment at https://review.openstack.org/#/c/47491513:52
jistrdprince, shardy_afk: ^ i think that patch is good to land, to get multinode CI going again. Hopefully the follow-up would fix OVB too.13:52
jaosoriord0ugal: could you check this out https://review.openstack.org/#/c/474573/ ?13:53
d0ugaljaosorior: sure13:54
openstackgerritChristopher Brown proposed openstack/tripleo-docs master: Remove K, L & M release docs  https://review.openstack.org/47497913:54
mdboothjaosorior: Thanks for looking13:54
dprincejistr: +2, +A13:54
jistrdprince: thanks13:55
jistrjaosorior: thanks13:55
d0ugaljaosorior: LGTM13:55
dciabrinshardy oh....13:57
jaosoriord0ugal: thanks :D13:58
*** akrivoka has quit IRC13:59
mandrejistr: we don't deploy multipathd either it seems, we should be safe on that side13:59
openstackgerritAlan Bishop proposed openstack/tripleo-quickstart master: Add ability to control the size of VM extradisks  https://review.openstack.org/47498514:00
*** limao_ has joined #tripleo14:00
openstackgerritChristopher Brown proposed openstack/tripleo-docs master: Remove K, L & M release docs  https://review.openstack.org/47497914:00
dciabrinshardy so only haproxy has this behaviour? or the other services as well?14:00
jistrmandre: yea i'm just looking at that... also i *think* it doesn't run as cinder user, so it should be oblivious to what UID in particular is /var/lib/cinder owned by, as either way it shouldn't be the same UID as multipathd runs as... unless i missed the user setting somewhere14:00
*** dtantsur|brb is now known as dtantsur14:01
mandrejistr: if it's not specified in the template or in kolla, then it defaults as root14:01
jistrmandre: yea i couldn't spot anything, so it probably does run as root14:02
honzarlandy: still getting "instackenv.json did not validate" errors with the latest patchset14:02
jaosoriorpradk: dammit. Seems that even with that patch, the issue is not entirely fixed :/ Seems we are gonna need that retry after all14:02
*** limao has quit IRC14:02
*** limao has joined #tripleo14:03
pradkjaosorior, is swift not up by then?14:03
pradkjaosorior, the dependency should have ensured that?14:04
jaosoriorpradk: that's what I thought O_o14:04
jaosoriorit's very strange14:04
pradkjaosorior, what error do you see? may be its different14:04
jaosoriorit's the same BadStatusLine stack trace14:04
jaosoriorI don't get it14:04
pradkhmm14:04
*** limao_ has quit IRC14:05
pradkjaosorior, do you have the logs i can look? want to check the timestamps14:06
rlandyhonza: my run is working so far  ... how did you apply the patchset?14:06
jaosoriorpradk: I don't; just whiped out that directory14:06
*** anshulbehl has quit IRC14:06
honzarlandy: git review -d <patch number>14:06
rlandyhonza: can you paste your https://github.com/openstack/tripleo-quickstart/blob/master/quickstart-extras-requirements.txt14:07
openstackgerritDamien Ciabrini proposed openstack/tripleo-heat-templates master: Use HAproxy config files copied by Kolla bootstrap  https://review.openstack.org/47494314:07
honzarlandy: a single line that reads "file:///home/hpokorny/code/tripleo-quickstart-extras/#egg=tripleo-quickstart-extras"14:07
honzarlandy: happy to try anything to help debug14:08
pradkjaosorior, may be the issue is with swift coming up properly? are swift services reachable14:08
*** shardy_afk is now known as shardy14:09
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Make a copy of files touched by puppet in container  https://review.openstack.org/47489114:09
jaosoriorpradk: right14:09
jaosoriorthat could be14:09
rlandyhonza: that looks right - each time before you run, do you delete your working dir? (default ~/.quickstart)?14:10
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169813614:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817214:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817514:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1698136 in tripleo "Containers deployment CI broken, logs missing" [Critical,In progress] - Assigned to Jiří Stránský (jistr)14:10
shardydciabrin: I suspect it's several services, but we hit haproxy first because it's in step114:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)14:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]14:10
honzarlandy: I use --working-dir .14:10
rlandyhonza: ok - do you delete and recreate it each time?14:10
honzarlandy: the latest run was actually from a clean checkout (and no ~/.quickstart)14:11
honzarlandy: yes; bash devmode.sh --ovb --no-gate --delete-all-stacks --working-dir .14:11
*** pchavva has quit IRC14:12
*** limao has quit IRC14:12
*** limao has joined #tripleo14:13
rlandyhonza: ok - no matter - as long as you are not reusing the working dir, you delete that dir and recreate it each time14:13
*** chlong has joined #tripleo14:14
openstackgerritChristopher Brown proposed openstack/tripleo-docs master: Remove K, L & M release docs  https://review.openstack.org/47497914:14
*** yamahata_ has joined #tripleo14:16
*** jlinkes has quit IRC14:16
rlandyhonza: in the working dir, you should see a file named <some prefix>env.yaml. pls paste the contents of that file. you can remove your password14:17
jaosoriorCould I get a review for this t-h-t patch https://review.openstack.org/#/c/474532/ ? :D14:18
honzarlandy: here http://paste.openstack.org/show/612868/14:18
openstackgerritCarlos Camacho proposed openstack/tripleo-ui master: Change tripleo-ui login images  https://review.openstack.org/47471014:20
jaosoriorthanks pradk14:20
*** katkapilatova has left #tripleo14:24
*** limao_ has joined #tripleo14:24
rlandyhonza: hmm .. you have the correct setting: dns_nameservers ...14:24
*** amoralej|lunch is now known as amoralej14:26
mandrecschwede: sorry, I don't have more info about https://github.com/openstack/tripleo-validations/commit/2e1a198c other than what's in the commit message14:26
*** limao__ has joined #tripleo14:27
mandrecschwede: I've looked at the clapper repo from where this validation originated, and there is no further info, https://github.com/rthallisey/clapper/pull/2014:27
pradksure14:27
*** limao has quit IRC14:27
honzarlandy: here is the error for completeness http://paste.openstack.org/show/612873/14:27
*** limao_ has quit IRC14:28
jaosoriormwhahaha, EmilienM: Could you check this out https://review.openstack.org/#/c/474946/14:28
cschwedemandre: ok, thx! I'll ask Tomas then, maybe he knows more about that14:28
rlandyhonza: it looks like you are doing things right  - the only thing that troubles me is putting your working dir within tripleo-quickstart14:29
EmilienMjaosorior: sure14:29
rlandybecause to rerun, you need to clean out that entire dir14:30
rlandyand there is room for confusion14:30
honzarlandy: ok, i'll try again without overriding that14:30
rlandyhonza: please remove everything in tripleo-quickstart, relcone, edit quickstart-extras-requirements.txt before you rerun14:31
*** limao__ has quit IRC14:31
*** limao has joined #tripleo14:31
honzarlandy: yep, on it14:31
rlandyhonza: to be clear - there is no issue overriding --working dir - I just would place it in a dir that you can delete and recreate14:32
jaosoriorHave a good weekend everyone!14:34
EmilienMjaosorior: no wait14:34
jaosorioraaaaah'14:34
EmilienMyou can't leave like that I mean14:34
shardyyou too jaosorior14:34
EmilienMjaosorior: enjoy :D14:35
openstackgerritCarlos Camacho proposed openstack/tripleo-ui master: Change tripleo-ui login images  https://review.openstack.org/47471014:35
openstackgerritMerged openstack/tripleo-common stable/ocata: Ensure Swift Rings backup container and URLs  https://review.openstack.org/47331914:36
openstackgerritMerged openstack-infra/tripleo-ci master: can_promote function: fix case for oooq when ZUUL_CHANGES is not defined  https://review.openstack.org/47450414:36
jaosoriorEmilienM: haha will do. you too14:37
*** cylopez has quit IRC14:37
*** jaosorior has quit IRC14:37
*** janki has quit IRC14:38
*** jprovazn has quit IRC14:40
*** jcoufal has joined #tripleo14:40
dciabrinshardy, https://review.openstack.org/#/c/474943/ updated btw14:41
*** jcoufal_ has quit IRC14:41
*** limao_ has joined #tripleo14:41
shardydciabrin: thanks, testing now14:41
*** limao has quit IRC14:42
openstackgerritMerged openstack/tripleo-heat-templates master: Only pass distinct services to enabled_services list  https://review.openstack.org/47486214:44
*** sid1 has quit IRC14:48
*** funzo has quit IRC14:50
*** limao has joined #tripleo14:50
*** agopi has quit IRC14:51
abishophttps://review.openstack.org/474955 (final "NAS secure" patch for ocata) is ready for review14:53
*** cdearborn__ has joined #tripleo14:53
*** hjensas has quit IRC14:54
*** limao_ has quit IRC14:54
*** hjensas has joined #tripleo14:54
*** salmankhan has joined #tripleo14:57
bogdandohttps://bugs.launchpad.net/tripleo/+bug/1698349/comments/3 sigh...14:59
openstackLaunchpad bug 1698349 in tripleo "mysql_init_tasks does not always provide .my.cnf" [Medium,Invalid] - Assigned to Bogdan Dobrelya (bogdando)14:59
*** akuznetsov has quit IRC15:00
*** limao has quit IRC15:01
ansiwenwhen deploying overcloud: "Property KeystoneFernetKeys not assigned"15:01
ansiwensomeone saw that before?15:01
*** limao has joined #tripleo15:02
*** mrchao has quit IRC15:03
*** agopi has joined #tripleo15:04
*** mcornea has quit IRC15:05
atothEmilienM, beagles, shardy, if I come across a deprecated attribute in a file I'm updated and it calls out that is was deprecated in a past release (newton in the case I'm looking at), should we remove that attribute or keep it for backwards compatibility?15:05
rlandyhonza: any luck?15:06
beaglesatoth, depends :) usually yes15:06
atothbeagles, yes to remove it?15:06
beaglesatoth, which one are you looking at?15:06
beaglesatoth: neutron::lbaas::service_plugins or whatever?15:07
atothbeagles, puppet-neutron/manifests/services/lbaas.pp  $package_ensure15:07
*** cdearborn has quit IRC15:08
openstackgerritMerged openstack/tripleo-heat-templates master: Fall back to non-containerized cinder-volume to unblock CI  https://review.openstack.org/47491515:08
beaglesatoth, yeah I think it's safe to submit a patch to remove that15:09
atothbeagles, that's just one of the deprecated references to package_ensure throughout the lbaas files I've run into15:09
*** ebarrera_ has joined #tripleo15:10
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817215:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817515:10
*** ooolpbot has quit IRC15:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]15:10
*** egonzalez has quit IRC15:10
*** ebarrera_ has quit IRC15:10
atothbeagles, should that change (to that and other files) be a single patch or can it be part of a larger change (for the enablement of the lbaas integration)15:11
beaglesatoth, if you have specific questions about deprecation in puppet modules, #openstack-puppet is the place to ask15:11
openstackgerritSteve Linabery proposed openstack/tripleo-puppet-elements master: scsi-target-utils is not used in RHEL7+  https://review.openstack.org/47500115:11
beaglesatoth, I'd put the removal of deprecated params in its own patch15:11
dciabrindprince, shardy when you have some time, could you help me make https://review.openstack.org/#/c/474183/1 a bit cleaner?15:11
atothbeagles, ok, good thing I have changed to much in that repo yet ;-)15:13
atothbeagles, haven't15:13
*** limao_ has joined #tripleo15:13
atothfriday typing skillz15:13
beagles:) for you maybe... for me that's everyday15:14
atoth:-)15:14
*** rcernin has quit IRC15:15
*** fzdarsky is now known as fzdarsky|afk15:16
*** jaganathan has joined #tripleo15:16
*** limao has quit IRC15:17
*** dciabrin has quit IRC15:17
*** jcoufal has quit IRC15:17
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Blacklist support for ExtraConfig  https://review.openstack.org/47500315:17
*** jcoufal has joined #tripleo15:18
ansiwenis there a way to use a custom tripleo-common for my overcloud deployment without redeploying the undercloud?15:19
*** salmankhan has quit IRC15:19
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Add subnet property to ctlplane network for server resources  https://review.openstack.org/47381715:19
*** salmankhan has joined #tripleo15:20
*** pcaruana has quit IRC15:20
abishopansiwen: just tried a fresh deployment, and got that same failure15:23
*** limao_ has quit IRC15:24
*** limao has joined #tripleo15:24
openstackgerritMarios Andreou proposed openstack/instack-undercloud master: WIP Adds execution of validations on undercloud upgrade  https://review.openstack.org/47457815:27
*** jbadiapa has quit IRC15:28
* EmilienM running massive Launchpad bug triage15:28
*** salmankhan has quit IRC15:31
*** dciabrin has joined #tripleo15:32
*** itlinux_ has joined #tripleo15:34
*** limao_ has joined #tripleo15:35
*** itlinux_ has quit IRC15:36
*** limao has quit IRC15:38
*** pchavva has joined #tripleo15:39
*** Goneri has joined #tripleo15:40
*** sid1 has joined #tripleo15:40
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: DNM - policy.json change test in container CI  https://review.openstack.org/47436015:41
*** lmiccini has quit IRC15:41
*** lmiccini has joined #tripleo15:41
*** itlinux has joined #tripleo15:44
dtantsurfolks, is there a way to set --compute-flavor etc via environment files?15:46
*** salmankhan has joined #tripleo15:47
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Use templated network-environment file for OVB  https://review.openstack.org/47478115:47
openstackgerritMerged openstack/tripleo-heat-templates master: Add templates to configure Ironic inspector  https://review.openstack.org/43615915:48
mwhahahadtantsur: Overcloud<role>Flavor i think15:49
mwhahahadtantsur: ie OvercloudComputeFlavor15:49
dtantsurmwhahaha: I thought so too, does not work15:49
mwhahahadtantsur: sounds like a bug then, https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/compute-role.yaml#L1315:50
paramitedoes anybody know what I have to do, to avoid 2017-06-16 15:47:20Z [overcloud]: CREATE_FAILED  Resource CREATE failed: resources[47]: resources.ControllerServiceChain.resources.PuppetServices.resources.ServiceChain.Property error: resources.KeystoneBase.properties: Property KeystoneFernetKeys not assigned15:50
mwhahahadtantsur: but don't we use it in CI?15:50
paramiteduring overcloud deploy?15:50
*** ebarrera has quit IRC15:50
dtantsurhmm, maybe it did, I need t olook in depth15:50
dtantsurparamite: I got it recently because of mistmatch between tripleo-common and tripleo-heat-templates versions..15:50
mwhahahaparamite: sounds like a you might have an outdated client? we recently added that in and it's supposed to be generated on deploy15:50
*** funzo has joined #tripleo15:51
paramitedtantsur, I've updated to latest common and templates and didn't help15:51
paramiteclient sounds promising15:51
paramitegonna update15:51
mwhahahaparamite: well it's a mistral action15:51
dtantsuryeah, maybe it was client, I did not look in depth15:51
mwhahahaparamite: so make sure you update the mistral workbooks when you updated common15:51
paramitemwhahaha, so just update openstackclient?15:51
paramiteoooh15:52
mwhahahaparamite: no it would be tripleoclient but it just calls a mistral action to generate the passwords15:52
paramitemistral workbooks ... how to update them? :)15:52
* mwhahaha goes looking for docs15:52
*** snecklifter_ has quit IRC15:52
mwhahahai know we have code to do it in the updates15:53
mwhahahabasiclaly it's removing all the tripleo actions and reimporting them15:53
mwhahahaparamite: https://github.com/openstack/instack-undercloud/blob/c984ea9b356b7e89748119122a97cfc54e532c47/elements/undercloud-install/os-refresh-config/post-configure.d/98-undercloud-setup#L93-L10215:54
paramitemwhahaha, thx15:54
itlinuxhello and good day everyone..15:54
paramiteis python-tripleoclient-7.1.1-0.20170613152038.127c35e.el7.centos.noarch.rpm new enough?15:54
itlinuxdoes anyone know where to get the latest images for the overcloud?15:54
itlinuxTY15:55
*** funzo has quit IRC15:56
*** ykarel has quit IRC15:56
mwhahahaparamite: no you need fome after the 14th15:56
paramitedoh ok15:56
paramitethanks15:56
mwhahahaparamite: you need https://review.openstack.org/#/c/473387/15:56
*** salmankhan has quit IRC15:56
paramiteack15:57
paramitemwhahaha, dtantsur: thanks a lot!15:57
*** sid1 has quit IRC15:57
*** nenad has quit IRC15:58
bnemecmwhahaha: We don't deploy with roles in ci right now.  That is a hole we should fill at some point though.16:00
bnemecIt might be addressed by the scale testing job, if we're okay with a periodic test instead of an every patch one.16:00
mwhahahabnemec: of course not, why would we cover that16:00
bnemecI think we're going to use roles there to keep the resource requirements as minimal as possible.16:00
bnemecmwhahaha: It's not a widely used feature, right? ;-)16:01
*** bnemec is now known as beekneemech16:01
*** sid1 has joined #tripleo16:01
* beekneemech wishes #tripleo a happy Friday16:01
*** sid1 has quit IRC16:01
*** jpich has quit IRC16:01
mwhahahafriday bitterness :D16:01
*** sid1 has joined #tripleo16:02
mwhahahaitlinux: in CI we use http://66.187.229.139/builds/current-tripleo/ evidently16:02
mwhahahano idea why we don't actually configure some sort of dns for that host16:02
itlinuxthank you! mwhahaha:16:02
*** dtantsur is now known as dtantsur|afk16:03
mwhahahaprobably the same reason we don't use roles in ci16:03
openstackgerritLokesh Jain proposed openstack/tripleo-heat-templates master: Addition of Nuage as mechanism driver for ML2  https://review.openstack.org/47478816:03
*** atheurer has joined #tripleo16:03
mwhahahaitlinux: fyi i pulled that from our tripleo-quickstart config https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/master.yml16:03
itlinuxthanks again..16:04
*** atheurer has quit IRC16:04
shardywe do test custom roles in CI, just not the new roles_data generation stuff16:04
shardythe undercloud job uses it, as does the upgrade job16:04
mwhahahashardy: do we test Overcloud<node>Flavor?16:05
itlinux@mwhahaha: do you have any tips on how to install the UI? TY16:05
*** ccamacho has quit IRC16:05
*** sid1 has quit IRC16:05
beekneemechitlinux: jtomasek and dtrainor are good people to ask about the UI.16:07
shardymwhahaha: not sure tbh - I thought the quickstart jobs used the quickstart generated flavors by default, but I could be mistaken16:07
dtrainorHowdy.  ui is enabled by default.16:08
*** ramishra has quit IRC16:08
shardyit's entirely possible we don't have an intesection between *Flavor overrides and custom roles coverage though16:08
dtrainoritlinux, do you have any specific questions I can help with?16:08
shardyintersection even16:08
itlinuxI will check the quickstart but I did not see the images..16:08
mwhahahashardy: yea but the question was about the compute flavor so just wondering if we have that in oooq or if we're using --control-flavor still (i have not checked)16:08
shardymwhahaha: yeah not certain tbh, would have to check logs16:09
mwhahahaitlinux: quickstart sill has some work pending around the ui and setting up tunneling, https://review.openstack.org/#/c/407733/3316:09
*** yprokule has quit IRC16:09
*** limao_ has quit IRC16:09
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817216:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817516:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]16:10
itlinuxthank you  @mwhahaha:16:10
*** sid1 has joined #tripleo16:12
*** trown is now known as trown|lunch16:13
mwhahahashardy: yea it looks like we're still using --*-flavor in oooq-extras16:13
mwhahahahttps://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-deploy/defaults/main.yml#L5616:13
jtomasekitlinux what is your setup? UI is installed with undercloud and available at <undercloudip>:3000 you just need to make sure that undercloud ip and public endpoints are publicly accessible (from your browser)16:13
*** artom has joined #tripleo16:14
*** zoliXXL is now known as zoli|gone16:14
*** suuuper has quit IRC16:17
*** bogdando has quit IRC16:18
openstackgerritJaganathan Palanisamy proposed openstack/python-tripleoclient master: Invoke the workflows specified in plan environment file.  https://review.openstack.org/46959616:19
*** sid1 has quit IRC16:20
*** sid1 has joined #tripleo16:20
itlinux@mwhahaha so this site is not good I guess :) https://docs.openstack.org/developer/tripleo-docs/installation/installation.html16:21
mwhahahaitlinux: it is. i use those directions to do installs16:22
*** sid1 has quit IRC16:22
mwhahahaitlinux: depends on what you're planning on doing, since you asked for current images i thought you were looking for development images (what i posted). if you're looking for stable versions just follow the instractions16:23
*** yamahata_ has quit IRC16:24
itlinuxi need to deploy ocata.. TY16:25
*** sid1 has joined #tripleo16:27
openstackgerritMichael Johnson proposed openstack/diskimage-builder master: Fix mkfs failure when loop device is not ready  https://review.openstack.org/47502216:30
*** agopi has quit IRC16:31
*** noslzzp has joined #tripleo16:31
honzarlandy: same error as before :(16:32
*** jaganathan has quit IRC16:32
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Dell SC: Add exclude_domain_ip option  https://review.openstack.org/47461316:33
honzarlandy: do you want to see the env.yaml again?16:33
openstackgerritMerged openstack/tripleo-heat-templates master: Make fernet max active keys configurable  https://review.openstack.org/47453216:34
openstackgerritMerged openstack/tripleo-common master: Add fernet keys purging based no t-h-t parameter  https://review.openstack.org/47457316:35
rlandyhonza: no - I trust you are doing things right. Are you running from within the RH network? I am trying to determine what difference there is16:35
*** lmiccini has quit IRC16:35
rlandyjaosorior reported a successful deploy yesterday16:36
Slowershardy: playing around with docker healthchecks.. it looks like it waits a minute before it checks the state..16:36
honzarlandy: not sure what you mean by that exactly16:36
honzarlandy: vpn?16:36
Slowershardy: is that going to be too long?16:36
Slowershardy: the other idea would be to do a docker exec..16:37
shardySlower: is there any way to make that faster, at least for the initial few checks?16:37
rlandyhonza: I am remote and running with vpn.16:37
honzarlandy: me, too16:37
Slowershardy: ya I'll look into it16:37
shardySlower: yes IMO it's too long if we want to interleave health checks with each deployment step16:37
Sloweryeah agreed16:37
shardyprobably OK for ongoing polling tho16:37
Sloweryeah I think for the main purpose of system health it's good16:38
rlandyhonza: when you ssh to your undercloud - what do you see in resolv.conf?16:38
shardySlower: yeah, just need some iburst style initial period while the service comes up16:38
rlandyI am guessing you have the right nameserver but maybe it's blocked as well16:38
shardyor maybe manually check status in a loop in the ansible playbook16:38
*** cdearborn_ has joined #tripleo16:39
Slowershardy: ah ya, --health-interval16:39
rlandyhonza: can you also check the private network created - you can do that from horizon - you should see the nameserver assigned16:39
Slowerbut that would affect it forever..16:39
SlowerI presume16:39
honzarlandy: http://paste.openstack.org/show/612880/16:40
honzarlandy: the network:dhcp device under ports?16:43
honzarlandy: not sure where to find the right info16:43
*** shardy has quit IRC16:44
*** florianf has quit IRC16:46
rlandyhonza: under your project (following the menu on the left hand side), Network -> Networks -> <prefix>-private -> Subnets -> <prefix>private ... you should see DNS Name Servers16:47
rlandylast thing I'll ask you to check :)16:47
honzarlandy: ah, yes, 208.67.222.22216:48
*** milan has quit IRC16:48
*** atheurer has joined #tripleo16:48
honzarlandy: you're doing me a favor, not the other way around :)16:48
*** milan has joined #tripleo16:50
*** athomas has quit IRC16:51
*** panda is now known as panda|bbl16:51
rlandyhonza: you have all the settings right ... will have to look into why this does not resolve for you16:51
openstackgerritMichael Johnson proposed openstack/diskimage-builder master: Fix mkfs failure when loop device is not ready  https://review.openstack.org/47502216:52
honzarlandy: the onboarding doc has a section on networking --- i completed it --- would that be a problem?16:53
honzarlandy: is there anything else apart from the oooq setup itself that i need to do?16:53
*** cdearborn__ has quit IRC16:54
rlandyhonza: networking?16:55
* rlandy checks networking section 16:56
*** agopi has joined #tripleo16:56
*** lucasagomes is now known as lucas-afk16:56
rlandyhinza: oh - I see ... reading16:57
*** sudipto has joined #tripleo16:57
*** sudipto_ has joined #tripleo16:57
*** fultonj has quit IRC16:58
rlandyhonza: ^^ ... ok - to run ovb, you don't need the networking section ... just the steps from "Create an OVB TripleO deployment for development and test."16:58
*** salmankhan has joined #tripleo16:59
honzarlandy: ok, great16:59
rlandyhonza: we run ovb with this heat template - it creates all the networks etc. you need ...16:59
rlandyhttps://github.com/cybertron/openstack-virtual-baremetal/blob/master/templates/quintupleo.yaml16:59
honzarlandy: right, i did think it was strange to have two private networks16:59
*** yamahata_ has joined #tripleo17:00
rlandyhonza; so you could have a pretty much blank tenant (access to  - get the V2 rc file (include the password) run17:01
rlandyyou don't need the section afterwards "OVB Details" - we take care of  that for you within devmode17:01
honzacool17:03
rlandyhonza: so when you look at the networks tab, you should see just the <prefix>* networks the OVB stack created and 38.145.32.0/2217:03
honzarlandy: correct17:03
rlandyok then17:03
rlandyyou can recognize all resources created by OVB - they are all prefix'ed17:04
*** artom has quit IRC17:04
rlandyhonza: ok- anything else we can look at to see differences?17:05
honzarlandy: i'm out of ideas17:05
*** tosky has quit IRC17:07
rlandyhonza: have you added additional security groups or routers?17:08
rlandyyou'll have a default security group17:08
rlandyand one created by the stack17:08
rlandyand one router created by the stack17:09
honzarlandy: only what is in the networking section --- i just deleted all of that and am retrying17:09
rlandyhonza: ok - great - if that fails, you can create a set of temporary credentials, and I can try working from you tenant17:09
rlandysort of last resort17:09
honzarlandy: awesome17:10
honzarlandy: thank you17:10
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817217:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817517:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]17:10
rlandyhonza: thank you for sticking with this17:10
*** gvrangan has joined #tripleo17:13
*** jtomasek has quit IRC17:14
EmilienMcan I get review on this doc patch please? https://review.openstack.org/#/c/474207/17:15
*** milan has quit IRC17:18
mdboothRandom question: why can't undercloud release N install an overcloud release N-M for any value of M? Why only M=1?17:18
itlinuxdum question what's the best way to create the undercloud.json file to use with instackenv.json ?17:18
openstackgerritDamien Ciabrini proposed openstack/tripleo-common master: Switch to RDO packages for HA support in Kolla images  https://review.openstack.org/47503517:18
*** dprince has quit IRC17:18
mdboothitlinux: There's a sample included in one of the packages. You can copy that and customise it.17:19
mdbooth/usr/share/instack-undercloud/undercloud.conf.sample17:20
itlinuxmdbooth ok let me check TY17:20
mdboothinstackenv.json is kinda orthogonal to that iiuc, because it describes the ironic nodes available to the undercloud, i.e. it's used by an installed undercloud. undercloud.conf is for installing the undercloud in the first place.17:21
itlinuxdum question that's a config in the doc I read it calls is undercloud.json ..17:21
*** sasha2 has quit IRC17:21
mdboothAh... undercloud.json.17:22
mdboothI don't know what that is, sorry.17:22
itlinuxyes I see the undercould.conf got created..17:22
itlinuxhttps://docs.openstack.org/developer/tripleo-docs/advanced_deployment/baremetal_overcloud.html?highlight=overcloud%20deployment17:22
itlinuxthat' s where I found it..17:22
itlinuxIf you already have an instackenv.json file with all nodes prepared, ....17:22
*** funzo has joined #tripleo17:23
*** gvrangan has quit IRC17:25
*** gvrangan has joined #tripleo17:25
*** sasha2 has joined #tripleo17:27
honzarlandy: another fail, same error17:27
*** dciabrin has quit IRC17:29
honzarlandy: shall i email you the credentials?17:29
*** pkovar has quit IRC17:29
openstackgerritMerged openstack/os-collect-config master: Updated from global requirements  https://review.openstack.org/47467517:29
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates master: Add Telemetry services to scenario002  https://review.openstack.org/47496917:32
openstackgerritMichael  Henkel proposed openstack/puppet-tripleo stable/ocata: This patch moves Contrail roles to OpenStack API communication from the public external to the internal_api network. It also adds the missing configuration option for dpdk.  https://review.openstack.org/47503917:32
*** jcoufal_ has joined #tripleo17:34
*** jcoufal has quit IRC17:36
rlandyhonza: yes - pls create switch to some temp creds and pm me - I'll look at your tenant17:36
*** tosky has joined #tripleo17:36
*** jcoufal has joined #tripleo17:36
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo master: Create the vhost root folder before creating the virtual host  https://review.openstack.org/47452617:38
*** jcoufal_ has quit IRC17:39
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo master: Create the vhost root folder before creating the virtual host  https://review.openstack.org/47452617:40
*** salmankhan has quit IRC17:40
*** salmankhan has joined #tripleo17:41
*** snecklifter has left #tripleo17:41
*** jcoufal_ has joined #tripleo17:48
*** jcoufal has quit IRC17:51
openstackgerritMichael  Henkel proposed openstack/puppet-tripleo master: This patch will move the Contrail roles communication towards OpenStack APIs from the public/external network to the internal_api network. I will also add the option to enable dpdk for Contrail.  https://review.openstack.org/47504517:51
*** bfournie has quit IRC17:53
*** itlinux has quit IRC17:53
*** itlinux_ has joined #tripleo17:55
*** trown|lunch is now known as trown17:55
*** salmankhan has quit IRC17:57
*** dprince has joined #tripleo18:01
*** ipsecguy has joined #tripleo18:02
*** salmankhan has joined #tripleo18:02
*** ipsecguy_ has quit IRC18:03
openstackgerritCarlos Camacho proposed openstack/tripleo-ui master: Change tripleo-ui login images  https://review.openstack.org/47471018:03
*** pcaruana has joined #tripleo18:04
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs  https://review.openstack.org/35921518:04
*** jcoufal has joined #tripleo18:04
openstackgerritMichael  Henkel proposed openstack/tripleo-heat-templates master: This patch moves Contrail roles communication from public/external to internal_api network for OpenStack API. It also adds the option to enable dpdk  https://review.openstack.org/47504818:04
*** sudipto has quit IRC18:04
*** sudipto_ has quit IRC18:04
*** jcoufal_ has quit IRC18:06
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817218:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817518:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]18:10
*** fragatin_ has quit IRC18:11
*** salmankhan has quit IRC18:17
*** salmankhan has joined #tripleo18:17
sshnaidm|offcores, please approve https://review.openstack.org/#/c/474891/18:22
*** pchavva has quit IRC18:22
*** jaganathan has joined #tripleo18:22
*** cdearborn_ has quit IRC18:22
*** salmankhan has quit IRC18:24
*** salmankhan has joined #tripleo18:26
openstackgerritClark Boylan proposed openstack/diskimage-builder master: On suse the python2 dev package is python-devel  https://review.openstack.org/47505118:27
*** sid1 has quit IRC18:27
itlinux_does anyone know what DELL and super micro boxes support for the pm_type ? TY18:29
*** artom has joined #tripleo18:30
*** salmankhan has quit IRC18:35
*** artom has quit IRC18:40
*** pcaruana has quit IRC18:41
*** salmankhan has joined #tripleo18:43
*** salmankhan has quit IRC18:50
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: testing featurset of containers  https://review.openstack.org/47308218:52
sshnaidm|offEmilienM, can you approve please? https://review.openstack.org/#/c/474891/18:53
EmilienMsshnaidm|off: looking18:53
sshnaidm|offEmilienM, it's solution to containers logs18:53
*** gvrangan has quit IRC18:53
EmilienMsshnaidm|off: yeah18:56
*** jtomasek has joined #tripleo19:10
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817219:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817519:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]19:10
*** jaganathan has quit IRC19:12
*** jtomasek has quit IRC19:16
*** pkovar has joined #tripleo19:20
*** pkovar has quit IRC19:20
*** amoralej is now known as amoralej|off19:25
*** sasha2 has quit IRC19:28
*** noslzzp has quit IRC19:30
*** ecerquei__ has quit IRC19:31
*** dciabrin has joined #tripleo19:46
*** tesseract has quit IRC19:53
openstackgerritMerged openstack/puppet-tripleo master: Fix redis when hostname has capital letters  https://review.openstack.org/47473719:54
*** dpivonka has quit IRC19:56
*** rhallisey has quit IRC19:57
openstackgerritMerged openstack/tripleo-heat-templates master: Add Ec2Api to container based deployment  https://review.openstack.org/46920019:58
*** jcoufal has quit IRC19:58
*** rhallisey has joined #tripleo20:00
*** jmelvin has quit IRC20:04
*** zzzeek has quit IRC20:07
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817220:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817520:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)20:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]20:10
itlinux_does anyone have any tips on creating it's own overcloud images to inject into the undercloud?20:11
*** f13o has quit IRC20:11
*** owalsh has quit IRC20:12
*** sbrzozow has quit IRC20:12
*** ecerquei__ has joined #tripleo20:15
*** itlinux_ has quit IRC20:18
*** agopi has quit IRC20:20
*** jtomasek has joined #tripleo20:22
openstackgerritGuil Barros proposed openstack/tripleo-heat-templates master: This patch moves Contrail roles communication from public/external to internal_api network for OpenStack API. It also adds the option to enable dpdk  https://review.openstack.org/47504820:26
*** f13o has joined #tripleo20:26
*** owalsh has joined #tripleo20:30
*** zzzeek has joined #tripleo20:31
*** morazi has quit IRC20:32
openstackgerritMerged openstack/puppet-tripleo stable/ocata: Add support for autofencing to Pacemaker Remote.  https://review.openstack.org/47373520:36
openstackgerritGuil Barros proposed openstack/tripleo-heat-templates master: Contrail: Fix controlplane/dataplane network asignments & enable optional dpdk  https://review.openstack.org/47504820:37
openstackgerritMerged openstack/puppet-tripleo master: Only set the stonith property on the pacemaker_master node  https://review.openstack.org/47163020:39
gbarrosEmilienM does that fix the commit? Sorry it's my first attempt at a commit msg ;)20:41
EmilienMgbarros: much better :D20:42
EmilienMthanks20:42
gbarroscool, thx20:42
gbarrosEmilienM i wasnt sure if i should set the heat_template_version to pike or ocata. I went with pike as that is current master.20:43
* mwhahaha looks at the install script and runs away20:44
gbarrosthough that same template should still work in ocata20:44
EmilienM            if [[ `echo $role |awk -F"-" '{print $2}'` == "contrailcontroller" || `echo $role |awk -F"-" '{print $2}'` == "contrailanalytics" || `echo $role |awk -F"-" '{print $2}'` == "contrailanalyticsdatabase" ]]; then20:47
EmilienMI mean20:47
EmilienMsuper easy folks20:47
* tosky flinks at the repeated code20:47
EmilienMecho "net.ipv4.tcp_keepalive_time = 5" >> /etc/sysctl.conf20:48
EmilienMgbarros: all of this can be done via puppet20:48
EmilienMwe already have the interface, etc20:48
gbarrosack. I'll poke michael about it.20:49
gbarrosmhenkel ^^20:49
EmilienMgbarros: -2 for now, I'm not happy with this patch20:50
gbarrosyup.20:50
gbarrosi reached out to mhenkel, we'll sort it out. thanks EmilienM20:52
*** bfournie has joined #tripleo20:53
EmilienMcool20:53
gbarrosim just fixing lint :)20:54
*** jmelvin has joined #tripleo20:54
*** bfournie has quit IRC20:54
*** abishop has quit IRC20:54
*** karimb has quit IRC20:57
*** owalsh has quit IRC21:05
openstackgerritGuil Barros proposed openstack/puppet-tripleo master: Contrail: Fix controlplane/dataplane network asignments & enable optional dpdk  https://review.openstack.org/47504521:06
*** owalsh has joined #tripleo21:07
*** dprince has quit IRC21:08
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817221:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817521:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]21:10
*** ecerquei__ has quit IRC21:13
*** mhenkel_ has joined #tripleo21:13
*** skramaja has quit IRC21:15
*** skramaja has joined #tripleo21:15
gbarrosEmilienM, got any ideas how else to do this? mhenkel indicates that the nasty bash configuration has to run before puppet runs on the node or os-net-config will bork.21:17
*** paramite has quit IRC21:18
*** itlinux_ has joined #tripleo21:20
*** iranzo has quit IRC21:27
*** chlong has quit IRC21:30
*** jtomasek has quit IRC21:31
*** jkilpatr has quit IRC21:31
*** fragatina has joined #tripleo21:35
*** itlinux_ has quit IRC21:38
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Add ignore_projects to filter gnocchi events  https://review.openstack.org/46920221:41
openstackgerritMerged openstack/puppet-tripleo master: For http service endpoints always redirect to https  https://review.openstack.org/47410821:43
* beekneemech looks at the check-tripleo queue21:46
* beekneemech wonders if he has anything that needs a recheck21:46
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817222:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817522:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)22:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]22:10
*** fragatina has quit IRC22:10
*** rlandy has quit IRC22:16
*** itlinux_ has joined #tripleo22:20
*** ansmith has joined #tripleo22:27
*** itlinux_ has quit IRC22:28
*** itlinux_ has joined #tripleo22:31
*** itlinux_ has quit IRC22:31
*** panda|bbl is now known as panda|off22:42
*** tosky has quit IRC22:44
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs  https://review.openstack.org/35921522:47
*** apetrich has quit IRC22:48
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Fix repositories for promotion jobs  https://review.openstack.org/47508422:49
sshnaidm|offpanda|off, ^^22:49
openstackgerritBen Nemec proposed openstack/tripleo-specs master: Clean up spec index  https://review.openstack.org/47508522:50
*** mmethot has quit IRC22:51
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Replace plan list table with a card view  https://review.openstack.org/46223022:52
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Fix repositories for promotion jobs  https://review.openstack.org/47508422:56
openstackgerritMerged openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/47469722:57
sshnaidm|offcores, take a look for promotion blocker:  https://review.openstack.org/47508422:57
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Fix mkfs failure when loop device is not ready  https://review.openstack.org/47502223:00
EmilienMsshnaidm|off: +223:03
EmilienMsshnaidm|off: I can ninja approve if it proves to pass CI23:03
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817223:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169817523:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/169847423:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] - Assigned to Steve Baker (steve-stevebaker)23:10
openstackLaunchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged]23:10
openstackLaunchpad bug 1698474 in tripleo "CI: promotion oooq jobs use current repos instead of consistent" [Critical,In progress] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)23:10
*** trozet has joined #tripleo23:11
*** brault_ has joined #tripleo23:17
*** spredzy_ has joined #tripleo23:18
*** george_goh|afk has joined #tripleo23:20
*** trown|brb has joined #tripleo23:20
*** liverpoo1er has joined #tripleo23:22
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Don't mangle code when building for production  https://review.openstack.org/47508923:23
*** therve_ has joined #tripleo23:24
*** kbyrne_ has joined #tripleo23:25
*** jmelvin has quit IRC23:25
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Add support for autofencing to Pacemaker Remote.  https://review.openstack.org/47373623:25
*** mjblack- has joined #tripleo23:25
openstackgerritMerged openstack/puppet-tripleo master: Ensure hiera step value is an integer  https://review.openstack.org/46655623:25
*** trozet has quit IRC23:26
*** liverpooler has quit IRC23:26
*** brault has quit IRC23:26
*** hamzy has quit IRC23:26
*** rwsu has quit IRC23:26
*** therve has quit IRC23:26
*** kbyrne has quit IRC23:26
*** spredzy has quit IRC23:26
*** cschwede has quit IRC23:26
*** sshnaidm|off has quit IRC23:26
*** jpena|off has quit IRC23:26
*** weshay has quit IRC23:26
*** trown has quit IRC23:26
*** mjblack has quit IRC23:26
*** mnaser has quit IRC23:26
*** george_goh has quit IRC23:26
*** kbyrne_ is now known as kbyrne23:26
*** spredzy_ is now known as spredzy23:26
*** mjblack- is now known as mjblack23:26
*** jpena|off has joined #tripleo23:26
*** rwsu has joined #tripleo23:27
*** hamzy has joined #tripleo23:27
*** sshnaidm|off has joined #tripleo23:28
*** weshay has joined #tripleo23:31
*** trozet has joined #tripleo23:31
*** cschwede has joined #tripleo23:31
*** cschwede has quit IRC23:31
*** cschwede has joined #tripleo23:31
*** weshay has quit IRC23:31
*** weshay has joined #tripleo23:31
*** trozet has quit IRC23:32
*** trozet has joined #tripleo23:32
openstackgerritMerged openstack/tripleo-heat-templates master: Fall back to non-containerized cinder-backup to unblock OVB CI  https://review.openstack.org/47497823:33
openstackgerritMerged openstack/tripleo-heat-templates master: Make a copy of files touched by puppet in container  https://review.openstack.org/47489123:33
*** mnaser has joined #tripleo23:34
openstackgerritMerged openstack/tripleo-quickstart-extras master: Limit collection of config-data to puppet-generated files  https://review.openstack.org/47489623:36
*** sid1 has joined #tripleo23:36
*** trozet has quit IRC23:45
*** mhenkel__ has joined #tripleo23:47
*** mhenkel_ has quit IRC23:50
*** mhenkel__ has quit IRC23:51
*** sid1 has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!