| *** owalsh has joined #tripleo | 00:13 | |
| *** dsneddon has quit IRC | 00:34 | |
| *** rh-jelabarre has joined #tripleo | 00:48 | |
| *** dsneddon has joined #tripleo | 01:01 | |
| *** bhagyashris has joined #tripleo | 01:11 | |
| openstackgerrit | Rico Lin proposed openstack/paunch master: Change cmd checking order for backward compatibility https://review.opendev.org/678117 | 01:24 |
|---|---|---|
| *** Goneri has quit IRC | 01:35 | |
| *** HardCase has joined #tripleo | 01:35 | |
| HardCase | Hey all, having a strange issue where when I try to run openstack overcloud deploy like I normally do, I get a strange error that states {u'errors': [u'Not enough baremetal nodes - available: 17, requested: 19'] ... This is not the normal Not enough nodes where I forgot to add the profile to a host or something like that. All my baremetal nodes are correct, but the strange thing is it doesn't even do the updating plan step, just goes | 01:40 |
| HardCase | straight to this error message. I even tried reducing my compute nodes to match the number of nodes it thinks i have. | 01:40 |
| *** gbarros has joined #tripleo | 01:42 | |
| *** ricolin has joined #tripleo | 01:48 | |
| *** mschuppert has quit IRC | 01:51 | |
| *** mmethot has joined #tripleo | 01:57 | |
| mcarden | I guess I'd be doing something like `openstack baremetal node list -f value | wc -l` to see how many nodes it thinks it has. | 02:01 |
| *** gbarros has quit IRC | 02:05 | |
| *** ekultails has quit IRC | 02:08 | |
| *** apetrich has quit IRC | 02:10 | |
| HardCase | Yeah its a long story, but rest assured I have done this and have validated my count correctly, made sure the baremetal nodes were set correctly so on and so forth | 02:21 |
| openstackgerrit | Merged openstack/tripleo-common master: Change ceph dashboard service name to meet puppet requirements https://review.opendev.org/677705 | 02:23 |
| openstackgerrit | Merged openstack/tripleo-common master: Bump the openstackdocstheme extension to 1.20 https://review.opendev.org/672854 | 02:23 |
| *** dsneddon has quit IRC | 02:24 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Configure nova_compute for vendordata https://review.opendev.org/677838 | 02:40 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/stein: Configure nova_compute for vendordata https://review.opendev.org/677836 | 02:40 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/stein: Allow combining system_upgrade_prepare and system_upgrade_run into system_upgrade https://review.opendev.org/677140 | 02:40 |
| openstackgerrit | Emilien Macchi proposed openstack/paunch master: WIP - introduce container_running() and use it before running podman exec https://review.opendev.org/677860 | 02:46 |
| EmilienM | cloudnull: https://zuul.opendev.org/t/openstack/build/4a1a6c73503f40bf84d8bde418d58f86 | 02:47 |
| EmilienM | from https://review.opendev.org/#/c/678110/ | 02:47 |
| EmilienM | fyi | 02:47 |
| HardCase | Hey team, getting a CHECK FAILED Instance None could not be found. Seems I have some ghost instances in my undercloud. Anyone have any thoughts on how to delete these? They don't show up when doing openstack server list | 02:53 |
| *** dsneddon has joined #tripleo | 02:56 | |
| mcarden | Well, you know your day is not going well if the answer starts with something like `sudo su -` `mysql -u root` `use nova;` `select display_name from instances;` ...just to see if there are any ghosts. | 02:59 |
| *** dsneddon has quit IRC | 03:00 | |
| *** psachin has joined #tripleo | 03:02 | |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-heat-templates master: Move GroupVars to overcloud.j2.yaml https://review.opendev.org/677218 | 03:10 |
| HardCase | mcarden: To true, thanks for the tip, I was just looking in the database but was pretty clueless as to which filed or table to look in. | 03:16 |
| mcarden | I'm clueless too. I just stumble around until I trip over stuff. | 03:17 |
| *** ramishra has joined #tripleo | 03:17 | |
| *** gkadam has joined #tripleo | 03:19 | |
| HardCase | Its interesting, I have two errors both for the same thing, but one says it can't find instance None, the other gave me a UUID, I did q select * from instances where uuid="theuuid" but it didn't return anything ... so fricken wierd | 03:19 |
| HardCase | bang head here ... | 03:19 |
| mcarden | Time to nuke it from orbit and start again. | 03:19 |
| HardCase | I had that thought, anyone know if its safe to nuke and reinstall the undercloud? Will that affect my deployed overcloud? | 03:21 |
| *** dsneddon has joined #tripleo | 03:29 | |
| *** dsneddon has quit IRC | 03:34 | |
| mcarden | Affect is such a strong term. I'm sure your deployed overcloud will keep on truckin' just fine. Until you need to interact with it via your redeployed Director. Then you'll be regretting your life choices. | 03:40 |
| *** HardCase has quit IRC | 03:41 | |
| *** HardCase has joined #tripleo | 03:42 | |
| *** dsneddon has joined #tripleo | 04:01 | |
| *** raukadah is now known as chkumar|rover | 04:04 | |
| *** dsneddon has quit IRC | 04:05 | |
| *** holser has joined #tripleo | 04:08 | |
| *** rh-jelabarre has quit IRC | 04:09 | |
| *** ricolin has quit IRC | 04:14 | |
| *** ricolin has joined #tripleo | 04:21 | |
| *** surpatil has joined #tripleo | 04:22 | |
| *** udesale has joined #tripleo | 04:23 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart-extras master: Move SnapshotsActionsTest to master skip list https://review.opendev.org/678141 | 04:24 |
| *** ratailor has joined #tripleo | 04:34 | |
| *** dsneddon has joined #tripleo | 04:34 | |
| *** dsneddon has quit IRC | 04:39 | |
| *** skramaja has joined #tripleo | 04:55 | |
| *** jaosorior has joined #tripleo | 05:01 | |
| *** ricolin has quit IRC | 05:02 | |
| *** ricolin has joined #tripleo | 05:03 | |
| *** janki has joined #tripleo | 05:08 | |
| *** dsneddon has joined #tripleo | 05:13 | |
| *** surpatil has quit IRC | 05:17 | |
| *** dsneddon has quit IRC | 05:18 | |
| openstackgerrit | Merged openstack/tripleo-upgrade stable/stein: Add post undercloud OS upgrade workarounds. https://review.opendev.org/676148 | 05:19 |
| openstackgerrit | Merged openstack/instack-undercloud stable/rocky: Explicitly set notification driver for novajoin https://review.opendev.org/678087 | 05:19 |
| openstackgerrit | Merged openstack/python-tripleoclient stable/stein: Pass deployment_options to config download https://review.opendev.org/675817 | 05:19 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/stein: Unescape IPv6 addresses for ceph_nfs_bind_addr https://review.opendev.org/678073 | 05:19 |
| *** apetrich has joined #tripleo | 05:42 | |
| *** yprokule has joined #tripleo | 05:42 | |
| *** dsneddon has joined #tripleo | 05:45 | |
| *** jbadiapa has quit IRC | 05:47 | |
| *** jbadiapa has joined #tripleo | 05:48 | |
| *** dsneddon has quit IRC | 05:50 | |
| *** aakarsh has quit IRC | 05:51 | |
| *** aakarsh has joined #tripleo | 05:51 | |
| *** holser has quit IRC | 05:52 | |
| *** udesale has quit IRC | 05:55 | |
| *** kopecmartin|off is now known as kopecmartin | 05:55 | |
| *** udesale has joined #tripleo | 05:55 | |
| *** surpatil has joined #tripleo | 06:02 | |
| *** udesale has quit IRC | 06:06 | |
| *** skramaja has quit IRC | 06:08 | |
| *** skramaja has joined #tripleo | 06:11 | |
| *** holser has joined #tripleo | 06:12 | |
| *** saneax has joined #tripleo | 06:18 | |
| *** dsneddon has joined #tripleo | 06:19 | |
| *** mschuppert has joined #tripleo | 06:21 | |
| *** florianf has joined #tripleo | 06:23 | |
| *** dsneddon has quit IRC | 06:24 | |
| *** holser has quit IRC | 06:29 | |
| *** surpatil has quit IRC | 06:36 | |
| *** gkadam has quit IRC | 06:43 | |
| *** HardCase has quit IRC | 06:46 | |
| *** shyamb has joined #tripleo | 06:49 | |
| shyamb | Hi | 06:50 |
| *** amoralej|off is now known as amoralej | 06:50 | |
| shyamb | We wanted to use servicenetmap to manage endpoints | 06:50 |
| shyamb | any documentation we have for this? | 06:51 |
| shyamb | Actually we are integrating a new service in osp13 | 06:53 |
| shyamb | so, on overcloud deployment, for this new service, endpoints are not getting created | 06:54 |
| shyamb | like for other services in openstack | 06:54 |
| shyamb | Which flag or config I need to enable to make sure, endpoints for my service also get created on overcloud deployment | 06:54 |
| *** jpena|off is now known as jpena | 06:59 | |
| *** dsneddon has joined #tripleo | 07:00 | |
| *** cylopez has joined #tripleo | 07:06 | |
| *** dsneddon has quit IRC | 07:07 | |
| *** ricolin has quit IRC | 07:15 | |
| *** rcernin has quit IRC | 07:15 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-ansible master: WIP Separate OVN VIP https://review.opendev.org/678168 | 07:16 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: WIP OVN separate VIP https://review.opendev.org/672673 | 07:19 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: WIP OVN DBS separate vip https://review.opendev.org/669847 | 07:19 |
| *** gfidente|afk is now known as gfidente | 07:20 | |
| *** jtomasek has joined #tripleo | 07:29 | |
| *** holser has joined #tripleo | 07:32 | |
| *** holser has quit IRC | 07:33 | |
| *** holser has joined #tripleo | 07:33 | |
| *** chem` has joined #tripleo | 07:36 | |
| *** xek has joined #tripleo | 07:37 | |
| *** chem has quit IRC | 07:37 | |
| *** dsneddon has joined #tripleo | 07:37 | |
| *** dsneddon has quit IRC | 07:42 | |
| *** jtomasek has quit IRC | 07:42 | |
| *** pbandark has joined #tripleo | 07:43 | |
| *** pbandark has quit IRC | 07:45 | |
| *** pbandark has joined #tripleo | 07:45 | |
| *** shyamb has quit IRC | 07:51 | |
| *** avivgts has joined #tripleo | 07:52 | |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart master: Adapt os_tempest in FS001 https://review.opendev.org/673400 | 07:55 |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-ansible master: Create tripleo-cellv2 role https://review.opendev.org/678022 | 08:02 |
| *** jpich has joined #tripleo | 08:04 | |
| *** lucasagomes has joined #tripleo | 08:06 | |
| *** aakarsh has quit IRC | 08:08 | |
| *** aakarsh has joined #tripleo | 08:09 | |
| *** holser has quit IRC | 08:09 | |
| *** dtantsur|afk is now known as dtantsur | 08:10 | |
| *** holser has joined #tripleo | 08:10 | |
| *** holser has quit IRC | 08:10 | |
| *** dsneddon has joined #tripleo | 08:12 | |
| *** rpittau|afk is now known as rpittau | 08:14 | |
| *** holser has joined #tripleo | 08:17 | |
| *** dsneddon has quit IRC | 08:17 | |
| *** tkajinam has quit IRC | 08:19 | |
| openstackgerrit | Jakub Libosvar proposed openstack/tripleo-heat-templates master: ml2-ansible: Set api_max_retries when net-ansible is used https://review.opendev.org/638119 | 08:20 |
| *** aakarsh|2 has joined #tripleo | 08:28 | |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Add the certificate specs in ceph_grafana composable service https://review.opendev.org/674556 | 08:28 |
| *** aakarsh has quit IRC | 08:29 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-ansible master: Remove static configs for the called playbook from the calling role https://review.opendev.org/678061 | 08:32 |
| *** ksambor has quit IRC | 08:39 | |
| *** ksambor has joined #tripleo | 08:40 | |
| xarlos | Are the tripleo docs ever coming back on docs.openstack? :-) | 08:41 |
| *** shyamb has joined #tripleo | 08:43 | |
| *** dsneddon has joined #tripleo | 08:44 | |
| *** ksambor has quit IRC | 08:47 | |
| *** ksambor has joined #tripleo | 08:47 | |
| *** janki has quit IRC | 08:48 | |
| *** dsneddon has quit IRC | 08:48 | |
| *** shyamb has quit IRC | 08:56 | |
| chkumar|rover | Hello | 09:00 |
| chkumar|rover | I appears that https://review.opendev.org/#/c/677705/ - Change ceph dashboard service name to meet puppet requirements is breaking check job | 09:00 |
| chkumar|rover | https://zuul.opendev.org/t/openstack/build/ca8e1f55314941a4941d616cb1521a1b/log/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#4922 | 09:00 |
| chkumar|rover | msg: you must add a [grafana-server] group and add at least one node.", | 09:00 |
| *** shyamb has joined #tripleo | 09:04 | |
| shyamb | Hi | 09:07 |
| shyamb | I have created log directory using host_prepare_tasks through template | 09:10 |
| shyamb | Directory getting created on host but it's ownership is root:root | 09:11 |
| shyamb | I want it to be owned by 42436:42436 | 09:11 |
| shyamb | http://paste.openstack.org/show/762374/ | 09:11 |
| shyamb | my log directory: /var/log/containers/trilio-datamover-api | 09:11 |
| Tengu | shyamb: you have to set that ownership during the container bootstrap part (step 1 probably). From within a container directly | 09:13 |
| Tengu | shyamb: there are examples of such thing iirc, check for "kolla" in tripleo-heat-templates | 09:14 |
| Tengu | there, some volumes see their ownership changed. | 09:14 |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-ansible master: Create tripleo-cellv2 role https://review.opendev.org/678022 | 09:16 |
| *** owalsh is now known as owalsh|away | 09:16 | |
| shyamb | Tengu: Sure | 09:16 |
| shyamb | will check | 09:17 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-heat-templates master: Set CephEnableDashboard to false https://review.opendev.org/678186 | 09:17 |
| shyamb | Tengu: thanks | 09:17 |
| chkumar|rover | gfidente: hello | 09:20 |
| chkumar|rover | gfidente: please have a look at this review https://review.opendev.org/#/c/678186/ seems to be this bug https://launchpad.net/bugs/1838460 | 09:20 |
| openstack | Launchpad bug 1838460 in tripleo "overcloud deploy fails with ou must add a [grafana-server] group and add at least one node" [High,In progress] - Assigned to chandan kumar (chkumar246) | 09:20 |
| gfidente | chkumar|rover yeah was talking about that with fmount a minute ago | 09:21 |
| gfidente | chkumar|rover I think best is to remove the bool from scenario001 for now | 09:21 |
| gfidente | fmount ^^ can you do it? | 09:21 |
| chkumar|rover | gfidente: https://review.opendev.org/#/c/678186/ set CephEnableDashboard to false | 09:22 |
| *** dsneddon has joined #tripleo | 09:22 | |
| chkumar|rover | gfidente: thank you! | 09:24 |
| *** abishop has quit IRC | 09:24 | |
| *** abishop has joined #tripleo | 09:24 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart-extras master: Move SnapshotsActionsTest to master skip list https://review.opendev.org/678141 | 09:25 |
| shyamb | Tengu: This should work? https://github.com/openstack/tripleo-heat-templates/blob/62082941bdf5ca76caca98e464094a98db7368c1/deployment/ovn/ovn-dbs-container-puppet.yaml#L135 | 09:25 |
| shyamb | It's setting log directory permissions through kolla_config permissions section | 09:26 |
| Tengu | shyamb: exactly :) | 09:26 |
| *** dsneddon has quit IRC | 09:27 | |
| shyamb | There is another way, that nova_api container is using | 09:27 |
| shyamb | https://github.com/openstack/tripleo-heat-templates/blob/14998e6a5dc0b641da91b5624998c6947586f5a6/deployment/logging/files/nova-api.yaml#L37 | 09:27 |
| shyamb | Tengu: ^^ | 09:27 |
| Tengu | hmm.... not sure about the difference, to be honest. you should probably try the kolla_config way first, I think it's cleaner. But I may be wrong on that. | 09:28 |
| shyamb | Yeah, I aggree | 09:28 |
| Tengu | ah. check comment line 30 | 09:28 |
| Tengu | shyamb: line 30 of the last file you linked explain why it's using that command :). | 09:29 |
| shyamb | Tengu: yes | 09:29 |
| shyamb | In my case, no db_sync | 09:29 |
| Tengu | so yeah. go with the kolla_config way, and you should be all good. | 09:30 |
| shyamb | so kolla_config approach should work | 09:30 |
| shyamb | Tengu: thanks a lot | 09:30 |
| Tengu | no problem | 09:30 |
| shyamb | this was quick | 09:30 |
| Tengu | :) | 09:30 |
| Tengu | I'm still learning ;) | 09:30 |
| *** florianf has quit IRC | 09:42 | |
| chkumar|rover | FYI, standalone scenario 01 is broken in check, https://review.opendev.org/#/c/678186/ will fix the issue | 09:50 |
| *** shyamb has quit IRC | 09:57 | |
| fmount | chkumar|rover: gfidente I've seen the review, thanks, we will revert it when the merge patch is completed | 09:59 |
| *** hjensas has quit IRC | 10:00 | |
| *** dsneddon has joined #tripleo | 10:00 | |
| *** psachin has quit IRC | 10:04 | |
| xarlos | Anyone got the link for the openstack docs for tripleo plesae? :-) | 10:05 |
| *** dsneddon has quit IRC | 10:05 | |
| chkumar|rover | xarlos: http://tripleo.org/ | 10:06 |
| *** psachin has joined #tripleo | 10:06 | |
| xarlos | chkumar|rover: Strangely, the docs.openstack link seemed a little more developed. | 10:07 |
| chkumar|rover | xarlos: https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/ | 10:10 |
| *** jaosorior has quit IRC | 10:10 | |
| xarlos | chkumar|rover: Ah yeah. Has the link format changed? | 10:11 |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-ansible master: Create tripleo-cellv2 role https://review.opendev.org/678022 | 10:11 |
| chkumar|rover | xarlos: yes, | 10:12 |
| xarlos | chkumar|rover: That explains it, thanks :-) | 10:14 |
| chkumar|rover | :-) | 10:14 |
| *** shyamb has joined #tripleo | 10:22 | |
| *** jaosorior has joined #tripleo | 10:27 | |
| *** bhagyashris has quit IRC | 10:30 | |
| *** dsneddon has joined #tripleo | 10:31 | |
| shyamb | Tengu: Hi | 10:35 |
| *** morazi has quit IRC | 10:35 | |
| shyamb | I have integrated my service in osp13 | 10:35 |
| shyamb | but its' endpoints are not getting created on overcloud deplot | 10:36 |
| *** dsneddon has quit IRC | 10:36 | |
| shyamb | Any idea, on what configuration I need to do to enable endpoint creation for my service | 10:36 |
| shyamb | any code reference will help a lot? | 10:37 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart master: Increase overcloud deploy timeout to 110 https://review.opendev.org/677458 | 10:39 |
| *** shyamb has quit IRC | 10:44 | |
| *** chudler has joined #tripleo | 10:55 | |
| *** chudler has quit IRC | 10:55 | |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: Introduce arch aware tag for container images https://review.opendev.org/663977 | 10:55 |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-ansible master: WIP: Create a role for OvS-DPDK host configuration https://review.opendev.org/677712 | 10:56 |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: DNM: Test multi-arch support for build-containers role https://review.opendev.org/678058 | 10:59 |
| chkumar|rover | Hello | 11:01 |
| chkumar|rover | one of the neutron tempest test is failing here https://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-full-tempest-master/3c94c86/logs/stestr_results.html | 11:01 |
| chkumar|rover | neutron_tempest_plugin.scenario.test_security_groups.NetworkSecGroupTest.test_two_sec_groups | 11:01 |
| chkumar|rover | As per the neutron server logs https://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-full-tempest-master/3c94c86/logs/undercloud/var/log/containers/neutron/server.log.txt.gz?level=ERROR | 11:02 |
| chkumar|rover | RevisionConflict: OVN revision number for d748ff0d-fe8c-47e5-bcb8-d2305119d1c6 (type: ports) is equal or higher than the given resource. Skipping update | 11:02 |
| chkumar|rover | please have a look, thanks! | 11:02 |
| *** shyamb has joined #tripleo | 11:04 | |
| gfidente | fmount chkumar|rover shall we just merge https://review.opendev.org/#/c/678186/ ? | 11:07 |
| shyamb | Hi | 11:07 |
| *** apetrich has quit IRC | 11:08 | |
| fmount | gfidente: chkumar|rover yes but | 11:08 |
| fmount | I'm wondering if merging https://review.opendev.org/#/c/674556/ | 11:08 |
| chkumar|rover | fmount: gfidente can we wait till it finishes http://zuul.openstack.org/stream/3fe86775915644adb36c0bf6afec27cc?logfile=console.log | 11:08 |
| fmount | can solve the issue | 11:09 |
| *** apetrich has joined #tripleo | 11:09 | |
| *** dsneddon has joined #tripleo | 11:09 | |
| gfidente | fmount right because we would get grafana nodes then | 11:09 |
| gfidente | fmount and don't have/need the haproxy part | 11:09 |
| gfidente | fmount cause it's not used in scenario001 | 11:09 |
| *** rfolco is now known as rfolco|brb | 11:10 | |
| fmount | so merging the 556 fixes the scenario001 | 11:10 |
| fmount | and we don't need https://review.opendev.org/#/c/678186/ | 11:10 |
| fmount | but I need to double check the status here: http://zuul.openstack.org/status | 11:10 |
| *** hjensas has joined #tripleo | 11:11 | |
| chkumar|rover | https://review.opendev.org/#/c/674556 standalone scenario01 passed | 11:12 |
| *** tesseract has joined #tripleo | 11:12 | |
| chkumar|rover | fmount: gfidente both patches fixes the issue | 11:13 |
| gfidente | chkumar|rover better to keep going with the actual changes then | 11:14 |
| chkumar|rover | gfidente: yes | 11:14 |
| *** dsneddon has quit IRC | 11:14 | |
| *** skramaja has quit IRC | 11:16 | |
| *** rfolco|brb is now known as rfolco | 11:17 | |
| openstackgerrit | Natal Ngétal proposed openstack/paunch master: [Configuration] Switch to stestr. https://review.opendev.org/629421 | 11:19 |
| *** apetrich has quit IRC | 11:24 | |
| *** apetrich has joined #tripleo | 11:25 | |
| *** jaosorior has quit IRC | 11:26 | |
| *** pbandark has quit IRC | 11:26 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-ansible master: Bump pre-commit hooks https://review.opendev.org/677012 | 11:29 |
| *** jpena is now known as jpena|lunch | 11:32 | |
| *** d0ugal has quit IRC | 11:33 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-ci master: Build fluentd containers for RHEL8 https://review.opendev.org/678202 | 11:35 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-ci master: Build fluentd containers for RHEL8 https://review.opendev.org/678202 | 11:38 |
| *** jpich has quit IRC | 11:39 | |
| *** jpich has joined #tripleo | 11:40 | |
| *** xarlos has quit IRC | 11:42 | |
| *** dsneddon has joined #tripleo | 11:43 | |
| *** jaosorior has joined #tripleo | 11:44 | |
| openstackgerrit | Jon Schlueter proposed openstack/tripleo-common master: cleanup fluentd overrides https://review.opendev.org/678206 | 11:46 |
| jschlueter | cloudnull: ^^ | 11:46 |
| *** paramite has joined #tripleo | 11:47 | |
| *** dsneddon has quit IRC | 11:48 | |
| *** d0ugal has joined #tripleo | 11:49 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ci master: DNM: test python https://review.opendev.org/678208 | 11:51 |
| *** morazi has joined #tripleo | 11:54 | |
| *** cylopez has quit IRC | 11:57 | |
| *** rh-jelabarre has joined #tripleo | 12:02 | |
| *** jpich has quit IRC | 12:02 | |
| *** cylopez has joined #tripleo | 12:02 | |
| *** jpich has joined #tripleo | 12:03 | |
| *** surpatil has joined #tripleo | 12:15 | |
| *** beagles has joined #tripleo | 12:16 | |
| *** rlandy has joined #tripleo | 12:20 | |
| *** rlandy is now known as rlandy|ruck | 12:20 | |
| *** shyamb has quit IRC | 12:21 | |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: DNM: Test multi-arch support for build-containers role https://review.opendev.org/678058 | 12:22 |
| *** dsneddon has joined #tripleo | 12:22 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-heat-templates master: Disable fluentd https://review.opendev.org/678211 | 12:25 |
| *** shyamb has joined #tripleo | 12:25 | |
| zbr | maybe someone can explain me why after runing `tox -e lint` on THT you endup with >100 untracked files inside your repository. | 12:26 |
| *** dsneddon has quit IRC | 12:27 | |
| *** hjensas has quit IRC | 12:27 | |
| openstackgerrit | Emilien Macchi proposed openstack/paunch master: WIP - introduce container_running() and use it before running podman exec https://review.opendev.org/677860 | 12:33 |
| *** shyamb has quit IRC | 12:41 | |
| *** udesale has joined #tripleo | 12:48 | |
| *** jpena|lunch is now known as jpena | 12:49 | |
| *** rascasoft has quit IRC | 12:50 | |
| *** rascasoft has joined #tripleo | 12:53 | |
| *** mcornea has joined #tripleo | 12:54 | |
| *** dsneddon has joined #tripleo | 12:54 | |
| *** ekultails has joined #tripleo | 12:55 | |
| *** gbarros has joined #tripleo | 12:57 | |
| *** fultonj has joined #tripleo | 12:59 | |
| *** dsneddon has quit IRC | 12:59 | |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-heat-templates stable/rocky: Redis metadata using incorrect network/service https://review.opendev.org/678218 | 13:00 |
| *** amoralej is now known as amoralej|lunch | 13:00 | |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates master: Remove fluentd composable service https://review.opendev.org/668851 | 13:02 |
| *** ratailor has quit IRC | 13:04 | |
| *** cdearborn has joined #tripleo | 13:05 | |
| *** jaosorior has quit IRC | 13:14 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Move SnapshotsActionsTest to master skip list https://review.opendev.org/678141 | 13:14 |
| *** aakarsh|2 has quit IRC | 13:15 | |
| *** Goneri has joined #tripleo | 13:20 | |
| *** amoralej|lunch is now known as amoralej | 13:22 | |
| ade_lee | mwhahaha, confirmed that the fix to undercloud yesterday (adding back notifications) works | 13:23 |
| ade_lee | mwhahaha, I see it already merged :) | 13:23 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Migrate scenario001 to haproxy/pacemaker https://review.opendev.org/678224 | 13:24 |
| mwhahaha | ade_lee: k thanks | 13:25 |
| ade_lee | mwhahaha, of course we probably want to craft something more refined for when telemetry is disabled and novajoin is enabled to prevent a buildup of messages in the [notifications] queue | 13:26 |
| *** dsneddon has joined #tripleo | 13:27 | |
| mwhahaha | ade_lee: probably but it shouldn't be too bad because it's the udnercloud. that can be a later follow up | 13:27 |
| *** ksambor has quit IRC | 13:27 | |
| ade_lee | mwhahaha, ack | 13:27 |
| *** ksambor has joined #tripleo | 13:28 | |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: DNM: Test multi-arch support for build-containers role https://review.opendev.org/678058 | 13:29 |
| *** rascasoft has quit IRC | 13:31 | |
| *** dsneddon has quit IRC | 13:31 | |
| *** rascasoft has joined #tripleo | 13:33 | |
| *** bnemec has joined #tripleo | 13:34 | |
| *** bnemec is now known as beekneemech | 13:35 | |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-common master: Add health check script for qdrouterd https://review.opendev.org/672967 | 13:36 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-common master: Activate health check script for qdrouterd image https://review.opendev.org/672969 | 13:36 |
| *** holser has quit IRC | 13:39 | |
| *** holser has joined #tripleo | 13:40 | |
| *** artom has joined #tripleo | 13:40 | |
| *** surpatil has quit IRC | 13:42 | |
| *** pbandark has joined #tripleo | 13:44 | |
| *** psachin has quit IRC | 13:47 | |
| cloudnull | mornings | 13:48 |
| cloudnull | jschlueter lgtm ! | 13:48 |
| cloudnull | thanks ! | 13:48 |
| *** aakarsh|2 has joined #tripleo | 13:48 | |
| cloudnull | EmilienM looking at that cherry-pick now | 13:48 |
| *** avivgts has quit IRC | 13:51 | |
| jschlueter | cloudnull: were you intended to block all plugins or just the plugins via gem install? | 13:55 |
| cloudnull | just the ones via gem install. I was attempting to return the fluentd container, for our purposes, back what kolla was providing before before they were using td-agent | 13:57 |
| cloudnull | https://review.opendev.org/#/c/675614/ | 13:57 |
| cloudnull | That's the review they're trying to land | 13:58 |
| *** ramishra has quit IRC | 14:01 | |
| openstackgerrit | Alex Schultz proposed openstack/instack-undercloud stable/queens: Explicitly set notification driver for novajoin https://review.opendev.org/678236 | 14:03 |
| dmsimard | cloudnull: I'll release the two nodes we had held unless we need them for anything else | 14:05 |
| *** dsneddon has joined #tripleo | 14:06 | |
| *** d0ugal has quit IRC | 14:06 | |
| *** d0ugal has joined #tripleo | 14:06 | |
| cloudnull | dmsimard ++ | 14:07 |
| cloudnull | sorry I should have followed up with you on that | 14:07 |
| dmsimard | all good | 14:07 |
| dmsimard | they served their purpose :) | 14:07 |
| cloudnull | ++ | 14:07 |
| cloudnull | it'll be a couple days before we can confirm this 401 nonsense as resolved, but its generally looking good | 14:08 |
| dmsimard | cloudnull: btw if there really was an order of magnitude of perf increase due to that utf8 thing, it'd probably be relnote worthy | 14:09 |
| cloudnull | I'd have to look into what the old successful runs were to prove that. | 14:09 |
| cloudnull | in the logging review, it went from 2+ hours to ~20 minutes. | 14:10 |
| *** yprokule has quit IRC | 14:10 | |
| *** dsneddon has quit IRC | 14:11 | |
| *** florianf has joined #tripleo | 14:11 | |
| *** dsneddon has joined #tripleo | 14:15 | |
| *** abishop is now known as abishop|afk | 14:15 | |
| openstackgerrit | Merged openstack/tripleo-docs master: Update references to deploy-guide https://review.opendev.org/678043 | 14:21 |
| *** chkumar|rover is now known as raukadah | 14:24 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: DNM Test review to allow vexxhost qualification to complete https://review.opendev.org/678242 | 14:28 |
| *** HardCase has joined #tripleo | 14:31 | |
| HardCase | Hey all, would anyone be able to tell me if it would hurt my overcloud deployment if I reinstalled my undercloud server? | 14:32 |
| mwhahaha | HardCase: you'd need to back it up correctly prior to reinstalling it to ensure you can manage your overcloud going forward | 14:36 |
| mwhahaha | we do have an undercloud backup & restore procedure | 14:36 |
| mwhahaha | so it shouldn't impact the overcloud unless you're using the undercloud to provide internet connectivity for your overcloud | 14:37 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: WIP - switch featureset037 to run on podman https://review.opendev.org/678244 | 14:38 |
| HardCase | Ok cool, maybe I missed that, is that in the docs somewhere? | 14:38 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-common stable/stein: Log exceptions when checking status https://review.opendev.org/678110 | 14:38 |
| cloudnull | mwhahaha EmilienM ^ updated to resolve the pep8 issue | 14:39 |
| mwhahaha | HardCase: https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/post_deployment/backup_and_restore/00_index.html | 14:39 |
| HardCase | K, I apologize I missed this but sincerest thanks | 14:39 |
| openstackgerrit | Merged openstack/tripleo-upgrade stable/stein: Handle when upgrade_prepare_extra_params is a string. https://review.opendev.org/676149 | 14:40 |
| mwhahaha | is it .Names or .Name | 14:40 |
| mwhahaha | oh so it's the config id | 14:40 |
| mwhahaha | can you show what teh labels are for that container? | 14:41 |
| mwhahaha | i mean the running labels | 14:41 |
| mwhahaha | like show the container info | 14:41 |
| mwhahaha | config_id is step2 | 14:42 |
| mwhahaha | why are we looking for the config id at all? | 14:43 |
| mwhahaha | because the container gets launched on step2 but this action is in step3 | 14:43 |
| mwhahaha | we shouldn't care abotu teh config_id | 14:43 |
| openstackgerrit | James Slagle proposed openstack/tripleo-ansible master: Choose first node as bootstrap node name/ip https://review.opendev.org/678246 | 14:45 |
| openstackgerrit | James Slagle proposed openstack/tripleo-ansible master: Restore AllNodesExtraMapData https://review.opendev.org/678247 | 14:45 |
| zzzeek | hey is it a thing that nearly all tripleo-heat-templates gerrits against master are failing on some of the tempest runs? https://review.opendev.org/#/q/project:openstack/tripleo-heat-templates | 14:46 |
| zzzeek | I dont want to keep typing "recheck" on mine if that is loading down the queue | 14:46 |
| mwhahaha | you mean is scenario001 broken? | 14:46 |
| mwhahaha | maybe | 14:46 |
| zzzeek | mwhahaha: yeah that one seems to be a problem | 14:47 |
| * mwhahaha pokes gfidente | 14:47 | |
| mwhahaha | rlandy|ruck, weshay: is scenario001 broken? | 14:47 |
| *** amoralej is now known as amoralej|off | 14:47 | |
| mwhahaha | https://zuul.opendev.org/t/openstack/builds?job_name=%09tripleo-ci-centos-7-scenario001-standalone seems to indicate yes | 14:48 |
| zzzeek | tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 / 035 / 053 seem to fail a lot too | 14:48 |
| gfidente | mwhahaha zzzeek yeah broken but the fix is merging | 14:48 |
| mwhahaha | ovb is questionable | 14:48 |
| gfidente | the fix is 674556 | 14:49 |
| gfidente | in gate | 14:49 |
| gfidente | seems it almost got merged | 14:49 |
| gfidente | but one of the jobs is probably timing out | 14:50 |
| gfidente | at collect-logs | 14:50 |
| mwhahaha | figures | 14:50 |
| *** holser has quit IRC | 14:50 | |
| *** ksambor_ has joined #tripleo | 14:50 | |
| zzzeek | gfidente: OK ill just ping my build to retry tomorrow or something | 14:51 |
| *** kopecmartin is now known as kopecmartin|off | 14:51 | |
| *** HardCase has quit IRC | 14:51 | |
| *** amoralej|off has quit IRC | 14:51 | |
| gfidente | zzzeek :D | 14:51 |
| *** ksambor has quit IRC | 14:53 | |
| *** ksambor_ is now known as ksambor | 14:53 | |
| *** xek has quit IRC | 14:58 | |
| rlandy|ruck | mwhahaha: yes | 14:58 |
| rlandy|ruck | as gfidente says - waiting on gate now | 14:58 |
| Tengu | folks, care to +W this one? https://review.opendev.org/672745 as well as its friends? https://review.opendev.org/#/q/status:open+project:openstack/tripleo-validations+branch:master+topic:bp/validation-framework-infra - that would be really cool :) | 14:59 |
| rlandy|ruck | https://bugs.launchpad.net/tripleo/+bug/1838460 | 14:59 |
| openstack | Launchpad bug 1838460 in tripleo "overcloud deploy fails with ou must add a [grafana-server] group and add at least one node" [High,In progress] - Assigned to chandan kumar (chkumar246) | 14:59 |
| Tengu | mwhahaha: thanks for the rebases btw :) | 14:59 |
| rlandy|ruck | zzzeek fs001,35,53 failures will not block your change | 15:01 |
| zzzeek | rlandy|ruck: intersting how come they dont say "non-voting" ? | 15:01 |
| rlandy|ruck | zzzeek: they run in third party | 15:01 |
| rlandy|ruck | it never votes | 15:01 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add the certificate specs in ceph_grafana composable service https://review.opendev.org/674556 | 15:01 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Replace ceph_grafana-server with ceph_grafana https://review.opendev.org/677708 | 15:01 |
| rlandy|ruck | there have been timeout issues there we are looking into | 15:01 |
| rlandy|ruck | yay - merged | 15:02 |
| zzzeek | raukadah: that's interting b.c. one of them has the "non-voting" indicator, guess that is idiosyncratic on their end thanks for the tip | 15:02 |
| *** abishop|afk is now known as abishop | 15:03 | |
| gfidente | luckily change just merged | 15:05 |
| gfidente | we should have scenario001 back working | 15:05 |
| gfidente | look at that | 15:06 |
| gfidente | awesome | 15:06 |
| *** mschuppert is now known as mschuppert|pto | 15:08 | |
| * gfidente runs | 15:08 | |
| cloudnull | if folks are in a reviewing mood, it would be great to get some eyes on - https://review.opendev.org/#/q/project:%255Eopenstack/tripleo-ansible+status:open+label:verified%253D%252B1%252Cuser%253Dzuul | 15:08 |
| *** morazi has quit IRC | 15:10 | |
| *** paramite has quit IRC | 15:17 | |
| *** dsneddon has quit IRC | 15:18 | |
| *** holser has joined #tripleo | 15:24 | |
| *** cfontain has joined #tripleo | 15:31 | |
| *** jpena is now known as jpena|off | 15:32 | |
| *** rpittau is now known as rpittau|afk | 15:32 | |
| rlandy|ruck | rdocloud jobs (fs001, 53, 35) should be better now - we've done some log cleanup | 15:33 |
| rlandy|ruck | zzzeek: ^^ if you want to retest | 15:33 |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-ansible master: WIP: Fix Octavia certificate lifecycle https://review.opendev.org/678257 | 15:33 |
| zzzeek | rlandy|ruck: thanks! | 15:33 |
| *** cfontain has quit IRC | 15:36 | |
| *** cfontain has joined #tripleo | 15:36 | |
| *** cfontain_ has joined #tripleo | 15:38 | |
| *** cfontain has quit IRC | 15:41 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Move GroupVars to overcloud.j2.yaml https://review.opendev.org/677218 | 15:48 |
| *** dsneddon has joined #tripleo | 15:48 | |
| *** Vorrtex has joined #tripleo | 15:51 | |
| *** dsneddon has quit IRC | 15:53 | |
| gfidente | bandini fmount btw https://review.opendev.org/678224 will initially fail because it started before we fixed scenario001 | 15:53 |
| gfidente | we'll recheck and see | 15:53 |
| fmount | gfidente: yes | 15:53 |
| fmount | I've seen and rechecked | 15:53 |
| fmount | gfidente: https://ef29a4a24daa6c7c76e0-6f0cb0ace11e3e1afebf268bdebe6e7b.ssl.cf5.rackcdn.com/678224/1/check/tripleo-ci-centos-7-scenario001-standalone/92cb2a6/logs/undercloud/home/zuul/undercloud-ansible-J6u0ty/inventory.yaml.txt.gz | 15:54 |
| openstackgerrit | Emilien Macchi proposed openstack/paunch master: WIP - introduce container_running() and use it before running podman exec https://review.opendev.org/677860 | 15:54 |
| gfidente | fmount oh cool in that run it does have Standalone | 15:55 |
| *** holser has quit IRC | 15:55 | |
| fmount | yeah | 15:56 |
| fmount | gfidente: https://ef29a4a24daa6c7c76e0-6f0cb0ace11e3e1afebf268bdebe6e7b.ssl.cf5.rackcdn.com/678224/1/check/tripleo-ci-centos-7-scenario001-standalone/92cb2a6/logs/undercloud/var/log/config-data/haproxy/etc/haproxy/haproxy.cfg.txt.gz | 15:56 |
| fmount | gfidente: so I think it's the right way | 15:56 |
| gfidente | cool | 15:56 |
| *** udesale has quit IRC | 15:57 | |
| *** udesale has joined #tripleo | 15:58 | |
| gfidente | so much awesomeness on a friday | 15:58 |
| fmount | ahahhaah yeah | 15:59 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/paunch master: WIP - introduce container_running() and use it before running podman exec https://review.opendev.org/677860 | 15:59 |
| *** hamzy has quit IRC | 15:59 | |
| *** dtantsur is now known as dtantsur|afk | 16:00 | |
| *** cylopez has quit IRC | 16:02 | |
| *** florianf has quit IRC | 16:05 | |
| *** holser has joined #tripleo | 16:06 | |
| *** beekneemech has quit IRC | 16:07 | |
| *** holser has quit IRC | 16:07 | |
| *** lucasagomes has quit IRC | 16:09 | |
| *** tesseract has quit IRC | 16:11 | |
| *** rascasoft has quit IRC | 16:14 | |
| mwhahaha | cloudnull: so where specifically is the reauth bits | 16:16 |
| *** rascasoft has joined #tripleo | 16:16 | |
| mwhahaha | cloudnull: i'm trying to use the uploader and i'm getting a 401 but it doesn't look like it's retrying teh token request | 16:16 |
| mwhahaha | which seems odd consider i just did an auth and it's bad | 16:16 |
| cloudnull | https://github.com/openstack/tripleo-common/blob/master/tripleo_common/image/image_uploader.py#L521-L532 | 16:17 |
| mwhahaha | hmm oh wait i'm getting insufficient_scope | 16:17 |
| mwhahaha | which seems odd | 16:17 |
| cloudnull | it will only re-auth when the docker api returns "invalid_token" | 16:17 |
| * mwhahaha flips tables | 16:18 | |
| cloudnull | we could open that up to any 401 but it was limited because the docker api is dumb | 16:18 |
| mwhahaha | i know what the problem is | 16:18 |
| mwhahaha | i was using docker.io/tripleomaster/ as my pull source | 16:18 |
| * cloudnull hopes its not a problem I created | 16:18 | |
| mwhahaha | so when it puts the url together it had docker.io/tripleomaster//container | 16:19 |
| mwhahaha | docker.io will 301 to the non-double slash version | 16:19 |
| mwhahaha | but it drops the auth token | 16:19 |
| mwhahaha | so it just loops on a 401 | 16:19 |
| mwhahaha | guess we need to cleanup the url a bit first | 16:19 |
| cloudnull | :\ | 16:20 |
| *** bnemec has joined #tripleo | 16:20 | |
| openstackgerrit | Merged openstack/tripleo-ci master: New vexxhost CI settings https://review.opendev.org/677270 | 16:26 |
| *** dsneddon has joined #tripleo | 16:28 | |
| *** cfontain_ has quit IRC | 16:28 | |
| *** Vorrtex has quit IRC | 16:29 | |
| weshay | cgoncalves you around /me looking at the infrared job for octavia | 16:30 |
| weshay | cgoncalves DFG-network-octavia-15_director-rhel-virthost-3cont_2comp-ipv4-geneve/15/ | 16:30 |
| cgoncalves | weshay, yo | 16:30 |
| weshay | cgoncalves hey.. is the issue atm, the containers are not getting updated? | 16:30 |
| cgoncalves | weshay, correct | 16:31 |
| weshay | ya.. same thing w/ teh latest run 15 | 16:31 |
| weshay | http://pastebin.test.redhat.com/791380 | 16:31 |
| cgoncalves | weshay, look at build 19 instead. it passed overcloud deploy. failed later at tempest | 16:31 |
| weshay | k | 16:31 |
| cgoncalves | tempest before was failing on way less tests than build 19, so I assume the container didn't get the patched rpms | 16:32 |
| * weshay checking | 16:32 | |
| *** dsneddon has quit IRC | 16:32 | |
| *** HardCase has joined #tripleo | 16:33 | |
| openstackgerrit | Merged openstack/puppet-tripleo stable/queens: Add ability to install kernel related packages https://review.opendev.org/678093 | 16:34 |
| weshay | cgoncalves ya.. again in 19 in the tripleo-container-image-prepare.log I do not see ansible-role-tripleo-modify-image invoked.. so containers afaict are not getting updated | 16:34 |
| cgoncalves | weshay, yeah. build 19 ran with this included: https://review.gerrithub.io/c/redhat-openstack/infrared/+/466083 | 16:35 |
| cgoncalves | not sure you're following that one so just in case | 16:35 |
| openstackgerrit | Emilien Macchi proposed openstack/paunch master: WIP - introduce container_running() and use it before running podman exec https://review.opendev.org/677860 | 16:36 |
| weshay | cgoncalves it didn't fix it | 16:37 |
| cgoncalves | right | 16:37 |
| weshay | cgoncalves should see something like.. the few hundred lines after 2019-08-22 20:16:27.093 18666 INFO tripleo_common.image.image_uploader [ ] Playbook variables: | 16:38 |
| weshay | in https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_83/678083/4/check/tripleo-ci-centos-7-containers-multinode/3c9cba9/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 16:38 |
| openstackgerrit | James Slagle proposed openstack/tripleo-ansible master: Restore AllNodesExtraMapData https://review.opendev.org/678247 | 16:39 |
| openstackgerrit | James Slagle proposed openstack/tripleo-ansible master: Choose first node as bootstrap node name/ip https://review.opendev.org/678246 | 16:39 |
| *** Vorrtex has joined #tripleo | 16:39 | |
| *** saneax has quit IRC | 16:43 | |
| cgoncalves | weshay, are you trying to fix it in infrared? if so, maybe better sync with abregman (Arie) | 16:44 |
| weshay | cgoncalves I don't think I have access | 16:44 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Fix trailing slashes for container locations https://review.opendev.org/678272 | 16:45 |
| weshay | this does mean any gerrit check/gating that has been done is probably invalid in the latest releases unfortunately | 16:45 |
| *** saneax has joined #tripleo | 16:45 | |
| cgoncalves | I'm afraid so, yeah | 16:46 |
| cgoncalves | hopefully we'll not have bad surprises once it starts patching the containers again | 16:46 |
| cgoncalves | we're confident the octavia patch blocked by this CI issue works (no code deviation from upstream stable/stein) but we shouldn't merge stuff without downstream CI say-so | 16:47 |
| weshay | cgoncalves getting the info for him htough | 16:51 |
| weshay | though | 16:52 |
| weshay | cgoncalves I'll include you in the email | 16:52 |
| *** bnemec has quit IRC | 16:53 | |
| cgoncalves | weshay, cool, thanks! | 16:53 |
| *** saneax has quit IRC | 16:56 | |
| *** dsneddon has joined #tripleo | 17:01 | |
| *** saneax has joined #tripleo | 17:01 | |
| *** udesale has quit IRC | 17:05 | |
| *** udesale has joined #tripleo | 17:06 | |
| *** dsneddon has quit IRC | 17:06 | |
| *** udesale has quit IRC | 17:11 | |
| *** udesale has joined #tripleo | 17:12 | |
| *** dsneddon has joined #tripleo | 17:12 | |
| *** saneax has quit IRC | 17:13 | |
| *** HardCase has quit IRC | 17:14 | |
| weshay | cgoncalves ok.. sorry about the spam.. but I think I see the issue, hopefully the email makes sense | 17:14 |
| *** saneax has joined #tripleo | 17:15 | |
| rlandy|ruck | EmilienM: hi - ping re: https://bugs.launchpad.net/tripleo/+bug/1840616 | 17:17 |
| openstack | Launchpad bug 1840616 in tripleo "Master/stein check and promotion OVB jobs are randomly giving ansible time out while overcloud deploy" [Critical,Triaged] - Assigned to Ronelle Landy (rlandy) | 17:17 |
| * EmilienM hides | 17:17 | |
| rlandy|ruck | it's just a question | 17:18 |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: Introduce arch aware tag for container images https://review.opendev.org/663977 | 17:18 |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: DNM: Test multi-arch support for build-containers role https://review.opendev.org/678058 | 17:18 |
| EmilienM | I'm not sure we want to increase the timeout | 17:18 |
| rlandy|ruck | I'm not suggesting that | 17:18 |
| EmilienM | good | 17:18 |
| EmilienM | what's the question? :) | 17:18 |
| rlandy|ruck | I'm looking into why the deploy takes so long | 17:19 |
| rlandy|ruck | so there are two theories going on | 17:19 |
| EmilienM | I see the fact caching being potential culprit | 17:19 |
| rlandy|ruck | EmilienM: so that was my first assumption | 17:19 |
| rlandy|ruck | even when the tasks skipped, time is taken there | 17:20 |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: Introduce arch aware tag for container images https://review.opendev.org/663977 | 17:20 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/rocky: Add ability to install kernel related packages https://review.opendev.org/678092 | 17:20 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Add ExtraKernelPackages https://review.opendev.org/678097 | 17:20 |
| EmilienM | rlandy|ruck: skipped by a dry run? | 17:20 |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: DNM: Test multi-arch support for build-containers role https://review.opendev.org/678058 | 17:20 |
| EmilienM | yes, because in dry run puppet is run (in noop mode) which requires the facts to be there, and therefore cached | 17:21 |
| rlandy|ruck | EmilienM: I'm just comparing passing and timing out jobs | 17:21 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Add ExtraKernelPackages https://review.opendev.org/678096 | 17:21 |
| rlandy|ruck | 10 minutes the deploy sits there | 17:21 |
| rlandy|ruck | syncing facts | 17:21 |
| rlandy|ruck | in jobs that timeout | 17:22 |
| EmilienM | mhh | 17:22 |
| rlandy|ruck | although 10 minutes in the greater scheme of things is not huge | 17:22 |
| EmilienM | the rsync is done locally, so it can't really be networking. Maybe the compress or facter itself | 17:22 |
| rlandy|ruck | the container images time may be a bigger factor | 17:22 |
| rlandy|ruck | it could just push us over the edge | 17:24 |
| *** chem` has quit IRC | 17:26 | |
| rlandy|ruck | anyways the patch timing and when we start to see timeouts also coincides - but that doesn't really mean much | 17:26 |
| EmilienM | in any case the rsync should not take that long | 17:27 |
| openstackgerrit | Emilien Macchi proposed openstack/paunch stable/stein: WIP - introduce container_running() and use it before running podman exec https://review.opendev.org/678071 | 17:31 |
| *** saneax has quit IRC | 17:31 | |
| *** udesale has quit IRC | 17:35 | |
| *** hamzy has joined #tripleo | 17:35 | |
| *** pbandark has quit IRC | 17:40 | |
| *** morazi has joined #tripleo | 17:42 | |
| *** morazi has quit IRC | 17:43 | |
| *** morazi has joined #tripleo | 17:43 | |
| *** gbarros has quit IRC | 17:45 | |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: Introduce arch aware tag for container images https://review.opendev.org/663977 | 17:45 |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: DNM: Test multi-arch support for build-containers role https://review.opendev.org/678058 | 17:45 |
| *** jpich has quit IRC | 17:56 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Add compute_feature_enabled.config_drive to configure-tempest.sh https://review.opendev.org/678008 | 18:03 |
| EmilienM | rlandy|ruck: total time for the facter cache tasks is less than 3 seconds in my env (which isn't fast) | 18:03 |
| rlandy|ruck | EmilienM: that may not be our only problem | 18:06 |
| EmilienM | right | 18:06 |
| rlandy|ruck | just the first possibility | 18:06 |
| rlandy|ruck | honestly, I think we were pretty on the edge with the deploy time | 18:06 |
| rlandy|ruck | and now we're over | 18:06 |
| rlandy|ruck | checking http://logs.rdoproject.org/79/667379/10/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053/d422e3f/report.html to see what else it reveals | 18:07 |
| *** gbarros has joined #tripleo | 18:08 | |
| EmilienM | nice logs | 18:09 |
| rlandy|ruck | tristanC's handy work | 18:10 |
| *** gbarros has quit IRC | 18:23 | |
| mwhahaha | rlandy|ruck: looks like scenario001 is also broken for queens | 18:27 |
| rlandy|ruck | looking | 18:27 |
| *** gbarros has joined #tripleo | 18:35 | |
| rlandy|ruck | mwhahaha: can you link where you see the failure? http://zuul.openstack.org/builds?branch=stable%2Fqueens&job_name=tripleo-ci-centos-7-scenario001-multinode-oooq-container has mixed success | 18:36 |
| rlandy|ruck | times out | 18:36 |
| rlandy|ruck | 2019-08-13 18:39:30.962266 | primary | TASK [overcloud-prep-containers : Prepare for the containerized deployment] **** here | 18:37 |
| mwhahaha | let me see if i can find it again | 18:39 |
| mwhahaha | i should have pasted it | 18:39 |
| mwhahaha | oh sorry, it was a master change | 18:40 |
| mwhahaha | though the timesout seem concerning | 18:40 |
| rlandy|ruck | mwhahaha: yeah - that is something to investigate | 18:41 |
| *** morazi has quit IRC | 18:43 | |
| cloudnull | mwhahaha mind taking a look at https://review.opendev.org/#/c/678110/ | 18:45 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-quickstart-extras master: Add a tasks to download amphora image for octavia deployments https://review.opendev.org/669344 | 18:54 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-quickstart-extras master: Add support for running Octavia tempest tasks https://review.opendev.org/670135 | 18:54 |
| *** pierreprinetti has joined #tripleo | 18:56 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-ci master: Change featurset for octavia standalone to support full tempest https://review.opendev.org/669740 | 18:57 |
| *** xek has joined #tripleo | 18:58 | |
| *** hamzy has quit IRC | 19:01 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates stable/stein: Explicitly set notification driver for novajoin https://review.opendev.org/678288 | 19:05 |
| *** gbarros has quit IRC | 19:07 | |
| *** bnemec has joined #tripleo | 19:10 | |
| *** bnemec is now known as beekneemech | 19:11 | |
| *** HardCase has joined #tripleo | 19:16 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Add settings from fs001 missing in fs035 https://review.opendev.org/678290 | 19:21 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Adds tripleo-clients-install role https://review.opendev.org/677452 | 19:21 |
| *** pierreprinetti has quit IRC | 19:21 | |
| *** gbarros has joined #tripleo | 19:22 | |
| EmilienM | https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/logs_37/677237/12/check/tripleo-ci-centos-7-undercloud-containers/6aa4883/logs/undercloud/home/zuul/undercloud_install.log.txt.gz | 19:22 |
| EmilienM | fatal: [undercloud]: FAILED! => {"msg": "Invalid data passed to 'loop', it requires a list, got this instead: 8088. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup."} | 19:22 |
| EmilienM | cloudnull: ^ have you seen that? | 19:22 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Creates tripleo-validations-package role https://review.opendev.org/677229 | 19:23 |
| cloudnull | EmilienM i have not | 19:23 |
| * cloudnull looking now | 19:23 | |
| *** hamzy has joined #tripleo | 19:26 | |
| cloudnull | hum . i guess I need to give that a look | 19:27 |
| *** gbarros has quit IRC | 19:30 | |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: Introduce arch aware tag for container images https://review.opendev.org/663977 | 19:32 |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: DNM: Test multi-arch support for build-containers role https://review.opendev.org/678058 | 19:32 |
| *** gbarros has joined #tripleo | 19:35 | |
| *** morazi has joined #tripleo | 19:36 | |
| weshay | cgoncalves you're not still around are you? | 19:39 |
| *** gbarros has quit IRC | 19:43 | |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: Introduce arch aware tag for container images https://review.opendev.org/663977 | 19:43 |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: DNM: Test multi-arch support for build-containers role https://review.opendev.org/678058 | 19:43 |
| cloudnull | EmilienM i guess that variable when provided by heat is a string, not an int | 19:44 |
| cloudnull | so, i will fix that and write a test for it | 19:44 |
| EmilienM | ++ | 19:45 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Add check for string type dport https://review.opendev.org/678295 | 19:50 |
| cloudnull | EmilienM ^ | 19:50 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-heat-templates master: Convert firewall rules to use TripleO-Ansible https://review.opendev.org/677237 | 19:51 |
| EmilienM | fun | 19:52 |
| cloudnull | rebased ^ and updated the depends-on, so it should validate the firewall review. | 19:53 |
| cloudnull | when folks get a moment, mind reviewing https://review.opendev.org/#/c/678246/2 | 19:54 |
| EmilienM | so demanding :D | 19:55 |
| EmilienM | it doesn't pass OVB jobs | 19:55 |
| mwhahaha | i'm looking at it (unrelated) | 19:56 |
| EmilienM | https://logs.rdoproject.org/46/678246/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/d34da3c/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz#_2019-08-23_17_47_23 | 19:56 |
| mwhahaha | rlandy|ruck: fyi i think the nodes just timeout on ovb. not related to teh caching bits https://logs.rdoproject.org/46/678246/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035/5a61c34/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-08-23_19_19_56 | 19:56 |
| EmilienM | indeed | 19:56 |
| cloudnull | thanks mwhahaha EmilienM | 19:57 |
| rlandy|ruck | k - was one thing to investigate. fs035 was missing some setting fs001 had. Spinning on that change right now | 19:58 |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-common master: Add python3 file for additional images https://review.opendev.org/678297 | 19:58 |
| *** morazi has quit IRC | 20:03 | |
| *** morazi has joined #tripleo | 20:04 | |
| *** weshay is now known as weshay_MOD | 20:10 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Parametrize UpgradeLeappDevelSkip to pass multiple env variables. https://review.opendev.org/675292 | 20:39 |
| openstackgerrit | Merged openstack/instack-undercloud stable/queens: Explicitly set notification driver for novajoin https://review.opendev.org/678236 | 20:39 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Move SnapshotsActionsTest to master skip list https://review.opendev.org/678141 | 20:39 |
| cloudnull | is github having a bad day for anyone else ? -- "2019-08-23 20:23:25.258348 | primary | fatal: unable to access 'https://github.com/redhat-openstack/rdoinfo/': Failed to connect to 140.82.113.3: Network is unreachable" | 20:39 |
| raukadah | cloudnull: https://github.com/redhat-openstack/rdoinfo.git is not working?, I mean with .git or you can clone it from https://review.rdoproject.org/r/rdoinfo | 20:44 |
| cloudnull | raukadah saw this https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_47/678247/2/gate/tripleo-ci-centos-7-scenario004-standalone/925ed7d/ | 20:44 |
| cloudnull | sorry https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_47/678247/2/gate/tripleo-ci-centos-7-scenario004-standalone/925ed7d/job-output.txt | 20:45 |
| cloudnull | that job failed due to the above error | 20:45 |
| cloudnull | could just be instance related. | 20:45 |
| *** Vorrtex has quit IRC | 20:47 | |
| raukadah | cloudnull: may be, there was some github issue yesterday, that's why it failed , there were some issue with gh api yesterday , we tracked it from ruck/rover cockpit http://dashboard-ci.tripleo.org/d/YRJtmtNWk/cockpit?orgId=1 Feel free to recheck it | 20:50 |
| *** holser has joined #tripleo | 20:55 | |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Implement tripleo container image push command https://review.opendev.org/674800 | 20:58 |
| *** holser has quit IRC | 20:59 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Fix bogus reference to conditional in octavia upgrade tasks https://review.opendev.org/673440 | 21:01 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: ceph-base: Disable ceph-ansible firewall tasks https://review.opendev.org/678045 | 21:01 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Fix bogus reference to conditional in octavia upgrade tasks https://review.opendev.org/673441 | 21:01 |
| *** holser has joined #tripleo | 21:03 | |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Implement tripleo container image push command https://review.opendev.org/674800 | 21:22 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Create the tripleo-sshd role https://review.opendev.org/675707 | 21:25 |
| *** xek has quit IRC | 21:25 | |
| openstackgerrit | Merged openstack/tripleo-ansible master: Create tripleo-cellv2 role https://review.opendev.org/678022 | 21:35 |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-upgrade master: Change Upgrade Run Script Generation to Per Host https://review.opendev.org/667610 | 21:40 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Create the tripleo-sshd role https://review.opendev.org/675707 | 21:47 |
| *** HardCase has quit IRC | 21:49 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Sort bootstrap node check for container_puppet_tasks https://review.opendev.org/678314 | 22:06 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Ensure container_startup_configs_json_stat is defined https://review.opendev.org/678315 | 22:06 |
| openstackgerrit | James Slagle proposed openstack/tripleo-ansible master: Choose first node as bootstrap node name/ip https://review.opendev.org/678246 | 22:07 |
| *** aakarsh|2 has quit IRC | 22:10 | |
| *** rcernin has joined #tripleo | 22:11 | |
| *** rlandy|ruck has quit IRC | 22:21 | |
| *** mcornea has quit IRC | 22:27 | |
| *** morazi has quit IRC | 22:27 | |
| *** mcornea has joined #tripleo | 22:27 | |
| *** mcornea has quit IRC | 22:27 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates stable/rocky: Explicitly set notification driver for novajoin https://review.opendev.org/678316 | 22:28 |
| openstackgerrit | Merged openstack/tripleo-ansible master: Create ansible environment variables in profile.d https://review.opendev.org/678072 | 22:30 |
| *** morazi has joined #tripleo | 22:33 | |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: Introduce arch aware tag for container images https://review.opendev.org/663977 | 22:40 |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: DNM: Test multi-arch support for build-containers role https://review.opendev.org/678058 | 22:41 |
| *** rcernin has quit IRC | 23:11 | |
| *** rcernin has joined #tripleo | 23:12 | |
| *** beekneemech is now known as keanu | 23:26 | |
| *** keanu is now known as beekneemech | 23:27 | |
| openstackgerrit | Merged openstack/tripleo-ansible master: Adds tripleo-clients-install role https://review.opendev.org/677452 | 23:39 |
| openstackgerrit | Merged openstack/tripleo-ansible master: Creates tripleo-validations-package role https://review.opendev.org/677229 | 23:40 |
| openstackgerrit | Merged openstack/tripleo-validations master: Switch to promote docs job https://review.opendev.org/677792 | 23:41 |
| openstackgerrit | Merged openstack/tripleo-ci master: Run scenario 001 or 004 for changes to Ceph deployment manifests https://review.opendev.org/676362 | 23:41 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Do not forcibly enable Glance multiple locations for RBD backend https://review.opendev.org/676392 | 23:43 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Use default value for NovaLiveMigrationWaitForVIFPlug https://review.opendev.org/677703 | 23:44 |
| *** holser has quit IRC | 23:44 | |
| *** cdearborn has quit IRC | 23:44 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!