14:00:04 <mwhahaha> #startmeeting tripleo
14:00:04 <mwhahaha> #topic agenda
14:00:04 <mwhahaha> * Review past action items
14:00:04 <mwhahaha> * One off agenda items
14:00:04 <mwhahaha> * Squad status
14:00:05 <mwhahaha> * Bugs & Blueprints
14:00:05 <mwhahaha> * Projects releases or stable backports
14:00:06 <mwhahaha> * Specs
14:00:06 <mwhahaha> * open discussion
14:00:07 <mwhahaha> Anyone can use the #link, #action and #info commands, not just the moderatorǃ
14:00:07 <mwhahaha> Hi everyone! who is around today?
14:00:09 <openstack> Meeting started Tue Aug 13 14:00:04 2019 UTC and is due to finish in 60 minutes.  The chair is mwhahaha. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:10 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:12 <openstack> The meeting name has been set to 'tripleo'
14:00:44 <Tengu> \o/
14:00:50 <rlandy|rover> o/
14:01:01 <chkumar|ruck> \o/
14:01:07 <ksambor> o/
14:01:27 <openstackgerrit> Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Add firewall pruning and protocal parsing  https://review.opendev.org/676003
14:01:37 <marios> o/
14:05:02 <gchamoul> o/
14:05:38 <mwhahaha> not a whole bunch of people today, let's get moving
14:05:45 <mwhahaha> #topic review past action items
14:05:45 <mwhahaha> None
14:06:12 <mwhahaha> #topic one off agenda items
14:06:13 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-meeting-items
14:06:26 <mwhahaha> chandankumar has a few things today
14:06:32 <mwhahaha> When to drop Pike TripleO jobs from promotion pipeline and check & gate queue?
14:06:33 <mschuppert> o/
14:06:39 <mwhahaha> https://releases.openstack.org/ - pike(Extended Maintenance currently) -> unmaintained in Next phase section
14:06:39 <mwhahaha> https://review.rdoproject.org/r/#/c/21165/
14:06:49 <mwhahaha> imho we should do an em release of pike to kill it
14:08:04 <rlandy|rover> mwhahaha: k - the infra team asked us to raise this
14:08:20 <mwhahaha> which infra team?
14:08:26 <chkumar|ruck> mwhahaha: ok, Any plan when we want to do the em release for tripleo?
14:08:27 <mwhahaha> openstack-infra?
14:08:44 <rlandy|rover> rdo-infra
14:09:06 <mwhahaha> we also need to properly eol qocata
14:09:35 <mwhahaha> so i'm +2 on doign the em release but i don't have the bandwidth to do it
14:09:50 <mwhahaha> so if someone wants to figure out what has to happen, feel free to proceed
14:10:04 <chkumar|ruck> mwhahaha: how much work is needed there?
14:10:15 <rlandy|rover> we can do some research and see if it's within our resources
14:10:15 <mwhahaha> i have no idea which is the issue :D
14:10:24 <mwhahaha> i think it's just proposing some release bits
14:10:32 <rlandy|rover> two things:
14:10:35 <rlandy|rover> eol ocata
14:10:41 <rlandy|rover> and end pike releases - em
14:10:57 <ykarel|away> mwhahaha, you mean push release tags for pike?
14:11:02 <mwhahaha> yea
14:11:05 <mwhahaha> i think that's really it
14:11:12 <ykarel|away> mwhahaha, but as per doc, here will be no more release in extended maintenance phase
14:11:15 <jfrancoa> o/
14:11:19 <ykarel|away> https://docs.openstack.org/project-team-guide/stable-branches.html#maintenance-phases
14:11:23 <mwhahaha> right but we want to officially mark it as -em
14:11:26 <mwhahaha> which is how you kill it
14:11:29 <chkumar|ruck> mwhahaha: do we have a list of projects to tackle in tripleo?
14:11:35 <chkumar|ruck> mwhahaha: that is doable
14:12:01 <ykarel|away> mwhahaha, okk, pike-em tag is already there, may be it needs -eol tag
14:12:08 <mwhahaha> https://opendev.org/openstack/governance/src/branch/master/reference/projects.yaml#L3770
14:12:28 <mwhahaha> i mean basically it's figure out what it means to end it. i think we still have stable/pike branches?
14:12:48 <rlandy|rover> we do
14:13:01 <mwhahaha> so not sure how we have pike-em and still having branches
14:13:05 <ykarel|away> may be worth to drop a mail to release team and get clearity over the process
14:13:12 <mwhahaha> indeed
14:14:02 <openstackgerrit> François Charlier proposed openstack/tripleo-quickstart-extras master: 'is in' is not a valid Jinja test  https://review.opendev.org/676203
14:15:19 <mwhahaha> any other questions about the pike stuff?
14:15:27 <ykarel|away> https://review.opendev.org/#/c/652847/
14:15:57 <chkumar|ruck> mwhahaha: nope
14:16:08 <mwhahaha> so maybe we can just kill the pike bits
14:16:20 <mwhahaha> anyway next item
14:16:27 <mwhahaha> Current gate issues:
14:16:27 <mwhahaha> http://status.openstack.org/elastic-recheck/
14:16:27 <mwhahaha> https://bugs.launchpad.net/tripleo/+bug/1839845
14:16:29 <openstack> Launchpad bug 1839845 in tripleo "TripleO gate jobs run on limestone nodes are failing "to retrieve repo file from https://trunk.rdoproject.org/centos7-<release>/current-tripleo/delorean.repo" after 10 retries " [High,Triaged]
14:16:29 <mwhahaha> https://bugs.launchpad.net/tripleo/+bug/1839532
14:16:30 <openstack> Launchpad bug 1839532 in tripleo "tripleo gate jobs are failing to pull containers when running on ovh provider with "UNAUTHORIZED" error" [Critical,Triaged]
14:16:47 <rlandy|rover> we have a large amount of gate failures lately
14:16:54 <rlandy|rover> as we are sure you have all noticed
14:17:10 * Tengu puts some WD-40 in the gate in order to unstuck it
14:17:34 <marios> mwhahaha: looks like that https://bugs.launchpad.net/tripleo/+bug/1837388 we poked at a while back the 401 unauth one ...
14:17:35 <openstack> Launchpad bug 1837388 in tripleo " tripleo-ci jobs failing intermittently with "Image prepare failed: 401 Client Error: Unauthorized" " [High,Triaged] - Assigned to Marios Andreou (marios-b)
14:17:38 <openstackgerrit> Lee Yarwood proposed openstack/tripleo-quickstart master: WIP Enable noVNC tests within featureset020  https://review.opendev.org/676205
14:17:48 <marios> mwhahaha: not clear whats going on there esp since we already have the retries in all the things afaics
14:17:50 <rlandy|rover> jpena is going to try help us out with repo retrieve error
14:18:01 <rlandy|rover> but the first is ovh related
14:18:29 <sshnaidm> I'm sure this 401 is OVH proxy issue, just need to convince openstack-infra..
14:19:37 <rlandy|rover> anyways we wanted to just raise it to the group that we're working on what can be done here
14:20:28 <cloudnull> o/
14:20:35 <mwhahaha> we added retries but that doesn't seem to have helped
14:20:52 <mwhahaha> cloudnull were you able to track down additional places we might need to do the retries?
14:21:03 <rlandy|rover> no - the number of failures has increased
14:21:11 <cloudnull> ive not looked at it in a bit, but I can
14:21:14 <mwhahaha> well number of failures or log lines
14:21:21 <mwhahaha> we added retries which would increase log lines
14:21:24 <mwhahaha> doesn't mean more failures
14:22:11 <rlandy|rover> number of tests failing the gate in 24 hrs
14:23:17 <openstackgerrit> Chandan Kumar (raukadah) proposed openstack/tripleo-ci master: Fixed grab kolla patch rhel8 task for check job  https://review.opendev.org/675563
14:25:04 <mwhahaha> ok so we've got gate problems. I guess i'll try and look into it a bit more this week
14:25:09 <openstackgerrit> Kevin Carter (cloudnull) proposed openstack/tripleo-common master: Log exceptions when checking status  https://review.opendev.org/674919
14:25:15 <mwhahaha> unfortunately they aren't reproducable locally so it'll be tough to diagnose
14:25:48 <mwhahaha> any other comments on gate problems?
14:25:54 <chkumar|ruck> mwhahaha: for second bug, we need help, for first one, we are working on shipping dlrn repo through afs mirror
14:26:00 <cloudnull> +1 its had to track down , that was the review I had up to try and log more, just rebased it to see if maybe it helps ?
14:26:11 <cloudnull> **its hard to track down
14:26:23 <mwhahaha> cloudnull: can you link the other review?
14:26:39 <chkumar|ruck> it might help to fix the first bug,
14:27:13 <cloudnull> #link https://review.opendev.org/674919
14:27:18 <cloudnull> #link https://review.opendev.org/#/c/674672/
14:27:49 <mwhahaha> k once the log improvements pass ci let's get it merged
14:29:49 <mwhahaha> any other comments?
14:30:13 <chkumar|ruck> nope
14:30:37 <mwhahaha> alright moving on
14:30:41 <mwhahaha> #topic Squad status
14:30:41 <mwhahaha> ci
14:30:41 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting
14:30:41 <mwhahaha> upgrade
14:30:41 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status
14:30:42 <mwhahaha> edge
14:30:42 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-edge-squad-status
14:30:42 <hjensas> For ansible services we cannot use hiera interpolation? (i.e this is failing https://review.opendev.org/#/c/676178/). Is there another way?
14:30:43 <mwhahaha> integration
14:30:43 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-integration-squad-status
14:30:44 <mwhahaha> validations
14:30:44 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-validations-squad-status
14:30:45 <mwhahaha> networking
14:30:45 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-networking-squad-status
14:30:46 <mwhahaha> transformation
14:30:46 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-ansible-agenda
14:31:05 <mwhahaha> hjensas: no you can't
14:31:18 <hjensas> mwhahaha: sorry, let's do it after the meeting.
14:31:20 <mwhahaha> hjensas: there's no hiera performed in ansible
14:32:54 <mwhahaha> any status related highlights?
14:33:50 <cloudnull> all clear on the transformation front :)
14:35:23 <mwhahaha> alright moving on then
14:35:26 <mwhahaha> #topic bugs & blueprints
14:35:26 <mwhahaha> #link https://launchpad.net/tripleo/+milestone/train-3
14:35:26 <mwhahaha> For Train we currently have 26 blueprints and 513 (-5) open Launchpad Bugs. 513 train-3.  172 (+0) open Storyboard bugs.
14:35:26 <mwhahaha> #link https://storyboard.openstack.org/#!/project_group/76
14:35:44 <mwhahaha> train m3 is in early september
14:35:54 <mwhahaha> so please keep that in mind
14:36:12 <mwhahaha> any other bug issues (other than the previously mentioned gate related ones)
14:37:20 <openstackgerrit> Ronelle Landy proposed openstack/tripleo-ci master: DNM - To test periodic  https://review.opendev.org/638756
14:38:57 <openstackgerrit> Bogdan Dobrelya (bogdando) proposed openstack/paunch master: runner: do not run exec if container not running  https://review.opendev.org/675637
14:40:30 <mwhahaha> sounds like nope
14:40:31 <mwhahaha> #topic projects releases or stable backports
14:40:42 <mwhahaha> so we talked about the pike em stuff.
14:40:51 <mwhahaha> and m3 is in a about a month
14:41:00 <mwhahaha> any one have any backports they need help with?
14:43:26 <mwhahaha> sounds like nope
14:43:29 <mwhahaha> #topic specs
14:43:29 <mwhahaha> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open
14:43:37 <mwhahaha> any spec related topics?
14:45:18 <mwhahaha> #topic open discussion
14:45:21 <mwhahaha> anything else?
14:46:53 <openstackgerrit> Luke Short proposed openstack/paunch master: runner: do not run exec if container not running  https://review.opendev.org/675637
14:48:24 <mwhahaha> alright thanks everyone
14:48:27 <mwhahaha> #endmeeting