| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (4 more messages) | 00:16 |
|---|---|---|
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (4 more messages) | 02:16 |
| *** gkadam has joined #oooq | 02:45 | |
| *** ykarel has joined #oooq | 03:22 | |
| *** skramaja has joined #oooq | 03:37 | |
| *** gkadam has quit IRC | 03:47 | |
| *** udesale has joined #oooq | 04:03 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010 (4 more messages) | 04:16 |
| *** jaosorior has joined #oooq | 05:52 | |
| *** apetrich has joined #oooq | 06:11 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010 (4 more messages) | 06:16 |
| *** saneax has joined #oooq | 06:45 | |
| *** jtomasek has joined #oooq | 06:50 | |
| *** jbadiapa has joined #oooq | 06:51 | |
| *** rascasoft has joined #oooq | 07:44 | |
| *** fmount has quit IRC | 07:53 | |
| *** fmount has joined #oooq | 08:06 | |
| *** fmount has quit IRC | 08:07 | |
| *** fmount has joined #oooq | 08:11 | |
| *** amoralej has joined #oooq | 08:12 | |
| *** hubbot1 has quit IRC | 08:17 | |
| *** gouthamr has quit IRC | 08:18 | |
| *** holser_ has joined #oooq | 08:18 | |
| *** dmellado has quit IRC | 08:20 | |
| *** holser_ has quit IRC | 08:22 | |
| *** holser_ has joined #oooq | 08:22 | |
| *** d0ugal has quit IRC | 08:25 | |
| *** d0ugal has joined #oooq | 08:25 | |
| *** saneax has quit IRC | 08:38 | |
| *** dtantsur|afk is now known as dtantsur | 08:38 | |
| *** tosky has joined #oooq | 08:41 | |
| *** zbr has joined #oooq | 08:52 | |
| *** jpena|off is now known as jpena | 08:56 | |
| *** holser_ has quit IRC | 08:56 | |
| *** saneax has joined #oooq | 08:57 | |
| zbr | hI! anything i missed in the last 24h? (offline due to bricked modem) | 08:58 |
| *** chem has joined #oooq | 09:00 | |
| *** bogdando has joined #oooq | 09:02 | |
| *** holser_ has joined #oooq | 09:05 | |
| *** ykarel has quit IRC | 09:10 | |
| *** ykarel has joined #oooq | 09:11 | |
| rfolco|ruck | hey panda cockpit is off? | 09:31 |
| rfolco|ruck | morning | 09:31 |
| ykarel | looks like https://review.openstack.org/#/c/639651/15 broke atleast phase 1 jobs | 09:33 |
| ykarel | example log:- https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-promote-rocky-rdo_trunk-minimal-118/console.txt.gz | 09:34 |
| ykarel | rfolco|ruck, ^^ | 09:34 |
| *** ykarel is now known as ykarel|lunch | 09:37 | |
| rfolco|ruck | ykarel, I'll check, thanks for pointing out | 09:40 |
| sshnaidm | rfolco|ruck, cockpit is off because rdo cloud outage | 09:41 |
| rfolco|ruck | sshnaidm, semaphore says rdo cloud is operational | 09:41 |
| sshnaidm | rfolco|ruck, hmm..just now got mail about outage from kforde | 09:42 |
| rfolco|ruck | sshnaidm, ok, thanks | 09:43 |
| *** derekh has joined #oooq | 09:43 | |
| zbr | i need some hints on f28 work, i got into http://logs.openstack.org/47/641447/6/check/tripleo-ci-fedora-28-scenario001-standalone/34f22ed/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-03-09_14_37_02 | 09:48 |
| zbr | and am not sure what causes this, i cannot browse this mirror from my location either. | 09:49 |
| *** panda is now known as panda|rover | 09:50 | |
| panda|rover | rfolco|ruck: what are you doing awake ? | 09:52 |
| *** jfrancoa has joined #oooq | 09:52 | |
| *** ykarel|lunch is now known as ykarel | 10:07 | |
| ykarel | zbr, so looks like image not even exists on docker.io, example:- https://hub.docker.com/r/tripleomaster/fedora-binary-aodh-api/tags/ | 10:09 |
| ykarel | the mirror you pointed are for docker.io | 10:10 |
| zbr | ykarel: afaik these are published to registry.rdoproject.org which is now down anyway. | 10:11 |
| ykarel | but that job is not looking at registry.rdoproject | 10:11 |
| ykarel | http://logs.openstack.org/47/641447/6/check/tripleo-ci-fedora-28-scenario001-standalone/34f22ed/logs/undercloud/home/zuul/containers-prepare-parameters.yaml.txt.gz | 10:12 |
| zbr | now I do not know what i need to do different in order to run the same jobs upstream. | 10:12 |
| rfolco|ruck | panda|rover, I wake up at 6am to uber son to school | 10:13 |
| panda|rover | rfolco|ruck: do you have to take the plane to get there ? | 10:13 |
| panda|rover | Old maccloud had a server farm. E I E I O | 10:13 |
| panda|rover | and on on server farm there was a ovs-switchd E I E I O | 10:14 |
| panda|rover | with a crash crash here and a crash crash there | 10:14 |
| rfolco|ruck | panda|rover, lol | 10:15 |
| *** udesale has quit IRC | 10:34 | |
| arxcruz | is review.rdoproject.org offline ? | 10:37 |
| zbr | arxcruz: yes. all rdo | 10:38 |
| arxcruz | wel... | 10:39 |
| arxcruz | well... | 10:39 |
| panda|rover | arxcruz: https://i.stack.imgur.com/YThly.jpg | 10:43 |
| arxcruz | LOL | 10:44 |
| rfolco|ruck | panda|rover, https://bugs.launchpad.net/tripleo/+bug/1819440 | 10:48 |
| openstack | Launchpad bug 1819440 in tripleo "phase 1 failing on No such file or directory: '/home/stack/tempest'" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 10:48 |
| rfolco|ruck | ykarel, arxcruz ^ | 10:48 |
| ykarel | ack | 10:48 |
| rfolco|ruck | arxcruz, YATB - Yet Another Tempest Bug | 10:48 |
| arxcruz | rfolco|ruck: https://i.stack.imgur.com/YThly.jpg | 10:49 |
| rfolco|ruck | panda|rover, I was not sure if assigned to you or arxcruz | 10:50 |
| *** jbadiapa has quit IRC | 10:52 | |
| *** jbadiapa has joined #oooq | 10:52 | |
| panda|rover | rfolco|ruck: to, me I'll eventually ping tempest guys | 10:53 |
| panda|rover | for example I have no idea why taht task was removed from the role | 10:53 |
| panda|rover | arxcruz: so tempest directory is now /var/lib/tempestdata ? | 10:54 |
| *** jbadiapa has quit IRC | 10:54 | |
| panda|rover | arxcruz: and we should move also stackviz to that ? | 10:54 |
| *** jbadiapa has joined #oooq | 10:54 | |
| panda|rover | arxcruz: or we can maintain {{ working_dir }} tempest for a while ? | 10:54 |
| arxcruz | it should copy the /var/lib/tempestdata to {{ working_dir }} | 10:55 |
| arxcruz | because the container | 10:55 |
| arxcruz | because everything needs container now, even tempest... | 10:55 |
| panda|rover | arxcruz: what's the subject, sorry ? | 10:55 |
| panda|rover | arxcruz: the role ? so stackviz tasks should happen after the tempest-results ? | 10:56 |
| sshnaidm | folks, please review https://review.rdoproject.org/r/#/c/19081/ | 10:56 |
| arxcruz | panda|rover: stackviz requires the subunit file | 10:56 |
| arxcruz | that is failing | 10:56 |
| rfolco|ruck | something is still expecting /home/stack/tempest | 10:57 |
| panda|rover | rfolco|ruck: it's stackviz | 10:59 |
| *** saneax has quit IRC | 10:59 | |
| panda|rover | arxcruz: sorry, I'm looking at this bug now https://bugs.launchpad.net/tripleo/+bug/1819440 | 10:59 |
| openstack | Launchpad bug 1819440 in tripleo "phase 1 failing on No such file or directory: '/home/stack/tempest'" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 10:59 |
| arxcruz | panda|rover: assign to me, i'll take a look | 10:59 |
| panda|rover | arxcruz: it's failing because the creation of {{ working_dir }}/tempest was removed | 10:59 |
| *** gouthamr has joined #oooq | 10:59 | |
| rfolco|ruck | symlink would be enough IMO | 11:00 |
| *** dmellado_ has joined #oooq | 11:04 | |
| panda|rover | no job is too big, no bug is too small! | 11:05 |
| panda|rover | CI patrol, to the lookout! | 11:05 |
| *** dmellado_ is now known as dmellado | 11:05 | |
| *** chem has quit IRC | 11:08 | |
| *** hubbot1 has joined #oooq | 11:09 | |
| panda|rover | a weekend at home with my daughter is devastating. I have all the musics and stories from the cartoons in my head | 11:14 |
| *** ykarel_ has joined #oooq | 11:22 | |
| *** ykarel has quit IRC | 11:24 | |
| *** saneax has joined #oooq | 11:46 | |
| *** ykarel_ is now known as ykarel | 11:49 | |
| zbr | @oooq if any of you want to have access to https://console.registry.rdoproject.org you need to be added to rdo-infra project on github by jpena. | 11:55 |
| zbr | just email him if you need that with your github account and email. | 11:56 |
| *** rlandy has joined #oooq | 11:57 | |
| arxcruz | panda|rover: well, the tempest directory should exist, i'm running a test now to identify why is failing | 11:58 |
| panda|rover | zbr: I think he said they're going to remove the support in the near future | 11:58 |
| arxcruz | panda|rover: rfolco|ruck the {{ working_dir }}/tempest should be copied from /var/lib/tempestdata | 11:58 |
| arxcruz | funny is that this directory exists in /var/lib/tempestdata since it's running and passing tempest | 11:59 |
| arxcruz | why isn't being copied, that's the question | 11:59 |
| rfolco|ruck | arxcruz, I thought chandan's patch was doing it | 11:59 |
| arxcruz | rfolco|ruck: yeah, it should | 11:59 |
| rfolco|ruck | arxcruz, permission wrong ? | 11:59 |
| panda|rover | arxcruz: yeah, the task taht was removedwas taking care of the permisison, not its existence | 12:00 |
| arxcruz | also, we are not running tempest containerized, that's why his patch pass | 12:00 |
| zbr | panda|rover: i think is more of setup issue, it was not designed to require team membership, only a side effect. anyway. I am glad is sorted for me. | 12:00 |
| arxcruz | panda|rover: because containerized tempest, run on /var/lib/tempestdata | 12:00 |
| arxcruz | the container create the tempest directory there | 12:00 |
| arxcruz | and then just copy to working_dir | 12:00 |
| panda|rover | zbr: yes, but it's not going to be a long term tool, so may better to rely on skopeo from the start ? | 12:01 |
| arxcruz | now i wondering why there's no check jobs running tempest containerized | 12:02 |
| arxcruz | was some bug that we switch to packages? | 12:02 |
| rfolco|ruck | panda|rover, open a new bug or reopen that existing one for pacemaker ? | 12:03 |
| *** skramaja_ has joined #oooq | 12:05 | |
| *** skramaja has quit IRC | 12:05 | |
| panda|rover | rfolco|ruck: reopen, it's exaclty the same. Ping jaosorior in the meantime, he was actively working on tis resolution, let's try to undertand if the fix landed already in packages | 12:08 |
| rfolco|ruck | panda|rover, done, thanks | 12:11 |
| weshay68802228 | zbr please start updating https://tree.taiga.io/project/tripleo-ci-board/task/685?kanban-status=1447275 | 12:13 |
| jaosorior | rfolco|ruck: what version of tripleo-common was used here? https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/7ced95b/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-03-10_17_16_01 | 12:14 |
| jaosorior | do we know if it included the fix? | 12:14 |
| rfolco|ruck | jaosorior, checking... openstack-tripleo-common-10.4.1-0.20190308095406.2616a99.el7.noarch | 12:14 |
| weshay68802228 | zbr I also don't see these jobs running in periodic https://review.rdoproject.org/r/#/c/19151/ | 12:15 |
| *** amoralej is now known as amoralej|lunch | 12:15 | |
| zbr | weshay68802228: i suspect this is the reasons: https://review.rdoproject.org/r/#/c/19155/ | 12:16 |
| zbr | adding the job definitions does not enabled them. i suspect that the trigger is enabled when we merge ^? | 12:17 |
| jaosorior | rfolco|ruck: did the issue only appear in the periodic job? or are we seeing it in other ovb runs as well? | 12:17 |
| zbr | weshay68802228: i tried to check that early today but rdo was down, see https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-fedora-28-scenario004-standalone-master --- it was triggered only by that change. | 12:18 |
| weshay68802228 | zbr k.. please go after that and open bugs on the f28 job tempest results. Work w/ arxcruz on the those bugs.. we probably need one bug per tempest failure unless arx thinks they are all related | 12:18 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (4 more messages) | 12:18 |
| rfolco|ruck | jaosorior, good point. double checking... still verifying the hash | 12:19 |
| arxcruz | weshay68802228: zbr where's the logs ? | 12:19 |
| arxcruz | weshay68802228: are you a bot? :P | 12:20 |
| weshay68802228 | lolz | 12:20 |
| weshay68802228 | zbr https://review.rdoproject.org/zuul/build/af35f7ca400043218d06c803da62eea7 | 12:20 |
| rfolco|ruck | split nick/password next time weshay68802228 | 12:20 |
| *** panda|rover is now known as panda|rover|lunc | 12:21 | |
| zbr | sshnaidm: if you could avoid the failed atop tasks it would be great, distracts me every time i check the logs https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-fedora-28-standalone-master/af35f7c/job-output.txt.gz#_2019-03-10_15_11_47_089238 | 12:22 |
| sshnaidm | zbr, can you submit a bug and assign to me please? | 12:23 |
| zbr | sshnaidm: sure, and thanks. | 12:23 |
| sshnaidm | zbr, do we have some special config file for fedora jobs? | 12:32 |
| zbr | sshnaidm: afaik no, is using the same config. | 12:32 |
| *** fultonj has joined #oooq | 12:33 | |
| zbr | added bug for f28 tempest failures: https://bugs.launchpad.net/tripleo/+bug/1819452 | 12:33 |
| openstack | Launchpad bug 1819452 in tripleo "f28: tempest failures" [Undecided,New] | 12:33 |
| zbr | weshay68802228: should I start looking deeper on ^ | 12:36 |
| weshay68802228 | zbr close that bug | 12:38 |
| weshay68802228 | zbr please open a bug per tempest test.. after consulting w/ arx | 12:38 |
| weshay68802228 | zbr and yes please go deep w/ arx.. | 12:39 |
| zbr | weshay68802228: ok. | 12:39 |
| *** dalvarez has joined #oooq | 12:39 | |
| weshay68802228 | zbr please add a bit of the trace to the bug summaries.. and not a generalization.. it makes it easier to find dupes | 12:42 |
| *** trown|back11mar is now known as trown | 12:46 | |
| weshay68802228 | rfolco|ruck panda|rover|lunc can you guys see if you can the fs001 vexxhost job passing consistently | 12:53 |
| weshay68802228 | arxcruz zbr can you join my blue in 5 to chat about f28 | 12:55 |
| zbr | weshay68802228: i am already with arxcruz on BJ, join us at https://bluejeans.com/2655417928 | 12:56 |
| panda|rover|lunc | weshay68802228: still as experimental, or we should run it more regularly for now ? | 12:56 |
| panda|rover|lunc | and now I have the keyboard full of crackers. | 12:57 |
| *** jpena is now known as jpena|lunch | 12:57 | |
| rfolco|ruck | tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-vexxhost ... mostly passing, but they're all rocky | 13:02 |
| rfolco|ruck | weshay68802228, ^ | 13:02 |
| weshay68802228 | panda|rover|lunc .. sync w/ sshnaidm I don't think the status of the job matters much, but def easier to debug and get it working there .. I think | 13:02 |
| weshay68802228 | not sure .. | 13:02 |
| weshay68802228 | I haven't seen them pass in a while | 13:02 |
| rfolco|ruck | last 10 jobs passed, a few timeouts | 13:03 |
| rfolco|ruck | 82% pass rate | 13:03 |
| rfolco|ruck | weshay68802228, ^ | 13:03 |
| panda|rover|lunc | rfolco|ruck: when is the last run ? | 13:06 |
| rfolco|ruck | | 2019-03-11 05:23 | | 13:07 |
| rfolco|ruck | panda|rover|lunc, ^ | 13:07 |
| *** bogdando has quit IRC | 13:11 | |
| rfolco|ruck | jaosorior, so the job used openstack-tripleo-common-10.4.1-0.20190308095406.2616a99.el7.noarch, which is commit 2616a99a2b428c4c543f13f165f466642de87366 and it has the fix https://review.openstack.org/#/c/641660/. | 13:18 |
| rfolco|ruck | panda|rover|lunc, weshay68802228 tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost (master) is failing 73%... so rocky is good, master is failing mostly | 13:22 |
| *** panda|rover|lunc is now known as panda|rover | 13:23 | |
| *** vinaykns has joined #oooq | 13:33 | |
| weshay68802228 | rfolco|ruck ya.. indicates we still have code issues in master | 13:33 |
| weshay68802228 | and debugging them on rdo will be hard | 13:33 |
| arxcruz | panda|rover: weshay68802228 rlandy https://review.rdoproject.org/r/#/c/18796/ please! :D | 13:39 |
| arxcruz | https://review.rdoproject.org/r/#/c/18795/ as well | 13:39 |
| sshnaidm | need a little survey, please try to browse to this link in chrome and firefox, and tell me if it redirects you to https: http://logs.rdoproject.org/45/560445/276/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/bcd5eab/logs/ara_oooq_oc/ | 13:45 |
| sshnaidm | zbr, weshay68802228 arxcruz panda|rover rfolco|ruck rlandy ^^ | 13:46 |
| rlandy | arxcruz: https://review.rdoproject.org/r/#/c/18796/ need rebase | 13:46 |
| panda|rover | sshnaidm: which link ? | 13:46 |
| rlandy | or something - can't merge | 13:46 |
| arxcruz | sshnaidm: not redirecting | 13:46 |
| sshnaidm | panda|rover, http://logs.rdoproject.org/45/560445/276/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/bcd5eab/logs/ara_oooq_oc/ | 13:47 |
| rfolco|ruck | I see some dirs and index.html.gz | 13:47 |
| panda|rover | sshnaidm: oh, the same link. Yes redirects in crhome | 13:47 |
| panda|rover | sshnaidm: Version 72.0.3626.96 | 13:48 |
| panda|rover | sshnaidm: does not redirect in firefox | 13:48 |
| *** dtrainor has quit IRC | 13:48 | |
| rlandy | both in fact need rebase | 13:48 |
| *** dtrainor has joined #oooq | 13:48 | |
| sshnaidm | amazing.. | 13:49 |
| *** jrist has quit IRC | 13:49 | |
| *** jrist has joined #oooq | 13:53 | |
| *** amoralej|lunch is now known as amoralej | 13:54 | |
| *** kopecmartin has joined #oooq | 13:54 | |
| *** agopi has joined #oooq | 13:56 | |
| *** jpena|lunch is now known as jpena | 13:59 | |
| panda|rover | scrum ? | 14:01 |
| panda|rover | or scram ! ? | 14:02 |
| weshay68802228 | zbr join scrum | 14:04 |
| weshay68802228 | zbr https://bluejeans.com/5878458097 | 14:05 |
| weshay68802228 | https://review.rdoproject.org/r/#/c/19081/ | 14:14 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (4 more messages) | 14:18 |
| *** skramaja_ has quit IRC | 14:30 | |
| zbr | weshay68802228: please check my last comment on https://tree.taiga.io/project/tripleo-ci-board/task/685?kanban-status=1447275 | 14:34 |
| weshay68802228 | sshnaidm can you join my blue for a sec | 14:53 |
| *** udesale has joined #oooq | 14:56 | |
| *** bogdando has joined #oooq | 15:25 | |
| zbr | weshay68802228: ping me when you have a sec | 15:27 |
| *** ykarel is now known as ykarel|afk | 15:28 | |
| rlandy | zbr: panda|rover: I'm creating the single base parent for periodic-tripleo-ci-<distro>-scenario00<X>-standalone-master | 15:28 |
| rlandy | generally we put base jobs on config | 15:28 |
| rlandy | any objection to those base hobs being in https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo-rdo-base.yaml? | 15:28 |
| zbr | rlandy: not sure what it means or look like but if we can easily run an existing job on a different distro by just overriding the nodeset, it would ok with me. | 15:29 |
| rlandy | ack - review coming up | 15:29 |
| rlandy | config can't be tested within a job review | 15:30 |
| zbr | rlandy: and that is a big downside. now what is the upside? | 15:31 |
| rlandy | zbr: base jobs are supposed to be config :) | 15:31 |
| * rlandy will submit and you'll see the change | 15:31 | |
| zbr | rlandy: is this one amendment to noah tablet that i missed? "base jobs on config" | 15:33 |
| rlandy | zbr: lol - it's a zuul rule - you'll notice all our jobs are set out like that | 15:34 |
| *** agopi is now known as agopi|brb | 15:39 | |
| panda|rover | zbr: base job usually interat with secrets, and we can use secret in run only in trusted repository | 15:44 |
| panda|rover | zbr: and did you just say noah's tablet ? | 15:46 |
| panda|rover | is it an iPad ? | 15:46 |
| zbr | panda|rover: so anything that does a push with need to be defined in a trusted repo, and we will not be able to test them before merging. | 15:46 |
| zbr | panda|rover: yeah, clay being a bit more long-lasting that what apples did. | 15:47 |
| panda|rover | zbr: I can't find it on amazon. Also seems it's using strange encodings for the character. What's this "Cuneiform". Bah everything should use utf-8. | 15:48 |
| panda|rover | zbr: yes in this case they are not pushing, but the periodic jobs are all reporting to dlrnapi with credentials | 15:49 |
| weshay68802228 | zbr what's up | 15:53 |
| *** weshay68802228 is now known as weshay | 15:53 | |
| zbr | weshay: last comment on https://tree.taiga.io/project/tripleo-ci-board/task/685?kanban-status=1447275 | 15:53 |
| zbr | apparently is using wrong mirrors | 15:54 |
| zbr | afaik is supposed to look at rdo registry, right? | 15:54 |
| weshay | zbr that's not the wrong mirror zbr | 15:55 |
| rlandy | zbr: panda|rover: pls see https://review.rdoproject.org/r/#/c/19269/ and https://review.rdoproject.org/r/#/c/19271/ - we would have to merge the first to see the second run | 15:55 |
| weshay | the fedora containers are not in docker.io | 15:55 |
| weshay | that's an upstream check job | 15:56 |
| weshay | and it won't work until we promote | 15:56 |
| weshay | w/ fedora | 15:56 |
| zbr | ok, so you confirmed my suspicion. | 15:56 |
| zbr | weshay: is rdo registry open so we can mirror it? | 15:57 |
| weshay | zbr I would suggest and chat w/ panda|rover about potentially pushing what we have to docker | 16:03 |
| weshay | zbr atm.. I would stop trying to run in check | 16:03 |
| zbr | weshay: i was trying to check if our nove hw architecture change was working.... i really don't like changes that are not tested. | 16:04 |
| zbr | panda|rover: rlandy : re https://review.rdoproject.org/r/#/c/19269/2/zuul.d/tripleo-rdo-base.yaml -- what worries me is that it will be impossible for us to test if any change mage to these, like updating the tempest_whitelist .... would not be testable before merging. too many bird-box fans around here... ;) | 16:06 |
| weshay | panda|rover should we push f28 containers to docker? | 16:06 |
| weshay | the deploy is working | 16:07 |
| weshay | for the base job at least | 16:07 |
| *** ykarel_ has joined #oooq | 16:07 | |
| rlandy | zbr: panda|rover: correct - weshay has that concern about anything in config | 16:07 |
| panda|rover | weshay: we can, I would not suggest to tag them as current-tripleo though | 16:07 |
| rlandy | zbr: panda|rover: we have two choices | 16:07 |
| rlandy | leave the parenting as is | 16:07 |
| weshay | panda|rover why not? | 16:08 |
| rlandy | add base jobs to a jobs repo | 16:08 |
| rlandy | ^^ not recommended | 16:08 |
| weshay | panda|rover the job is currently failing anyway | 16:08 |
| rlandy | I will leave the reviews open for voting | 16:08 |
| rlandy | zbr: all options have pluses and minuses | 16:08 |
| weshay | panda|rover zbr http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-fedora-28-standalone-master/af35f7c/logs/tempest.html.gz | 16:08 |
| weshay | we could push the containers w/ the hash used here | 16:08 |
| weshay | a13151c3c151d051e2bffb848265c93609995d64_9ef9aea4 | 16:09 |
| *** ykarel|afk has quit IRC | 16:09 | |
| weshay | $ sudo docker pull trunk.registry.rdoproject.org/tripleomaster/centos-binary-nova-compute-ironic:a13151c3c151d051e2bffb848265c93609995d64_9ef9aea4 | 16:10 |
| weshay | is there | 16:10 |
| weshay | zbr btw.. that patch will be tested by centos just fine | 16:11 |
| panda|rover | zbr: what I"ve tried to push in the past is to make changes on config that don't affect any existing job, then configure a job in rdo-job, trying to tst as much as possibile from config. And when it works, inject the new configuration to the existing. | 16:11 |
| panda|rover | weshay: ok about current-tripleo, I remember we blindly promoted the hash in the past to with even less evidence its was sane. | 16:12 |
| weshay | zbr http://logs.openstack.org/43/642443/1/check/tripleo-ci-centos-7-standalone/583b6e8/logs/tempest.html.gz | 16:12 |
| weshay | I'm going to merge | 16:12 |
| zbr | panda|rover: me kinda lazy, trying to find simple workflows, not complex ones. i wonder if there is a way to make zuul test a change from config. | 16:12 |
| panda|rover | zbr: no | 16:13 |
| zbr | weshay: i am confident that it would work, but still. | 16:13 |
| zbr | panda|rover: if security is of concern, zuul could rely on gerrit labels for checking trust (like a new label or a +2 at least). now i do not know if there are any zuul achitecture limitations that prevent this. | 16:15 |
| weshay | arxcruz zbr we need nova compute input on https://review.openstack.org/#/c/642443/ | 16:15 |
| panda|rover | zbr: the main concern on trusted project is leaking of credentials, and stuff taht needs to stay secret. No job can be tested publically on those changes. you could put a change that dumps the ENV in a job or just write out all the secrets for a project. No automatic job shoudl start on those. | 16:16 |
| panda|rover | zbr: I will not try here to redeisgn the zuul + gerrit interaction. | 16:16 |
| zbr | panda|rover: yeah, this is why I am saying that zuul could test this change only AFTER people vet for it (review it) | 16:16 |
| panda|rover | I remember some discussion in infra about that, maybe 1,5 years ago. I don't know the conclusion of that discussion | 16:17 |
| zbr | panda|rover: like a 2nd-check that runs only after getting the reviews. | 16:17 |
| zbr | panda|rover: thanks for the background info on that. | 16:18 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (4 more messages) | 16:18 |
| weshay | zbr arxcruz /me chatting in #openstack-nova | 16:21 |
| weshay | zbr can you please put in another review that updates the params, and doesn't override | 16:21 |
| *** ykarel_ is now known as ykarel | 16:23 | |
| panda|rover | wow vexxhost is really sensitive on the tripleo-container-prepare task. Fails mostly for that reason | 16:25 |
| *** jfrancoa has quit IRC | 16:25 | |
| zbr | weshay: doing this before getting an answer from nova? I doubt the proposed value is wrong, especially as it does not change centos-7 behavior. | 16:27 |
| *** ykarel is now known as ykarel|away | 16:29 | |
| *** trown is now known as trown|lunch | 16:30 | |
| panda|rover | weshay: rfolco|ruck pre-planning + bug triage ? | 16:31 |
| rfolco|ruck | panda|rover, weshay I am there | 16:31 |
| rfolco|ruck | https://bluejeans.com/5878458097 | 16:31 |
| panda|rover | oops wrong bj | 16:31 |
| *** holser_ has quit IRC | 16:32 | |
| panda|rover | rfolco|ruck: you diconnected ? i disconnected ? | 16:32 |
| weshay | zbr https://review.openstack.org/#/c/642517/ | 16:33 |
| sshnaidm | weshay, rlandy why do we need "update: no" there? | 16:41 |
| rlandy | sshnaidm: if you rerun | 16:41 |
| rlandy | and have local changes | 16:42 |
| rlandy | was a request | 16:42 |
| sshnaidm | rlandy, uh, I see.. | 16:42 |
| *** agopi|brb is now known as agopi | 16:47 | |
| *** udesale has quit IRC | 16:52 | |
| weshay | rlandy can you chat w/ panda|rover and rfolco|ruck and myself now | 17:02 |
| rlandy | ack | 17:02 |
| weshay | https://bluejeans.com/5878458097 | 17:02 |
| rlandy | bj? | 17:02 |
| weshay | ya | 17:02 |
| *** kopecmartin is now known as kopecmartin|off | 17:10 | |
| *** tosky__ has joined #oooq | 17:14 | |
| *** tosky has quit IRC | 17:14 | |
| *** rfolco has joined #oooq | 17:14 | |
| *** rfolco|ruck has quit IRC | 17:16 | |
| *** tosky__ is now known as tosky | 17:19 | |
| *** dtantsur is now known as dtantsur|afk | 17:19 | |
| *** bogdando has quit IRC | 17:22 | |
| *** ykarel|away has quit IRC | 17:29 | |
| *** trown|lunch is now known as trown | 17:49 | |
| *** derekh has quit IRC | 18:00 | |
| *** jpena is now known as jpena|off | 18:13 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (4 more messages) | 18:18 |
| weshay | https://review.openstack.org/#/c/505673/9/ | 18:20 |
| *** panda|rover is now known as panda|rover|off | 18:20 | |
| *** amoralej is now known as amoralej|off | 18:57 | |
| *** saneax has quit IRC | 19:23 | |
| *** fmount has quit IRC | 19:24 | |
| rlandy | zbr: pls see QE notes added to https://tree.taiga.io/project/tripleo-ci-board/task/808?kanban-status=1538166 | 19:25 |
| rlandy | ^^ if that is sufficient confirmation, should we move the card? | 19:25 |
| rlandy | pls comment on note on fedora | 19:25 |
| weshay | panda|rover|off rfolco rdo and the ovb jobs are green again!! | 19:29 |
| rfolco | weshay, \o/ | 19:29 |
| *** dsneddon has quit IRC | 19:29 | |
| rfolco | weshay, you mean, they just started passing 1st round | 19:30 |
| *** dsneddon has joined #oooq | 19:31 | |
| zbr | weshay: what do you think on rlandy comment? ^ i personally do not thing that http://zuul.openstack.org/builds?job_name=tripleo-build-containers-fedora-28 is called often enough. | 19:31 |
| weshay | zbr /me looks | 19:32 |
| weshay | zbr what are you reerring to? | 19:33 |
| weshay | https://tree.taiga.io/project/tripleo-ci-board/task/808?kanban-status=1538166 | 19:33 |
| weshay | ? | 19:33 |
| zbr | weshay: i will let you decide if is done or not. i kinda feel I could have enabled it on more conditions (similar to the c7 ones) | 19:35 |
| weshay | my keyboard is going crazy today | 19:35 |
| weshay | zbr ya.. it should be on the same repos and triggers as centos-7 probably | 19:36 |
| zbr | i am putting it in progress. fix should be easy. | 19:36 |
| weshay | zbr can you remove your -1 on https://review.rdoproject.org/r/#/c/19269/ | 19:37 |
| weshay | zbr would be better to submit a review that moves ALL the base jobs | 19:38 |
| weshay | rather than -1 where precedent has been set | 19:38 |
| weshay | fair? | 19:38 |
| zbr | i am not convinced but i removed it. i value a lot the ability to test changes before merging. | 19:39 |
| weshay | zbr ya.. but that is not about that review | 19:40 |
| weshay | rlandy is just following prcedent | 19:41 |
| weshay | zbr you are suggesting base jobs don't belong there | 19:41 |
| weshay | which is fine, just not in that review | 19:41 |
| zbr | weshay: i do find tripleo-ci as a good place to keep jobs definitions. | 19:42 |
| weshay | rlandy my feedback to you would be | 19:42 |
| weshay | why are you not making abstract jobs there | 19:42 |
| weshay | zbr ok.. I think that is right | 19:43 |
| * weshay grabs zbr's reviw for upstream | 19:44 | |
| weshay | zbr rlandy maybe we can all chat for a sec re: https://review.openstack.org/#/c/641447/6/zuul.d/standalone-jobs.yaml | 19:45 |
| rlandy | k | 19:45 |
| weshay | and https://review.rdoproject.org/r/#/c/19269/2/zuul.d/tripleo-rdo-base.yaml | 19:45 |
| zbr | sure | 19:46 |
| weshay | my blue | 19:46 |
| weshay | zbr https://bluejeans.com/u/whayutin/ | 19:49 |
| rfolco | jaosorior, I reopened https://bugs.launchpad.net/tripleo/+bug/1818994, still don't see pacemaker configured and your fix is applied. | 19:49 |
| openstack | Launchpad bug 1818994 in tripleo "ovb jobs broken because pacemaker is unconfigured" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 19:49 |
| weshay | https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo.yaml#L39 | 20:18 |
| weshay | zbr ^ | 20:18 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container, tripleo-ci- (4 more messages) | 20:18 |
| *** fmount has joined #oooq | 20:21 | |
| zbr | rlandy: weshay : https://review.rdoproject.org/r/#/c/19294/ | 20:31 |
| * rlandy looks | 20:32 | |
| *** vinaykns has quit IRC | 20:34 | |
| weshay | let's merge that | 20:36 |
| rlandy | zbr: you're totally functional after drinking beer - looks good | 20:36 |
| rlandy | weshay: ack | 20:36 |
| rlandy | will watch next pipeline kick | 20:37 |
| zbr | rlandy: copy/paste muscle memory ;) | 20:37 |
| weshay | https://review.openstack.org/#/c/642546/1/docs/tripleo-quickstart-logs.html | 20:51 |
| * weshay reboots | 20:51 | |
| fultonj | i'd like to +w https://review.openstack.org/#/c/641479/ it has the necessary votes | 20:59 |
| fultonj | i see sshnaidm had a -1 but i replied to his comments and have not heard back | 20:59 |
| fultonj | weshay: ? | 20:59 |
| * weshay looks | 21:00 | |
| fultonj | weshay: do you have a position on it? it should make a trello card go away :) | 21:00 |
| rlandy | zbr: weshay: https://review.rdoproject.org/r/#/c/19271/ better? | 21:00 |
| zbr | rlandy++ | 21:02 |
| hubbot1 | zbr: rlandy's karma is now 48 | 21:02 |
| zbr | rlandy: btw, we need to trigger some jobs when we touch that file, because if we run only tox-docs we will break stuff when we do refactor them. | 21:03 |
| weshay | fultonj /me still here, checking a few things.. we have a minus 1 so want to be mindful | 21:04 |
| rlandy | zbr: same with all periodics | 21:04 |
| weshay | fultonj did you trigger a tripleo-ci-centos-7-containers-multinode-pike job that depends on that change? | 21:05 |
| fultonj | weshay: i did, and it turned green | 21:05 |
| fultonj | https://review.rdoproject.org/r/#/c/19141/ | 21:05 |
| weshay | https://review.rdoproject.org/r/#/c/19141/ | 21:05 |
| *** trown is now known as trown|outtypewww | 21:05 | |
| weshay | ya.. see it | 21:06 |
| fultonj | sshnaidm: if you see this laster, would you please comment on https://review.openstack.org/#/c/641479 ? thanks. | 21:14 |
| weshay | fultonj let's merge that and address sshnaidm's concern which is valid in some follow up patches | 21:15 |
| fultonj | weshay: ok, thanks | 21:16 |
| rlandy | zbr: what's the story here: https://review.openstack.org/#/c/641447 - DNM? | 21:18 |
| zbr | rlandy: ask weshay, afaik I cant be merged. | 21:21 |
| rlandy | k - qe continues | 21:21 |
| zbr | rlandy: if you want I can split it, it looks uguly because of deduplication offiles | 21:21 |
| weshay | rlandy tomorrow | 21:22 |
| weshay | after containers are pushed | 21:22 |
| rlandy | ok | 21:22 |
| rlandy | weshay: what's the deal with the QE cards- I move them to done if the reviews are pushed and used in jobs? | 21:32 |
| rlandy | we just usually do that ourselves | 21:32 |
| weshay | ya.. | 21:32 |
| rlandy | example: https://tree.taiga.io/project/tripleo-ci-board/task/803?kanban-status=1538166 | 21:32 |
| rlandy | jobs are in and used in pipeline | 21:32 |
| rlandy | done | 21:32 |
| weshay | done right | 21:32 |
| weshay | most of it should be fairly trivial for qe | 21:33 |
| weshay | as we see the job(s) they are running | 21:33 |
| weshay | tempest is failing | 21:33 |
| weshay | but ok | 21:33 |
| *** dsneddon has quit IRC | 21:38 | |
| *** dsneddon has joined #oooq | 21:50 | |
| *** dsneddon has quit IRC | 21:55 | |
| *** jtomasek has quit IRC | 21:56 | |
| *** dsneddon has joined #oooq | 22:04 | |
| *** rfolco is now known as rfolco|ruck|off | 22:05 | |
| rlandy | weshay: stupid question - where is the the container push work - I see build and tag | 22:06 |
| weshay | it's working.. I'll add a screenshot | 22:07 |
| weshay | which task | 22:07 |
| weshay | https://tree.taiga.io/project/tripleo-ci-board/task/773?kanban-status=1538166 | 22:07 |
| *** dsneddon has quit IRC | 22:09 | |
| weshay | rlandy see my comments | 22:12 |
| *** dsneddon has joined #oooq | 22:14 | |
| weshay | rlandy thanks for rocking these out to done | 22:17 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container, tripleo-ci- (4 more messages) | 22:18 |
| rlandy | weshay: https://tree.taiga.io/project/tripleo-ci-board/task/711?kanban-status=1538166 is the only task I left. I don't think I have access to view rdo container registry to paste evidence. | 22:24 |
| weshay | rlandy k.. we'll just leave it for marios | 22:24 |
| rlandy | the rest I could verify from the job logs/definitions etc. | 22:24 |
| weshay | the board is looking good | 22:24 |
| *** tosky has quit IRC | 22:26 | |
| zbr | rlandy: send email to jpena|off with your github account and he will give you access | 22:45 |
| zbr | i got my access today | 22:45 |
| rlandy | zbr: cool - thanks | 22:45 |
| zbr | in fact, do we have a list of all team members with out github accounts? like stored in code. | 22:46 |
| zbr | we could make his life easier and send one more request for all | 22:46 |
| *** rascasoft has quit IRC | 23:12 | |
| *** rascasoft has joined #oooq | 23:52 | |
| *** dsneddon has quit IRC | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
