| mnaser | s/playbook/image/ | 00:00 |
|---|---|---|
| mordred | clarkb: https://zuul.opendev.org/t/openstack/build/52f37432128747bea37c791194671d79 | 00:03 |
| mordred | clarkb: did something go wrong with that playbook update? | 00:04 |
| mordred | mnaser: it does not seem to have run - but I'm looking in to an overall issue there | 00:05 |
| clarkb | mordred: oh I probably need a become | 00:05 |
| openstackgerrit | Clark Boylan proposed opendev/system-config master: Become root when fixing bridge logging https://review.opendev.org/718280 | 00:07 |
| mnaser | is the opendev pattern: executor adds bridge to inventory and runs playbooks on it? curious as to why its not adding the host-to-run-things-on (im sure there is logical reasoning) | 00:07 |
| clarkb | mordred: ^ that shouldn't have affected anything about the running of the inner playbook though | 00:07 |
| clarkb | mnaser: our ssh configs don't allow it and we'd need to manage a zuul user on all the servers | 00:08 |
| mordred | clarkb: no - but it affected the _next_ job | 00:08 |
| clarkb | mordred: ah | 00:08 |
| mordred | so we didnt' run the etherpad job because update-system-config failed | 00:08 |
| mnaser | clarkb: oh ok, right | 00:08 |
| mordred | I'm guessing the same thing is the reason mnaser's job didn't run | 00:08 |
| clarkb | mnaser: basically keeping the bastion as a bastion simplifies a lot of stuff | 00:08 |
| mordred | mnaser: you're getting all the luck here :) | 00:08 |
| clarkb | mnaser: we don't have to retrofit literally everything this way | 00:09 |
| mnaser | :D | 00:09 |
| clarkb | it also gives us a place to put logs safely | 00:09 |
| mnaser | it's interesting to hear the reasoning behind the decision | 00:09 |
| mordred | clarkb: also - I responded on remote-puppet-else - the short answer is "no, it doesn't need to depend on update-system-config" - the long answer is "wow it's super broken and it's parented on itself, but if it were parented on the right base job, the base job would depend on update-system-config" | 00:09 |
| clarkb | mordred: so it does need that dep to exist, but not directly? | 00:10 |
| mordred | yah | 00:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run AFS in zuul https://review.opendev.org/717056 | 00:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run remote-puppet-else in zuul https://review.opendev.org/717057 | 00:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Remove run_all.sh and ansible cron job https://review.opendev.org/717058 | 00:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Remove ansible-cron role https://review.opendev.org/717059 | 00:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Trigger everything on inventory changes https://review.opendev.org/717114 | 00:11 |
| mordred | clarkb: there ya go ^^ | 00:12 |
| mordred | that shoudl fix the afs and else jobs - which were in fact both broken | 00:12 |
| mordred | corvus: fwiw - zuul did not complain that we pushed up patches where the job listed itself as its own parent | 00:12 |
| corvus | mordred: it'll complain if you try to run them | 00:14 |
| corvus | but i agree, that's probably an easy extra check we could add to catch things earlier :) | 00:14 |
| fungi | mnaser: another thing hopping through the bastion and running ansible there gets us is a central place we can temporarily disable all updates for a server on the fly, which tends to be useful in emergencies | 00:14 |
| fungi | though probably something similar could be done with static nodes in nodepool | 00:15 |
| mordred | corvus: yeah - we may just not have thought about that as a thing to check for :) | 00:20 |
| mordred | fungi, corvus : mind +Aing https://review.opendev.org/#/c/718280 - we're not running any prod playbooks | 00:21 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 00:22 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure-pip: export ensure_pip_virtualenv_command https://review.opendev.org/718224 | 00:22 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] fetch-zuul-cloner: use ensure-pip https://review.opendev.org/717882 | 00:23 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] use ensure-pip in fetch-subunit-output test https://review.opendev.org/718225 | 00:23 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure-tox: use ensure-pip role https://review.opendev.org/717663 | 00:23 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 00:23 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-tox: make idempotent and update testing https://review.opendev.org/718284 | 00:23 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run review and review-dev in zuul https://review.opendev.org/717054 | 00:23 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run gitea in zuul https://review.opendev.org/717055 | 00:23 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run AFS in zuul https://review.opendev.org/717056 | 00:23 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run remote-puppet-else in zuul https://review.opendev.org/717057 | 00:23 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Remove run_all.sh and ansible cron job https://review.opendev.org/717058 | 00:23 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Remove ansible-cron role https://review.opendev.org/717059 | 00:23 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Trigger everything on inventory changes https://review.opendev.org/717114 | 00:23 |
| ianw | clarkb / mordred: interested in early thoughts on -> https://review.opendev.org/718284 | 00:24 |
| ianw | updates ensure-tox to always export tox_exectuable; however for the stack ontop of it, the most important bit is removing the testing assumption that tox is pre-installed | 00:24 |
| mordred | ianw: I think it's a great idea | 00:25 |
| clarkb | ianw: for some reason I thought it already did that | 00:26 |
| clarkb | so ya ++ good idea | 00:26 |
| clarkb | now just working my way through the test changes | 00:27 |
| ianw | thanks ... i shall see how it falls out with the stack ontop of it. having the plain node in there for testing is good, and by the top of the stack should hopefully go green | 00:28 |
| clarkb | ya I agree on the test change it seems the first two cases are effectively the same | 00:28 |
| clarkb | ianw: I think you can self approve it if ready now or wait for more test info? | 00:29 |
| ianw | clarkb: thanks, i'll wait for the stack ontop ... it should help https://review.opendev.org/#/c/717663/ go green on the -plain nodes | 00:30 |
| openstackgerrit | Merged opendev/system-config master: Become root when fixing bridge logging https://review.opendev.org/718280 | 01:09 |
| *** ysandeep|away is now known as ysandeep|rover | 01:17 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: ensure-tox: make idempotent and update testing https://review.opendev.org/718284 | 01:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure pip role https://review.opendev.org/717639 | 01:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure-pip: export ensure_pip_virtualenv_command https://review.opendev.org/718224 | 01:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] fetch-zuul-cloner: use ensure-pip https://review.opendev.org/717882 | 01:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] use ensure-pip in fetch-subunit-output test https://review.opendev.org/718225 | 01:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure-tox: use ensure-pip role https://review.opendev.org/717663 | 01:20 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 01:20 |
| ianw | mordred / clarkb : testing order was wrong; first step was install tox to a virtualenv, then because we made it idempotent, when we ran it again it (correctly) found the existing install and didn't do anything | 01:22 |
| *** ysandeep|rover is now known as ysandeep|brb | 02:10 | |
| ianw | 2020-04-08 01:26:53.352582 | gentoo-17-0-systemd | Fetching file gentoo-20200228.tar.gz.md5sum ... | 02:30 |
| ianw | 2020-04-08 01:26:53.405965 | gentoo-17-0-systemd | Fetching file portage-20200228.tar.gz.md5sum ... | 02:30 |
| ianw | 2020-04-08 01:26:53.457829 | gentoo-17-0-systemd | 20200228 snapshot was not found | 02:30 |
| ianw | prometheanfire: ^ | 02:30 |
| ianw | https://zuul.opendev.org/t/zuul/build/1e072187006d46a8afa452d2c77d5f20/log/job-output.txt | 02:30 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: [wip] ensure-tox: use ensure-pip role https://review.opendev.org/717663 | 02:30 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 02:30 |
| prometheanfire | sup | 02:38 |
| prometheanfire | hmm, snapshot being the portage I imagine | 02:38 |
| prometheanfire | why are you downloading 20200228? | 02:39 |
| prometheanfire | tar.gz seems odd too | 02:39 |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: Move suse builds to nb04, drop pip-and-virtualenv https://review.opendev.org/718299 | 02:39 |
| prometheanfire | it's bz2 or xz now http://distfiles.gentoo.org/snapshots/ | 02:40 |
| prometheanfire | ianw: looks like I'm still waiting on a new stage3 for systemd (and general) | 02:40 |
| prometheanfire | at least got the gcc build time fix | 02:41 |
| * prometheanfire is really looking to move to stage3 files instead of stage4 (gentoo releng likes that as well) | 02:41 | |
| ianw | prometheanfire: i have no idea, this is just one of the zuul-jobs tests | 02:42 |
| ianw | happens out of TASK [configure-mirrors : Update Gentoo cache] | 02:42 |
| prometheanfire | I'd call it ephemeral | 02:42 |
| prometheanfire | emerge-webrsync should be calling the xz or bz2 | 02:43 |
| prometheanfire | it would have failed anyway with the gcc build | 02:44 |
| *** ysandeep|brb is now known as ysandeep|rover | 03:42 | |
| *** ykarel|away is now known as ykarel | 04:24 | |
| *** calcmandan has joined #opendev | 04:53 | |
| *** sgw has quit IRC | 05:13 | |
| *** DSpider has joined #opendev | 05:53 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 06:05 |
| *** slittle1 has quit IRC | 06:07 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: Update Fedora to 31 https://review.opendev.org/717657 | 06:27 |
| AJaeger | ianw: with your tox changes, do we need zbr's https://review.opendev.org/690057 ? | 06:36 |
| ianw | AJaeger: i'm ... not sure. i'm not quite sure what that is going for | 06:37 |
| AJaeger | neither am I ;( | 06:38 |
| ianw | AJaeger: i think the stack from https://review.opendev.org/#/c/717657/ is actually in pretty much final shape. i haven't written change descriptions yet | 06:40 |
| *** lpetrut has joined #opendev | 07:09 | |
| *** rpittau|afk is now known as rpittau | 07:17 | |
| *** ysandeep|rover is now known as ysandeep|lunch | 07:24 | |
| *** ralonsoh has joined #opendev | 07:33 | |
| zbr | super cool: zuul returns 500 if you try to post an emoji comment | 07:41 |
| zbr | ianw: i see your work overlaps with mine on ensure-tox a lot, what can I do to help getting our changes merged? | 07:44 |
| zbr | i had a big interest in making tox upgradable because it blocks me on ansible-zuul on few repos that need newer tox | 07:45 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 07:52 |
| *** tosky has joined #opendev | 08:00 | |
| *** dpawlik has joined #opendev | 08:08 | |
| *** sshnaidm has joined #opendev | 08:23 | |
| sshnaidm | hi, how can I delete the stein branch for tripleo-ansible repo? | 08:27 |
| AJaeger | infra-root, can either of you help, sshnaidm, please? | 08:28 |
| AJaeger | sshnaidm: why do you need to delete it? Isn't stein still under maintenance? | 08:28 |
| *** ysandeep|lunch is now known as ysandeep|rover | 08:29 | |
| ianw | zbr: sorry i'm out of time for today. | 08:29 |
| sshnaidm | it doesn't contain anything meaningful and makes problems for other stein jobs by its existence | 08:30 |
| ianw | zbr: the stack at https://review.opendev.org/#/c/717657 downwards is pretty much what i'm thinking, but i don't expect you to look at it yet as it has no explanation at all ... i will write proper change logs tomorrow | 08:30 |
| ianw | i think it will probably eliminate the need to upgrade things; we can avoid pre-installing tox until the job phase. but we will have to see | 08:31 |
| zbr | ianw: i already reviewed that one, afaik is ready to review. | 08:31 |
| zbr | i can understand the fedora bump and the removal of the workaround is welcomed. | 08:32 |
| zbr | ianw: but the problem is is the chain.... | 08:33 |
| zbr | but it would be great to fix and merge https://review.opendev.org/#/c/718284/ | 08:34 |
| zbr | which is not working on plain ubuntu | 08:34 |
| zbr | ianw: it would be so cool to have more zuul cores in EU timezones, nowadays not much gets reviewed or merged while US is sleeping. | 08:45 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 08:48 |
| *** ykarel is now known as ykarel|lunch | 08:53 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 09:26 |
| *** ykarel|lunch is now known as ykarel | 10:12 | |
| *** rpittau is now known as rpittau|bbl | 10:24 | |
| *** ysandeep|rover is now known as ysandeep|afk | 10:30 | |
| *** sshnaidm is now known as sshnaidm|afk | 10:56 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 10:59 |
| openstackgerrit | Slawek Kaplonski proposed openstack/project-config master: Update Neutron Granafa dashboard https://review.opendev.org/718392 | 11:02 |
| *** calcmandan has quit IRC | 11:25 | |
| *** calcmandan has joined #opendev | 11:25 | |
| *** slittle1 has joined #opendev | 11:28 | |
| *** ysandeep|afk is now known as ysandeep|rover | 11:31 | |
| *** sshnaidm|afk is now known as sshnaidm | 12:19 | |
| *** rpittau|bbl is now known as rpittau | 12:23 | |
| *** roman_g has joined #opendev | 12:24 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 12:32 |
| *** ykarel is now known as ykarel|afk | 12:56 | |
| *** sgw has joined #opendev | 13:02 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Depend on infra-prod-update-system-config https://review.opendev.org/718434 | 13:03 |
| *** ysandeep|rover is now known as ysandeep|away | 13:25 | |
| *** hashar has joined #opendev | 13:28 | |
| fungi | sshnaidm: before we can delete a branch, check with the release team to make sure it's not listed in the releases repo (or else release jobs will helpfully recreate it), and also abandon any open changes in gerrit targeting that branch | 13:32 |
| sshnaidm | fungi, and who is the release team? | 13:33 |
| AJaeger | sshnaidm: #openstack-release | 13:33 |
| AJaeger | sshnaidm: https://governance.openstack.org/tc/reference/projects/release-management.html | 13:34 |
| fungi | sshnaidm: the *openstack* release team, since tripleo-ansible is an official openstack deliverable | 13:34 |
| fungi | sshnaidm: i see there's also a stable/train branch, isn't that causing you similar problems? | 13:34 |
| sshnaidm | fungi, train is fine | 13:34 |
| sshnaidm | fungi, only stein to delete | 13:34 |
| fungi | why do you have a train branch but not want a stein branch? | 13:35 |
| fungi | to be fair, i don't really care, these are probably just questions the release team is going to be asking you | 13:35 |
| sshnaidm | because stein doesn't contain anything meaningful | 13:36 |
| fungi | as long as *openstack* (represented in this case by either the release team or the technical committee) is okay with us deleting a stable branch from an official deliverable on basically no notice, then i'm happy to help | 13:36 |
| fungi | i'm not one to judge what is meaningful for one of openstack's repositories | 13:36 |
| fungi | usually branch deletions come with some sort of discussion and advance notice | 13:37 |
| *** ykarel|afk is now known as ykarel | 13:37 | |
| fungi | which is why i'm trying to make sure we follow proper channels for authorization | 13:38 |
| openstackgerrit | Sean McGinnis proposed openstack/project-config master: Add tooling to update python jobs on branch creation https://review.opendev.org/673019 | 13:44 |
| AJaeger | fungi, could you review the change above for release team, please? ^ | 13:50 |
| mordred | AJaeger: seems like we should maybe move that stuff into the release repo at some point, no/ | 13:56 |
| mordred | ? | 13:56 |
| openstackgerrit | Merged opendev/system-config master: Depend on infra-prod-update-system-config https://review.opendev.org/718434 | 14:02 |
| AJaeger | mordred: I runs in post or so and we wanted that here for security reasons AFAIK (or it did not work in another repo since not trusted). | 14:03 |
| mnaser | good morning, it is me again asking about zuul-preview | 14:06 |
| mordred | mnaser: that patch that just merged ^^ should fix it (famous last words) | 14:10 |
| fungi | mordred: AJaeger: also we'd probably need to create a separate gerrit account for their repo's use if we moved those jobs into it | 14:12 |
| AJaeger | fj | 14:12 |
| AJaeger | fungi: understood | 14:12 |
| fungi | (those jobs do a lot of writes to gerrit, creating branches, pushing changes, et cetera) | 14:12 |
| fungi | (...pushing tags...) | 14:12 |
| openstackgerrit | Merged openstack/project-config master: Add tooling to update python jobs on branch creation https://review.opendev.org/673019 | 14:19 |
| *** mlavalle has joined #opendev | 14:41 | |
| *** lpetrut has quit IRC | 14:46 | |
| openstackgerrit | Thierry Carrez proposed opendev/system-config master: [WIP] Disable global Github replication https://review.opendev.org/718478 | 14:47 |
| openstackgerrit | Thierry Carrez proposed openstack/project-config master: Introduce job for granular GitHub mirroring https://review.opendev.org/718479 | 14:48 |
| ttx | alright, first draft | 14:49 |
| *** ykarel is now known as ykarel|away | 15:04 | |
| mordred | mnaser: look at the status page - infra-prod-service-zuul-preview is in the opendev-prod-hourly pipeline! | 15:12 |
| mordred | mnaser: we're actually going to run the playbook | 15:12 |
| fungi | the evolution of zuul-driven continuous deployment | 15:14 |
| mordred | fungi: it really helps when you get the job names right | 15:15 |
| fungi | feh | 15:15 |
| fungi | infra-root: our certcheck notified us that the tarballs.opendev.org cert is a month from expiring. i took a look at the certs on static.opendev.org and most of them haven't been renewed since february, but the acme.sh log says there are no changes... i'll start looking deeper into this mystery shortly, but if anybody has ideas as to what might have silently broken this, i'm happy to hear them and shorten my | 15:17 |
| fungi | investigation | 15:17 |
| clarkb | fungi: we renew after 60 days | 15:19 |
| clarkb | so depending when in february they were created it may still be within that window. For tarballs if it is tripping the certcheck it has missed that 60 day window I think | 15:20 |
| mordred | also maybe due to zuul cd the letsencrypt playbook could have not run recently? | 15:22 |
| clarkb | mordred: oh ya we may want to make that a daily periodic job if it isn't already? | 15:22 |
| mordred | https://zuul.opendev.org/t/openstack/builds?job_name=infra-prod-service-letsencrypt | 15:23 |
| fungi | clarkb: oh, okay, in that case maybe we need to tune our renewal frequency or our certcheck warning window | 15:23 |
| mordred | clarkb: it is - but it was skipped on its last run due to something before it not running I think | 15:23 |
| fungi | clarkb: because certcheck is set to warn us at 30 days remaining, and the certs are good for 90 | 15:23 |
| clarkb | fungi: ya I think its set up where we should renew before certcheck runs, but if we miss a single day it will trip | 15:23 |
| clarkb | since we normally renew just fine without tripping certcheck | 15:24 |
| clarkb | maybe a few extra days in between is worthwhile just to accomodate changes to the config management :) | 15:25 |
| fungi | right, minor race condition there | 15:25 |
| fungi | is renewing monthly too frequent for letsencrypt's rate limit? | 15:26 |
| mordred | clarkb: we had a timeout on running base - which then caused the other nightlies to skip | 15:26 |
| mordred | clarkb: oh - we only ran it with 5 forks - I thought I'd fixed that | 15:26 |
| clarkb | fungi: mordred to tl;dr we need to continue to tune the periodic jobs but once those are running properly that should fix any LE issues | 15:29 |
| mordred | clarkb: I'm confused - the ansible_forks: 50 in .zuul.yaml for infra-prod-base doesn't seem to have overridden the ansible_forks: 5 in infra-prod-playbook | 15:29 |
| clarkb | the good thing about warning early is we've got the warning early :) | 15:29 |
| mordred | ++ | 15:29 |
| fungi | clarkb: i wouldn't rely on certcheck to let us know overall reconfiguration is broken, because there's no guarantee we've got a cert ready to expire | 15:31 |
| mordred | corvus: when you're awake/around - am I doing something wrong in infra-prod-base wrt ansible_forks? | 15:31 |
| fungi | but yeah, it's at least warning us before it becomes a cert expiration emergency | 15:31 |
| clarkb | fungi: right, its letting us know the cert is going to expire and we have ~a month to fix that | 15:31 |
| clarkb | regardless of what may be causing it | 15:31 |
| fungi | though maybe this is also an argument for leaving the certcheck cronjob in actual cron and not running it from zuul, as belt and suspenders since relying on the same mechanism both to update certs and to check that certs are getting updated is not that robust | 15:33 |
| clarkb | ++ (note I don't think there has been any work or plan to put it into zuul as it isn't a playbook) | 15:34 |
| corvus | mordred: have a link to a run? | 15:34 |
| mordred | corvus: no - the log file is on bridge | 15:34 |
| mordred | corvus: oh - wait - one sec | 15:34 |
| mordred | corvus: on bridge the playbook log is /var/log/ansible/base.yaml.log | 15:34 |
| corvus | mordred: i'm looking for a job | 15:34 |
| corvus | a build | 15:34 |
| mordred | corvus: in zuul, the log is https://zuul.opendev.org/t/openstack/build/17b539528e6b4976827606d1ab955566 | 15:35 |
| mordred | yeah | 15:35 |
| corvus | mordred: thanks | 15:35 |
| mordred | corvus: I see 50 in the inventory - but still 5 on the invocation - could this be an issue with the variable being named ansible_ ? | 15:38 |
| corvus | mordred: yep: https://docs.ansible.com/ansible/latest/reference_appendices/special_variables.html#magic | 15:42 |
| corvus | mordred: it's a "magic" variable which apparently always means "the number of forks allowed by this (ie, the executor) ansible process" | 15:42 |
| corvus | mordred: so i think we just need to rename it everywhere | 15:43 |
| mordred | corvus: kk. fixing | 15:43 |
| clarkb | ttx: looks like fungi and AJaeger have some minor tweaks for the github replication job suggested | 15:46 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run review and review-dev in zuul https://review.opendev.org/717054 | 15:47 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run gitea in zuul https://review.opendev.org/717055 | 15:47 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run AFS in zuul https://review.opendev.org/717056 | 15:47 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run remote-puppet-else in zuul https://review.opendev.org/717057 | 15:47 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Remove run_all.sh and ansible cron job https://review.opendev.org/717058 | 15:47 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Remove ansible-cron role https://review.opendev.org/717059 | 15:47 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Trigger everything on inventory changes https://review.opendev.org/717114 | 15:47 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Rename ansible_forks to infra_prod_ansible_forks https://review.opendev.org/718497 | 15:47 |
| mordred | clarkb, corvus : ^^ 718497 should fix the forks issue, which should let base run in a reasonable amount of time | 15:48 |
| fungi | clarkb: ttx: well, it doesn't hurt to add it to reno initially and then add it to more repos after it merges | 15:48 |
| mordred | I also had to update the afs and else changes | 15:48 |
| clarkb | mordred: k will look shortly | 15:48 |
| clarkb | fungi: there is also a suggestion to disambiguate the jobs purpose | 15:48 |
| AJaeger | clarkb: I'm wondering how to limit the job only to official projects. Could somebody add that in-tree for a random project? | 15:50 |
| openstackgerrit | Slawek Kaplonski proposed openstack/project-config master: Update Neutron Grafana dashboard https://review.opendev.org/718392 | 15:51 |
| openstackgerrit | Slawek Kaplonski proposed openstack/project-config master: Update Neutron Grafana dashboard https://review.opendev.org/718392 | 15:51 |
| mordred | AJaeger: well- if they did and the target repo didn't exist, it wouldn't work | 15:51 |
| clarkb | AJaeger: there are a couple protections. The first is on the zuul side. You could make the job protected and only allow it to be used in project-config then have reviewers enforce that. The other is I think github now forces you to acknowlege new group memberships so random repos woudln't be able to add that user to their github repos | 15:51 |
| AJaeger | ah, ok | 15:51 |
| fungi | also there's the fact that the job itself hard-codes the openstack/ namespace | 15:52 |
| AJaeger | clarkb: sshould we ask fungi to make it protected? | 15:52 |
| AJaeger | ask ttx I mean | 15:52 |
| clarkb | (it should be noted I'm not a github expert I just seem to recall them no longer allowing anyone to add you to various things anymore) | 15:53 |
| fungi | i thought the only way to successfully add that job to a project-pipeline was in the project-config repo anyway? | 15:53 |
| mordred | mnaser: the zuul-preview playbook has run, so I would expect it to be running latest now | 15:53 |
| clarkb | fungi: ya I think the secret use may already imply that | 15:53 |
| AJaeger | ttx, do you want to merge the change as is or make some changes? I'm fine either way... | 15:56 |
| clarkb | mordred: forks fix lgtm and has been approved | 15:59 |
| mnaser | mordred: http://site.925bfe37815144d0859f260605d5fb98.zuul.zuul-preview.opendev.org well i guess its still broken | 15:59 |
| clarkb | the process has been running since yesterday | 16:00 |
| *** sshnaidm is now known as sshnaidm|afk | 16:04 | |
| mordred | hrm | 16:04 |
| clarkb | the ansible log claims that it updated the image and that it ran docker compose up | 16:05 |
| clarkb | but maybe the implication is the image was up to date after all | 16:05 |
| *** hashar is now known as hasharAway | 16:05 | |
| clarkb | ah yup: Status: Image is up to date for zuul/zuul-preview:latest | 16:05 |
| mordred | the sha of the docker image on zp shows the correct sha | 16:06 |
| clarkb | I think that means the deployment stuff is probably not the problem | 16:06 |
| mordred | let's look at logs | 16:06 |
| mordred | I don't see any new coredumps at least | 16:07 |
| mordred | zuul-preview_1 | [Wed Apr 08 16:07:30.498638 2020] [proxy:warn] [pid 310:tid 140643527550720] [client 2600:1700:72e2:4810:6cf7:8f33:6360:1849:60229] AH01144: No protocol handler was valid for the URL / (scheme 'https'). If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule. | 16:08 |
| corvus | did we never proxy to an https url? | 16:08 |
| mordred | I'm guessing not - proxy_ssl is not enabled in the image | 16:09 |
| corvus | ok, you can leave this to me if you want | 16:09 |
| corvus | (it's easy to test changes to zuul-preview locally) | 16:10 |
| mordred | kk | 16:10 |
| mordred | more and more of this is working though | 16:10 |
| mordred | infra-root: I'm running the etherpad playbook manually again since there was a different issue yesterday with the auto-triggered one | 16:14 |
| fungi | out of curiosity, will we need to change how we interact with the admin api on the new deployment? | 16:19 |
| clarkb | fungi: you'll need to retrieve the admin token value out of the container `sudo docker exec etherpadcontainername cat path_to_token_file` | 16:20 |
| clarkb | fungi: but then because we use host networking all of your curling can be done as usual | 16:20 |
| clarkb | fungi: if the path_to_token_file is part of a bind mount from the host you'll be able to cat it off the host side too | 16:21 |
| clarkb | (I'm just not sure if that is the case) | 16:21 |
| fungi | so http://localhost:9001/api/ will still be reachable outside the container? | 16:21 |
| clarkb | fungi: yes, beacuse the container and the host share the same network stack (so localhost is the same to each of them) | 16:21 |
| clarkb | mordred: and other people that grok ansible better than me. I elft a comment on https://review.opendev.org/#/c/718224/4/roles/ensure-pip/tasks/main.yaml about some ansible stuff and how it interacts with zuul. Would be great if someone can double chck that and or provide better ideas | 16:22 |
| fungi | looks like it works just fine on the new server, yep | 16:22 |
| fungi | mostly just making sure we aren't missing anything we need to be able to do manual fixing of stuff via the api, as i rely on it ~weekly | 16:22 |
| clarkb | fungi: if we weren't using host networking we'd probably do NAT. If we did NAT we'd probably do a 9001:9001 forward and it would still work | 16:22 |
| fungi | cool | 16:23 |
| *** hasharAway is now known as hashar | 16:23 | |
| *** rpittau is now known as rpittau|afk | 16:23 | |
| fungi | looks like /var/lib/docker/overlay2/049ba88dab1cd8a7cb3971688b4707114560a732cc5f97f3f556bf88e56d659d/merged/opt/etherpad-lite/APIKEY.txt is it | 16:24 |
| clarkb | fungi: and if you check /etc/etherpad-docker/docker-compose.yaml that inner path isn't bind mounted from the host | 16:26 |
| clarkb | fungi: the easiest way to get at it will be using docker exec though | 16:26 |
| clarkb | (then you don't need to care where docker is storing the filesystem) | 16:26 |
| mordred | clarkb, fungi : /etc/letsencrypt-certs/etherpad.opendev.org/etherpad.opendev.org.cer | 16:27 |
| mordred | is missing - but the key file is there | 16:27 |
| clarkb | mordred: that means LE hasn't run successfully | 16:27 |
| clarkb | mordred: we generate the key, then need to verify the the domain with LE to get the .cer | 16:27 |
| mordred | yeah. the other thing that failed on the LE playbook was restarting the apache on etherpad.opendev.org | 16:28 |
| clarkb | /var/log/acme.sh is where I'd start | 16:28 |
| mordred | on etherpad? | 16:28 |
| clarkb | ya | 16:28 |
| clarkb | thoguh why would it restart apache if it didn't make a .cer /me looks at the LE dir | 16:28 |
| mordred | h - it can't find the openstack.org dns record | 16:28 |
| mordred | I could have sworn I added one | 16:29 |
| *** dpawlik has quit IRC | 16:29 | |
| clarkb | I agree that is what acme.sh says | 16:29 |
| fungi | according to /etc/etherpad-docker/docker-compose.yaml the service is named etherpad, but is that not also the name of the container? docker exec seems to believe there's "No such container" | 16:29 |
| clarkb | fungi: it is etherpaddocker_etherpad_1 discoverable via `sudo docker ps -a` | 16:30 |
| clarkb | I think if you used docker-compose exec you can use the 'etherpad' name and it would figure it out for you | 16:31 |
| fungi | ahh, thanks | 16:31 |
| fungi | clarkb: looks like the answer is: | 16:35 |
| fungi | wget -qO- 'http://localhost:9001/api/1/getText?apikey='$(sudo docker-compose -f /etc/etherpad-docker/docker-compose.yaml exec etherpad cat /opt/etherpad-lite/APIKEY.txt)'&padID=Virtual_PTG_Planning&rev=640' | 16:35 |
| fungi | (as an example) | 16:35 |
| fungi | that pad doesn't actually exist so the api returns a "padID does not exist" error | 16:36 |
| fungi | but that gets the api key plumbed in effectively | 16:36 |
| fungi | i'll make a note in my to do list to update our runbook | 16:37 |
| mordred | DOH | 16:38 |
| mordred | I made an acme cname of _acme-challenge.etherpad01.openstack.org - not very useful | 16:39 |
| mordred | clarkb: re: ensure-pip | 16:42 |
| mordred | clarkb: this works because fact cache | 16:42 |
| mordred | clarkb: (we use the pattern in several places) | 16:42 |
| clarkb | mordred: aha | 16:42 |
| clarkb | mordred: do we need to explicitly cache the fact? | 16:43 |
| mordred | clarkb: yes | 16:44 |
| mordred | cacheable: true | 16:44 |
| clarkb | ah that is in your comment. Thank you for that | 16:46 |
| mordred | infra-root: https://etherpad.opendev.org/ is up ... I will now run in the db dump | 16:51 |
| mordred | SIGH | 16:52 |
| mordred | oh wait | 16:52 |
| mordred | clarkb: why is this: https://review.opendev.org/#/c/718497 not in the gate? | 17:09 |
| mordred | nevermind | 17:09 |
| mordred | clean check | 17:09 |
| mordred | it's still in check | 17:09 |
| openstackgerrit | Ivan Kolodyazhny proposed openstack/project-config master: Add publish-to-pypi jobs for new Vitrage xstatic-* projects https://review.opendev.org/716630 | 17:20 |
| mnaser | corvus, mordred: thank you for the work on zuul-preview, looks like we're good to go :) | 17:39 |
| mordred | mnaser: woot! | 17:39 |
| mordred | mnaser: wow - so we landed a change and it properly ran in prod! | 17:40 |
| mnaser | yes, i guess :D | 17:41 |
| openstackgerrit | Merged opendev/system-config master: Rename ansible_forks to infra_prod_ansible_forks https://review.opendev.org/718497 | 17:46 |
| mordred | mnaser: thanks for being patient on that ... I think the end state is going to be very pleasing | 17:47 |
| openstackgerrit | Merged opendev/system-config master: Run review and review-dev in zuul https://review.opendev.org/717054 | 17:53 |
| openstackgerrit | Merged opendev/system-config master: Run gitea in zuul https://review.opendev.org/717055 | 17:53 |
| *** ralonsoh has quit IRC | 18:01 | |
| *** diablo_rojo has quit IRC | 18:02 | |
| *** diablo_rojo has joined #opendev | 18:06 | |
| mordred | clarkb, corvus : when you get a sec, https://review.opendev.org/#/c/717057/ and https://review.opendev.org/#/c/717056 need re-review due to rebasing and propagatig a couple of fixes | 18:22 |
| mordred | infra-root: I have applied the db dump from etherpad.openstack.org to etherpad.opendev.org | 18:23 |
| mordred | so - we can commence to test it | 18:23 |
| fungi | mordred: are you able to load any old pads on it? i see that https://etherpad.opendev.org/p/Virtual_PTG_Planning doesn't have the content from https://etherpad.openstack.org/p/Virtual_PTG_Planning (picked from my recent history) | 18:26 |
| mordred | fungi: yes - but I made the dump several days ago | 18:26 |
| mordred | so if it's content from this week it's probablynot there | 18:26 |
| mordred | https://etherpad.opendev.org/p/gerrit-2020-03-20 loads fine | 18:26 |
| fungi | oh, hrm, yeah maybe too new. i'll pick another one | 18:27 |
| clarkb | https://etherpad.opendev.org/p/clarkb-test loads for me | 18:28 |
| clarkb | we should find a 4 byte utf8 glyph and test that it works | 18:28 |
| fungi | yep, okay, security-sig-newsletter is there | 18:29 |
| fungi | and i can query the api for the pad text from it too | 18:29 |
| mordred | yay | 18:31 |
| clarkb | my browser seems to have a hard time with the example sheets I've found (likely because my font simply lacks chinese characters?) | 18:31 |
| mordred | clarkb: example? | 18:31 |
| mordred | (clarkb-test is fine for me) | 18:31 |
| clarkb | http://www.i18nguy.com/unicode/supplementary-test.html | 18:31 |
| clarkb | mordred: I want to take a 4 byte utf8 char and put it in a pad and make sure it can be ready back out again | 18:32 |
| mordred | clarkb: I just pasted in one of teh chunks from that | 18:32 |
| mordred | (to clarkb-test) | 18:32 |
| clarkb | mordred: that caused my browser to reconnet and the error glyphs that were there when you pasted are gone | 18:33 |
| clarkb | I'm not sure that is working quite right | 18:33 |
| mordred | weird | 18:33 |
| clarkb | mordred: if you refresh the pad are they there for you? | 18:33 |
| *** hashar has quit IRC | 18:33 | |
| clarkb | it could be this is all client side issues because my font lacks the characters | 18:34 |
| mordred | oh - nope | 18:34 |
| clarkb | of course this isn't necessarily a regression | 18:34 |
| clarkb | we could test similar on the old server | 18:34 |
| mordred | worked on the old server | 18:34 |
| mordred | so yea - I thnk there's something going on there | 18:34 |
| *** hashar has joined #opendev | 18:36 | |
| clarkb | k | 18:37 |
| corvus | fwiw, li confirms the translation of "project management combat operations" as "more or less right if you break it down" :) [as jargon it may mean something a little different] | 18:41 |
| mordred | corvus: I feel like project management combat operations are something we should perhaps practice around here | 18:42 |
| corvus | mordred: you know how we've been frustrated at the changing definition of gitops? here's a solution staring us in the face. | 18:42 |
| mordred | ++ | 18:43 |
| clarkb | I feel like I'm lacking context but combat operations is not quite how I would describe what we do with computers :) | 18:43 |
| corvus | line 9-15 of the etherpad | 18:44 |
| clarkb | ah | 18:44 |
| mordred | clarkb: I agree- but apparently people feel that "gitops" should imply a convergence engine now (kind of lie how cloud native somehow became synonymous with containers) - so maybe calling what we do 项目管理实战操作 would be cooler | 18:44 |
| mordred | although it is more letters than gitops | 18:44 |
| clarkb | more bytes too | 18:44 |
| fungi | and for some reason all blank in my irc client | 18:45 |
| corvus | clarkb: i dunno, i feel "no project management plan survives contact with the production system" seems apt | 18:45 |
| clarkb | corvus: ha | 18:45 |
| clarkb | fungi: your font likely lacks the glyphs | 18:45 |
| fungi | likely | 18:45 |
| clarkb | mine has those but not the 4 byte ones mordred was testing with | 18:45 |
| clarkb | mordred: for the 4 byte problem we should maybe double check that mariadb hasn't changed the behavior around utf8 with respect to mysql behavior? | 18:45 |
| corvus | anyway, sorry to distract -- anything i can do to help? :) | 18:46 |
| fungi | well, except my terminal is supposed to actually incorporate glyphs form other fonts if my preferred font is missing them, so i presume i just entirely lack any font containing representations of those codepoints | 18:46 |
| clarkb | corvus: I think its looking good except for the 4 byte character handling | 18:46 |
| clarkb | corvus: basically modrred pasted in 4 byte utf8 chars and the pad lost them | 18:47 |
| clarkb | (and my client was forced to reconnect) | 18:47 |
| JayF | A great 4-byte emoji that you can use, (which I've confirmed blows up on mysql `utf8` but not `utf8_mb4`) is the poop emoji. | 18:47 |
| corvus | clarkb: oh, so they're not in the pad at all; that's why i can't find it? did we confirm that works on the current one? | 18:47 |
| clarkb | corvus: yes mordred reports this works on the old one | 18:47 |
| clarkb | and ya not in the pad at all | 18:48 |
| corvus | JayF: yeah, that one killed the prod server during a summit | 18:48 |
| clarkb | they were pasted in at line 18 | 18:48 |
| clarkb | etherpad basically ate the line | 18:48 |
| clarkb | I wonder how dangerous googling "poop emoji" will be | 18:48 |
| corvus | clarkb, mordred: what pad on the current prod server has the 4bytes? | 18:48 |
| clarkb | mordred: ^ | 18:49 |
| corvus | clarkb: i think the main danger would be if the kids discovered there was a poop emoji and started wanting to use it everywhere | 18:49 |
| clarkb | ya the results are pretty tame | 18:50 |
| corvus | otherwise, i think you're clear from a 'goatse' pov | 18:50 |
| clarkb | unfortunately my terminal font does not have that glyph | 18:50 |
| clarkb | otherwise I'd share it here just for fun | 18:50 |
| corvus | people also ask: "Is poop emoji really poop?" | 18:50 |
| clarkb | hahahaha | 18:51 |
| mordred | I restarted the opendev etherpad server | 18:51 |
| fungi | i'm personally surprised "poop emoji" wasn't one of the stars of the popular animated emoji movie | 18:51 |
| mordred | with the mb4 charsets in the my.cnf | 18:51 |
| corvus | mordred: can i see your 4byte test on current etherpad prod? | 18:51 |
| mordred | nope. because it doesn't write | 18:52 |
| mordred | I just tried pasting it in to clarkb-test | 18:52 |
| clarkb | mordred: on openstack.org | 18:52 |
| mordred | oh - one sec | 18:52 |
| mordred | I just used a throwaawy - lemme do it on clarkb-test there | 18:52 |
| mordred | corvus: you see it on https://etherpad.openstack.org/p/clarkb-test ? | 18:53 |
| corvus | mordred: line 17? | 18:53 |
| *** diablo_rojo has quit IRC | 18:53 | |
| mordred | corvus: yes | 18:54 |
| mordred | so - on the new etherpad, we're getting an unhandled promise exception | 18:54 |
| mordred | lemme recreate | 18:54 |
| corvus | something like 60 han-like chars? | 18:54 |
| mordred | etherpad_1 | [2020-04-08 18:54:40.245] [ERROR] console - Error: ER_TRUNCATED_WRONG_VALUE_FOR_FIELD: Incorrect string value: '\xF0\xA0\x9C\x8E \xF0...' for column `etherpad-lite`.`store`.`value` at row 1 | 18:54 |
| mordred | corvus: yup | 18:54 |
| fungi | han solo is a han-like character | 18:54 |
| corvus | mordred: i see them on opendev.org too | 18:55 |
| mordred | that's the issue we're getting etherpad side when I paste those | 18:55 |
| mordred | corvus: you do? | 18:55 |
| corvus | oh wow, i reloaded and they disappeared | 18:55 |
| mordred | yeah | 18:55 |
| corvus | but they must have shown up when you pasted them | 18:55 |
| mordred | yeah - I thnk it's a db write issue | 18:55 |
| mordred | now the question is - why | 18:55 |
| corvus | agreed | 18:55 |
| fungi | that would make sense given the symptom | 18:55 |
| clarkb | corvus: yup they showed up for me as error glpyhs then I was force reconencted and they were gone | 18:56 |
| corvus | for me, they showed up correctly, and i was not forced to reconnect (i chose to reload) | 18:56 |
| corvus | not sure if that's because of a server-side change or just differing local conditions | 18:57 |
| * mordred is going to restart again | 18:58 | |
| mordred | ok - I think I fixed it | 18:59 |
| mordred | I'll push up the changes | 19:00 |
| corvus | mordred: lgtm! | 19:00 |
| clarkb | the error glpyhs are there for me now | 19:00 |
| clarkb | whcih is what I'd expect given my font situation | 19:00 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Configure etherpad to use utf8mb4 https://review.opendev.org/718536 | 19:02 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Configure etherpad to use utf8mb4 https://review.opendev.org/718536 | 19:02 |
| mordred | clarkb, corvus : I applied that ^^ directly on the server | 19:02 |
| mordred | oh - there's another thing | 19:02 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Set production NODE_ENV https://review.opendev.org/718538 | 19:05 |
| mordred | clarkb, corvus : ^^ that | 19:05 |
| AJaeger | mnaser: could you abandon these, please? https://review.opendev.org/#/q/project:openstack/openstack-ansible-pip_install+is:open | 19:07 |
| AJaeger | noonedeadpunk: or you ^ | 19:07 |
| noonedeadpunk | yeah, sure | 19:07 |
| noonedeadpunk | done | 19:09 |
| AJaeger | config-core, 2 repos are ready to retire: please review https://review.opendev.org/#/c/717775 to finalize | 19:14 |
| AJaeger | noonedeadpunk: thanks | 19:14 |
| openstackgerrit | Merged openstack/project-config master: Add vexxhost/ansible-role-base-server https://review.opendev.org/718199 | 19:28 |
| *** hashar has quit IRC | 20:19 | |
| mordred | corvus, fungi : if you have a sec, https://review.opendev.org/#/c/718536/ and https://review.opendev.org/#/c/718538/ would be nice | 20:23 |
| corvus | what's the deal with "Comments left for invalid file tools/irc_checks.py" ? | 20:24 |
| fungi | so far i've assumed that's related to one of the linters but since zuul doesn't say which one left the comments i haven't taken time to go looking through every result link | 20:25 |
| mordred | corvus: https://zuul.opendev.org/t/openstack/build/0aa2330d82bd4fa79b16f540edb68f54/log/job-output.txt#581 | 20:25 |
| corvus | it looks like this is showing up in the tox output: | 20:25 |
| corvus | tools/irc_checks.py:25: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. | 20:25 |
| corvus | config = yaml.load(open('hiera/common.yaml', 'r')) | 20:25 |
| corvus | yeah that | 20:25 |
| mordred | should we make that safe_load just to shut up the warning? | 20:26 |
| corvus | that's a python warnings module outuput from running the script | 20:26 |
| corvus | which i guess happens to match the tox inline comment regex? | 20:26 |
| mordred | yeah | 20:26 |
| fungi | neat! | 20:26 |
| corvus | mordred: yeah, i can go ahead and fix the script -- but do you think we should make any changes to the regex? | 20:27 |
| fungi | somebody raised a similar point today on the openstack-discuss thread about inline comments | 20:27 |
| mordred | corvus: I'm not sure we can ... that's pretty much the exact output we'd see from flake8 | 20:28 |
| fungi | http://lists.openstack.org/pipermail/openstack-discuss/2020-April/013989.html | 20:28 |
| corvus | mordred: yeah, and that file *is* in the repo | 20:28 |
| fungi | example here: https://review.opendev.org/#/c/717879/1/oslo_policy/policy.py@677 | 20:28 |
| mordred | corvus: yup | 20:28 |
| corvus | for the oslo policy issue -- i think the solutions there would be to do something other than the warning module, or turn off inline comments for the tox-py* and lower-constraints jobs in that repo | 20:30 |
| mordred | corvus: maybe we should add a flag somewhere to tell zuul not to warn on non-matching files for a given set of inline comments, then set that in our tox inline return? since it's a general comment returner, the warning about non-matching isn't usually useful to the person writing the patch | 20:30 |
| fungi | there's also an outstanding patch to have zuul mention which job is responsible for that comment, right? | 20:31 |
| mordred | no - not for the warnings | 20:31 |
| fungi | ahh | 20:31 |
| mordred | we updated the inline comments | 20:31 |
| corvus | mordred: then no one would ever see the error? | 20:32 |
| mordred | corvus: well - they would whenver they make a patch that touches the file in question | 20:32 |
| corvus | to be clear, i'm not complaining about the warning, it's doing it's job -- it's alerting us to something not working as expected | 20:33 |
| corvus | and we're going to act on it | 20:33 |
| corvus | so for me, this is an unconvincing reason to suppress the warning :) | 20:33 |
| mordred | I think maybe the fact that it shows up as a warning from zuul is the thing the's troubling. it makes it seem like there is a zuul issue, rather than "I have detected an issue in the output but there is no file to report it against so I'll tell you about it in the comment" | 20:34 |
| corvus | well, it's saying that the job is flawed | 20:34 |
| mordred | because the _warning_ aspect is "this job might be misconfigured and reporting information incorrectly" | 20:35 |
| corvus | mordred: i think we can tighten up the regex | 20:35 |
| mordred | ok | 20:35 |
| corvus | a pep8 error looks like "tests/test_discovery.py:219:46: E128 continuation line under-indented for visual indent" | 20:36 |
| corvus | but this has an extra "...Warning:" section | 20:37 |
| mordred | corvus: we should also check the sphinx error lines | 20:37 |
| corvus | yeah, i think a check for ".*Warning:" will clear those too | 20:38 |
| mordred | corvus: but if we tighten the regex to exclude those warnings, isn't that just more suppression? like - this did report a thing that we're going to fix, and the fix isn't that the job is broken | 20:38 |
| corvus | mordred: yes, i'm trying to keep these two things separate :) | 20:38 |
| corvus | 1) the job is returning inline comments that are invalid for the change which we did not intend to be included. that's why i think the zuul warning is working appropriately. 2) there's a problem that we arguably should fix. we could decide that we want to get information like that from inline comments, and i'm somewhat ambivalent about that, but i think the oslo folks would suggest that they just want | 20:40 |
| corvus | to see pep8 errors from the pep8 job | 20:40 |
| openstackgerrit | Sean McGinnis proposed openstack/project-config master: Fix branch to series matching logic in job updates https://review.opendev.org/718549 | 20:41 |
| mordred | corvus: nod, that's helpful | 20:42 |
| corvus | mordred: i think i'll propose a change to zuul-jobs with a test case, and a regex explicitly for user warnings, and we can bikeshed about whether it should be included or excluded :) | 20:43 |
| mordred | :) | 20:43 |
| mordred | corvus: just what we all need, more bikeshedding! | 20:43 |
| corvus | mordred: i'm confused; i wrote the unit test for this and it passed, because the pep8 regex should already exclude these comments | 20:55 |
| mordred | corvus: uhm | 20:56 |
| corvus | i'll keep digging | 20:56 |
| corvus | oh i think it's hitting the sphinx matcher | 20:57 |
| mordred | yeah - was just about to say that | 20:57 |
| openstackgerrit | Merged opendev/system-config master: Configure etherpad to use utf8mb4 https://review.opendev.org/718536 | 21:16 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: tox: Don't inline python warnings https://review.opendev.org/718554 | 21:18 |
| corvus | mordred: regarding the case of https://review.opendev.org/#/c/717879/1/oslo_policy/policy.py@677 -- do you think we should toggle the flag for the 3 jobs that are reporting just on that repo, or do it for all of 'openstack-tox-py36' etc? | 21:20 |
| mordred | corvus: I don't think we should turn if off for openstack-tox-py36 broadly - I think landing your warnings patch while we ponder the broader topic would be better | 21:22 |
| corvus | mordred: oh, so you don't think i should turn it off just for oslo.policy? | 21:23 |
| corvus | (ie, just land the warnings patch in zuul-jobs, then think about re-expanding it later?) | 21:23 |
| openstackgerrit | James E. Blair proposed opendev/system-config master: Use SafeLoader in irc_checks https://review.opendev.org/718558 | 21:25 |
| mordred | corvus: yeah | 21:25 |
| corvus | okay, then i think that bottoms out this rabbit hole for now :) | 21:25 |
| mordred | \o/ | 21:25 |
| openstackgerrit | Merged openstack/project-config master: Fix branch to series matching logic in job updates https://review.opendev.org/718549 | 21:31 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Don't log the public loop on master-nameserver https://review.opendev.org/718559 | 21:38 |
| *** DSpider has quit IRC | 21:50 | |
| openstackgerrit | Sean McGinnis proposed openstack/project-config master: Add expression switch to job update sed statement https://review.opendev.org/718564 | 22:08 |
| mnaser | btw: i know we've been using more and more of opendev, so i'm happy to help drive some things (probably the small quirks and not major things like docker-ifying a whole service as that's a bit hard without having visiblity of root -- for now at least) | 22:42 |
| openstackgerrit | Merged openstack/project-config master: Add expression switch to job update sed statement https://review.opendev.org/718564 | 22:47 |
| openstackgerrit | Merged opendev/system-config master: Set production NODE_ENV https://review.opendev.org/718538 | 23:08 |
| fungi | mnaser: the help is most welcome! (not that you haven't already been helping in a multitude of ways) | 23:13 |
| fungi | whatever we've got going on that you want to pitch in on would be cool | 23:13 |
| mnaser | i'll need to get a bit of the hang of system-config | 23:14 |
| fungi | take all the time you need. i still feel like i'm getting the hang of system-config and i've been here longer than it has | 23:14 |
| *** tosky has quit IRC | 23:20 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!