| *** rlandy|bbl is now known as rlandy|out | 01:10 | |
| opendevreview | Ian Wienand proposed opendev/system-config master: bootstrap-bridge: drop pip3 role, add venv https://review.opendev.org/c/opendev/system-config/+/856593 | 01:37 |
|---|---|---|
| *** Guest166 is now known as diablo_rojo_phone | 01:47 | |
| diablo_rojo1 | fungi: clarkb finally got this sorted - https://review.opendev.org/c/openstack/project-config/+/847364 | 02:15 |
| diablo_rojo1 | When you have a spare moment, a review would be super helpful. | 02:15 |
| * diablo_rojo1 realizes she is in the same tz as ianw currently and waves :) | 02:16 | |
| opendevreview | OpenStack Proposal Bot proposed openstack/project-config master: Normalize projects.yaml https://review.opendev.org/c/openstack/project-config/+/857798 | 02:16 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Run jobs with a focal bridge.openstack.org https://review.opendev.org/c/opendev/system-config/+/857799 | 02:18 |
| ianw | diablo_rojo1: welcome to the future :) | 02:18 |
| diablo_rojo1 | Its a bit weird, I must admit. | 02:18 |
| opendevreview | Merged openstack/project-config master: Normalize projects.yaml https://review.opendev.org/c/openstack/project-config/+/857798 | 02:44 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Run jobs with a focal bridge.openstack.org https://review.opendev.org/c/opendev/system-config/+/857799 | 02:45 |
| *** ysandeep|out is now known as ysandeep | 02:49 | |
| opendevreview | Ian Wienand proposed opendev/system-config master: Run jobs with a focal bridge.openstack.org https://review.opendev.org/c/opendev/system-config/+/857799 | 02:53 |
| diablo_rojo1 | ianw: I guess if you have a moment - https://review.opendev.org/c/openstack/project-config/+/847364 would you review this? | 03:42 |
| ianw | diablo_rojo1: lgtm | 03:45 |
| diablo_rojo1 | thanks ianw @ | 03:48 |
| diablo_rojo1 | ! | 03:48 |
| diablo_rojo1 | lol | 03:48 |
| opendevreview | Merged openstack/project-config master: Setup #openinfra-envirosig IRC Channel https://review.opendev.org/c/openstack/project-config/+/847364 | 03:57 |
| opendevreview | Ian Wienand proposed opendev/system-config master: install-ansible: remove testinfra version install workaround https://review.opendev.org/c/opendev/system-config/+/852475 | 04:01 |
| opendevreview | Ian Wienand proposed opendev/system-config master: testinfra: install with ansible extras https://review.opendev.org/c/opendev/system-config/+/852476 | 04:01 |
| opendevreview | Ian Wienand proposed opendev/system-config master: install-ansible: remove stevedore workaround https://review.opendev.org/c/opendev/system-config/+/852477 | 04:01 |
| opendevreview | Ian Wienand proposed opendev/system-config master: install-ansible: remove stub install for ARA https://review.opendev.org/c/opendev/system-config/+/852478 | 04:01 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bootstrap-bridge: drop pip3 role, add venv https://review.opendev.org/c/opendev/system-config/+/856593 | 04:01 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Run jobs with a focal bridge.openstack.org https://review.opendev.org/c/opendev/system-config/+/857799 | 04:01 |
| opendevreview | Ian Wienand proposed opendev/system-config master: run-selenium: Use latest tag on firefox image https://review.opendev.org/c/opendev/system-config/+/857803 | 04:01 |
| ianw | it looks like our jobs are generally ok with a Focal node as bridge.openstack.org, but something is up with the screenshots. that runs a firefox container on bridge and takes shots of the remote nodes. i'll have to debug that, but it's not in the production path | 04:06 |
| opendevreview | Ian Wienand proposed opendev/system-config master: run-selenium: Use latest tag on firefox image https://review.opendev.org/c/opendev/system-config/+/857803 | 04:10 |
| opendevreview | Ian Wienand proposed opendev/system-config master: run-selenium: Use latest tag on firefox image https://review.opendev.org/c/opendev/system-config/+/857803 | 04:12 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Run jobs with a focal bridge.openstack.org https://review.opendev.org/c/opendev/system-config/+/857799 | 04:12 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bootstrap-bridge: drop pip3 role, add venv https://review.opendev.org/c/opendev/system-config/+/856593 | 04:28 |
| opendevreview | Ian Wienand proposed opendev/system-config master: run-selenium: Use latest tag on firefox image https://review.opendev.org/c/opendev/system-config/+/857803 | 04:28 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Run jobs with a focal bridge.openstack.org https://review.opendev.org/c/opendev/system-config/+/857799 | 04:28 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bootstrap-bridge: drop pip3 role, add venv https://review.opendev.org/c/opendev/system-config/+/856593 | 04:53 |
| opendevreview | Ian Wienand proposed opendev/system-config master: run-selenium: Use latest tag on firefox image https://review.opendev.org/c/opendev/system-config/+/857803 | 04:53 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Run jobs with a focal bridge.openstack.org https://review.opendev.org/c/opendev/system-config/+/857799 | 04:53 |
| *** ysandeep is now known as ysandeep|afk | 05:13 | |
| opendevreview | Ian Wienand proposed opendev/system-config master: bootstrap-bridge: drop pip3 role, add venv https://review.opendev.org/c/opendev/system-config/+/856593 | 05:15 |
| opendevreview | Ian Wienand proposed opendev/system-config master: run-selenium: Use latest tag on firefox image https://review.opendev.org/c/opendev/system-config/+/857803 | 05:15 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Run jobs with a focal bridge.openstack.org https://review.opendev.org/c/opendev/system-config/+/857799 | 05:15 |
| *** bhagyashris_ is now known as bhagyashris|ruck | 05:51 | |
| *** ysandeep|afk is now known as ysandeep | 06:25 | |
| *** jpena|off is now known as jpena | 07:21 | |
| *** ysandeep is now known as ysandeep|afk | 08:07 | |
| frickler | amorin: infra-root: some updates on the nested-kvm issue: good news is that the c2-N nodes do not show the issue. it also turns out that the trigger is not nested-kvm per se, but neutron's addition of cpu_mode=host-passthrough, I'm running a check without that now as final confirmation https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/854910/5..6/zuul.d/base-nested-switch.yaml | 08:26 |
| amorin | ack | 08:28 |
| frickler | if you compare cpu flags, there are two additional ones set in "our" flavor: https://paste.opendev.org/show/b52xVaeN45Dg5GGrmKbf/ not sure to which host setup this correlates | 08:30 |
| frickler | but I'm rather sure that "ssbd" is what triggers the failing cirros kernel behavior | 08:30 |
| frickler | also while I initially only saw failures on gra1, the situation seems to be the same on bhs1 | 08:32 |
| amorin | that make sense, I think the hypervisor are the same on bhs1 and gra1 | 08:36 |
| frickler | amorin: can you tell if the difference from c2-30 to ssd-osFoundation-3 is because it runs on different hardware or is that just some flavor thing? | 08:45 |
| amorin | the hardware is different | 08:46 |
| amorin | I will check, but if I am correct, the flags in cpu info are directly taken from the hypervisor + some extra | 08:46 |
| amorin | let me check that | 08:46 |
| amorin | cpu_model_extra_flags=vmx,ssbd,pdpe1gb,pcid | 08:47 |
| amorin | we have this for os foundation aggregate | 08:48 |
| amorin | not sure why | 08:48 |
| frickler | seems the middle two are exactly what I see as delta | 08:48 |
| amorin | it seems we enabled that in the paste especially for foundation | 08:50 |
| amorin | we enabled that for kuryr project issues it seems | 08:51 |
| amorin | "<kashyap> dpawlik: For Intel hosts, don't forget: "spec-ctrl", "ssbd" as well." | 08:53 |
| amorin | taken from a conversation | 08:53 |
| frickler | interesting, do you have a timestamp for that so I can look up more context? | 08:53 |
| *** ysandeep|afk is now known as ysandeep | 08:53 | |
| amorin | it seems it was done in 03/2019 | 08:55 |
| amorin | I have this in my internal ticketing system: | 08:55 |
| amorin | http://logs.openstack.org/56/629856/3/gate/kuryr-kubernetes-tempest-daemon-containerized-octavia-py36/153248a/controller/logs/screen-etcd.txt.gz#_Jan_14_08_02_23_379837 | 08:55 |
| amorin | but link is down now | 08:55 |
| amorin | and this | 08:55 |
| amorin | http://logs.openstack.org/29/630629/1/check/kuryr-kubernetes-tempest-multinode-daemon-octavia-containerized/83152b6/controller/logs/screen-etcd.txt.gz#_Jan_14_09_57_16_449855 | 08:56 |
| amorin | based on the git history, we did this change on 2019-01-18 | 08:57 |
| amorin | note that, it seems we did it on some other aggreates as well, but I reverted it because of live migration issues | 08:58 |
| amorin | we kept it only on OSF aggregate | 08:59 |
| amorin | should I try to remove it as well? | 08:59 |
| amorin | that would solve your issue | 08:59 |
| dpawlik | amorin: o/ | 09:00 |
| amorin | hey daniel :) | 09:00 |
| frickler | maybe wait for feedback from other infra-root first | 09:00 |
| dpawlik | amorin: is it related to me ? | 09:01 |
| amorin | yup | 09:01 |
| amorin | it's a commit from you yes :) | 09:01 |
| dpawlik | woo, good that git got function "blame" :) | 09:04 |
| amorin | yup :) | 09:08 |
| dpawlik | ianw: hey, do you still use grafyaml to maintain grafana? | 09:12 |
| *** ysandeep is now known as ysandeep|brb | 09:51 | |
| *** ysandeep|brb is now known as ysandeep | 10:03 | |
| opendevreview | Alfredo Moralejo proposed zuul/zuul-jobs master: Use AFS mirrors for extras-common in CS9 https://review.opendev.org/c/zuul/zuul-jobs/+/857730 | 10:17 |
| ianw | dpawlik: yes, we still do, but we also support importing dashboards exported directly from grafana. documentation is @ https://docs.opendev.org/opendev/system-config/latest/grafana.html | 10:30 |
| *** bhagyashris is now known as bhagyashris|ruck | 10:59 | |
| frickler | amorin: just as a theoretical question for now: could you create a new flavor for us with the two additional flags dropped? so that would could run some tests on possibly affected projects before we switch everything? or is this an either-or thing? | 11:00 |
| *** dviroel|afk is now known as dviroel | 11:31 | |
| *** pojadhav is now known as pojadhav|afk | 11:39 | |
| *** rlandy is now known as rlandy|mtg | 11:42 | |
| amorin | I dont think it's possible on flavor side, this is an nova.conf setting (cpu_extra_flag) | 12:23 |
| amorin | frickler ^ | 12:23 |
| *** ysandeep is now known as ysandeep|afk | 12:26 | |
| frickler | ah, then I misunderstood this, o.k. | 12:26 |
| *** rlandy|mtg is now known as rlandy | 13:12 | |
| *** ysandeep|afk is now known as ysandeep | 13:24 | |
| *** dasm|off is now known as dasm | 13:29 | |
| johnsom | FYI 5.16 made a bunch of changes around ssbd handling. I suspect by passing it through, but not the other related CPU flags we have an invalid CPU. | 13:56 |
| johnsom | https://www.phoronix.com/news/Linux-5.16-Spectre-SECCOMP-To-P | 13:57 |
| fungi | johnsom: that could certainly explain why we didn't notice initially until we started trying to use newer ubuntu versions, which of course use recent linux kernels | 14:12 |
| opendevreview | Clark Boylan proposed opendev/system-config master: More aggresively enable ansible pipelining https://review.opendev.org/c/opendev/system-config/+/857239 | 15:21 |
| fungi | i've been running a scripted import of all the mailman sites on the new held node for the past hour or so, and it's finally on lists.openstack.org's data (it's doing the sites in alpha order) | 15:35 |
| fungi | logging all of stdout and stderr this time, with timing data for each command as well as overall start/end timestamps for each site | 15:36 |
| *** marios is now known as marios|out | 15:40 | |
| *** pojadhav is now known as pojadhav|out | 15:41 | |
| corvus | it seems like zuul is not on fire... yeah? if my pyrotechnic estimation is correct, i'll make that release.... | 15:48 |
| clarkb | corvus: I have not heard or seen any reports of fire and the jobs I expected to run ran and produced results that I could interrogate | 15:48 |
| clarkb | corvus: did we want to test a job with ansible 6 first? | 15:49 |
| clarkb | zuul's ci does that already so probably not critical | 15:49 |
| corvus | still, a good sanity check. i'll do that real quick | 15:55 |
| *** dviroel is now known as dviroel|lunch | 15:59 | |
| dtantsur | hi folks! not sure if you can help, but apparently if you trigger an update of grub-pc on debian testing nodes (which bifrost does), it fails: https://zuul.opendev.org/t/openstack/build/062f0c9f6a764cce869b6a3f597c6889/log/logs/bifrost.log#15233 | 16:01 |
| *** ysandeep is now known as ysandeep|out | 16:03 | |
| dtantsur | I found some explanation on https://forum.openmediavault.org/index.php?thread/38006-recent-updates-and-upgrades-can-fail-when-updating-to-grub-pc-2-02-dfsg1-20-deb1/ | 16:03 |
| opendevreview | Alfredo Moralejo proposed zuul/zuul-jobs master: Use extras-common repo in CS9 from package_mirror https://review.opendev.org/c/zuul/zuul-jobs/+/857730 | 16:03 |
| fungi | dtantsur: i'm not sure we have a good answer to projects wanting to install a bootloader on test nodes, unless they're also going to adapt the boot configuration for those nodes accordingly. can you clarify the problem you're having? | 16:04 |
| dtantsur | fungi: it's not that we need to install a bootloader, we need some grub files though | 16:05 |
| fungi | ahh, and the maintscripts for that package seem to care | 16:05 |
| dtantsur | namely, EFI binaries | 16:05 |
| corvus | clarkb: do you want to try throwing ansible 6 at the zuul job and check timings? | 16:05 |
| dtantsur | so, grub-efi-amd64-bin pulls in grub-pc, and grub-pc tries to ensure we can boot | 16:05 |
| opendevreview | Alfredo Moralejo proposed zuul/zuul-jobs master: Use extras-common repo in CS9 from package_mirror https://review.opendev.org/c/zuul/zuul-jobs/+/857730 | 16:06 |
| clarkb | corvus: the one I was testing with 2.9? I can do that | 16:06 |
| corvus | ya | 16:06 |
| fungi | dtantsur: you may be able to coerce it to skip that check and/or its other maintscript activities and triggers. there's an apt policy which can be set to indicate you don't want some of that stuff to happen i think, which is commonly used for installing packages into a chroot for example. i'll see if i can find some docs | 16:07 |
| JayF | fungi: the latest version of the grub-pc package specifically errors if you use that mechanism | 16:07 |
| JayF | fungi: it's explicitly checking and failing if you have a non-interactive DEBIAN_FRONTEND | 16:07 |
| JayF | afaict | 16:07 |
| fungi | ahh, okay so maybe you need another way of getting the efi binaries. maybe you can fetch and unpack the package without installing it? | 16:08 |
| clarkb | how is debian building cloud images? The underlying issue is we build the image on a different machine which has different devices. For this reason dib does everything by label instead | 16:08 |
| JayF | I mean, we could do that, but then we're not really testing what the bifrost user would be testing | 16:08 |
| clarkb | there must be some workaround though otherwise debian wouldn't be able to have cloud iamges | 16:08 |
| dtantsur | right | 16:08 |
| dtantsur | JayF: well, we could switch to only downloading the DEB/RPM files | 16:08 |
| JayF | vm images are a bit easier because you can statically set the debconf value | 16:08 |
| fungi | do bifrost users typically install grub-pc onto cloud virtual machines? | 16:08 |
| fungi | or are you hoping that's similar to something a bifrost user actually does? | 16:09 |
| dtantsur | fungi: the users don't care, the problem is about getting EFI artefacts | 16:09 |
| fungi | it seems to me that getting efi artifacts as a side effect of installing a particular bootloader is a risky choice | 16:09 |
| fungi | and this is just evidence of that | 16:10 |
| JayF | https://packages.debian.org/stretch/grub-efi-amd64-bin | 16:10 |
| JayF | we didn't make that choice though; the debian packager did | 16:10 |
| JayF | I don't disagree with you that the dep link is dubious here; but we can't really control that | 16:10 |
| fungi | you didn't make the choice to use efi binaries? | 16:10 |
| dtantsur | apt --download-only --assume-yes | 16:10 |
| dtantsur | fungi: well, we definitely did not invent UEFI :D maybe we do need to look into another way of getting them | 16:10 |
| JayF | dtantsur: I'm really keen of the fact we're relying on the OS for security updates to such things | 16:11 |
| fungi | i'm saying installing a package to get the efi binaries even though you don't necessarily intend to use that bootloader is dubious, and leads to situations like this | 16:11 |
| JayF | I don't want an EFI bootloader vuln to become a CVE in Ironic||bifrost | 16:11 |
| dtantsur | JayF: I suspect we bake them into an image anyway | 16:11 |
| dtantsur | so you won't get new versions without updating bifrost | 16:12 |
| JayF | if we bake them into an image, I'm game to try something less deb-y | 16:12 |
| fungi | getting the efi binaries from the package does make sense. installing a bootloader just to get access to those binaries is the part what i'm questioning | 16:12 |
| fungi | there are ways to get files from a package other than installing it | 16:12 |
| dtantsur | JayF: I can give it a try and see where it leads us? | 16:12 |
| JayF | I agree with you in technical fact, I disagree with you in terms of it being a good practice | 16:12 |
| JayF | but the debian maintainers have forced our hand | 16:12 |
| JayF | dtantsur: bluntly, i'm not sure I'm up to speed enough on bifrost and ansible generally to be a good choice for this; but I can try if we have literally nobody else invested in fixing it | 16:13 |
| * dtantsur wonders of shim falls in the same bucket | 16:13 | |
| clarkb | Is this in some nested image the job builds for testing or are you trying to modify the image we provide to the job? | 16:13 |
| fungi | well, the debian maintainers have decided what the purpose of those packages is, and it's to set up the bootloader for the system you're installing them on. your use of those packages is... for lack of a better term, off-label | 16:13 |
| dtantsur | JayF: I have a relatively calm evening, so only ify ou want to | 16:13 |
| dtantsur | clarkb: we're testing bifrost, it does not know that it's running in the CI | 16:14 |
| JayF | dtantsur: I already have a big list, and first on it is "run even more benchmarks for the sqla 2.0 migration", which I think is more urgent | 16:14 |
| clarkb | dtantsur: right I understand that. I'm trying to understand he context of the package install | 16:14 |
| clarkb | are you trying to install a debian package on the host in order to grab some files out of the pcakge that you later inject somewhere else? If so I agree with fungi | 16:15 |
| dtantsur | clarkb: it's the easiest way we've found to get the UEFI artefacts that we use for building EFI-compatible boot ISOs | 16:15 |
| clarkb | why not extract the files out of the package directly? | 16:15 |
| JayF | e.g., from grub-efi-amd64-bin > /usr/lib/grub/x86_64-efi/monolithic/grubx64.efi | 16:15 |
| fungi | if those efi binaries are vendored blobs, you can probably fetch them more easily from the debian source package which builds that binary package | 16:15 |
| clarkb | ya or that | 16:15 |
| dtantsur | gonna give it a try | 16:16 |
| JayF | I'm going to file a bug trying to convince upstream to change the hard dep into a recommendation | 16:16 |
| JayF | because IMO that's where the real disconnect between what this package should be and what it is actually trying to do is | 16:17 |
| fungi | just be aware even if they agree, they probably won't change that until after bookworm releases, since tey're coming up on the start of release freeze soon | 16:17 |
| * dtantsur sees suse support still in tree and screams | 16:17 | |
| fungi | and they try to stabilize things which affect the base install as early as possible | 16:17 |
| *** jpena is now known as jpena|off | 16:19 | |
| fungi | mm3 test import has reached the longest part (openstack-stable-maint achive) | 16:51 |
| corvus | i'm going to enable "keep" on all the executors to help debug an issue with ansible 6 | 16:57 |
| fungi | thanks for the heads up! | 17:01 |
| *** dviroel|lunch is now known as dviroel | 17:09 | |
| corvus | i'm turning off keep now | 17:16 |
| clarkb | fungi: re mm3 migrations. Do you know if we can do a migration and then followup with an update to that migration? If so maybe we can potentially prime the migration ahead of time? That might be more complicated than it is worth though | 17:26 |
| fungi | we may be able to, but i'd avoid it unless absolutely necessary. the less we complicate things, the better | 17:27 |
| fungi | delaying deliveries and having the archives unavailable for a little while is not the end of the world | 17:27 |
| clarkb | ++ | 17:28 |
| fungi | we could think about importing openstack-stable-maint last when doing that site, since it only gets automated posts from a specific time each day | 17:29 |
| fungi | and just switch dns over once the other lists are done | 17:29 |
| *** rlandy is now known as rlandy|ruck | 17:34 | |
| opendevreview | Felipe Reyes proposed openstack/project-config master: Mirror keystone-openidc to github https://review.opendev.org/c/openstack/project-config/+/857940 | 17:42 |
| fungi | migration of openstack-stable-maint took 1h34m this time, which was ~53% of the 2h56m to migrate all of the lists.openstack.org site | 18:28 |
| fungi | by comparison, the lists.opendev.org site took 8m24s for migration | 18:30 |
| fungi | it's almost done with lists.starlingx.io and then lists.zuul-ci.org should go fairly quickly, and i'll see about putting the log up in a paste | 18:33 |
| opendevreview | James E. Blair proposed zuul/zuul-jobs master: Remove shebang from all python ansible modules https://review.opendev.org/c/zuul/zuul-jobs/+/857948 | 19:06 |
| fungi | clarkb: here's a new error some of the config imports hit... 'Length of value for nonmember_rejection_notice is 299 which is too long for MySQL.' | 19:10 |
| fungi | i guess we have a limited field size for that | 19:11 |
| clarkb | fungi: ya we'll probably need to dump the field size of nonmember_rejection_notice and decide if we need upstream to make it bigger or reduce the length of that value | 19:13 |
| opendevreview | James E. Blair proposed zuul/zuul-jobs master: DNM: check sfio sphinx jobs https://review.opendev.org/c/zuul/zuul-jobs/+/857970 | 19:14 |
| fungi | we only hit that error on 3 lists, fwiw | 19:14 |
| fungi | one was for member_moderation_notice and the other two were nonmember_rejection_notice | 19:15 |
| fungi | and they look like (old) boilerplate probably | 19:15 |
| opendevreview | James E. Blair proposed zuul/zuul-jobs master: DNM: check generate-manifest job https://review.opendev.org/c/zuul/zuul-jobs/+/857973 | 19:56 |
| opendevreview | James E. Blair proposed zuul/zuul-jobs master: DNM: check generate-manifest job 2.9 https://review.opendev.org/c/zuul/zuul-jobs/+/857974 | 19:58 |
| *** rlandy|ruck is now known as rlandy|ruck|biab | 20:44 | |
| opendevreview | James E. Blair proposed zuul/zuul-jobs master: Remove shebang from all python ansible modules https://review.opendev.org/c/zuul/zuul-jobs/+/857948 | 20:49 |
| opendevreview | Ian Wienand proposed zuul/zuul-jobs master: linters: lint that library files don't start with #! https://review.opendev.org/c/zuul/zuul-jobs/+/857981 | 21:10 |
| opendevreview | Ian Wienand proposed zuul/zuul-jobs master: linters: lint that library files don't start with #! https://review.opendev.org/c/zuul/zuul-jobs/+/857981 | 21:12 |
| *** rlandy|ruck|biab is now known as rlandy|ruck | 21:31 | |
| opendevreview | Merged zuul/zuul-jobs master: Remove shebang from all python ansible modules https://review.opendev.org/c/zuul/zuul-jobs/+/857948 | 21:39 |
| corvus | infra-root: ^ heads up -- that change is hopefully a noop, but if we're wrong, it will affect all jobs | 21:43 |
| corvus | if it fails, it will fail with post_failure | 21:43 |
| *** dviroel is now known as dviroel|out | 21:45 | |
| clarkb | I've seen ansible 6 jobs are happy | 21:50 |
| clarkb | still waiting on ansible 5 jobs to report back | 21:50 |
| *** rlandy|ruck is now known as rlandy|ruck|bbl | 21:50 | |
| fungi | yeah, eyes peeled | 21:54 |
| clarkb | the zuul tox docs change used ansible 5 and was happy | 21:56 |
| clarkb | everything I'm seeing so far looks good | 21:56 |
| fungi | mainly worried about log uploads, but those seem to be fine | 21:57 |
| clarkb | there are a number of cinder jobs that have run under ansible 5 in the agte that are fine too | 21:57 |
| *** dasm is now known as dasm|off | 21:58 | |
| corvus | i think at this point we've seen success in the two critical cases we expected... so any errors at this point would be coming from unexpected conditions. | 21:58 |
| corvus | based on that, i'm going to take a break and check back in later | 21:59 |
| fungi | thanks! | 22:00 |
| *** dmitriis0 is now known as dmitriis | 22:12 | |
| fungi | log from the latest mm3 import test of the lists.openstack.org site: https://paste.opendev.org/show/816758 | 22:42 |
| fungi | https://paste.opendev.org/show/816759 is a better, more trimmed-down one | 22:53 |
| Clark[m] | fungi: paste should default to the anonymized pastes now. Any idea why yours aren't? Wondering if we have a bug in that | 23:04 |
| corvus | some zuul image jobs failed, maybe because of the registry, i'm checking | 23:09 |
| corvus | ssl.SSLError: [SSL: BAD_KEY_SHARE] bad key share (_ssl.c:1131) | 23:10 |
| corvus | does that error translate to "restart the server" ? | 23:10 |
| clarkb | corvus: I would check that the LE cert updated as expected. | 23:10 |
| clarkb | (and if we aren't checking that cert with our certcheck we shoudl add it) | 23:11 |
| clarkb | internet says that could just be noise from scanners | 23:11 |
| clarkb | injecting bad data trying to do bad things | 23:11 |
| corvus | yeah, but if that's the last thing and it's not responding on the port... | 23:12 |
| corvus | #status log restarted zuul-registry container | 23:13 |
| corvus | it's responding now | 23:13 |
| opendevstatus | corvus: finished logging | 23:13 |
| fungi | clarkb: pastebinit seems to call the api in a way that generates the old numbered style | 23:16 |
| fungi | i still need to look into why that is | 23:17 |
| clarkb | ah | 23:17 |
| clarkb | ya when we patched it we only did the web ui I tink | 23:17 |
| clarkb | I forgot there are cli tools | 23:17 |
| opendevreview | James E. Blair proposed opendev/system-config master: Add Jaeger tracing server https://review.opendev.org/c/opendev/system-config/+/855983 | 23:42 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!