| opendevreview | Ian Wienand proposed opendev/system-config master: install-ansible: remove stub install for ARA https://review.opendev.org/c/opendev/system-config/+/852478 | 00:07 |
|---|---|---|
| opendevreview | Ian Wienand proposed opendev/system-config master: bootstrap-bridge: drop pip3 role, add venv https://review.opendev.org/c/opendev/system-config/+/856593 | 00:07 |
| opendevreview | Ian Wienand proposed opendev/system-config master: run-selenium: Use latest tag on firefox image https://review.opendev.org/c/opendev/system-config/+/857803 | 00:07 |
| opendevreview | Ian Wienand proposed opendev/system-config master: afs-release: better info when can not get lockfile https://review.opendev.org/c/opendev/system-config/+/858009 | 00:07 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Run jobs with a jammy bridge.openstack.org https://review.opendev.org/c/opendev/system-config/+/857799 | 00:07 |
| opendevreview | Ian Wienand proposed opendev/system-config master: testinfra: Update selenium calls https://review.opendev.org/c/opendev/system-config/+/858003 | 00:07 |
| ianw | jeez we have hardcoded "bridge.openstack.org" in a lot of places | 00:37 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Abstract name of bastion host for testing path https://review.opendev.org/c/opendev/system-config/+/858476 | 02:00 |
| diablo_rojo | So... it seems the meetbot hasn't appeared in the envirosig channel despite the patch being merged? | 02:37 |
| fungi | diablo_rojo: i think we need to manually restart its container, it doesn't restart automatically so that we don't accidentally interrupt any running meetings | 03:07 |
| fungi | i'll do it now | 03:07 |
| diablo_rojo | oh thats a good safety mechanism | 03:15 |
| diablo_rojo | thank you fung! | 03:16 |
| fungi | any time | 03:25 |
| opendevreview | Ian Wienand proposed opendev/system-config master: install-ansible: remove testinfra version install workaround https://review.opendev.org/c/opendev/system-config/+/852475 | 03:29 |
| opendevreview | Ian Wienand proposed opendev/system-config master: testinfra: install with ansible extras https://review.opendev.org/c/opendev/system-config/+/852476 | 03:29 |
| opendevreview | Ian Wienand proposed opendev/system-config master: install-ansible: remove stevedore workaround https://review.opendev.org/c/opendev/system-config/+/852477 | 03:29 |
| opendevreview | Ian Wienand proposed opendev/system-config master: install-ansible: remove stub install for ARA https://review.opendev.org/c/opendev/system-config/+/852478 | 03:29 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bootstrap-bridge: drop pip3 role, add venv https://review.opendev.org/c/opendev/system-config/+/856593 | 03:29 |
| opendevreview | Ian Wienand proposed opendev/system-config master: run-selenium: Use latest tag on firefox image https://review.opendev.org/c/opendev/system-config/+/857803 | 03:29 |
| opendevreview | Ian Wienand proposed opendev/system-config master: afs-release: better info when can not get lockfile https://review.opendev.org/c/opendev/system-config/+/858009 | 03:29 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Run jobs with a jammy bridge.openstack.org https://review.opendev.org/c/opendev/system-config/+/857799 | 03:30 |
| opendevreview | Ian Wienand proposed opendev/system-config master: testinfra: Update selenium calls https://review.opendev.org/c/opendev/system-config/+/858003 | 03:30 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Abstract name of bastion host for testing path https://review.opendev.org/c/opendev/system-config/+/858476 | 03:30 |
| tkajinam | hello. it seems jobs using nodes with nested-virt-centos-9-stream labal is currently failing because of node failure. do we need to make any change in job definition or is that some issue within infra ? | 04:14 |
| tkajinam | example: https://zuul.opendev.org/t/openstack/build/e3cab8e04c4a4d668d5029b40bf307b9 | 04:14 |
| tkajinam | the current nodeset definition can be found here: https://github.com/openstack/puppet-octavia/blob/master/.zuul.yaml#L15-L25 | 04:15 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Abstract name of bastion host for testing path https://review.opendev.org/c/opendev/system-config/+/858476 | 04:23 |
| *** ysandeep|out is now known as ysandeep | 05:04 | |
| *** pojadhav is now known as pojadhav|PTO | 05:09 | |
| opendevreview | Ian Wienand proposed opendev/system-config master: Abstract name of bastion host for testing path https://review.opendev.org/c/opendev/system-config/+/858476 | 05:38 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Convert production playbooks to bastion host group https://review.opendev.org/c/opendev/system-config/+/858486 | 05:38 |
| opendevreview | lotorev vitaly proposed opendev/project-config master: Update link to zuul gating docs https://review.opendev.org/c/opendev/project-config/+/858454 | 05:42 |
| vlotorev[m] | <fungi> "vlotorev: looks like 858454 is..." <- fungi: I forget to publish edited file via web. Done now. | 05:46 |
| vlotorev[m] | I cdn modify pipeline description in other tenants if 858454 gets positive review. | 05:46 |
| vlotorev[m] | * fungi: I forget to publish edited file via web. Done now. | 05:46 |
| vlotorev[m] | I can modify pipeline description in other tenants if 858454 gets positive review. | 05:46 |
| *** ysandeep is now known as ysandeep|lunch | 07:24 | |
| *** jpena|off is now known as jpena | 07:30 | |
| *** ysandeep|lunch is now known as ysandeep | 08:04 | |
| *** ysandeep is now known as ysandeep|afk | 09:00 | |
| *** ysandeep|afk is now known as ysandeep | 09:36 | |
| *** ysandeep is now known as ysandeep|afk | 10:26 | |
| *** rlandy|out is now known as rlandy|rover | 10:34 | |
| *** ysandeep|afk is now known as ysandeep | 10:58 | |
| rlandy|rover | hello ... we have a NODE_FAILURE showing up on the showing up tripleo-ci-centos-9-scenario010-standalone | 11:23 |
| rlandy|rover | since 09/17 | 11:23 |
| rlandy|rover | https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-9-scenario010-standalone+&skip=0 | 11:23 |
| rlandy|rover | there are no logs so had to tell what is happening | 11:24 |
| rlandy|rover | this job uses nodeset: single-centos-9-node-nested-virt | 11:24 |
| akahat|ruck | we are testing to see if 8 is also problem. | 11:24 |
| rlandy|rover | so assumption is that it's the nest-virt nodeset | 11:24 |
| rlandy|rover | can you see anything in the logs? if not, would it be possible to hold a node and investigate | 11:25 |
| akahat|ruck | https://review.rdoproject.org/r/c/testproject/+/45162 | 11:26 |
| rlandy|rover | https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-scenario010-standalone+&skip=0 - looked ok until 15th | 11:28 |
| rlandy|rover | akahat|ruck will check it now | 11:28 |
| opendevreview | Merged opendev/project-config master: Update link to zuul gating docs https://review.opendev.org/c/opendev/project-config/+/858454 | 11:41 |
| fungi | tkajinam: rlandy|rover: akahat|ruck: still waking up, but i was looking into a similar issue with fedora-36 nodes yesterday as well. seems like both problems may have started around the same time | 11:43 |
| fungi | are you seeing similar problems with non-nestedvirt centos-9-stream labels? | 11:43 |
| rlandy|rover | fungi: hi - good morning ... and no those are running fine | 11:44 |
| fungi | interesting | 11:44 |
| rlandy|rover | ie: non-nested run everywhere - so those seems ok | 11:44 |
| rlandy|rover | idk if c8 ones are impacted yet | 11:44 |
| rlandy|rover | the last run on c8 was on 09/15 | 11:45 |
| rlandy|rover | so before the start of trouble | 11:45 |
| rlandy|rover | just created https://bugs.launchpad.net/tripleo/+bug/1990269 to start tracking and report gate blocker to openstack-discuss | 11:47 |
| rlandy|rover | akahat|ruck: ^^ fyi | 11:47 |
| akahat|ruck | rlandy|rover, ack | 11:47 |
| fungi | just checking up on the fedora-36 situation first, it's still what i was seeing yesterday. pulling the instance console log from nova it's booted up to a login prompt on tty1 and the boot output says networkmanager brought up networking, but nodepool is never able to reach the sshd on it | 11:49 |
| fungi | now to see if the nested virt stream 9 is at all similar | 11:49 |
| rlandy|rover | fungi: thanks | 11:50 |
| fungi | interestingly, nl04 which provides the bulk of those nodes (handles ovh) has no debug logs since 01:22 utc on the 17th | 12:00 |
| Clark[m] | fungi: nl04 is the only launcher providing those nodes. Sjc1 and limestone have max servers 0. | 12:01 |
| fungi | and the launcher daemon isn't running on it | 12:01 |
| *** rlandy|rover is now known as rlandy|rover|mtg | 12:01 | |
| fungi | docker-compose logs is full of tracebacks | 12:01 |
| fungi | KeyError: 'rockylinux-9' | 12:02 |
| fungi | let's see what config change rolled out around that time | 12:02 |
| *** rlandy|rover|mtg is now known as rlandy|rover | 12:03 | |
| Clark[m] | Or changes to nodepool | 12:04 |
| fungi | nothing merged over the weekend, so must be cropping up from a restart yes | 12:04 |
| *** amorin_ is now known as amorin | 12:07 | |
| fungi | https://paste.opendev.org/show/816812 is the cleaned up traceback from the docker-compose logs output | 12:07 |
| fungi | the /etc/nodepool/nodepool.yaml file on the server last updated 2022-09-01, so long before the container failed to start | 12:09 |
| Clark[m] | Need to add that image to https://opendev.org/openstack/project-config/src/branch/master/nodepool/nl04.opendev.org.yaml#L197 | 12:10 |
| Clark[m] | The other launcher file diskimage lists appear more complete. I wonder why nl04 is out of sync | 12:11 |
| fungi | oh indeed. thanks for spotting that | 12:11 |
| fungi | patch inbound | 12:11 |
| Clark[m] | fungi: I think rockylinux-8 is missing too. May need to cross check all the images listed in the pools | 12:13 |
| *** tosky is now known as Guest1012 | 12:14 | |
| *** tosky__ is now known as tosky | 12:14 | |
| fungi | they're all out of sync | 12:14 |
| fungi | nl01 has one more entry than nl02 for that matter | 12:14 |
| fungi | yeah, openeuler only got added to nl01 | 12:16 |
| *** tosky is now known as Guest1013 | 12:30 | |
| *** tosky__ is now known as tosky | 12:30 | |
| opendevreview | Jeremy Stanley proposed openstack/project-config master: Add missing rockylinux and openeuler entries https://review.opendev.org/c/openstack/project-config/+/858523 | 12:35 |
| fungi | infra-root: that's ^ fairly urgent since we've basically not been using ovh or vexxhost for nodes so far this week (which makes nested virt flavors completely unavailable too) | 12:36 |
| fungi | i'll get the config on nl04 patched locally by hand so i can start the service in the meantime | 12:36 |
| rlandy|rover | fungi: thanks for the quick action on this :) | 12:40 |
| fungi | #status log Manually patched configuration on nl04 in order to get the nodepool-launcher service running again while we wait for change 858523 to be reviewed | 12:41 |
| opendevstatus | fungi: finished logging | 12:41 |
| fungi | you're welcome | 12:41 |
| Clark[m] | fungi: that change lgtm but I'm not logged into Gerrit on this device | 12:41 |
| Clark[m] | Note the hourly infra-prod-service- ode pool will likely undo the hand fix | 12:42 |
| fungi | turns out i +2'd https://review.opendev.org/854431 which introduced this error, so i feel partly responsible for it being broken | 12:42 |
| fungi | Clark[m]: undo yes, but not restart the container probably? | 12:43 |
| fungi | seems like it broke on saturday because we added nodepool to the zuul restart playbook | 12:43 |
| fungi | (the configuration had been broken on that server since the first of the month) | 12:44 |
| fungi | so while ansible will probably undo the temporary addition, we'll hopefully deploy the fix before the next launcher restart | 12:44 |
| Clark[m] | Nodepool rereads it's config on each pass through the run loop. No restart required | 12:45 |
| fungi | yeah, but seems it only crashed when it was restarted? | 12:45 |
| Clark[m] | I suspect something changed in nodepool to make this more fatal than before. And that did require a restart (which we do automatically when nodepool has new images) | 12:46 |
| fungi | oh, perhaps it will crash again due to newer nodepool. i'll keep an eye out for that | 12:46 |
| Clark[m] | Now that the new code is running I suspect a revert of the on disk config will cause it to crash again | 12:46 |
| fungi | i can test that easily now | 12:46 |
| frickler | I thought I had done a check for the config error in https://review.opendev.org/c/zuul/nodepool/+/852264 , or was that a different scenario? | 12:47 |
| fungi | frickler: this was the launcher crashing, not the builder. but similar i think, yes | 12:48 |
| fungi | and it was present in the provider pools labels list not the provider diskimages list (but still missing from the top level diskimages list) | 12:56 |
| fungi | i think the pools is what the launcher uses, it doesn't seem to care about the provider diskimages list | 12:57 |
| fungi | Clark[m]: so it looks like undoing my edit does cause the launcher to start complaining that it's unable to reload the new config, but it doesn't terminate | 13:05 |
| fungi | if i restart the container i expect the service will fail to come up | 13:05 |
| Clark[m] | Ah ok | 13:06 |
| fungi | but for now it's just stuck using the working config because it's unable to parse the broken one | 13:06 |
| Clark[m] | Re the validator update we could add similar checks to that for the launchers | 13:06 |
| fungi | yes | 13:06 |
| *** dasm|off is now known as dasm | 13:30 | |
| *** ysandeep is now known as ysandeep|dinner | 14:12 | |
| *** soniya is now known as soniya|dinner | 14:14 | |
| opendevreview | Merged openstack/project-config master: Add missing rockylinux and openeuler entries https://review.opendev.org/c/openstack/project-config/+/858523 | 14:20 |
| clarkb | fungi: it looks like ovh may actually be able to occasionally boot fedora 36 (I decided to take a quick look in case the fedora issues were related to ovh being down) | 14:52 |
| clarkb | which makes this problem more subtle and probably more difficult to debug. Looks like 2/3 of the ovh gra boots for the one request I checked failed then the third succeeded | 14:53 |
| fungi | yeah | 15:04 |
| fungi | and explains why the node_failure got worse when nl04 was out of commission | 15:05 |
| clarkb | network manager is failing to start checkout the console log for clarkb-test-36 in iweb mtl01 | 15:18 |
| clarkb | the logs for that are in journald though according to the console log. Not sure how we're going to get those out. I guess chroot on a rescue instance | 15:19 |
| fungi | odd, the console logs i looked at said that nm started and systemd switched to network-ready or whatever | 15:19 |
| clarkb | yes, it does that eventually after marking it failed | 15:19 |
| clarkb | why it marks network online when NM fails I do not know | 15:19 |
| clarkb | https://src.fedoraproject.org/rpms/NetworkManager/c/22b2a6c081e8f3c3d30d459703de185857e3fca1?branch=f36 perhaps but that doesn't give a detailed changelog. Maybe nm does | 15:21 |
| clarkb | https://github.com/NetworkManager/NetworkManager/blob/1.38.4/NEWS nothing super noteworthy there | 15:22 |
| clarkb | I can't do a rescue in that cloud | 15:24 |
| fungi | clarkb: this is what i saw on one i caught failing in that same region: https://paste.opendev.org/show/816822 | 15:24 |
| fungi | (pulled from my terminal buffer) | 15:25 |
| clarkb | yup line 168 | 15:25 |
| clarkb | shows it failed | 15:25 |
| clarkb | I think its a best effort ot bring up networking before it gives up and then proceeds with booting | 15:25 |
| fungi | yeah, i completely overlooked that line | 15:25 |
| fungi | my eye was drawn to "Network is Online." | 15:26 |
| fungi | so anyway, yes your reproduction seems to match | 15:26 |
| clarkb | its possible this never worked and we didn't really notice until ovh's launcher stopped. OVH is a dhcp cloud so any sane fallback there with NM would end up dhcping and be happy | 15:27 |
| fungi | clarkb: i saw a similar failure in rackspace, so presumably we could rescue a node there | 15:27 |
| clarkb | that said I did boot a node in ovh and nm seemed to be happy with the glean config. Not likely that it is falling back there and instead maybe something directly wrong with static network config | 15:28 |
| clarkb | oh good point | 15:28 |
| clarkb | ok I'll clean this one up and try in rax | 15:28 |
| *** ysandeep|dinner is now known as ysandeep | 15:35 | |
| clarkb | it looks like NM is attempting DHCP which of course returns no responses on the non DHCP clouds | 15:45 |
| fungi | that would seem to match the places we see it fail | 15:46 |
| clarkb | /etc/sysconfig/network-scripts/ifcfg-enX0 has BOOTPROTO=static and IPADDR set | 15:47 |
| clarkb | https://docs.fedoraproject.org/en-US/fedora/latest/release-notes/sysadmin/Networking/ I think that is our answer | 15:47 |
| clarkb | I can write a change to dib to add that package when glean is used | 15:48 |
| clarkb | it would be nice if that doc stated the package name though | 15:48 |
| fungi | sheesh | 15:49 |
| fungi | so i guess 36 was never working outside dhcp clouds | 15:49 |
| clarkb | https://packages.fedoraproject.org/pkgs/NetworkManager/NetworkManager-initscripts-ifcfg-rh/ | 15:51 |
| clarkb | fungi: yes I think that is correct | 15:51 |
| fungi | indeed, that's not a package name i would have easily guess from the release note saying "include the ifcfg subpackage" | 15:53 |
| fungi | you know, *the* ifcfg subpackage. NetworkManager-initscripts-ifcfg-rh, duh! | 15:53 |
| opendevreview | Clark Boylan proposed openstack/diskimage-builder master: Install Fedora ifcfg NM compat package https://review.opendev.org/c/openstack/diskimage-builder/+/858547 | 16:00 |
| clarkb | something like that maybe | 16:00 |
| *** marios is now known as marios|out | 16:00 | |
| clarkb | I'll leave the rescued node in rax dfw for now so that anyone else can chroot into /mnt and journalctl -u NetworkManager | 16:04 |
| clarkb | you do have to ssh as the 'ubuntu' user but your keys should work | 16:05 |
| *** ysandeep is now known as ysandeep|out | 16:15 | |
| clarkb | if anyone has a pagure account for fedora it looks like you can edit https://pagure.io/fedora-docs/release-notes/blob/f36/f/modules/release-notes/pages/sysadmin/Networking.adoc to add the package name to that doc | 16:20 |
| clarkb | oh wait its a third party system thta apparently write sthose files. THat explains the extreme line length | 16:21 |
| opendevreview | Michal Nasiadka proposed openstack/project-config master: Add rockylinux-9-arm64 https://review.opendev.org/c/openstack/project-config/+/858554 | 16:30 |
| fungi | i'll make sure i triple-check that ^ one for missing parts :/ | 16:33 |
| fungi | clarkb: for _the_ ifcfg subpackage, we could also temporarily stuff it into infra-package-needs if we're in a hurry (i'm not, but just pointing it out) | 16:35 |
| *** jpena is now known as jpena|off | 16:37 | |
| mnasiadka | fungi: I'm doing that for first time (adding arm64) if there's anything else that needs to be done first - give me a shout, happy to do everything is needed | 16:38 |
| mnasiadka | While we're at it - I don't think there are Rocky Linux mirrors, right? | 16:38 |
| fungi | mnasiadka: that one looks right. at least it doesn't miss the thing that nested-virt-rockylinux-9 did | 16:39 |
| fungi | mnasiadka: no, we don't normally mirror every distro, we've been trying to keep it to the ones which are more heavily used, though if we see rockylinux approach centos levels of utilization in our jobs we should probably consider it | 16:39 |
| mnasiadka | fungi: oops, sorry for missing what I missed (wasn't aware) | 16:40 |
| fungi | mnasiadka: no need to apologize, i should have caught it when i reviewed the change originally | 16:40 |
| mnasiadka | fungi: Kolla projects group is mainly going to utilise Rocky Linux 9 in CI after Zed release, so probably the usage will raise in couple of months | 16:41 |
| fungi | mnasiadka: yeah, i have a feeling antelope will have a lot of demographic shift in jobs, and we should probably look at it again in a few months to see where we stand | 16:41 |
| fungi | one preliminary bit of info though, does rocky have public rsync access to any mirrors? | 16:42 |
| fungi | all our other rpm-based distros are mirrored via rsync, so consistency there would be nice | 16:42 |
| mnasiadka | A lot of mirrors on their website mention rsync (https://mirrors.rockylinux.org/mirrormanager/mirrors) but I don't know if it's public | 16:43 |
| mnasiadka | NeilHanlon should know a lot more than me | 16:43 |
| fungi | no worries, there's plenty of time to find out | 16:43 |
| mnasiadka | yup, thanks | 16:44 |
| NeilHanlon | you can pull from msync.rockylinux.org via rsync; we'll be restructuring soon, though, so that'll line up with the opendev mirrors too, probably | 17:39 |
| fungi | NeilHanlon: good to know, thanks! | 17:40 |
| *** dhill is now known as Guest1039 | 18:19 | |
| clarkb | fungi: that is a good point. I think we can wait until ianw has a chance to look later today. If we think dib update can be done relatively quickly that is good enough | 18:43 |
| chris7dohmen | Hello folks! I went to use `pbr==5.10.0` which had the symbol fix for squashed messages. I saw that it was not working and trying to track down why, and it looks like the commit did not make it into the build, although it was in master. The commit in question is https://opendev.org/openstack/pbr/commit/45d4c9f02fafb425e61a18bbd7a20ef456293a56 which was merged Aug 5th with the build on Aug 11th. Did I do something wrong or was it just not picked up b | 19:10 |
| fungi | chris7dohmen: sorry, we're having our weekly meeting at the moment, but i can take a look when it wraps up in about 50 minutes | 19:11 |
| clarkb | looks like the commit that was tagged is before the commit you want | 19:13 |
| clarkb | situation normal as far as git and the build system are concerned | 19:14 |
| slittle1 | hi all, has opendev ever hosted a copy of the Linux kernel git? | 19:15 |
| clarkb | slittle1: no | 19:15 |
| slittle1 | Would there be any concerns? | 19:16 |
| clarkb | yes, we'd need to make sure gerrit and our replica giteas (and possibly zuul depending on how it is used) will handle it | 19:16 |
| fungi | i expect it would be very large. cloning it repeatedly in ci jobs would also likely be taxing on our servers | 19:16 |
| clarkb | the gerrit mailing list is full of users struggling to manage that repo in gerrit | 19:16 |
| clarkb | it is doable but require fiddling | 19:16 |
| clarkb | https://groups.google.com/g/repo-discuss/c/TJ4Pdg3KHUI for example | 19:18 |
| slittle1 | One of my StarlingX colleagues is looking for a place to host the kernel. Not sure if Vefa has joined the channel yet or not. Might still be getting his IRC set up | 19:19 |
| clarkb | slittle1: is there some reason that the existing kernel hosting doesn't work? | 19:19 |
| clarkb | I think OSUOSL hosts it and they let people have forks on it. But I'm not sure what the criteria for that are | 19:19 |
| slittle1 | There are a few drivers. One was a desire for a uniform code inspection framework, i.e. gerrit | 19:20 |
| clarkb | if we ignore that the linux kernel is huge and presents unique problems with git hosting, I think it is a good idea to avoid using opendev to host forks of projects that already have homes | 19:20 |
| chris4dohmen | fungi: thanks, no rush! I was updating our tooling and was rather confused by the version I was getting out after this change. After reviewing the files I had downloaded from pypi I noticed and confirmed the issue (or my mistake when I made the review) | 19:21 |
| clarkb | chris4dohmen: https://opendev.org/openstack/releases/src/branch/master/deliverables/_independent/pbr.yaml#L231 is what was tagged. Which is the sixth commit listed at https://opendev.org/openstack/pbr/commits/branch/master (and I think you want the first?) | 19:23 |
| slittle1 | The second was an ability to create branches and tags on the Linux git. | 19:24 |
| slittle1 | If OSUOSL allows branches/tags, that might be an option | 19:24 |
| fungi | linus mirrors his tree into github, so forking there could also be an option i suppose | 19:26 |
| clarkb | ya for that I'm not sure. I know some people have their kernels hosted there, but that may just be for maintainers? For us this wuold be non trivial and isn't something I think we can commit to now. It would require significant investigation. (like stand up a whole new gerrit and gitea do benchmarking) | 19:27 |
| clarkb | https://discourse.gitea.io/t/issues-cloning-a-big-repository/63 gitea may have resolved those problems as this is a really old post, but it definitely struggled at one time too | 19:33 |
| chris4dohmen | clark: Ah ok I see what happened then. commit `1eadf23b94846a0ebc8402e09190ed1de6dbffd0` is the head of `pbr==5.10.0`, not what was in the latest master at the time. That makes sense why it was not included then as I was just going off dates. thanks for the clarification! what is the typical release schedule for new releases? | 19:33 |
| clarkb | chris4dohmen: OpenStack is working on putting its "Zed" release out the door right now. I suspect that you could propose a PBR release to openstack/releases and have it land as soon as that is done in a few weeks. https://releases.openstack.org/zed/schedule.html They likely won't want to release PBR until afterwards unless it is a release breaking fix | 19:34 |
| chris4dohmen | clark: nope, just a bugfix. I will talk with our devops team and try to convince them to use a dev build for now. When you say propose a change, you mean submit a review for an update to that file, or is there something else involved? thanks for all of your help! | 19:37 |
| clarkb | chris4dohmen: yes push an update to https://opendev.org/openstack/releases/src/branch/master/deliverables/_independent/pbr.yaml#L231 with the new version and its associated commit hash. That gets reviewed and when it lands the build system makes the release | 19:38 |
| chris4dohmen | clarkb: awesome, thanks for clarifying that and answering all of my questions! | 19:39 |
| ianw | i can make a dib release with the f36 fix. not sure how around to be i'll watch it by the time builds make it to production | 19:41 |
| fungi | i don't think it's too urgent, mainly f36 nodes will keep failing to boot in non-dhcp providers and lead to delays filling node requests and a bit of wasted resources in those providers, and could start leading to new node_failure results if ovh is unusable for us | 19:44 |
| *** rlandy|rover is now known as rlandy|rover|biab | 20:02 | |
| opendevreview | Merged openstack/diskimage-builder master: Install Fedora ifcfg NM compat package https://review.opendev.org/c/openstack/diskimage-builder/+/858547 | 20:26 |
| *** rlandy|rover|biab is now known as rlandy|rover | 20:30 | |
| ianw | fungi: if you have a sec to double-check things in https://review.opendev.org/c/openstack/diskimage-builder/+/855589 ; that is a linters fix as i think we dropped dib-lint | 20:31 |
| ianw | mnasiadka / fungi: i'm not fully -1 on https://review.opendev.org/c/openstack/project-config/+/858554 but i also don't think we know if it works. i've put a suggestion in there about func test builds in dib. it would need monitoring either way | 20:41 |
| zigo | Hi there! | 20:51 |
| zigo | There's no release notes for keystone for Zed. | 20:51 |
| zigo | https://docs.openstack.org/releasenotes/keystone/unreleased.html | 20:51 |
| Clark[m] | zigo: I think they need to push a change to add a file like this one for zed https://opendev.org/openstack/keystone/raw/branch/master/releasenotes/source/yoga.rst | 21:03 |
| zigo | ok | 21:04 |
| fungi | Clark[m]: zigo: https://review.opendev.org/c/openstack/keystone/+/857565 | 21:05 |
| fungi | i think that's it | 21:05 |
| opendevreview | Merged openstack/diskimage-builder master: Allow flake8 version 5 https://review.opendev.org/c/openstack/diskimage-builder/+/854670 | 21:28 |
| opendevreview | Ian Wienand proposed openstack/diskimage-builder master: Start running dib-lint again https://review.opendev.org/c/openstack/diskimage-builder/+/855589 | 21:57 |
| ianw | fungi: ^ that needed some fixes after a rebase and a flake8 update change merging before it | 21:58 |
| ianw | fungi: another for when you have time is https://review.opendev.org/q/topic:configure-mirrors-centos to add the flag to enable that repo to our configure-mirrors job. there's a couple underneath it to keep the test job in sync | 22:27 |
| *** dhill is now known as Guest1048 | 22:35 | |
| ianw | "usermod: user zuul is currently used by process 749" | 22:39 |
| ianw | this is i think a new one on system-config tests ... maybe it's something i've done | 22:39 |
| ianw | it must be specific to https://review.opendev.org/c/opendev/system-config/+/858476/; we've had good runs of system-config-run-base this morning | 22:40 |
| ianw | ahh it is my fault; from the shuffling i'm doing. i've included zuul as an extra user in the bridge bringup | 22:42 |
| opendevreview | Merged opendev/base-jobs master: Revert "Switch base-test to test-prepare-workspace-git" https://review.opendev.org/c/opendev/base-jobs/+/858473 | 23:22 |
| opendevreview | Merged opendev/base-jobs master: base-test: add descriptive names https://review.opendev.org/c/opendev/base-jobs/+/858474 | 23:27 |
| opendevreview | Merged opendev/base-jobs master: configure-mirrors: enable extras-common for 9-stream https://review.opendev.org/c/opendev/base-jobs/+/858255 | 23:27 |
| *** rlandy|rover is now known as rlandy|out | 23:40 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!