| opendevreview | Ian Wienand proposed opendev/system-config master: inventory: add host keys https://review.opendev.org/c/opendev/system-config/+/862762 | 00:07 |
|---|---|---|
| corvus | infra-root: there seems to be a zuul issue | 00:35 |
| corvus | i think nodepool is not connected to zk | 00:36 |
| corvus | 2022-10-27 00:35:45,250 WARNING kazoo.client: Connection dropped: socket connection error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997) | 00:36 |
| ianw | this seems like suspiciously something a new bridge would have done ... | 00:36 |
| corvus | ah -- maybe it created a new CA | 00:37 |
| corvus | i think if the ca directory wasn't manually copied over, it would probably do that | 00:37 |
| ianw | we just fixed the nodepool job -> https://zuul.opendev.org/t/openstack/builds?job_name=infra-prod-service-nodepool&project=opendev/system-config | 00:38 |
| corvus | if that's the case, then we might have restarted nodepool but not the zk servers; so we might need to restart everything? | 00:38 |
| ianw | i did not copy a CA directory. which one so i can update https://etherpad.opendev.org/p/bastion-upgrade-nodes-2022-10 ? | 00:38 |
| ianw | (sorry just pulling up roles, not 100% familiar with the layout) | 00:39 |
| ianw | /var/zookeeper/tls i guess? | 00:40 |
| corvus | on bridge it's /var/zk-ca and /var/jaeger-ca | 00:40 |
| corvus | (we have 2 cas now) | 00:40 |
| ianw | mea culpa i totally overlooked migrating those :/ | 00:41 |
| corvus | i didn't think about it either :| | 00:41 |
| ianw | if i copy them from old bridge -> new bridge and re-ansible nodepool, that should work? | 00:41 |
| corvus | yep i think so | 00:41 |
| ianw | ok one sec ... | 00:42 |
| corvus | and if i'm right about the sequencing, then that may be the only thing necessary -- i think zuul<->zk is still using existing connections and so hasn't noticed or cared about any files changing out from under it. | 00:42 |
| corvus | (so we might be able to get away with not touching the rest of the system and it will just resume once nodepool is supplying nodes) | 00:43 |
| ianw | i've put them in place (old versions in /root/old-ca) and am running a manual run of ansible-playbook service-nodepool now | 00:45 |
| corvus | nl01 is doing work now | 00:48 |
| corvus | and zuul is running new jobs | 00:48 |
| ianw | <phew> | 00:49 |
| corvus | so we're probably back in service | 00:49 |
| ianw | we would have noticed almost straight away -- but the nodepool playbook wasn't working due to the old !!binary data in the inventory, for which our best guess is that python3.10's json encoder is more unhappy about | 00:50 |
| ianw | c.f. https://review.opendev.org/c/opendev/system-config/+/862759 | 00:50 |
| ianw | thanks for that, and i will be putting together notes from that etherpad to make it a bit easier to switch hosts next time | 00:51 |
| ianw | playbook just finished | 00:52 |
| corvus | thank you! | 00:52 |
| *** rlandy|bbl is now known as rlandy|out | 01:25 | |
| opendevreview | Ian Wienand proposed opendev/system-config master: bastion host: add global known_hosts values https://review.opendev.org/c/opendev/system-config/+/862764 | 02:00 |
| opendevreview | Ian Wienand proposed opendev/system-config master: base: restrict bastion login to bridge01.opendev.org https://review.opendev.org/c/opendev/system-config/+/862765 | 02:10 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Remove old bridge testing https://review.opendev.org/c/opendev/system-config/+/862766 | 02:10 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bastion host: add global known_hosts values https://review.opendev.org/c/opendev/system-config/+/862764 | 02:21 |
| opendevreview | Ian Wienand proposed opendev/system-config master: base: restrict bastion login to bridge01.opendev.org https://review.opendev.org/c/opendev/system-config/+/862765 | 02:21 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Remove old bridge testing https://review.opendev.org/c/opendev/system-config/+/862766 | 02:21 |
| opendevreview | Ian Wienand proposed opendev/system-config master: inventory: add host keys https://review.opendev.org/c/opendev/system-config/+/862762 | 02:49 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bastion host: add global known_hosts values https://review.opendev.org/c/opendev/system-config/+/862764 | 02:49 |
| opendevreview | Ian Wienand proposed opendev/system-config master: base: restrict bastion login to bridge01.opendev.org https://review.opendev.org/c/opendev/system-config/+/862765 | 02:49 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Remove old bridge testing https://review.opendev.org/c/opendev/system-config/+/862766 | 02:49 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bastion host: add global known_hosts values https://review.opendev.org/c/opendev/system-config/+/862764 | 03:28 |
| opendevreview | Ian Wienand proposed opendev/system-config master: base: restrict bastion login to bridge01.opendev.org https://review.opendev.org/c/opendev/system-config/+/862765 | 03:28 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Remove old bridge testing https://review.opendev.org/c/opendev/system-config/+/862766 | 03:28 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bastion host: add global known_hosts values https://review.opendev.org/c/opendev/system-config/+/862764 | 03:49 |
| opendevreview | Ian Wienand proposed opendev/system-config master: base: restrict bastion login to bridge01.opendev.org https://review.opendev.org/c/opendev/system-config/+/862765 | 03:49 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Remove old bridge testing https://review.opendev.org/c/opendev/system-config/+/862766 | 03:49 |
| *** dasm|rover is now known as dasm|off | 03:50 | |
| *** marios is now known as marios|ruck | 04:57 | |
| opendevreview | Ian Wienand proposed opendev/system-config master: bastion host: add global known_hosts values https://review.opendev.org/c/opendev/system-config/+/862764 | 05:35 |
| opendevreview | Ian Wienand proposed opendev/system-config master: base: restrict bastion login to bridge01.opendev.org https://review.opendev.org/c/opendev/system-config/+/862765 | 05:35 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Remove old bridge testing https://review.opendev.org/c/opendev/system-config/+/862766 | 05:35 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bootstrap-bridge: Codify allowed Zuul logins https://review.opendev.org/c/opendev/system-config/+/862761 | 05:35 |
| opendevreview | Ian Wienand proposed opendev/system-config master: base: restrict bastion login to bridge01.opendev.org https://review.opendev.org/c/opendev/system-config/+/862765 | 05:35 |
| opendevreview | Ian Wienand proposed opendev/system-config master: Remove old bridge testing https://review.opendev.org/c/opendev/system-config/+/862766 | 05:36 |
| opendevreview | Ian Wienand proposed opendev/system-config master: inventory: add host keys https://review.opendev.org/c/opendev/system-config/+/862762 | 05:36 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bootstrap-bridge: Codify allowed Zuul logins https://review.opendev.org/c/opendev/system-config/+/862761 | 05:38 |
| opendevreview | Ian Wienand proposed opendev/system-config master: inventory: add host keys https://review.opendev.org/c/opendev/system-config/+/862762 | 05:38 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bastion host: add global known_hosts values https://review.opendev.org/c/opendev/system-config/+/862764 | 05:38 |
| *** jpena|off is now known as jpena | 06:58 | |
| noonedeadpunk | frickler: yes, https://pubmirror1.math.uh.edu/fedora-buffet/ is broken indeed | 09:26 |
| noonedeadpunk | https://pubmirror1.math.uh.edu/fedora-buffet/epel/9/Everything/x86_64/repodata/repomd.xml contains links that does not exist in fact | 09:26 |
| noonedeadpunk | Should we jsut try switching mirrors then? | 09:27 |
| mnasiadka | We should, Kolla is also affected | 09:27 |
| mnasiadka | question to which one | 09:27 |
| noonedeadpunk | Try to open metalink from where rsync is ran? | 09:29 |
| noonedeadpunk | Ie `curl https://mirrors.fedoraproject.org/metalink?repo=epel-9&arch=x86_64` ? | 09:29 |
| noonedeadpunk | this will get list of best-matching mirrors based on the location | 09:29 |
| mnasiadka | Yes, but we would need infra-root help with that I guess ;-) | 09:31 |
| mnasiadka | 6 years ago it was changed from the mirror on kernel.org - https://opendev.org/opendev/system-config/commit/8e3bfee4ee740cb59bb800c309ae73e19d7a05c9 | 09:32 |
| noonedeadpunk | well, or anybody who has access to rax or vexxhost I guess ?:) | 09:32 |
| frickler | this is what I get on mirror-update, where we run the rsync https://paste.opendev.org/show/bLeOZxhSnT4q7JtcDLUd/ | 09:58 |
| frickler | feel free to propose a change to use one of those | 09:58 |
| noonedeadpunk | I like rsync://download-ib01.fedoraproject.org/ or rsync://mirrors.mit.edu/ | 10:16 |
| noonedeadpunk | fedoraproject sounds like smth reference... | 10:16 |
| noonedeadpunk | Will propose patch | 10:16 |
| noonedeadpunk | but not sure how they're permanent though... | 10:18 |
| mnasiadka | fedoraproject.org seems like it's at least close to source and maintained by fedora | 10:20 |
| noonedeadpunk | well, they have download-ib01.fedoraproject.org and download-cc-rdu01.fedoraproject.org and dunno if they mean to make these persistant... | 10:22 |
| noonedeadpunk | I used mit.edu mirrors for other repos and they tend to work nicely and reliably for years... | 10:24 |
| noonedeadpunk | and domain doesn't look like smth that can change in a week and for metalink usage only... | 10:24 |
| opendevreview | Dmitriy Rabotyagov proposed opendev/system-config master: Switch epel mirror to mirrors.mit.edu https://review.opendev.org/c/opendev/system-config/+/862793 | 10:26 |
| *** rlandy|out is now known as rlandy | 10:34 | |
| *** dviroel_ is now known as dviroel | 11:30 | |
| fungi | we did use mit mirrors for a while for something, but then they had prolonged issues and we switched, as usually tends to happen | 11:43 |
| fungi | https://review.opendev.org/811832 Revert "Switch Fedora mirror to mirrors.mit.edu" | 11:44 |
| fungi | roughly a year ago we switched our fedora mirroring from mit to uh for similar reasons to the problem at uh now for epel | 11:45 |
| noonedeadpunk | ugh | 11:49 |
| noonedeadpunk | I didn't know that :( | 11:49 |
| noonedeadpunk | should we go with download-ib01.fedoraproject.org/ then? | 11:50 |
| noonedeadpunk | Or mirror.cogentco.com ? :D | 11:51 |
| * noonedeadpunk don't trust cogent too much | 11:51 | |
| noonedeadpunk | but might be worth a shot... | 11:52 |
| noonedeadpunk | I would say that issue with MIT was different. | 11:53 |
| noonedeadpunk | as uh now have repomod.xml jsut out of sync with actuall content | 11:53 |
| frickler | the ideal solution would be to find a mirror with operators someone has a direct contact to | 11:58 |
| fungi | yeah, the point is that every mirror we ever try eventually gets swapped out for another due to prolonged issues, so i don't expect that to necessarily change | 12:11 |
| fungi | sometimes we even do have responsive mirror operators, but when they respond with something like "we're aware and are rebuilding our mirror but it's going to take another week to finish" then we still end up needing to switch to something else | 12:12 |
| noonedeadpunk | Well, it's ideal solution indeed... | 12:19 |
| noonedeadpunk | But unfortunatelly I don't think I do have such contacts :( | 12:20 |
| *** dasm|off is now known as dasm|rover | 13:10 | |
| kayo | hello everyone, if I want to start coding around openstack where is the best place to do it? and we have a tag like 'easy hacks' to start? | 13:44 |
| fungi | kayo: this channel is for a collaboration services community. our systems host a variety of projects, one of which is openstack. the openstack community has a guide for getting started contributing to their projects though, and you can find it here: https://docs.openstack.org/contributors/ | 13:46 |
| kayo | got it, thanks fungi | 13:47 |
| fungi | my pleasure | 13:47 |
| fungi | and if you have questions generally relevant to the opendev collaboratory, this would be the place to ask | 13:48 |
| noonedeadpunk | sooo..... should we jsut wait for current mirrors to re-sync (hopefully they're taking care of it) | 13:53 |
| fungi | noonedeadpunk: this code comment suggests that tibbs in #fedora-admin might know the status: https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/mirror-update/files/epel-mirror-update#L28-L29 | 14:02 |
| fungi | looks like we switched from mirrors.kernel.org to pubmirror1.math.uh.edu for epel mirroring 5.5 years ago in https://review.openstack.org/453274 | 14:05 |
| fungi | with the rationale that "we have a point of contact for future failures" | 14:06 |
| noonedeadpunk | surprisingly, person still seems hanging in same channel... | 14:07 |
| noonedeadpunk | I haven't treated this comment and point of contact - shame on me | 14:08 |
| fungi | we seem to be using mirror.facebook.net for centos and fedora, not sure if they also mirror epel, but also it's not as if they haven't had stale/corrupt mirrors for extended periods in the past year too | 14:09 |
| noonedeadpunk | they do | 14:10 |
| noonedeadpunk | facebook was in the list | 14:10 |
| noonedeadpunk | `rsync://mirror.facebook.net/fedora/epel/` | 14:10 |
| fungi | we've tried lots of mirrors for these things over the years, and the only thing they seem to have in common is that they all suffer issues from time to time | 14:10 |
| fungi | as evidenced by the fact that we have this discussion about one or another of them every few months | 14:11 |
| noonedeadpunk | well, that is true. But they ususally recover in ~24 hours I'd say | 14:11 |
| noonedeadpunk | And it's third day it's broken which a bit annoying | 14:12 |
| noonedeadpunk | I pinnged tibbs so let's see | 14:13 |
| fungi | thanks! | 14:14 |
| fungi | and if it's going to be a while (or they're unresponsive), we can try a different mirror yet again | 14:14 |
| *** knikolla[m] is now known as knikolla | 14:38 | |
| clarkb | I went ahead a +2'd the mirror change since that seems reasonable if we don't hear back | 15:08 |
| clarkb | also I'm having a very slow start this morning | 15:08 |
| clarkb | note that it is my understanding that we cannot use any of the mirrors under fedoraproject.org as those are all top level and only full public mirrors may sync from there | 15:10 |
| noonedeadpunk | I can put -W to wait for it for a while | 15:12 |
| noonedeadpunk | *for answer | 15:13 |
| mnasiadka | I moved Kolla builds to use download-ib01.fedoraproject.org mirror and it works | 15:13 |
| noonedeadpunk | I think you can reelase -W with core | 15:13 |
| mnasiadka | (for now) | 15:13 |
| clarkb | ya end users using those mirrors is fine. It is our mirrors that shouldn't sync from there | 15:14 |
| clarkb | infra-root anyone have time to review https://review.opendev.org/c/opendev/system-config/+/862631 to create python3.11 base images now that 3.8 has been dropped? | 15:30 |
| clarkb | corvus: ianw: do we need to double check that the auth for tracing didn't get affected like nodepool? | 15:35 |
| noonedeadpunk | um, should 3.8 be dropped? | 15:35 |
| clarkb | noonedeadpunk: yes, opendev doesn't use it anymore on an of our container images | 15:35 |
| noonedeadpunk | ah, ok, fair. it's internal | 15:36 |
| clarkb | this is independnet of the platforms Zuul provides for testing and is used for running our services | 15:36 |
| noonedeadpunk | yeah-yeah, fair | 15:36 |
| noonedeadpunk | sorry | 15:36 |
| clarkb | separately looking at the zuul job runtime improvements between 3.8 and 3.10/3.11 I think everyone should drop it :) | 15:37 |
| clarkb | it will be interesting to see if openstack's 3.10 testing shows similar results or not | 15:37 |
| noonedeadpunk | well, after AA | 15:37 |
| corvus | clarkb: i mentioned both CAs at the time and was operating under the assumption ianw copied both over. jaeger is receiving traces, so i think we can wait for him to confirm. | 15:39 |
| clarkb | ack | 15:41 |
| *** dviroel is now known as dviroel|lunch | 15:50 | |
| opendevreview | Dmitriy Rabotyagov proposed opendev/system-config master: Switch epel mirror to pubmirror3.math.uh.edu https://review.opendev.org/c/opendev/system-config/+/862793 | 16:08 |
| *** marios|ruck is now known as marios|out | 16:09 | |
| noonedeadpunk | so, repo maintainer has responded and suggested using pubmirror3 as it's lighter and less loaded and less likely to fail | 16:09 |
| fungi | thanks! expediting then | 16:09 |
| fungi | still has rsync available, i take it? | 16:10 |
| noonedeadpunk | Well look like it does | 16:10 |
| fungi | i'll know momentarily | 16:11 |
| noonedeadpunk | I checked with rsync -rltvz rsync://pubmirror3.math.uh.edu/fedora-buffet/epel/8 /tmp/epel | 16:11 |
| noonedeadpunk | but would be great if you could double-checked as I'm quite annoyingly side-pinged | 16:11 |
| fungi | sync in progress now | 16:13 |
| fungi | noonedeadpunk: mnasiadka: our epel mirrors should be updated to match what's on pubmirror3.math.uh.edu now. please double-check they have the content you're hoping for | 16:15 |
| fungi | if they do, i'll approve 862793 and release my lock session once that deploys | 16:16 |
| noonedeadpunk | this means basically issue recheck somewhere? | 16:21 |
| fungi | unless you know how it was breaking and can more directly check the files there seem to have the correction | 16:25 |
| noonedeadpunk | issued recheck for 861601 | 16:27 |
| noonedeadpunk | it will fall into retry_limit if did not help | 16:27 |
| fungi | hopefully should be able to see fairly quickly if it gets past installing things from epel or not? | 16:28 |
| noonedeadpunk | well... I think it takes ~15 mins before our stuff start running at all | 16:29 |
| noonedeadpunk | but yes, realtively quickly we can confirm if it fixed issue or not | 16:29 |
| *** jpena is now known as jpena|off | 16:32 | |
| mnasiadka | fungi: pubmirror3 looks good | 16:37 |
| fungi | thanks for confirming! | 16:39 |
| fungi | once noonedeadpunk is satisfied, i'll make it permanent | 16:39 |
| * noonedeadpunk needs another 5 mins | 16:41 | |
| *** dviroel|lunch is now known as dviroel | 16:45 | |
| noonedeadpunk | fungi: works for us! | 16:49 |
| fungi | awesome, thanks! | 16:50 |
| opendevreview | Merged opendev/system-config master: Switch epel mirror to pubmirror3.math.uh.edu https://review.opendev.org/c/opendev/system-config/+/862793 | 17:15 |
| clarkb | fungi: do you know if I write config to disable phased updates on all debuntu machines if older versions like bionic will complain because tehy don't recognize the config entries? | 17:46 |
| clarkb | I've remembered we probably want to disable phased updates for our prod jammy servers and trying to decide if I need to only apply that file to jammy and newer | 17:46 |
| clarkb | oh I can use the held gitea server to check | 18:00 |
| clarkb | hrm that server doesn't seem to exist anymore. I don't think I cleaned it up | 18:02 |
| fungi | yeah, i'm really not sure to be honest, though i expect apt will just ignore things it doesn't recognize | 18:02 |
| fungi | the held mm3 listserv is jammy | 18:03 |
| fungi | 862793 has deployed, so i'm releasing the lock now | 18:04 |
| clarkb | I've just remembered my local fileserver is bionic. I'll test really quickly on that | 18:05 |
| clarkb | seems like it does get ignored (or if it doesn't then it does what we want either way). I was able to apt-get update and dist-upgrade my local machine without complaint | 18:08 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Remove snapd from our servers https://review.opendev.org/c/opendev/system-config/+/862834 | 18:09 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Don't install phased package updates with apt https://review.opendev.org/c/opendev/system-config/+/862835 | 18:09 |
| clarkb | Those are two changes related to booting new jammy servers. Though the first applies to our older servers too | 18:09 |
| *** dasm|rover is now known as dasm|off | 19:21 | |
| *** dviroel is now known as dviroel|afk | 19:39 | |
| ianw | clarkb/corvus: yep i copied both of those ca's to the new bridge | 19:54 |
| clarkb | ianw: for https://review.opendev.org/c/opendev/system-config/+/862761 is that necessary because the normal mechanism only runs after bridge is bootstrapped? | 19:58 |
| clarkb | wondering if we need to remove the old mechanism to reduce confusion | 19:58 |
| ianw | clarkb: so afaict there is no old method, other than manually copying in keys to zuul's authorised keys? | 20:00 |
| ianw | that was why i thought it was better to keep track of it in system-config | 20:01 |
| clarkb | I thought we did a zuul user instanll | 20:02 |
| clarkb | ianw: ya its via extra_users in the bastion group file | 20:03 |
| clarkb | but I think that does occur after bootstrapping, but maybe its ok? | 20:03 |
| ianw | clarkb: hrm -- that doesn't install the authorized keys for the other projects? | 20:05 |
| clarkb | ianw: it should install them. They are listed for the user in the all.yaml group var file | 20:06 |
| ianw | system-config : inventory/service/group_vars/all.yaml ? | 20:08 |
| ianw | oh i see | 20:09 |
| clarkb | ya, extra_users indicates which users listed in that file should be added | 20:10 |
| ianw | ok, i missed that path | 20:11 |
| clarkb | I think the chicken and egg here is once you've booted a new server how do you add the zuul user to it since we don't normall do that. Maybe this hsould be a flag to launch node instead | 20:11 |
| clarkb | and have base install it normally rathe rthan something we do in that chnage? | 20:11 |
| ianw | yeah i will document this in the "how to replace bastion host" | 20:12 |
| ianw | the launch node runs base manually, maybe? | 20:13 |
| ianw | the problem might be that the new host isn't in the bastion group | 20:14 |
| clarkb | yes | 20:16 |
| clarkb | so the normal user setup happens which includes our users, but it won't do host specific extra users | 20:16 |
| clarkb | but maybe we can set a flag for that as part of launch node? like setting a group | 20:16 |
| clarkb | then still only run the base playbook, but it can preapply things like that (I think iptables would be the other big one?) | 20:17 |
| ianw | thanks, i'll flip that to WIP and look into that, i'm sure we can figure something there | 20:20 |
| ianw | https://review.opendev.org/c/opendev/system-config/+/862762/4 and https://review.opendev.org/c/opendev/system-config/+/862764/7 to add the host keys automatically i think are orthogonal to that, and hopefully useful | 20:20 |
| fungi | i think i fell asleep while the nodepool zk config problem was being ironed out... is that solved now or should i review some changes? | 20:22 |
| ianw | fungi: i don't think there's anything more -- i migrated the old keys over and restarted nodepool and i believe that was it. also added a note to https://etherpad.opendev.org/p/bastion-upgrade-nodes-2022-10 so i can write up something that doesn't forget it for next time | 20:23 |
| fungi | oh, i meant the replacement for the ansible module. i saw you did something purely with templating/var expansion but i was losing coherency before test results came back | 20:25 |
| ianw | fungi: oh, that -- that is https://review.opendev.org/c/opendev/system-config/+/862759 | 20:27 |
| ianw | that seems to have reviews, thank you. i think it should be safe to merge, but low priority, it's just slightly slow | 20:28 |
| fungi | thanks, so nodepool deployments are still broken without that, right? | 20:30 |
| fungi | oh, the extra binary copy of the zk keys was the underlying issue | 20:31 |
| fungi | now i remember | 20:31 |
| fungi | anyway, approved that module replacement. it seems like a net win all around | 20:33 |
| fungi | many times faster with much less code | 20:33 |
| ianw | fungi: yes, to be clear, that's correct. the thing that was breaking was that we had (unused) keys in our private ansible with !!binary which couldn't be serialised to json. the theory is that <python3.10 was for some reason less picky about it | 20:36 |
| *** dviroel|afk is now known as dviroel | 20:46 | |
| opendevreview | Merged opendev/system-config master: nodepool-base: don't call out to find zk_hosts https://review.opendev.org/c/opendev/system-config/+/862759 | 21:05 |
| *** rlandy is now known as rlandy|bbl | 21:12 | |
| *** dviroel is now known as dviroel|out | 22:10 | |
| clarkb | fungi: or ianw any interest in python3.11 docker base images ? https://review.opendev.org/c/opendev/system-config/+/862631 | 22:29 |
| fungi | oh, right i had that up earlier and then got distracted | 22:30 |
| ianw | lgtm | 22:30 |
| clarkb | thanks! | 22:30 |
| fungi | i could have sworn i reviewed that earlier, but anyway +3 | 22:30 |
| fungi | i probably just got distracted by something in between looking through the diff and actually voting on it | 22:31 |
| clarkb | once that is in I can update the zuul and nodepool changes to switch most of their testing over to 3.11 and then zuul can decide if that is worthwhile (I like zuul being able to show off how good CI/CD enable changes like that | 22:31 |
| fungi | thanks | 22:32 |
| clarkb | on the gitea side of things I was going to look into their new proxy protocol support. Has anyone set that up before and if so any idea if apache can speak it too? | 22:32 |
| clarkb | (I think the proxy protocol may give us the logs we want of being abel to trace requests through the proxies properly) | 22:33 |
| fungi | we have it set up between haproxy and apache2, right? | 22:34 |
| fungi | i thought i remembered discussing it anyway, where proxy info is embedded into the socket setup | 22:34 |
| clarkb | I don't think we're doing the PROXY protocol | 22:35 |
| clarkb | its really only interesting because gitea's xforwarded for handling is broken | 22:37 |
| clarkb | maybe it is best to just wait for or try to get that fixed instead | 22:37 |
| clarkb | kevinz: hey, it looks like we're still getting alerts that the cloud ssl cert will expire in a few days | 22:40 |
| clarkb | another thing that just occured to me is adding that may cause direct requests to the backend to not work the way we expect | 22:45 |
| opendevreview | Ian Wienand proposed opendev/system-config master: inventory: add host keys https://review.opendev.org/c/opendev/system-config/+/862762 | 23:09 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bastion host: add global known_hosts values https://review.opendev.org/c/opendev/system-config/+/862764 | 23:09 |
| opendevreview | Merged opendev/system-config master: Add python 3.11 docker images https://review.opendev.org/c/opendev/system-config/+/862631 | 23:11 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!