| opendevreview | Merged openstack/project-config master: linaro-us : set max-servers to 0 https://review.opendev.org/c/openstack/project-config/+/870876 | 00:00 |
|---|---|---|
| clarkb | it does look like networking to that mirror node is unhappy for some reason | 00:01 |
| ianw | rhrm | 00:06 |
| ianw | nothing interesting on the console log | 00:07 |
| ianw | PING 147.28.149.111 (147.28.149.111) 56(84) bytes of data. | 00:09 |
| ianw | From 147.75.47.67 icmp_seq=2 Redirect Host(New nexthop: 147.28.149.111) | 00:09 |
| ianw | interesting response | 00:09 |
| fungi | that does look like a routing issue | 00:18 |
| fungi | is that the new environment or the old one? | 00:18 |
| ianw | that's the new one | 00:23 |
| ianw | one tangential thing i've noticed is that css is broken on older lists.openstack.org archives -> e.g. https://lists.openstack.org/pipermail/openstack-dev/2017-August/thread.html | 00:23 |
| ianw | the css links are http, which my firefox refuses to load from the https site | 00:23 |
| ianw | anyway the instanance shows a power state of "paused" ... but i don't think it is | 00:24 |
| ianw | OS-EXT-STS:power_state | Paused | 00:30 |
| ianw | OS-EXT-STS:vm_state | active | 00:30 |
| ianw | it's both active and paused | 00:30 |
| ianw | kevinz: ^ if around | 00:31 |
| ianw | fungi: do you see that? I only seem to see that icmp redirect if i ping from RAX | 00:36 |
| ianw | I have rebooted the server, and it is now active. i don't know what happened to it :/ | 00:37 |
| *** rlandy is now known as rlandy|out | 00:45 | |
| ianw | there seems to be plenty of arm64 jobs queued, but i can't see that nodepool is choosing to try and start nodes on the new cloud .... | 00:51 |
| Clark[m] | Does grafana show errors? | 01:01 |
| opendevreview | Ian Wienand proposed opendev/system-config master: make-tarball: role to archive directories https://review.opendev.org/c/opendev/system-config/+/865784 | 01:04 |
| opendevreview | Ian Wienand proposed opendev/system-config master: tools/make-backup-key.sh https://review.opendev.org/c/opendev/system-config/+/866430 | 01:04 |
| opendevreview | Ian Wienand proposed opendev/system-config master: bridge: Install common key and test tarball artifact generation https://review.opendev.org/c/opendev/system-config/+/866432 | 01:04 |
| ianw | Clark[m]: it doesn't seem to get that far even | 01:09 |
| ianw | 2023-01-18 01:05:06,011 INFO nodepool.PoolWorker.linaro-regionone-main: [e: 2686ac2061784e33b808c4385fa5e9f3] [node_request: 300-0020204239] Declining node request | 01:09 |
| ianw | 'node_types': ['ubuntu-jammy-arm64', 'ubuntu-bionic-arm64', 'ubuntu-focal-arm64', 'ubuntu-jammy-arm64'] | 01:10 |
| ianw | this was a request for some nodes it should have been able to provide | 01:10 |
| ianw | ohhhh, i bet this is a quota issue | 01:16 |
| *** dasm is now known as dasm|off | 01:24 | |
| fungi | ianw: whether you see nexthop responses is going to depend on what icmp types your firewall allows to arrive | 01:36 |
| fungi | or whether they're being filtered upstream of you | 01:37 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Switch Gerrit to Java 17 https://review.opendev.org/c/opendev/system-config/+/870877 | 03:03 |
| *** yadnesh|away is now known as yadnesh | 04:43 | |
| opendevreview | Daniel Blixt proposed zuul/zuul-jobs master: Reset connection before testing build ssh-keys https://review.opendev.org/c/zuul/zuul-jobs/+/788826 | 07:26 |
| opendevreview | Daniel Blixt proposed zuul/zuul-jobs master: Allow mirror push to delete current branch https://review.opendev.org/c/zuul/zuul-jobs/+/764152 | 07:37 |
| opendevreview | Daniel Blixt proposed zuul/zuul-jobs master: Make build-sshkey handling windows compatible https://review.opendev.org/c/zuul/zuul-jobs/+/780662 | 07:39 |
| *** jpena|off is now known as jpena | 08:21 | |
| *** yadnesh is now known as yadnesh|afk | 08:26 | |
| *** ysandeep is now known as ysandeep|lunch | 08:44 | |
| *** soniya29|rover is now known as soniya29|rover|lunch | 08:55 | |
| *** soniya29|rover|lunch is now known as soniya29|rover | 10:06 | |
| *** ysandeep|lunch is now known as ysandeep | 10:36 | |
| *** rlandy|out is now known as rlandy | 11:11 | |
| *** dviroel|afk is now known as dviroel | 11:19 | |
| *** dviroel|afk is now known as dviroel | 11:54 | |
| *** ysandeep is now known as ysandeep|brb | 12:33 | |
| *** ysandeep|brb is now known as ysandeep | 12:45 | |
| dpawlik | fungi, clarkb o/ | 13:01 |
| dpawlik | seems that Facebook fedora mirror is not working as expected | 13:02 |
| dpawlik | https://mirror.facebook.net/fedora/linux/releases/35/ is empty | 13:02 |
| dpawlik | http://mirror.dfw.rax.opendev.org/fedora/releases/35/ | 13:02 |
| dpawlik | they have move it to http://archives.fedoraproject.org/pub/archive/fedora/ ... | 13:03 |
| *** gthiemon1e is now known as gthiemonge | 13:09 | |
| fungi | dpawlik: i think that's a fedora change, not facebook? | 13:15 |
| fungi | fedora periodically "archives" their old releases | 13:15 |
| fungi | probably means we're behind in moving our fedora images to a newer version | 13:15 |
| Tengu | ah, yeah, 35 is eol | 13:16 |
| dpawlik | yeah, that's it | 13:16 |
| dpawlik | did not read the README | 13:17 |
| dpawlik | thanks fungi | 13:17 |
| fungi | yw | 13:17 |
| *** soniya29|rover is now known as soniya29|rover|brb | 13:20 | |
| *** dasm|off is now known as dasm | 13:34 | |
| *** ysandeep is now known as ysandeep|afk | 14:54 | |
| *** dviroel is now known as dviroel|lunch | 15:32 | |
| *** dviroel|lunch is now known as dviroel | 16:23 | |
| clarkb | fungi: frickler: https://review.opendev.org/c/opendev/system-config/+/869763 is an easy one to drop gerrit 3.5 images | 16:41 |
| clarkb | frickler: and then https://review.opendev.org/c/opendev/system-config/+/870118/ should fix our 3.6 images (they can land in either order though removing 3.5 first runs fewer jobs in total) | 16:42 |
| fungi | yeah, i'm stepping through that series now. thanks! | 16:43 |
| clarkb | with those two changes in I might end up merging my add 3.7 images stack and java 17 stack. I think we should probably move to java 17 first? | 16:43 |
| clarkb | *with those two changes landed | 16:43 |
| clarkb | actually I guess the order there doesn't matter a whole lot. We can add 3.7 then java 17 and it will just apply to both 3.6 and 3.7 | 16:44 |
| fungi | i approved the 3.7 images change too. should i wait on the upgrade job or will it complicate the java 17 work? | 16:45 |
| amorin | hello opendev team, we are struggling with a test on mistral, about dnspython, we are not sure how to fix it, would you mind giving us a hint to follow good rules? | 16:46 |
| amorin | https://review.opendev.org/c/openstack/mistral/+/870840 | 16:46 |
| amorin | this is what we proposed so far, but we have the feeling it's wrong | 16:46 |
| clarkb | fungi: I don't think it will. since both 3.6 and 3.7 can do java 17 | 16:46 |
| fungi | amorin: i have a feeling it will be very similar to the one swift had to fix yesterday. will look as soon as the tc meeting wraps up | 16:46 |
| amorin | ack, thanks! | 16:46 |
| clarkb | fungi: I also had to implement a hack to make java 17 work due to https://bugs.chromium.org/p/gerrit/issues/detail?id=15994 its interesting thatthey say they fully support java 17 in 3.6 then this happens | 16:47 |
| fungi | amorin: in fact, it's identical. right down to only showing up in the docs builds | 16:47 |
| amorin | yup | 16:48 |
| fungi | the problem is that testenv:docs installs the documentation build requirements with constraints, but tox is also installing mistral's dependencies in a separate step and no constraints are app | 16:48 |
| fungi | lied | 16:48 |
| amorin | yes, a fix in tox.ini maybe? | 16:49 |
| fungi | yes, exactly, the solution the swift team went with was to add an explicit requirements.txt as well in the deps list for testenv:docs | 16:49 |
| amorin | ok, will check swift gerrit logs to figure that out | 16:50 |
| fungi | i'll find you a link to their change | 16:50 |
| amorin | https://review.opendev.org/c/openstack/swift/+/870783 | 16:50 |
| amorin | maybe | 16:50 |
| fungi | amorin: https://review.opendev.org/c/openstack/swift/+/870853 | 16:50 |
| fungi | yes that's the one | 16:50 |
| amorin | thanks! | 16:51 |
| opendevreview | Corey Bryant proposed zuul/zuul-jobs master: Adapt tox_extra_args to tox 4.x https://review.opendev.org/c/zuul/zuul-jobs/+/870978 | 16:57 |
| opendevreview | Merged opendev/system-config master: Remove Gerrit 3.5 images https://review.opendev.org/c/opendev/system-config/+/869763 | 17:07 |
| *** marios is now known as marios|out | 17:08 | |
| opendevreview | Merged opendev/system-config master: Convert Verified MaxWithBlock to submit-requirement in testing https://review.opendev.org/c/opendev/system-config/+/870109 | 17:12 |
| *** yadnesh|afk is now known as yadnesh|away | 17:13 | |
| frickler | amorin: fyi seems eventlet has released a fixed version and I've proposed an u-c bump now https://review.opendev.org/c/openstack/requirements/+/870988 . fixing your doc builds still is a good thing | 17:25 |
| *** gmann is now known as gmann_afk | 17:29 | |
| *** rlandy is now known as rlandy|laptop_return | 17:38 | |
| *** gmann_afk is now known as gmann | 17:41 | |
| fungi | yeah, there's basically two issues there. one is the dnspython+eventlet incompatibility, but the other is that the docs builds for some projects weren't correctly applying constraints | 17:41 |
| *** jpena is now known as jpena|off | 17:42 | |
| opendevreview | Merged opendev/system-config master: Add Gerrit 3.7 images https://review.opendev.org/c/opendev/system-config/+/869765 | 17:43 |
| *** gmann is now known as gmann_afk | 18:06 | |
| *** rlandy|laptop_return is now known as rlandy | 18:32 | |
| *** gmann_afk is now known as gmann | 18:56 | |
| amorin | ack frickler, thanks for info, we applied the patch like swift team and that worked | 19:38 |
| *** prometheanfire is now known as Guest1769 | 20:12 | |
| *** dviroel is now known as dviroel|afk | 21:09 | |
| *** Guest1769 is now known as prometheanfire | 21:24 | |
| ianw | ok, nl03 is now reporting https://keyoxide.org/FE3366210A6D0989C247FD6EB6A6FB8B9615AEC8 | 22:04 |
| ianw | bah, it is not, that is my copy buffer | 22:04 |
| ianw | it is reporting "Compute service reports fault: No valid host was found." | 22:05 |
| clarkb | this is trying to boot in the new cloud? | 22:06 |
| ianw | yeah | 22:06 |
| clarkb | could be a placement issue if things are not accounted for properly basically it thinks there is no room when there is ? | 22:06 |
| ianw | yeah it seems hard to diagnose from this side. or we're somehow asking for something it doesn't have | 22:07 |
| ianw | i can start a m1.medium with a standard image | 22:16 |
| ianw | and we do have opendev-control node running ... the common theme i'm sensing here here is that both don't have ephemeral disk | 22:17 |
| ianw | while the ci node flavor does | 22:17 |
| ianw | ok, if i try to boot an image with 8cpu/8gb/no ephemeral i get a different error | 22:39 |
| ianw | Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance | 22:39 |
| ianw | ... but if i do the same with the plain ubuntu-22.04 image i *don't* get a failure | 22:41 |
| ianw | ... so something about the nodepool uploaded images | 22:42 |
| ianw | ubuntu-22.04 is raw, and we're uploading .qcow2. i wonder if we want raw | 22:54 |
| ianw | i think what i need to do here is shutdown nb03 and let nb04 upload raw images. we already upload raw to osuosl | 23:08 |
| ianw | shutting down nb03 is just so that i don't have to worry about anything parallel uploading | 23:09 |
| *** rlandy is now known as rlandy|out | 23:19 | |
| ianw | ... interesting, nb03 has shutoff itself anyway, along with the mirror | 23:21 |
| fungi | how... fun | 23:21 |
| ianw | i asked kevinz to cleanup some leaked hosts there, which he reported he did, i imagine that had something to do with it | 23:22 |
| ianw | i've put both nb03 and the linaro-us mirror in emergency. the region is already turned off with max-nodes: 0 from yesterday | 23:22 |
| opendevreview | Ian Wienand proposed openstack/project-config master: nb04: use linaro region mirror https://review.opendev.org/c/openstack/project-config/+/871006 | 23:28 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Update git in gitea images https://review.opendev.org/c/opendev/system-config/+/871009 | 23:53 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!