| ianw | #status log redirected old jjb at docs.o.o/infra/jenkins-job-builder to RTD site; moved old content to ./attic/* and added a .htaccess and README in the original dir. see https://groups.google.com/g/jenkins-job-builder/c/U6VL3_ajoMA/m/SlYzxDJcAwAJ | 00:11 |
|---|---|---|
| opendevstatus | ianw: finished logging | 00:11 |
| ianw | AdamR: ^ i replied on list, but let me know if issues | 00:11 |
| opendevreview | Merged openstack/project-config master: Add Zuul performance metrics dashboard https://review.opendev.org/c/openstack/project-config/+/831071 | 00:23 |
| fungi | ianw: if you're good with 830912 i can do the extra steps it needs in a gerrit restart once it's merged/published | 01:39 |
| ianw | fungi: oh, sorry i thought it was in! | 02:10 |
| ianw | i can clean those up when it merges too if you like | 02:10 |
| fungi | oh, if you're up for it that'd be great. it's getting fairly late here so i'm less sure i'll be awake once the images update | 02:11 |
| fungi | per the commit message, the logfiles have to be removed/moved while the service is offline | 02:12 |
| ianw | ++ | 02:13 |
| ianw | we moved the periodic jobs forward, i guess most people don't care but it does slow things down in the pacific afternoon | 02:25 |
| ianw | i feel like two or so hours from now might be even quieter | 02:26 |
| ianw | a constant battle with daylight savings times | 02:26 |
| fungi | sounds fine to me. i'll try to check on it first thing when i wake up in that case | 02:28 |
| opendevreview | Merged opendev/system-config master: Remove Gerrit's JVM GC logs https://review.opendev.org/c/opendev/system-config/+/830912 | 03:28 |
| ianw | still note seeing ^ on hub.docker | 03:44 |
| opendevreview | Ian Wienand proposed zuul/zuul-jobs master: ensure-pip: fix typo in ensure_pip_virtualenv_command documentation https://review.opendev.org/c/zuul/zuul-jobs/+/831136 | 03:53 |
| ianw | oh doh; 830912 doesn't need to push an image. i can just quickly restart per the instructions now | 05:07 |
| ianw | #status log stopped gerrit, moved jvm log files to ~gerrit2/tmp/jvm-logs, restarted gerrit to apply 830912 | 05:11 |
| opendevstatus | ianw: finished logging | 05:11 |
| ianw | not seeing the errors in logs | 05:14 |
| *** ysandeep|out is now known as ysandeep | 05:59 | |
| *** ykarel_ is now known as ykarel | 06:23 | |
| *** luigi is now known as luigi-training | 06:39 | |
| *** luigi-training is now known as luigi | 06:39 | |
| *** ysandeep is now known as ysandeep|lunch | 07:18 | |
| *** amoralej|off is now known as amoralej | 07:24 | |
| *** bhagyashris_ is now known as bhagyashris|ruck | 07:56 | |
| *** jpena|off is now known as jpena | 09:42 | |
| *** ysandeep|lunch is now known as ysandeep | 09:53 | |
| *** rlandy is now known as rlandy|ruck | 11:15 | |
| *** sshnaidm|off is now known as sshnaidm | 11:38 | |
| fungi | yeah, sorry, i meant deployed, since it was a change to the docker-compose config | 13:13 |
| fungi | and yes it looks like the jvm_gc.log has not returned either | 13:19 |
| fungi | and gerrit is still running steadily as far as I can see | 13:20 |
| opendevreview | Maksim Malchuk proposed openstack/diskimage-builder master: Correctly create DIB_ENV variable and dib_environment file https://review.opendev.org/c/openstack/diskimage-builder/+/828899 | 14:26 |
| *** ysandeep is now known as ysandeep|dinner | 14:43 | |
| *** rlandy|ruck is now known as rlandy|ruck|mtg | 14:54 | |
| *** ysandeep|dinner is now known as ysandeep | 15:12 | |
| diablo_rojo | Looks like ethercalc is maybe down? | 15:14 |
| diablo_rojo | Was going to look at the PTG ethercalc: https://ethercalc.openstack.org/7yxdas7suqnd | 15:15 |
| fungi | thanks, checking | 15:28 |
| fungi | /opt/ethercalc/node_modules/redis/index.js:602 | 15:31 |
| fungi | Error: Can't set headers after they are sent. | 15:31 |
| fungi | i don't think that's the issue we've seen in the past | 15:32 |
| fungi | #status log Started the Ethercalc service after it crashed at 11:29:07 UTC | 15:34 |
| opendevstatus | fungi: finished logging | 15:34 |
| fungi | diablo_rojo: ^ | 15:34 |
| fungi | looks like it was down for roughly 4 hours | 15:34 |
| fungi | https://github.com/audreyt/ethercalc/issues/626 | 15:36 |
| fungi | maybe that is the one we keep running into after all | 15:37 |
| clarkb | fungi: ianw: so the mergability chcek change is in as is the jvm gc log cleanup to remove the errors from our logs? Did the apache update manage to get in somehow? I was going to look at the zuul registry more closely today once caught up generally | 16:17 |
| fungi | clarkb: the first two yes, the apache vhost config update to block access to gitiles has not merged yet since it's still blocked on the registry bug, but also not terribly urgent | 16:18 |
| clarkb | fungi: ok, I don't know if you saw but the testinfra test is why we rebuild the image. We could either stop running testinfra on image updates (I think taht is a bad idea since new images should get testinfra'd) or split the testing from the change if we like. Based on what I found Friday I'm not so sure the registry will be a quick and easy fix | 16:19 |
| fungi | yeah, i don't mind splitting the test to a second change, but the test is testing the adjusted config works, so i'm also fine with trying to address the registry bug first | 16:21 |
| clarkb | ok. Once I'ev settled in a bit and caught up on some reviews and other stuff (gitea fixed our bug I need to check if that is in 1.16.2 for example) I'll look at the registry and see if I can spy why two uploads of the same object is problematic in the registry | 16:22 |
| clarkb | Looks like the gitea bugfix is on 1.16.3. I'm happy to wait a bit logner and upgrade to that version if it shows up soon. Looks like it has a few other bugfixes too | 16:30 |
| *** rlandy|ruck|mtg is now known as rlandy|ruck | 16:31 | |
| fungi | yeah, that seems prudent and saves us some work | 16:35 |
| opendevreview | James E. Blair proposed openstack/project-config master: Correct typo in zuul-performance dashboard https://review.opendev.org/c/openstack/project-config/+/831232 | 16:53 |
| corvus | clarkb: fungi ^ that's a trivial fix if you have a sec | 16:55 |
| clarkb | done | 16:59 |
| *** ysandeep is now known as ysandeep|out | 16:59 | |
| clarkb | I think I understand the zuul-registry bug | 17:03 |
| clarkb | the paste I made last week was on the right track but for the wrong object. Bsaically what happens is if a second client pushes the same object while the other one is writing the object we return to the client and say it is done. It immediately does a HEAD against the object. Before my atomic move change this could be a short read depending on how much had been written. After the | 17:05 |
| clarkb | atomic move it was always a 404 | 17:05 |
| clarkb | I'll work on a fix. Though I'm not sure what the correct fix is yet. I think with the atomic moves we can stop returning early and just have each client move the file in place atomically and avoid the race entirely | 17:05 |
| clarkb | but I need to update my change to not use a consistent tmpname | 17:06 |
| opendevreview | Merged openstack/project-config master: Correct typo in zuul-performance dashboard https://review.opendev.org/c/openstack/project-config/+/831232 | 17:12 |
| *** jpena is now known as jpena|off | 17:31 | |
| clarkb | https://review.opendev.org/c/zuul/zuul-registry/+/831235 here's a first draft of the zuul-registry fixup | 17:36 |
| clarkb | I've managed to skip breakfast. I'm going to go eat then likely dig into reviews. Looks like there are a few in zuul land that need attention | 17:57 |
| diablo_rojo | fungi, thanks for looking into it! Can confirm its back up now. | 17:59 |
| opendevreview | Clark Boylan proposed zuul/zuul-jobs master: DNM testing depends-on https://review.opendev.org/c/zuul/zuul-jobs/+/831243 | 18:28 |
| *** artom__ is now known as artom | 19:02 | |
| opendevreview | Merged openstack/diskimage-builder master: Revert "Revert "Detect boot and EFI partitions in extract-image"" https://review.opendev.org/c/openstack/diskimage-builder/+/830900 | 22:07 |
| opendevreview | Clark Boylan proposed zuul/zuul-jobs master: DNM testing depends-on https://review.opendev.org/c/zuul/zuul-jobs/+/831243 | 22:22 |
| *** rlandy|ruck is now known as rlandy|ruck|biab | 22:51 | |
| *** rlandy|ruck|biab is now known as rlandy|ruck | 23:13 | |
| ianw | is it just me or is element/matrix not happy? | 23:14 |
| clarkb | ianw: seems to be working for me | 23:24 |
| clarkb | though I've not tried to send messages for a bit | 23:24 |
| clarkb | ianw: once https://review.opendev.org/c/zuul/nodepool/+/831108 is in we should expect happy rocky? | 23:38 |
| ianw | clarkb: oh, that hit so many transient issues yesteday :/ | 23:39 |
| ianw | if it's gone in, i think we can expect one less problem with rocky -- and hopefully it only has one problem remaining :) | 23:39 |
| clarkb | no the nodepool update hasn't happened yet. But sounds like that is the next step at least | 23:40 |
| clarkb | I'm putting together a meeting agenda tomorrow and that was one of the thigns to catch up on. Please update the wiki if I've missed anything important for tomorrow | 23:41 |
| clarkb | I'll try to send that out in about 20 minutes or so | 23:42 |
| fungi | i'm not aware of anything that needs adding | 23:42 |
| clarkb | ianw: there asn't any non rocky fallout from the glean release was there? | 23:43 |
| clarkb | looks like no recent glean changes at least if s | 23:43 |
| ianw | i don't believe so ... i watches some builds and haven't seen reports of failures to boot, etc. | 23:44 |
| clarkb | thanks! I meant to followup on that and got sniped by the registry | 23:44 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!