| opendevreview | Ian Wienand proposed opendev/system-config master: gitea: set custom avatars for orgs https://review.opendev.org/c/opendev/system-config/+/834085 | 00:06 |
|---|---|---|
| ianw | acessing the gitea api reminded me of https://review.opendev.org/c/opendev/system-config/+/782887 which switches our ansible things to use token auth | 00:23 |
| ianw | not sure if that's worth restoring? it was to avoid problems with the hashing algorithm that i think upstream reverted. anyone think of other advantages? | 00:24 |
| Clark[m] | Ya upstream reverted iirc and it was to make auth quicker and use less memory. I don't think it is necessary due to upstream's pivot | 00:53 |
| opendevreview | Merged opendev/system-config master: Test gitea 1.16 partial clones https://review.opendev.org/c/opendev/system-config/+/834187 | 01:02 |
| ianw | logos! https://751e8ffd5efe929acb06-2d54b4071a8d065f516b028813c4bd96.ssl.cf1.rackcdn.com/834085/14/check/system-config-run-gitea/f813d1d/bridge.openstack.org/screenshots/gitea-org-expore.png | 01:38 |
| ianw | our ci system really is something :) | 01:38 |
| mnaser | ianw: aw man we want a logo too now :p | 02:34 |
| fungi | very nice! | 02:39 |
| fungi | mnaser: nobody has a logo up yet, but 834085 should make that possible | 02:39 |
| ianw | mnaser: i did a vexxhost logo :) | 02:51 |
| mnaser | oh no way, that's awesome | 02:51 |
| ianw | https://review.opendev.org/c/opendev/system-config/+/834085/14/playbooks/roles/gitea-set-org-logos/files/vexxhost.png | 02:51 |
| ianw | if it should be something else, feel free to update | 02:52 |
| mnaser | thats more than ok :) | 02:52 |
| *** arxcruz is now known as arxcruz|rover | 07:47 | |
| *** jpena|off is now known as jpena | 07:48 | |
| *** arxcruz|rover is now known as arxcruz|ruck | 09:00 | |
| *** rlandy|PTO is now known as rlandy | 10:26 | |
| *** dviroel|afk is now known as dviroel | 10:58 | |
| priteau | Good morning. Sorry, I have another issue today :) The cloudkitty-dashboard release notes are not getting published. | 11:02 |
| priteau | See https://docs.openstack.org/releasenotes/cloudkitty-dashboard/, yoga is missing although we merged the patch a while ago | 11:03 |
| priteau | I've also just merged https://review.opendev.org/c/openstack/cloudkitty-dashboard/+/834272 which should have changed the title | 11:03 |
| priteau | promote-openstack-releasenotes https://zuul.opendev.org/t/openstack/build/ca7f765f43bf4f44bee7c6bc7c89b5ae : SUCCESS in 1m 01s | 11:04 |
| priteau | Maybe I should ask in #openstack-infra instead? | 11:05 |
| frickler | priteau: asking here is fine, I can take a look after my current meeting | 11:15 |
| priteau | Thanks frickler | 11:18 |
| frickler | priteau: I do see https://docs.openstack.org/releasenotes/cloudkitty-dashboard/yoga.html but it is empty, do you expect actual content to be there? | 12:14 |
| priteau | I swear it wasn't there earlier | 12:15 |
| priteau | The index page has updated too | 12:15 |
| frickler | maybe it just took a bit to publish after that last promote job? or browser cache? | 12:15 |
| priteau | And yes, I am expecting it to be empty | 12:15 |
| priteau | I tried in an incognito window, maybe this still uses browser cache | 12:16 |
| fungi | priteau: frickler: note that there is a delay between when documentation files are written to the read-write volume in afs and when that update is synchronized to the read-only replicas like the one the docs site is served from. normally we run a `vos release` every 5 minutes on a timer out of cron, but that's a serialized set of calls for all the static afs volumes and some of them (e.g. the | 12:23 |
| fungi | tarballs volume) can have large updates which take longer to copy across the network so can in some rare cases delay publication of updates for other volumes by an hour or more | 12:23 |
| priteau | ok, I must have check right after the promote job finished | 12:23 |
| priteau | Still doesn't explain why the initial reno update didn't publish, its promote job was successful: https://review.opendev.org/c/openstack/cloudkitty-dashboard/+/832582 | 12:24 |
| fungi | yeah, generally expect uploads from a promote job to be reflected on the docs site within 5-10 minutes, but longer delays are possible. if it's more than an hour, definitely reach out because something might be stuck (it happens, because clouds... like an unexpected hypervisor host reboot impacting an afs server could leave some volumes in a dirty state) | 12:25 |
| priteau | Anyway, problem solved for now, thanks frickler and fungi | 12:25 |
| fungi | priteau: was the page a 404, or just didn't have any notes in it? | 12:35 |
| fungi | priteau: the promote job uploaded the release notes built in the gate job, which you can see a preview of here: https://80d2536878f1bc1b0f54-21add993a714f438a33591436aac6270.ssl.cf1.rackcdn.com/832582/1/gate/build-openstack-releasenotes/3c92081/docs/yoga.html | 12:36 |
| fungi | and that's empty | 12:36 |
| fungi | so maybe your question is more why 832582 built an empty set of release notes? | 12:40 |
| *** dhill is now known as Guest2481 | 12:50 | |
| *** TheJulia is now known as needsleep | 12:57 | |
| *** needsleep is now known as TheJulia | 12:57 | |
| priteau | fungi: the empty yoga page is expected, there were no changes in the release | 13:01 |
| priteau | But the notes built by 832582 were not published, yoga.html was a 404 until this morning when I merged another patch to publish again | 13:02 |
| priteau | You can see it fail on this releases patch: https://review.opendev.org/c/openstack/releases/+/834268, https://5edd5383f0a935c91c95-83c6da06e0a1e2eb40957e6f08135c7a.ssl.cf5.rackcdn.com/834268/2/check/openstack-tox-validate/5280f85/job-output.txt | 13:03 |
| priteau | Could not fetch release notes page https://docs.openstack.org/releasenotes/cloudkitty-dashboard/yoga.html: 404 | 13:03 |
| fungi | yeah, that's weird. the promote job claims to have uploaded it to days before that 404 was logged | 13:17 |
| fungi | er, ten days before | 13:17 |
| priteau | When I checked the release notes this morning, the timestamp at the bottom was from 2021 | 13:27 |
| priteau | The upload must have failed silently somehow | 13:27 |
| fungi | doesn't look like it from the promote build log | 13:35 |
| fungi | https://zuul.opendev.org/t/openstack/build/980b657afcce4475ba4dda9e78a62148/console#1/0/28/localhost | 13:35 |
| fungi | that has the list of files copied to /afs/.openstack.org/docs/releasenotes/cloudkitty-dashboard and includes "<<CHANGED>>>f+++++++++ yoga.html" | 13:36 |
| fungi | the only think i can think of is that some other build came along after it and removed that file (maybe building from an earlier repository state?) | 13:37 |
| fungi | looking at https://zuul.opendev.org/t/openstack/builds?job_name=promote-openstack-releasenotes&project=openstack/cloudkitty-dashboard there were two builds with identical start times that day, one for master and one for stable/yoga | 13:38 |
| *** TheJulia is now known as needssleep | 13:38 | |
| fungi | i wonder if the https://review.opendev.org/832581 promote job uploaded moments after and removed yoga.html because that wasn't included in its gate build | 13:39 |
| fungi | https://zuul.opendev.org/t/openstack/build/18d0ea7d4c2a4be9be5e0a52dfcc5fbc/console#1/0/28/localhost | 13:40 |
| fungi | "<<CHANGED>>*deleting yoga.html" | 13:41 |
| fungi | priteau: mystery solved | 13:41 |
| fungi | we probably need to think about how to address races between these sorts of publication jobs | 13:42 |
| *** sboyron_ is now known as sboyron | 13:43 | |
| *** sboyron is now known as Guest2485 | 13:43 | |
| *** Guest2485 is now known as sboyron_ | 13:44 | |
| opendevreview | Takashi Kajinami proposed openstack/project-config master: Add puppet-manila-core https://review.opendev.org/c/openstack/project-config/+/834318 | 14:17 |
| *** dviroel is now known as dviroel|lunch | 15:23 | |
| *** dviroel|lunch is now known as dviroel | 16:10 | |
| clarkb | infra-root https://review.opendev.org/c/opendev/system-config/+/821780 is a fairly straightforward review that improves our testing of firewall rules | 16:11 |
| * clarkb is going through system-config change sagain to try and prune that list furhter | 16:12 | |
| corvus | the rolling restart of executors completed yesterday; i plan on rolling the schedulers and web servers later today | 16:15 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Finish removing git.openstack.org references https://review.opendev.org/c/opendev/system-config/+/654040 | 16:23 |
| gmann | fungi: frickler clarkb fixed config error in legacy job file https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/834331/1 | 16:26 |
| gmann | and l-c job moving to focal https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/834179/2 | 16:27 |
| *** jpena is now known as jpena|off | 16:48 | |
| *** marios is now known as marios|out | 16:49 | |
| opendevreview | Merged opendev/system-config master: Add firewall behavior assertions to testinfra testing https://review.opendev.org/c/opendev/system-config/+/821780 | 17:12 |
| clarkb | thanks! | 17:12 |
| clarkb | we're down to like 165 open changes now | 17:12 |
| fungi | that's remarkable | 17:13 |
| fungi | down by an order of magnitude, i'm betting | 17:13 |
| clarkb | I think we were around 300 when this started | 17:13 |
| fungi | okay, so not an order of magnitude, but down by almost half anyway | 17:15 |
| clarkb | we're definitely hitting the set that is more difficult to wade through. Having to do a bit more investigation for each one that I think is abandonable and half of them I've left open as they represent some value | 17:15 |
| fungi | i had just assumed we had far more open system-config changes than we really did | 17:16 |
| fungi | it was more than my watership down counting threshold anyway | 17:16 |
| clarkb | fungi: any thought son whether https://review.opendev.org/c/opendev/system-config/+/360710 should stay open in the hopes of it getting reimplemented in ansible? I don't know that we see connection pressure against gerrit over http today so this is probably fine to remove? | 17:17 |
| fungi | mmm, yeah, i pushed that in 2016. i think it's safe to say if we needed it we would have done it in the intervening 6 years. i'll abandon that | 17:19 |
| clarkb | topic:system-config-cleanup has a number of change sthat I evaluated as being worthwhile and I either reabsed them or rewrote them against the current state of things too | 17:21 |
| clarkb | I don't think any of them are particularly urgent though as they have been hanging out for a while | 17:21 |
| fungi | we did merge my thread tuning change for gerrit around the same time, so apparently that was sufficient for whatever slowdowns we were experiencing at the time | 17:21 |
| clarkb | Does anyone know what the motiviation for https://review.opendev.org/c/opendev/system-config/+/641079 and https://review.opendev.org/c/opendev/system-config/+/641073 was? That log file doesn't seem to exist in the current containers | 17:39 |
| clarkb | mordred: ^ do you recall | 17:39 |
| opendevreview | Jeremy Stanley proposed opendev/system-config master: Add (lack of) license information for org logos https://review.opendev.org/c/opendev/system-config/+/834343 | 17:46 |
| *** dviroel_ is now known as dviroel | 17:46 | |
| clarkb | fungi: https://gerrit-review.googlesource.com/c/gerrit/+/333304 fyi that change is one we're going to pick up when https://review.opendev.org/c/opendev/system-config/+/834244 lands and we restart gerrit. Not sure if elections tooling and similar scripts may rely on that | 18:01 |
| fungi | codesearch says we don't call --no-limit | 18:03 |
| clarkb | fungi: what about just no-limit? it is useable in both ssh and http iirc and http would not hvae the -- | 18:04 |
| fungi | yeah, that's actually what i queried: https://codesearch.opendev.org/?q=no-limit | 18:04 |
| clarkb | cool we should be fine then | 18:04 |
| fungi | pretty sure all the implementations i've written have relied on normal paging | 18:05 |
| fungi | and offsets | 18:05 |
| fungi | in the opendev/engagement implementation i actually constrain the queries as much as possible (so basically one batch per repo) and do overlapping offsets with client-side deduplication in order to try to eliminate undercount from nondeterministic paging | 18:06 |
| fungi | it roughly doubles the amount of data pulled, but increases accuracy to near perfect | 18:08 |
| fungi | stateless pagination in query apis is such a mess | 18:08 |
| clarkb | ooof ya | 18:08 |
| fungi | i also use the largest page sizes allowed by the api in order to minimize the number of requests, but i think i may not have even been aware there was a no-limit option | 18:09 |
| clarkb | I think it must be newer | 18:11 |
| fungi | i would believe that | 18:13 |
| fungi | i may have written most of the engagement query script prior to our 3.x upgrade | 18:14 |
| fungi | can't exactly recall now | 18:14 |
| opendevreview | Merged opendev/system-config master: gitea: set custom avatars for orgs https://review.opendev.org/c/opendev/system-config/+/834085 | 18:28 |
| clarkb | the logo change seems to have applied ok | 19:17 |
| clarkb | (yay for testing) | 19:17 |
| clarkb | the gerrit image rebuild change will finally +1 after a couple of network related failures | 19:18 |
| mnaser | anyone ever thought into where the 'updated' time comes from gitea? | 19:37 |
| clarkb | mnaser: I think it updates when changes are replicated to it | 19:39 |
| mnaser | hmm, i see it the same time for all vexxhost projects, maybe it has to do when the system was redeployed | 19:40 |
| clarkb | one thing that makes it weird is that gerrit stores votes and similar in git no wtoo so it isn't just when you push changes but when reviews are made aiui. https://opendev.org/opendev/system-config is listed as updating an hou rago but the last chnge merged more than that | 19:42 |
| mnaser | yeah i have stuff which havent had any reviews but still showing last updates all same time | 19:52 |
| clarkb | mnaser: I wonder if that update is from the logo update | 19:55 |
| clarkb | I wonder if the db update trips a timestamp update | 19:56 |
| clarkb | the timing seems to line up for that | 19:57 |
| clarkb | and the airship repos show the same thing which lends more evidence to that being the case | 19:58 |
| *** dviroel is now known as dviroel|out | 21:37 | |
| fungi | #status log Rebooted all 12 Zuul executors for Linux kernel updates to address CVE-2022-25636 | 21:41 |
| opendevstatus | fungi: finished logging | 21:41 |
| fungi | corvus: ^ in doing that, the executors have restarted onto 5.0.1.dev188 2a509898 | 21:42 |
| mordred | clarkb: I honestly don't remember. I believe it has to do primarily with clearing out that tc.log file, which exists for XA transaction support and will be happily re-created if needed if it doesn't exist. | 22:25 |
| mordred | the file is created on start ... that location would be in the volume we bind-mount in | 22:27 |
| clarkb | mordred: ya the location is bind mounted but the file doesn't exist. I wonder if we just don't make use of whatever generates it | 22:59 |
| clarkb | fungi: I think if we can get https://review.opendev.org/c/opendev/system-config/+/834244 landed an early week restart of gerrit next week is a good idea | 23:00 |
| clarkb | though happy for tha tto get approved early next week if preferred | 23:00 |
| clarkb | you're probably enjoying a beer and bbq | 23:00 |
| clarkb | so no rush :) | 23:01 |
| fungi | nah, nothing so exciting as all that. i can take a look | 23:02 |
| clarkb | that will get us the release that they just made today | 23:02 |
| clarkb | nothing super important out of it for our uses that I can see but just general fixes which are good to keep up with | 23:03 |
| * fungi accidentally nicks himself on the bleeding edge | 23:03 | |
| opendevreview | Guillaume Chauvel proposed opendev/system-config master: Move xenial distutils check in OS specific files. https://review.opendev.org/c/opendev/system-config/+/772420 | 23:41 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
