| melwitt | does anyone know whether https://opendev.org/opendev/jeepyb/commit/faca72ce59bfce0758f5f3f7eed3e71f98a35a18 has been released yet? asking bc blueprints aren't getting updated when expected. if it's been released can someone please share any error messages that may have been logged? | 00:27 |
|---|---|---|
| ianw | that is https://review.opendev.org/c/opendev/jeepyb/+/795912 and so should be incorporated | 00:29 |
| fungi | have we updated our gerrit container image and restarted into it since new jeepyb would have ended up in it? | 00:33 |
| Clark[m] | We have restarted since that change landed | 00:34 |
| Clark[m] | Multiple times | 00:34 |
| ianw | yeah the hook is there, i'm just poking at the container now | 00:36 |
| ianw | do we have an example change so i can try replaying the hook? | 00:37 |
| ianw | the command "/usr/local/bin/update-blueprint patchset-created --change 809020 --change-url 'https://review.opendev.org/c/openstack/nova-specs/+/809020' --commit a5e78811c65d72870d4b758d9038398a3a1b11f7 --project openstack/nova-specs" | 00:45 |
| ianw | appears to complete without failure | 00:45 |
| ianw | hrm, i just replayed for https://review.opendev.org/c/openstack/nova-specs/+/787458 | 01:20 |
| ianw | it parsed and saved and update to the whiteboard correctly | 01:22 |
| ianw | the problem is the last update to that was mid july when possibly the fix wasn't in | 01:28 |
| ianw | melwitt: so i see updates to 809020 correct, and i replayed an old one (787458) and it seemed to update. so a priori i don't see anything wrong | 01:29 |
| *** odyssey4me is now known as Guest7290 | 04:14 | |
| *** ysandeep|out is now known as ysandeep | 04:18 | |
| *** bhagyashris is now known as bhagyashris|off | 05:35 | |
| *** jpena|off is now known as jpena | 07:26 | |
| kopecmartin | hi, opendev.org doesn't properly renderer .rst tables - https://opendev.org/osf/interop/src/branch/master/doc/source/process/2017A.rst (rst syntax should be ok based on an online rst editor), should I file a bug somewhere? | 08:04 |
| *** dviroel|out is now known as dviroel | 11:08 | |
| *** ykarel_ is now known as ykarel | 11:11 | |
| *** ykarel_ is now known as ykarel | 11:28 | |
| *** jpena is now known as jpena|lunch | 11:28 | |
| *** arxcruz is now known as arxcruz|pto | 11:55 | |
| *** ysandeep is now known as ysandeep|brb | 12:12 | |
| *** odyssey4me is now known as Guest7335 | 12:17 | |
| *** jpena|lunch is now known as jpena | 12:21 | |
| *** odyssey4me is now known as Guest7338 | 12:31 | |
| ttx | It's probably a gitea bug / limitation | 12:44 |
| fungi | well, limitation in the rst renderer it's using anyway | 13:04 |
| fungi | i'll see if i can jog my memory as to which one it is | 13:05 |
| *** ysandeep|brb is now known as ysandeep | 13:08 | |
| fungi | kopecmartin: ttx: it's using `pandoc -f rst` according to https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/gitea/templates/app.ini.j2#L90-L97 | 13:09 |
| ttx | thumbsup emoji | 13:16 |
| *** ykarel is now known as ykarel|afk | 13:18 | |
| kopecmartin | thanks, i'll try a few experiments, let's see if i can find a syntax which will be accepted by pandoc and will render a proper table | 13:20 |
| fungi | kopecmartin: alternatively, if pandoc is just outright limited in its capability, we could fairly trivially substitute any other renderer which can be called in a similar fashion | 13:33 |
| fungi | modulo increases in resource consumption and so on, of course | 13:33 |
| *** artom_ is now known as artom | 13:35 | |
| *** lukas is now known as Guest7348 | 13:43 | |
| *** ykarel|afk is now known as ykarel | 14:07 | |
| *** odyssey4me is now known as Guest7352 | 14:07 | |
| fungi | clarkb: one more upcoming nail in the lists.o.o pv instance's coffin: https://salsa.debian.org/glibc-team/glibc/-/blob/sid/debian/debhelper.in/libc.NEWS#L3-6 | 14:13 |
| fungi | that just popped up this morning when i was upgrading some of my systems | 14:13 |
| fungi | oh, though i guess that's 32-bit only, not 64-bit, so *maybe* not relevant? | 14:15 |
| fungi | if anything, it's an indication that support for pv xen is falling away | 14:16 |
| fungi | bit by bit | 14:16 |
| clarkb | ya certainly seems like it isn't a platform anyone expects to really exist out there in the wild anymore | 14:40 |
| melwitt | ianw: thank you for checking the update_blueprint script! it appears that I wrongly assumed that bc I did not receive a notification from launchpad, that the blueprint did not get updated. sorry for that 😓 | 14:58 |
| fungi | melwitt: well, thanks for helping us confirm it's actually working now! ;) | 14:59 |
| melwitt | np 😂 | 14:59 |
| clarkb | fungi: did you catch my bits of research on using node exporter from ubuntu packages? tldr is that prior to the 1.0 reelase every release was changing metric names around which would probably get very clunky to manage with our heterogenous deployments | 15:03 |
| clarkb | fungi: I think if the packages were >=1.0 it wouldn't be a problem | 15:04 |
| fungi | clarkb: yep, that makes sense as a reason to go with the containerized approach if we were to decide on node_exporter. however with snmp_exporter i guess we don't need to worry about it since that gets installed centrally on the prometheus host, and snmp is already standardized since decades anyway | 15:14 |
| clarkb | yup | 15:14 |
| *** ykarel is now known as ykarel|away | 15:20 | |
| *** ysandeep is now known as ysandeep|away | 15:23 | |
| opendevreview | Lajos Katona proposed openstack/project-config master: Add neutron-dynamic-routing-stable-maint group https://review.opendev.org/c/openstack/project-config/+/809232 | 15:53 |
| kopecmartin | fungi: rst syntax is the culprit at the end, it turned out that pandoc is less forgiving than rst online tools , the issue was a multiline cell - https://review.opendev.org/c/osf/interop/+/808943/6/doc/source/process/2021A.rst | 15:57 |
| kopecmartin | fungi: thank you | 15:58 |
| *** marios is now known as marios|out | 16:01 | |
| fungi | kopecmartin: awesome, thanks for testing that out! | 16:05 |
| fungi | kopecmartin: it looks like you had a stray + compared to the multi-row span cell examples at https://docutils.sourceforge.io/docs/user/rst/quickref.html#tables | 16:10 |
| fungi | could that have been the culprit? | 16:10 |
| fungi | seems to me the + to the left of PTG should have been a | instead | 16:11 |
| *** jpena is now known as jpena|off | 17:04 | |
| fungi | now that lists.o.o has been running on focal for a bit, the memory utilization actually looks a lot better than it was on xenial: http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=219&rra_id=all | 17:19 |
| fungi | even its swap utilization isn't all that bad by comparison: http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=221&rra_id=all | 17:19 |
| clarkb | that might be the first time in history where running newer software used less memory :) | 17:20 |
| fungi | well, it's still somewhat early, but looks promising at least | 17:23 |
| clarkb | infra-root heads up there appears to be a lot of openstack release activity today. Probably a good day to be very slushy | 17:36 |
| kopecmartin | fungi: i thought so too, however, it didn't help - seems like pandoc can't handle multiline cells, it didn't process correctly even the example tables with multiline cells from the documentation | 17:36 |
| fungi | clarkb: good, i was feeling very slushy anyway | 17:37 |
| fungi | kopecmartin: good to know. i wonder whether there's a bug open against pandoc to support that | 17:37 |
| fungi | kopecmartin: indeed, web search for the win... https://github.com/jgm/pandoc/issues/1024 | 17:38 |
| kopecmartin | well, luckily we don't really need multiline cells | 17:38 |
| clarkb | also gitea probably shouldn't be considered your primary publication location | 17:39 |
| kopecmartin | clarkb: i was thinking about that .. any suggestions except for wiki pages and docs.openstack.org? | 17:41 |
| fungi | kopecmartin: looks like pandoc's html writer supports rowspan and colspan as of 2.11 (almost a year ago) so we may not have new enough pandoc to support it anyway | 17:42 |
| fungi | kopecmartin: we also have docs.opendev.org | 17:42 |
| * kopecmartin checking that out | 17:42 | |
| clarkb | fungi: our gitea images are still buster iirc but we have bullseye available now we could update too. I'm not sure if pandoc is new enough on bullseye though | 17:43 |
| fungi | kopecmartin: but i think the board of directors is looking at having a cms of some sort to provide a web presence for foundation board level efforts, so that may also fit the interop wg | 17:43 |
| clarkb | I guess pushing up a change to see if everything runs on bullseye in testing is worthwhile. I'll do that | 17:43 |
| fungi | clarkb: looks like bullseye has pandoc 2.9 still, so no | 17:44 |
| fungi | 2.11 isn't in debian at all yet | 17:44 |
| clarkb | ok. Still worth making that change and seeing what breaks | 17:45 |
| fungi | absolutely, just won't address the rst table rendering limitations | 17:45 |
| fungi | though as far as pandoc is concerned, it will take us from 2.2.1-3+b2 to 2.9.2.1-1+b1 | 17:46 |
| fungi | so we may see some rendering improvements at least (or regressions!) | 17:47 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Update our gitea images to bullseye https://review.opendev.org/c/opendev/system-config/+/809269 | 17:51 |
| clarkb | something like that I Think | 17:52 |
| clarkb | hrm that change is only building images and not running the tests against them. If images build successfully I guess I can add a noop edit to testinfra to run the tests | 17:53 |
| fungi | interesting that we don't include the test jobs when the image configs change | 17:58 |
| clarkb | oh wait we do run the job I'm just not capable of reading | 18:11 |
| clarkb | my brain wants to treat the job list as ordered by dependencies and the system-config-run-gitea job is listed first | 18:11 |
| clarkb | thinking out loud here maybe we want to start writing changes like that in bulk so that we can work through issues and then start landing them all after the openstack release (and probably sooner for certain services) | 18:14 |
| fungi | corvus: looks like alembic 1.7 breaks gertty on startup with "AttributeError: module 'alembic' has no attribute 'migration'" | 18:24 |
| fungi | rolling alembic back to 1.6.5, everything's gine | 18:24 |
| fungi | s/gine/fine/ | 18:24 |
| fungi | i'll see if i can find a fix | 18:25 |
| clarkb | fungi: zuul had that issue, it is a missing import | 18:25 |
| fungi | ahh, thanks for the hint! | 18:26 |
| fungi | indeed, zuul side fix was https://review.opendev.org/806783 so i'll propose the same | 18:27 |
| fungi | yeah, that does the trick | 18:28 |
| fungi | unfortunately https://alembic.sqlalchemy.org/en/latest/changelog.html#change-1.7.0 makes no mention of that | 18:31 |
| opendevreview | Jeremy Stanley proposed ttygroup/gertty master: Import alembic.migration https://review.opendev.org/c/ttygroup/gertty/+/809279 | 18:33 |
| fungi | corvus: ^ that got me working again, fyi | 18:33 |
| clarkb | fungi: ya the blame is more on the user side than the alembic side | 18:35 |
| clarkb | fungi: a name was being used that was present as an import side effect then they refactored and the side effect no longer happened | 18:35 |
| fungi | aha, well, overheating spacebar regressions and all, but if a module imports other modules into its namespace it's not surprising when consumers inadvertently treat them as objects declared within the module | 18:37 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Build our gerrit images on Bullseye https://review.opendev.org/c/opendev/system-config/+/809286 | 20:46 |
| clarkb | In that one we use python-builder on our side which is buster and openjdk:11 currently. openjdk:11 is buster I think but will in theory become bullseye in the future. By being explicit about it we ensure we don't end up with a msimatch between those images that are used. | 20:46 |
| fungi | yeah, bullseye has openjdk-11 packages as well | 20:48 |
| clarkb | yup they have images for bullseye it is just a matter of being explicit about it which the above chagne does | 20:48 |
| *** dviroel is now known as dviroel|out | 20:48 | |
| fungi | there doesn't seem to be an openjdk-12 yet, or at least not one carried in debian | 20:49 |
| clarkb | fungi: they just released 17 actually. 11 is a sort of defacto standard old version aiui and is the latest that gerrit supports | 20:49 |
| clarkb | I think 12 - 14 are EOL? | 20:49 |
| clarkb | I don't really keep up with that ecosystem anymore | 20:50 |
| fungi | aha, yeah there's also an openjdk-17 in nullseye | 20:53 |
| fungi | bullseye | 20:53 |
| fungi | https://packages.debian.org/bullseye/openjdk-17-jdk | 20:53 |
| clarkb | ya so whenever gerrit adds new java support we'll be ready :) | 20:54 |
| clarkb | supposedly its a fair bit quicker than java 11 at certain types of GC tasks | 20:54 |
| *** timburke_ is now known as timburke | 20:59 | |
| fungi | that's be nice, assuming it translates to gerrit not getting as bogged down on garbage collection | 21:18 |
| *** osmanlicilegi is now known as Guest0 | 21:20 | |
| *** prometheanfire is now known as Guest1 | 21:20 | |
| opendevreview | Marco Vaschetto proposed openstack/diskimage-builder master: Allowing use local image https://review.opendev.org/c/openstack/diskimage-builder/+/809009 | 21:49 |
| *** promethe- is now known as prometheanfire | 21:53 | |
| clarkb | Both of the bullseye updates I pushed today pass testing which is good to see. But in both cases we don't really rely on the operating system for much so shouldn't be surprised either | 22:56 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Set a gerrit replication timeout of 15 minutes https://review.opendev.org/c/opendev/system-config/+/809297 | 23:09 |
| clarkb | infra-root ^ I think we should consider that as we appear to have some leaky replication tasks again | 23:10 |
| clarkb | I'm going to manually clean up the queue again but hopefully setting a timeout like that will make things happier | 23:10 |
| ianw | clarkb: am i missing something or does that need to also update https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/gerrit/templates/replication.config.j2#L9 to write the timeout value? | 23:22 |
| clarkb | oh you're right. For some reason I thought it iterated over all the values in the dict and wrote them if present | 23:23 |
| clarkb | I'll push an update shortly | 23:23 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Set a gerrit replication timeout of 15 minutes https://review.opendev.org/c/opendev/system-config/+/809297 | 23:25 |
| clarkb | we'll need to restart gerrit to pick that up after it lands and applies | 23:26 |
| clarkb | I can't remember why we set autoReload to false anymore but there aws a good reason for it iirc | 23:27 |
| ianw | clarkb: i'm just reading now, should we maybe be setting the ssh timeouts in https://gerrit.googlesource.com/plugins/replication/+doc/v3.3.0/src/main/resources/Documentation/config.md#file-1 | 23:31 |
| ianw | gerrit.sshConnectionTimeout : Timeout for SSH connections. If 0, there is no timeout and the client waits indefinitely. By default, 2 minutes | 23:31 |
| ianw | so we're clearly waiting longer than 2 minutes, which seems to mean | 23:31 |
| ianw | it must be in | 23:31 |
| ianw | gerrit.sshCommandTimeout : Timeout for SSH command execution. If 0, there is no timeout and the client waits indefinitely. By default, 0. | 23:31 |
| clarkb | ya I kind of figured that the network timeout would cover both of those bases since in theory if there is no network transmission it would cover both of those? | 23:33 |
| clarkb | it isn't claer to me what ssh command execution covers. Is that the entire replication over ssh? | 23:33 |
| clarkb | or just establishing the ssh connection ? | 23:34 |
| clarkb | ianw: I guess we could set sshCommandTimeout to 15 minutse as well? | 23:35 |
| clarkb | it shouldnt hurt to be at least as long as the network transmission timeout | 23:35 |
| ianw | git grep sshCommandTimeout doesn't seem to give me anything | 23:38 |
| clarkb | ianw: I'm going to context switch to the space x launch here, feel free to update the change if you can decipher the replication plugin better. Or I'll see if RTFSing helps | 23:39 |
| ianw | i'll have a poke and see if anything becomes clear, but in general ++ for setting something here | 23:39 |
| clarkb | looks like src/main/java/com/googlesource/gerrit/plugins/replication/SshHelper.java uses the ssh command timeout | 23:40 |
| clarkb | Process proc = ssh.exec(cmd, commandTimeout); | 23:40 |
| clarkb | sshSessionFactoryProvider.get().getSession(uri, null, FS.DETECTED, connectionTimeout); | 23:41 |
| clarkb | so ya it seems connection timeout is the syn ack synack + maybe authorizing and all the to establish the connection then the command timeout is everything that happens after? | 23:41 |
| ianw | ahh, of course so it's not in my gerrit source tree | 23:42 |
| clarkb | its in https://gerrit.googlesource.com/plugins/replication/ that repo if you clone that | 23:42 |
| opendevreview | Marco Vaschetto proposed openstack/diskimage-builder master: Allowing use local image https://review.opendev.org/c/openstack/diskimage-builder/+/809301 | 23:43 |
| opendevreview | Marco Vaschetto proposed openstack/diskimage-builder master: Allowing use local image https://review.opendev.org/c/openstack/diskimage-builder/+/809009 | 23:45 |
| ianw | yeah too much java to parse :) i say go with the simple thing and work from there | 23:46 |
| opendevreview | Marco Vaschetto proposed openstack/diskimage-builder master: Allowing use local image https://review.opendev.org/c/openstack/diskimage-builder/+/809009 | 23:46 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!