| ianw | yeah now we have the buster container we might need to reevaludate | 00:56 |
|---|---|---|
| ianw | reevauluate | 00:56 |
| ianw | it's probably a question of what else it drags in | 00:58 |
| opendevreview | Ian Wienand proposed openstack/diskimage-builder master: Test building jammy https://review.opendev.org/c/openstack/diskimage-builder/+/836228 | 01:11 |
| *** lbragstad2 is now known as lbragstad | 02:37 | |
| *** raukadah is now known as chandankumar | 03:40 | |
| *** raukadah is now known as chandankumar | 04:22 | |
| *** ysandeep|out is now known as ysandeep | 04:26 | |
| opendevreview | Merged openstack/project-config master: Match configure-mirrors for CentOS wheel URLs https://review.opendev.org/c/openstack/project-config/+/836200 | 04:26 |
| ianw | fungi: ^ thanks for that. i've remounted the wheel volumes without the -stream, and released | 04:28 |
| opendevreview | Merged zuul/zuul-jobs master: ensure-kubernetes: fix missing 02-crio.conf https://review.opendev.org/c/zuul/zuul-jobs/+/835162 | 06:24 |
| frickler | ianw: not sure why this doesn't show in the build log, testing locally I got: E: Extracting .//var/cache/apt/archives/base-files_12ubuntu3_amd64.deb requires the zstdcat command, which is not available | 06:29 |
| frickler | zstd is only suggested by debootstrap, explicitly added it now for the next attempt | 06:29 |
| *** ysandeep is now known as ysandeep|afk | 06:49 | |
| opendevreview | Merged zuul/zuul-jobs master: ensure-pip: fix typo in ensure_pip_virtualenv_command documentation https://review.opendev.org/c/zuul/zuul-jobs/+/831136 | 07:26 |
| ianw | huh, weird. the package format has changed in unstable? | 07:30 |
| ianw | clarkb: i made a glean 1.21.0 release with your fix and unpased centos 9; centos-9-stream-0000005135 is the fixed version. it's uploading now | 07:34 |
| ianw | also i had a poke at some of the deleting images nodepool is stuck on, seems a lot of leaks in osuosl. i've sent an email to support about that | 07:35 |
| *** jpena|off is now known as jpena | 07:36 | |
| frickler | Ramereth: ^^ | 08:40 |
| frickler | ianw: seems to be a different issue. maybe I have to try with the docker image locally | 08:42 |
| *** ysandeep|afk is now known as ysandeep | 09:11 | |
| *** arxcruz|out is now known as arxcruz | 09:34 | |
| *** jpena is now known as jpena|off | 09:35 | |
| *** jpena|off is now known as jpena | 09:44 | |
| *** pojadhav- is now known as pojadhav | 10:58 | |
| *** pojadhav- is now known as pojadhav | 11:13 | |
| *** pojadhav is now known as pojadhav|afk | 11:34 | |
| *** dviroel|out is now known as dviroel | 11:38 | |
| fungi | ianw: thanks for moving the volumes too! | 11:49 |
| fungi | frickler: interesting, zstd is only a suggests in the debootstrap in debian/unstable, neither cdebootstrap nor mmdebstrap (what i usually use) even have it as a suggests. wonder if this is an ubuntu-specific modification? | 11:54 |
| fungi | as in maybe ubuntu switched from zlib to zstd compression for their debs recently | 11:56 |
| fungi | frickler: yep, that's exactly it, apparently... http://bugs.debian.org/996019 | 12:03 |
| fungi | the version in bookworm is too old, the version in bullseye/sid has support for unpacking ubuntu's newer format but seems to need zstd for doing so | 12:05 |
| fungi | er, te version in bullseye is too old, the version in bookworm/sid is new enough | 12:05 |
| fungi | too many "b" release names in a row | 12:05 |
| fungi | a new enough version migrated to bookworm (testing) in november, so adding to bullseye-backports would probably be possible if requested | 12:07 |
| fungi | https://tracker.debian.org/pkg/debootstrap | 12:08 |
| fungi | right now there are older versions in old-bpo (buster-backports) and o-o-bpo (stretch-backports) indicating it frequently gets backported for such reasons | 12:09 |
| frickler | fungi: interesting, seems debootstrap on jammy itself doesn't have this dependency and works fine without zstd. | 12:21 |
| fungi | probably it uses libzstd instead, or has it statically linked | 12:23 |
| frickler | fungi: it uses dpkg-deb if that has zstd support, which it seems to have on ubuntu but not debian https://git.launchpad.net/ubuntu/+source/debootstrap/tree/scripts/gutsy?h=applied/ubuntu/jammy#n47 | 12:32 |
| frickler | and that in turn indeed uses libzstd | 12:34 |
| *** ysandeep is now known as ysandeep|break | 12:40 | |
| *** pojadhav- is now known as pojadhav | 12:45 | |
| *** jpena is now known as jpena|off | 12:54 | |
| *** ysandeep|break is now known as ysandeep | 13:00 | |
| *** jpena|off is now known as jpena | 13:03 | |
| marios | o/ folks can someone point me to the host pin for diablo? so we can start recording (for tripleo ) | 13:04 |
| *** timburke_ is now known as timburke | 13:04 | |
| fungi | marios: zoom? maybe ping diablo_rojo in #openinfra-events, we don't manage that | 13:08 |
| fungi | infra-root: lists.kata-containers.io ended up back on the spamhaus pbl... it looks like spamhaus has changed their processes, now the listing page mentions "NOTE: Exclusions are only valid for 1 year." | 13:09 |
| fungi | we probably need to revisit all our pbl exclusions | 13:10 |
| marios | fungi: yeah thanks found it | 13:11 |
| marios | fungi: thankyou for checking | 13:11 |
| marios | sorry for noise | 13:11 |
| fungi | no worries | 13:12 |
| fungi | #status log Requested Spamhaus PBL exclusion for the IPv4 address of lists.kata-containers.io | 13:21 |
| opendevstatus | fungi: finished logging | 13:21 |
| opendevreview | Merged zuul/zuul-jobs master: Add tox-py310 job https://review.opendev.org/c/zuul/zuul-jobs/+/821247 | 13:51 |
| fungi | meetpad, jvb01 and jvb02 seem to be holding up well so far. can see some load on meetpad but nothing looks worrisome | 13:53 |
| opendevreview | Merged opendev/git-review master: Switch to unittest.mock https://review.opendev.org/c/opendev/git-review/+/832496 | 14:58 |
| jrosser | i got an unexpected -2 from zuul on this patch https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/833865 | 15:09 |
| jrosser | and the message was a warning that `Change 835548 in project openstack/openstack-ansible does not share a change queue with 833865 in project openstack/openstack-ansible-os_nova` | 15:09 |
| jrosser | and i'm OK with those not sharing a queue, and didn't expect a warning to fail the job completely | 15:10 |
| jrosser | once the depends-on patch was merged, it went through and merged as expected but at the expense of rechecking | 15:11 |
| clarkb | jrosser: corvus can confirm but I think that may be zuuls new behavior of being more explicit about why things aren't enqueing? Either that or unexpected fallout from the topic submisson stuff /me checks if they shared a topic | 15:16 |
| clarkb | no does not look like they shared a topic | 15:16 |
| clarkb | ianw: thank you for getting that release pushed. I see booted in-use instances in rax now | 15:21 |
| clarkb | I need to cleanup my test node now | 15:21 |
| clarkb | and done | 15:34 |
| clarkb | frickler: left a question on the nodepool change for debootstrap update. I'm +2 since its mostly just a thing about the comment | 15:36 |
| clarkb | I think I'm mostly caught up on things that happened over the weekend. I'm half avoiding PTG esssions this morning since I've got a DMV appointment midday today and don'y want to get sucked into anything but ping me if I should jump on a call | 15:39 |
| prometheanfire | fyi, I have to hold back paramiko due to it not working with dropbear anymore (closed as wontfix) https://github.com/paramiko/paramiko/issues/1961 | 15:41 |
| clarkb | prometheanfire: thats not entirely true. It will work with dropbear but you either need a newre dropbera than is currently in cirros or you need to use a different key type than rsa | 15:42 |
| clarkb | prometheanfire: we've been suggesting that people use non rsa keys | 15:42 |
| prometheanfire | ah, kk | 15:45 |
| clarkb | prometheanfire: it does look like paramiko is trying sha2 as the fallback which is good. Openssh doesn't do this and falls back to sha1 then fails due to its new sha1 policy which is a bit silly imo. I think that all clients should fallback to sha2 if they disallow sha1 then error as paramiko does if the server cannot sha2 | 15:45 |
| prometheanfire | as long as it's known about | 15:45 |
| clarkb | prometheanfire: basically I wouldn't pin paramiko I would force tempest et al to use a different key type. The support is already there for fips testing | 15:46 |
| prometheanfire | ya, tempest is the right place for a fix | 15:46 |
| prometheanfire | if I see a meeting during the ptg I'll bring it up | 15:47 |
| clarkb | fungi: looks like virtualenv still hasn't updated for new setuptools. | 15:48 |
| fungi | nope | 15:52 |
| clarkb | I was hoping we could revert some of thos ework around today. Oh well | 15:53 |
| fungi | yeah, tell me about it | 15:57 |
| *** marios is now known as marios|out | 16:17 | |
| gibi | it is just me or gerrit has issues managing ssh keys? | 16:22 |
| gibi | I tried to delete old keys via the UI | 16:23 |
| gibi | first it seems it is deleted but after a refresh they are back | 16:23 |
| clarkb | gibi: you have to click save iirc | 16:23 |
| clarkb | but no I've not had any issues removing keys | 16:23 |
| gibi | /o\ | 16:24 |
| gibi | clarkb: thanks. that was the trick | 16:25 |
| clarkb | ya I'm not sure I like that ui but it allows you to queue up a bunch of deletion sand additions and apply them at once | 16:26 |
| clarkb | I'm guessing that was useful to some user | 16:26 |
| gibi | clarkb: do you happen to know if our gerrit supports ed25519 keys? | 16:29 |
| gibi | I can add such key via the gui but it does not seems to be accepted via the ssh handshake | 16:30 |
| *** jpena is now known as jpena|off | 16:31 | |
| clarkb | gibi: it does | 16:31 |
| clarkb | I use one now | 16:31 |
| gibi | hm | 16:32 |
| clarkb | fungi: I'm reading the tc ptl discussion notes and I think some of that feedback should be to opendev. Its a bit frustrating that we've done office hours for multiple PTGs and when we decide not to becuase no one engages us there is a bunch of feedback to the wrong group of people :/ | 16:32 |
| fungi | yep | 16:33 |
| clarkb | fungi: I wonder how we can communicate that better. Like hey we tried for a ocuple years and didn't get any input so stopped. How can we better collect that feedback? type messaging to the mailing list maybe | 16:33 |
| gibi | I use a yubikey with ed25519_sk resident key so it might be a problem on my side unlocking the ssh key with the yubikey during ssh | 16:33 |
| clarkb | gibi: _sk keys are different than regular keys. I don't know that gerrit supports them | 16:33 |
| clarkb | both sides need to support them iirc | 16:33 |
| fungi | support would need to be in apache mina-sshd i guess | 16:34 |
| clarkb | fungi: yup | 16:34 |
| gibi | clarkb, fungi: thanks for the info. I guess I will stick to a non _sk key then with gerrit for now | 16:35 |
| clarkb | yup looks like the sshd has to be aware of the key type at least for negotiation purposes | 16:38 |
| clarkb | it may be that the client does the heavy lifting, but seems like that key type must be explicitly recognized by both sides of the connection | 16:38 |
| *** dviroel is now known as dviroel|lunch | 17:01 | |
| *** dviroel|lunch is now known as dviroel | 17:55 | |
| opendevreview | Merged zuul/zuul-jobs master: run-buildset-registry: Drop extra install packages task https://review.opendev.org/c/zuul/zuul-jobs/+/835156 | 18:08 |
| *** rlandy is now known as rlandy|brb | 19:20 | |
| *** rlandy|brb is now known as rlandy | 19:44 | |
| johnsom | Hi, I'm having an odd issue with the openstack-tox-docs job on Wallaby. I see pip running a second time unconstrained, which pulls in a newer (broken) version of jinja2: https://zuul.opendev.org/t/openstack/build/d1b2e504012a42b7948723e47d771ef6/log/job-output.txt#734 | 20:27 |
| johnsom | I am not seeing where that pip install is coming from. My code-search-foo isn't working today. | 20:27 |
| johnsom | Does anyone have any thoughts? | 20:27 |
| fungi | the tox jobs normally run pip install twice, in order to separately pull in siblings projects for source installs. something there is probably interacting questionably | 20:28 |
| fungi | looking at the report | 20:28 |
| johnsom | Yeah, I found the task in siblings.yaml: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/tox/tasks/siblings.yaml | 20:29 |
| fungi | johnsom: for further reference, the module it's running is this: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/tox/library/tox_install_sibling_packages.py | 20:30 |
| johnsom | Thanks for the second set of eyes btw | 20:30 |
| fungi | it expects to handle constraints if supplied | 20:30 |
| fungi | https://zuul.opendev.org/t/openstack/build/d1b2e504012a42b7948723e47d771ef6/console#3/0/10/ubuntu-focal | 20:31 |
| clarkb | it should only update the packages that are found to be siblings iirc and then do that install with contsraints applied | 20:31 |
| fungi | it looks like a constraints file was supplied | 20:31 |
| fungi | https://zuul.opendev.org/t/openstack/build/d1b2e504012a42b7948723e47d771ef6/console#0/4/13/ubuntu-focal | 20:34 |
| fungi | that indicates it checked out the requirements repo at that path to stable/wallaby | 20:34 |
| clarkb | looking at the install siblings task it doesn't look like it installed any siblings? | 20:36 |
| johnsom | The wallaby upper-constraint is: Jinja2===2.11.3 which is what it pulled the first time. | 20:36 |
| clarkb | I don't think it is the siblings install doing it | 20:38 |
| clarkb | since it happens in the task before that | 20:38 |
| clarkb | Run tox without tests is where it happens | 20:38 |
| johnsom | Yes | 20:39 |
| johnsom | https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/tox/tasks/siblings.yaml#L2 | 20:39 |
| johnsom | That task name matched | 20:39 |
| clarkb | I think it is installing the deps and that works with constraints but then when it installs the package itself it fails to use constraints | 20:41 |
| *** dviroel is now known as dviroel|afk | 20:41 | |
| fungi | the unconstrained pip install command seems to be coming from tox itself, yeah? ubuntu-focal | [4348] /home/zuul/src/opendev.org/openstack/designate$ /home/zuul/src/opendev.org/openstack/designate/.tox/docs/bin/pip install --exists-action w -e . | 20:43 |
| fungi | er, https://zuul.opendev.org/t/openstack/build/d1b2e504012a42b7948723e47d771ef6/log/job-output.txt#734 | 20:43 |
| clarkb | ya | 20:43 |
| clarkb | its the step that installs designate | 20:43 |
| clarkb | after it installs all the deps | 20:43 |
| clarkb | I think that implies the package requirements want newer jinja and so pip pulls it in rather than seeing it s already satisifed? | 20:44 |
| fungi | this is tox's usedevelop behavior at work, right? | 20:45 |
| clarkb | hrm except the requirment is jinja2>=2.10 and 2.11.3 was installed previously | 20:45 |
| clarkb | fungi: yes I think so | 20:45 |
| johnsom | I see that set in the tox.ini for Designate: https://github.com/openstack/designate/blob/stable/wallaby/tox.ini#L27 | 20:46 |
| clarkb | I suspect this is reproduceable locally | 20:46 |
| johnsom | I have no idea why | 20:46 |
| clarkb | just run tox -edevos | 20:46 |
| clarkb | heh if I can spoell | 20:46 |
| clarkb | tox -edocs | 20:46 |
| fungi | johnsom: you might see if dropping the testenv.usedevelop option from tox.ini results in the correct behavior there | 20:46 |
| fungi | this might be a tox regression | 20:46 |
| fungi | i can't remember if usedevelop ever obeyed install_command | 20:47 |
| clarkb | what is expected to happen is we install all the deps with constraints then when it installs the package itself it hsould see all thoes packages are already installed at valid versions and not reinstall them | 20:47 |
| fungi | oh, actually install_command isn't setting a constraints option, that's being set in the deps | 20:47 |
| fungi | so this makes a bit of sense | 20:47 |
| clarkb | fungi: well install_command isn't even used by projects like nova | 20:48 |
| fungi | usedevelop doesn't consult the deps which is where -c is being set | 20:48 |
| clarkb | oh! | 20:48 |
| johnsom | so, locally, tox -e docs runs just fine for me | 20:48 |
| clarkb | johnsom: you may need a -r too | 20:48 |
| johnsom | But I probably don't have the latest tox | 20:48 |
| johnsom | Yeah, it was a new clone, no previous env | 20:48 |
| clarkb | I think the issue is that the doc requirements don't fully overlap with the requirements.txt to install the pcake | 20:48 |
| clarkb | *package | 20:49 |
| clarkb | and when pip reconciles all of those other missing package deps to install designate it pulls more stuff in | 20:49 |
| clarkb | I think you want to not install the package at all | 20:49 |
| clarkb | whatever flag that is in tox | 20:49 |
| clarkb | or stop using doc specific requirements | 20:49 |
| fungi | though if you're using a sphinx extension which runs the program to capture cli output, for example, you may need it installed in the env. depends on what's in the project's docs as to whether that approach is needed | 20:50 |
| * fungi needs to start preparing for the ptg to resume | 20:50 | |
| johnsom | Hmm, this just "magically" broke sometime between March 23rd and April 1st. | 20:50 |
| clarkb | ya and if that is the case then you shouldn't have doc specific requirements that are a subset of normal isntall requirements | 20:51 |
| clarkb | johnsom: tox hasn't updated since december, but maybe virtualenv/setuptools/pip shifted behavior | 20:51 |
| clarkb | OR | 20:51 |
| clarkb | jinja2 updated to be non backward compatible and it just worked installing the wrong version until then | 20:51 |
| fungi | it probably broke on april 1, just to prank you ;) | 20:51 |
| clarkb | anyway I'm pretty sur ethis is because it installs a lot of packages to install designate | 20:51 |
| clarkb | and the constrained install is only the doc deps and that doesn't install enough under constraints | 20:52 |
| fungi | jinja2 3.1.0 was released on march 24, and 3.1.1 a day after | 20:53 |
| fungi | johnsom: we probably have build history prior to that, if you want to check a successful run for that branch and see what version of jinja2 actually got installed in the docs builds | 20:54 |
| fungi | i.e. whether it was 3.0.3 or what's in constraints | 20:55 |
| johnsom | Ah, we do have logs for the last successful. https://zuul.opendev.org/t/openstack/build/62a1130e153b40acace1a82feb11bd39/logs | 20:55 |
| johnsom | Looking | 20:55 |
| johnsom | Yeah, it was Jinja2 3.0.3 | 20:57 |
| fungi | so constraints was never actually applying there | 20:57 |
| fungi | jinja2<3.1 was simply working | 20:58 |
| johnsom | Still, I'm trying to get my head around why it would install unconstrained. I'm going to read up on this usedevelop=True setting I don't know why is in tox.ini | 20:58 |
| johnsom | Yeah, I think you are right | 20:58 |
| clarkb | johnsom: is develop is not the problem | 20:58 |
| fungi | https://github.com/pallets/jinja/pull/1544 was what broke you, according to the changelog for 3.1.0 | 20:58 |
| clarkb | johnsom: the problem is that your doc requirements are not a super set of the designate requirements | 20:58 |
| clarkb | I suspect this problem exists for much of openstack too | 20:59 |
| fungi | and is specific to the docs jobs if the project is being installed into the env | 20:59 |
| johnsom | Yeah, I don't think any of the project I have seen are setup that way | 20:59 |
| clarkb | because the constraints only apply when installing the dependencies. Then when it installs designate (the package) the expectation is that all the deps are already installed so it will noop the installs at that point | 20:59 |
| clarkb | but that doesn't work here becuase most of the dependencies are not listed as doc requirements and thus weren't installed and thus get pulled in unconstrained | 21:00 |
| johnsom | We see in the section above it installed 2.11.3 then this second run uninstalled that and installed 3.1.1 | 21:02 |
| clarkb | johnsom: yes becaus esome other dep depends on jinja2 and caused pip to decide to install the newer version | 21:02 |
| clarkb | likely because their dep is >2.11.3 | 21:03 |
| johnsom | https://zuul.opendev.org/t/openstack/build/d1b2e504012a42b7948723e47d771ef6/log/job-output.txt#876 | 21:03 |
| clarkb | I think there are two reasonable fixes here. First is to stop intsalling the package (designate) in doc builds. The other is to drop the docs specific requirements and install all the deps constraints | 21:03 |
| clarkb | johnsom: ya so maybe pip dep solver is being to smart then | 21:04 |
| clarkb | eithre way the reason is some other package depends on jinja2 that wasn't preinstalled forcing jinja2 to get reinstalled and for some reason pip ignores the existing install | 21:04 |
| johnsom | designate has to be installed for the code indexes and such. | 21:05 |
| clarkb | then you need to drop the doc requirements | 21:05 |
| clarkb | or rather do doc requirements + project requirements | 21:05 |
| clarkb | since I guess there may be doc specific requirements that only get installed for documentation | 21:05 |
| johnsom | It's definitely something wrong in pip. Above it sees it's installed https://zuul.opendev.org/t/openstack/build/d1b2e504012a42b7948723e47d771ef6/log/job-output.txt#748 then gets another >=2.10 and it goes and pulls it again. | 21:06 |
| clarkb | johnsom: add a line here https://opendev.org/openstack/designate/src/branch/master/tox.ini#L46 to add requirements | 21:06 |
| clarkb | johnsom: well even if you fix the jinja2 problem some other dep could hvae that problem | 21:07 |
| clarkb | or if the thing pulling in jinja2 later says >=3.0 | 21:07 |
| clarkb | yes pip is maybe not doing the correct thing here but blaming pip and them fixing this specific issue is only a temporary/half fix | 21:07 |
| clarkb | since jinja2>=3.0 in a dep that wasn't preinstalled would cause the same problem | 21:07 |
| clarkb | and any other dep could do the same thing | 21:07 |
| clarkb | my hunch i sthat the pip dep solver is related to why pip does that | 21:08 |
| johnsom | Well, that also passes local, so I will give that a try. | 21:10 |
| clarkb | I've updated the meeting agenda on the wiki. I didn't really have anything new to add and trimmed out some stuff that occurred. Feel free to add topics for the next little bit and I'll get that sent out at the end of my day | 21:22 |
| johnsom | clarkb You are the pip whisperer. That worked. Thank you all for the insight. | 21:30 |
| clarkb | cool, it might be worth a thread on openstack-discuss to make sure that others are aware of this problem if they install the package and use a subset of deps in the doc requirements | 21:31 |
| clarkb | johnsom: is that something you might want to draft up or should I send an email about it/ | 21:31 |
| johnsom | I'm not sure I understand the issue well enough to be clear. | 21:32 |
| johnsom | If you give me a blurb on the why/how it's happening I can fill in the how-to-fix parts | 21:33 |
| clarkb | johnsom: can do. I'll start an etherpad | 21:33 |
| johnsom | +1 | 21:33 |
| clarkb | johnsom: https://etherpad.opendev.org/p/wnJGdyc7AdE0Z4SNYuqQ | 21:37 |
| johnsom | clarkb Cool, on it. | 21:37 |
| clarkb | johnsom: I edited line 10 of that etherpad with a smal ledit to make it more clear | 21:55 |
| johnsom | clarkb Ok, I haven't poked at that yet. | 21:56 |
| clarkb | I'm going to drop off the security sig call now since it is noisy here at home today and I got my bit in :) I will see if I can join the qa things tomorrow but that beginning of that slot conflicts with school run | 21:59 |
| clarkb | and other meetings | 21:59 |
| fungi | explicitly installing designate with constraints before (or instead of) tox's usedevelop behavior could be another workaround | 22:19 |
| johnsom | fungi This worked: https://review.opendev.org/c/openstack/designate/+/836407/1/tox.ini | 22:20 |
| johnsom | I'm currently writing up the commit message for the real change and we will send out the email to the discuss list. | 22:20 |
| fungi | in a personal project i add {toxinidir} to the deps= in my testenv:docs instead of relying on usedevelop | 22:20 |
| fungi | so it's effectively installing "." but that way it can do so along with the normal deps installation rather than in a separate usedevelop phase | 22:21 |
| fungi | since your deps contain the -c, doing it that way would apply constraints to the designate installation and also avoid having the separate usedevelop invocation of pip | 22:22 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!