| clarkb | :/ | 00:00 |
|---|---|---|
| *** mlavalle has quit IRC | 00:46 | |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: Restrict wheel publish jobs https://review.opendev.org/731632 | 00:52 |
| *** Meiyan has joined #opendev | 00:58 | |
| *** diablo_rojo has quit IRC | 01:02 | |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: Move wheel build role/job outside project-config https://review.opendev.org/731401 | 01:30 |
| openstackgerrit | Merged openstack/diskimage-builder master: Prepare to drop Python 2 support https://review.opendev.org/731378 | 02:38 |
| ianw | infra-root: i've redrafted https://etherpad.opendev.org/p/rm-pip-and-virtualenv with more of a call-to-action as discussed, please leave any comments and i'll get a mail out | 03:46 |
| *** ykarel|away is now known as ykarel | 04:31 | |
| AJaeger | ianw: https://review.opendev.org/731637 is needed for other stable branches for requirements as well, isn't it? | 05:12 |
| *** bolg has quit IRC | 05:12 | |
| *** sgw has quit IRC | 05:48 | |
| *** ysandeep|away is now known as ysandeep | 05:55 | |
| *** dpawlik has joined #opendev | 05:55 | |
| jrosser | clarkb: it was an embedded ansiible, 2.9.9, and it picked /usr/bin/python over anything more obviously python3-ish | 05:57 |
| *** DSpider has joined #opendev | 06:57 | |
| *** rpittau|afk is now known as rpittau | 07:04 | |
| *** hashar has joined #opendev | 07:08 | |
| *** tosky has joined #opendev | 07:12 | |
| *** factor has quit IRC | 07:21 | |
| *** dtantsur|afk is now known as dtantsur | 07:30 | |
| *** xiaolin has quit IRC | 07:34 | |
| *** njohnston has quit IRC | 07:47 | |
| *** moppy has quit IRC | 08:01 | |
| *** moppy has joined #opendev | 08:01 | |
| hrw | looks like I have to run one of those cache-wheel jobs on local machine and update bindep to get build-time requirements installed | 08:16 |
| hrw | numpy.distutils.system_info.NotFoundError: No lapack/blas resources found. | 08:17 |
| AJaeger | hrw: check bindep.txt in requirements repository - that list should cover everything you need. And if not: please update ;) | 08:20 |
| hrw | AJaeger: that's the plan | 08:21 |
| hrw | need to add blas/lapack and few other | 08:22 |
| frickler | hmm, running download-logs.sh from raxcdn doesn't mind the // conjunctions, that might explain why this wasn't noticed in initial testing earlier. but now it silently stops on file 269/313, can someone verify the same thing happening for them? https://980825aa441338566c98-f5b371aed546eed3ac03c77e9adc0c03.ssl.cf2.rackcdn.com/704831/11/check/devstack-platform-focal/a5aca94/download-logs.sh | 08:24 |
| hrw | AJaeger: having to check xenial/bionic/buster/c7/c8 take some time. focal should just work with whatever bionic/buster get | 08:26 |
| hrw | (vevn3) root@38e04e24663e:~/zuul/requirements# apt -y install `bindep|tail -n1` | 08:29 |
| hrw | and go on | 08:29 |
| AJaeger | hrw: don't check c7, we have no arm c7 images and will not support c7 on ARM. So, one less ;) | 08:31 |
| hrw | ;) | 08:31 |
| hrw | AJaeger: for now c8 on aarch64 is only partially supported | 08:32 |
| hrw | AJaeger: kolla can not be built for that combo until c8.2 | 08:32 |
| AJaeger | I see | 08:36 |
| *** ysandeep is now known as ysandeep|lunch | 08:43 | |
| frickler | curl: (61) Unrecognized content encoding type. libcurl understands deflate, gzip, br content encodings. | 09:07 |
| frickler | this has "Content-Encoding: xz" ... https://980825aa441338566c98-f5b371aed546eed3ac03c77e9adc0c03.ssl.cf2.rackcdn.com/704831/11/check/devstack-platform-focal/a5aca94/controller/logs/devstack.journal.xz | 09:08 |
| frickler | so a gather nobody is really using download-logs.sh or we should have seen complaints about it earlier :-/ | 09:08 |
| frickler | and that's even with curl on focal, I first assumed my local gentoo version to be too old | 09:09 |
| hrw | good to have 46 cpu cores. 5 distros build all python packages from openstack/requirements at same time. each in own container | 09:23 |
| *** ykarel is now known as ykarel|lunch | 09:35 | |
| *** roman_g has joined #opendev | 09:52 | |
| *** hashar has quit IRC | 09:59 | |
| *** Meiyan has quit IRC | 10:01 | |
| *** ysandeep|lunch is now known as ysandeep | 10:03 | |
| *** rpittau is now known as rpittau|bbl | 10:07 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/diskimage-builder master: Do not fail in a venv when activate_this.py is not found https://review.opendev.org/731719 | 10:23 |
| openstackgerrit | Dmitry Tantsur proposed openstack/diskimage-builder master: Add handling for no activate_this.py in venv https://review.opendev.org/704478 | 10:32 |
| *** ykarel|lunch is now known as ykarel | 10:38 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Split inventory into multiple dirs and move hostvars https://review.opendev.org/730991 | 11:43 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Override bridge hostvars directly https://review.opendev.org/731258 | 11:43 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run iptables in service playbooks instead of base https://review.opendev.org/730999 | 11:54 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Rename service-letsencrypt to just letsencrypt https://review.opendev.org/731617 | 11:54 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Split inventory into multiple dirs and move hostvars https://review.opendev.org/730991 | 11:54 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Override bridge hostvars directly https://review.opendev.org/731258 | 11:54 |
| *** hashar has joined #opendev | 11:56 | |
| openstackgerrit | Merged openstack/project-config master: Add root_marker_dir for releasenotes publish https://review.opendev.org/731462 | 11:56 |
| *** elod has quit IRC | 12:10 | |
| hrw | AJaeger: present? | 12:16 |
| hrw | AJaeger, smcginnis: trying to sort out all those dependencies for requirements and wheels cache builds | 12:16 |
| hrw | AJaeger, smcginnis: they either go to bindep in requirements or to playbook in openstack-zuul-jobs | 12:16 |
| hrw | and I got lost where to put them | 12:16 |
| *** elod has joined #opendev | 12:17 | |
| hrw | on one side https://review.opendev.org/#/c/731728/ updated bindep in requirements project to cover all build-deps for whatever is in upper-constraints.txt | 12:17 |
| hrw | on the other wasn't bindep to list what is needed to fulfill 'pip install -r requirements.txt' needs? | 12:19 |
| hrw | copied that into https://review.opendev.org/#/c/731728 | 12:22 |
| AJaeger | hrw: that was my understanding but did not double check so far | 12:22 |
| hrw | that's why I left comment as Sean asked there | 12:24 |
| AJaeger | hrw you havebot hthe same review - the first 731728 is wrong | 12:24 |
| AJaeger | hrw: just left a comment, bindep.txt documents it nicely ;) | 12:25 |
| hrw | ok | 12:25 |
| hrw | AJaeger: anyway build-wheel-cache role in openstack-zuul-jobs looks like ignoring bindep anyway | 12:27 |
| hrw | will add there few lines for it | 12:28 |
| AJaeger | hrw: best discuss with ianw as well. But ignoring bindep means your requirements change is a noop. Adding that might be good idea | 12:30 |
| AJaeger | hrw playbooks/wheel-cache/pre.yaml uses bindep | 12:31 |
| AJaeger | hrw: so, if bindep is ignored, let's debug why | 12:31 |
| hrw | ok thanks | 12:32 |
| AJaeger | hrw: on https://review.opendev.org/#/c/731630 we use the wrong bindep - fixing... | 12:35 |
| *** rpittau|bbl is now known as rpittau | 12:35 | |
| hrw | AJaeger: https://review.opendev.org/#/c/728798/26/playbooks/build-kafka.yaml will be needed too. but have to improve it | 12:36 |
| hrw | smcginnis: elo | 12:37 |
| hrw | AJaeger: as simple version does not cover xenial/buster/bionic/focal ;( | 12:37 |
| smcginnis | I think there may be a way to have different "targets" in bindep. In that case, I think we could include things in there, as long as they are not installed in the default case. | 12:38 |
| AJaeger | smcginnis: we can tag them with "wheel" | 12:38 |
| smcginnis | I haven't had to set up binary requriements for specific jobs before though, so I am not sure. | 12:38 |
| smcginnis | AJaeger: OK, that could make sense then. | 12:39 |
| hrw | smcginnis: x86 world is simple as most of wheels you get ready to use ;D | 12:39 |
| *** tkajinam has quit IRC | 12:39 | |
| AJaeger | hrw, updated https://review.opendev.org/731630 to use bindep with profile wheels. | 12:41 |
| AJaeger | I think your build-kafka should add the binaries to bindep.txt as well. Let's get 731630 passing first ;) | 12:42 |
| hrw | smcginnis: you are aware that entries in requirements.txt of openstack/requirements project require EMPTY bindep.txt? | 12:44 |
| hrw | none of entries there need anything | 12:44 |
| hrw | I just installed openstack/requiements on clean focal | 12:44 |
| hrw | so whole bindep.txt is for listing build deps of wheels basically... | 12:44 |
| hrw | AJaeger: there are no binaries for librdkafka ;( | 12:46 |
| AJaeger | hrw, smcginnis, so we can change lines from "krb5-devel [platform:redhat]" to "krb5-devel [platform:redhat wheel]" | 12:46 |
| hrw | AJaeger: not new enough ones | 12:46 |
| hrw | AJaeger: s/\[/[wheel /g | 12:46 |
| AJaeger | hrw: yep | 12:46 |
| hrw | so what for that file would be? | 12:46 |
| fungi | hrw: i think the requirements.txt in openstack/requirements is probably irrelevant here. wheels are built for the global-requirements.txt file instead | 12:46 |
| hrw | fungi: and for global-requirements.txt current bindep.txt is not enough on !x86 | 12:47 |
| AJaeger | fungi: yes, as stated in bindep.txt from rquirements repo | 12:47 |
| hrw | that's my patch does - updates bindep.txt to really have all deps | 12:47 |
| fungi | hrw: sure, i'm not surprised, far fewer people publish arm wheels to pypi so there will be more we have to build | 12:47 |
| hrw | fungi: yep | 12:48 |
| AJaeger | fungi, please review 731630 so that we can test this (part of ianw's wheel build rework) | 12:48 |
| fungi | i'll put it on my queue, still catching up on e-mail and any emergencies for the morning | 12:48 |
| hrw | AJaeger: please add debian-buster ones too | 12:49 |
| AJaeger | hrw: one step at a time ;) | 12:49 |
| hrw | AJaeger: 731360 adds all entries so can add all we want ;d | 12:49 |
| AJaeger | hrw: let me first get it working for those entries we have ;) | 12:51 |
| AJaeger | hrw: needs iteration | 12:52 |
| hrw | ok | 12:53 |
| *** ykarel is now known as ykarel|afk | 13:01 | |
| *** elod has quit IRC | 13:06 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/diskimage-builder master: Do not fail in a venv when activate_this.py is not found https://review.opendev.org/731719 | 13:09 |
| *** elod has joined #opendev | 13:13 | |
| hrw | ok. master packages built with whatever cache we plan to have | 13:17 |
| mordred | infra-root: I'm out this morning - will be back online this afternoon | 13:19 |
| *** hashar has quit IRC | 13:37 | |
| *** ysandeep is now known as ysandeep|afk | 13:41 | |
| AJaeger | hrw: libmysqlclient-dev is missing on buster - what's its name? That needs updating in requirements/bindep.txt | 13:43 |
| hrw | AJaeger: libmariadb-dev | 13:44 |
| hrw | AJaeger: https://review.opendev.org/#/c/731728/ | 13:45 |
| AJaeger | ;) thanks | 13:45 |
| *** olaph has joined #opendev | 13:47 | |
| *** hashar has joined #opendev | 13:52 | |
| *** mlavalle has joined #opendev | 14:00 | |
| openstackgerrit | Merged zuul/zuul-jobs master: upload-artifactory: fix variable misspelling https://review.opendev.org/731535 | 14:15 |
| *** abhishekk has joined #opendev | 14:21 | |
| openstackgerrit | Merged zuul/zuul-jobs master: ensure-bazel: replace ignore_errors with failed_when https://review.opendev.org/730733 | 14:21 |
| abhishekk | I have question related to meetpad | 14:21 |
| fungi | abhishekk: hopefully we have answers | 14:22 |
| abhishekk | If I create a room on meetpad then to invite others I just need to share that url with the members, right? | 14:22 |
| abhishekk | for example; https://meetpad.opendev.org/<room-name> | 14:22 |
| abhishekk | 2. Right now when I joined one room, I am seeing one etherpad there that is there for testing purpose? | 14:25 |
| corvus | clarkb, mordred: can you check my comments on https://review.opendev.org/730999 ? | 14:29 |
| *** sgw has joined #opendev | 14:29 | |
| *** ykarel|afk is now known as ykarel | 14:32 | |
| abhishekk | fungi, got the answers, thank you | 14:32 |
| fungi | abhishekk: cool, but just to confirm yes every meetpad room corresponds to a like-named etherpad document | 14:33 |
| fungi | so meetpad.o.o/foo uses the same pad as you get by going to etherpad.o.o/foo | 14:34 |
| abhishekk | oh, cool | 14:34 |
| smcginnis | I haven't checked, but I would assume if you called a meetpad meeting the same as an existing etherpad, you would end up automatically picking up that existing etherpad. For better or worse. | 14:34 |
| fungi | smcginnis: correct, we relied on that for some of our testing even | 14:35 |
| abhishekk | smcginnis, yes you get the same one | 14:35 |
| fungi | the expectation is that teams who are using their ptg planning etherpads for taking notes could just use that same name for their meetpad room | 14:35 |
| fungi | similar to putting that pad up on a projection screen at an in-person ptg | 14:36 |
| abhishekk | +1 | 14:36 |
| fungi | except that you'll be able to edit the pad directly through your jitsi-meet window rather than needing to have it in a separate browser window/tab | 14:36 |
| openstackgerrit | Oleksandr Kozachenko proposed zuul/zuul-jobs master: Add the namespace in wait-for-pods role https://review.opendev.org/731292 | 14:44 |
| openstackgerrit | Oleksandr Kozachenko proposed zuul/zuul-jobs master: Add namespace in the collect-k8s-logs role https://review.opendev.org/731319 | 14:44 |
| abhishekk | fungi, I can see force reconnect to etherpad message frequently, is it due to less network speed? | 14:51 |
| fungi | abhishekk: it might be that you have another browser window/tab open to that same etherpad directly? | 14:51 |
| abhishekk | fungi, not till now | 14:52 |
| abhishekk | also in meetpad etherpad window I have written some contents which are still there, but when I opened it using etherpad.o.o/foo I am not seeing those | 14:53 |
| fungi | mordred: have we possibly broken ptgbot updates on eavesdrop? | 14:54 |
| fungi | discussing in #openstack-ptg but the deployment job doesn't seem to have updated the git repo in /opt.ptgbot | 14:55 |
| fungi | er, /opt/ptgbot | 14:55 |
| fungi | (when you're back this afternoon) | 14:55 |
| fungi | i'll dig deeper in the meantime | 14:55 |
| fungi | abhishekk: in that case it sounds like something might be wrong with the websocket connection back to the etherpad api | 14:56 |
| fungi | what browser are you using? | 14:56 |
| abhishekk | firefox | 14:57 |
| corvus | abhishekk: what's the meetpad url? i can try to join and see if i have similar problems | 14:58 |
| abhishekk | corvus, https://meetpad.opendev.org/Konan | 14:59 |
| clarkb | note firefox is expected to not work as well with jitsi meet | 14:59 |
| abhishekk | clarkb, let me shift to chrome then | 15:00 |
| clarkb | thats from my own experience testing but also the perf docs frickler found | 15:00 |
| *** hashar has quit IRC | 15:00 | |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul-jobs master: WIP: improve ensure-tox, ensure-twine, upload-pypi https://review.opendev.org/731784 | 15:01 |
| abhishekk | clarkb, ack | 15:01 |
| corvus | abhishekk: i think the issue with the etherpad text not propagating is case sensitiveti | 15:02 |
| corvus | abhishekk: https://etherpad.opendev.org/p/konan | 15:02 |
| corvus | that's where we ended up | 15:02 |
| corvus | fungi, clarkb: ^ | 15:02 |
| abhishekk | corvus, let me check | 15:02 |
| abhishekk | corvus, yes, now I can see it | 15:03 |
| corvus | notice the captial K for the meetpad url gets translated to lowercase for the etherpad | 15:03 |
| fungi | interesting | 15:04 |
| abhishekk | aha, so it will be case for all the rooms who will use capital letters in room names | 15:04 |
| fungi | we likely didn't think to test with capital letters | 15:04 |
| corvus | yeah, though i'd like to see if we can fix that; i'm not sure where the translation is happening | 15:04 |
| corvus | i'll be back after breakfast | 15:04 |
| fungi | okay, so update on the ptgbot deployment, looks like we may have regressed infra-prod-remote-puppet-else | 15:05 |
| abhishekk | corvus, thank you | 15:06 |
| fungi | hrm, nope, maybe not the infra-prod-remote-puppet-else failures, it looks like we're not applying it at all on eavesdrop.o.o | 15:07 |
| fungi | i guess infra-prod-service-eavesdrop is replacing it for that server, but doesn't do ptgbot | 15:12 |
| openstackgerrit | Oleksandr Kozachenko proposed zuul/zuul-jobs master: Add namespace in the collect-k8s-logs role https://review.opendev.org/731319 | 15:14 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul-jobs master: WIP: improve ensure-tox, ensure-twine, upload-pypi https://review.opendev.org/731784 | 15:16 |
| fungi | infra-root: should i manually recreate the vcsrepo and exec steps here for now, or do we have something which is supposed to be running this still? https://opendev.org/opendev/puppet-ptgbot/src/branch/master/manifests/init.pp | 15:16 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul-jobs master: WIP: improve ensure-tox, ensure-twine, upload-pypi https://review.opendev.org/731784 | 15:24 |
| corvus | fungi: looking | 15:26 |
| fungi | oh, playbooks/service-eavesdrop.yaml is still supposed to run-puppet for eavesdrop.pp | 15:29 |
| fungi | so maybe it just needs to run on changes to the ptgbot repo | 15:30 |
| fungi | it hasn't built since the most recent fix merged | 15:30 |
| corvus | when was that? | 15:30 |
| ttx | 3 hours ago | 15:31 |
| fungi | patch for openstack/ptgbot merged 12:42z but infra-prod-service-eavesdrop last ran at 07:24z | 15:31 |
| fungi | we're running it in periodic | 15:31 |
| ttx | ah it's periodic, not hourly | 15:31 |
| corvus | probably the quickest way to get it deployed would be to run the service plabyook manually on bridge | 15:32 |
| fungi | yep, i can give that a whirl | 15:32 |
| fungi | thanks for looking! | 15:33 |
| corvus | and yeah, we should be able to run it on changes to ptgbot repo, but it might take us a few days to get that nailed down, so i'd be in favor of just running that manually as necessary until we do | 15:34 |
| clarkb | corvus: that plan wfm | 15:35 |
| fungi | where do we keep the system-config playbooks on bridge.o.o nowadays? | 15:36 |
| fungi | the /opt/system-config checkout looks stale | 15:36 |
| clarkb | fungi: /home/zuul/src/opendev.org/opendev/system-config | 15:36 |
| clarkb | fungi: its like a zuul node | 15:36 |
| fungi | ahh, thanks! | 15:36 |
| fungi | yep | 15:36 |
| fungi | so should i run `ansible-playbook -l eavesdrop.openstack.org playbooks/service-eavesdrop.yaml` as zuul from there? | 15:37 |
| clarkb | I'm having a bit of a slow start today, but if no oen else is wanting to do it I'll work on spinning up jvb02-04 after some breakfast | 15:37 |
| clarkb | fungi: I think you may need to sudo it (the zuul user bit is less important) | 15:37 |
| fungi | er, i guess it's eavesdrop01 | 15:37 |
| fungi | ahh, okay, so zuul runs it with root privs anyway? | 15:37 |
| clarkb | fungi: yes | 15:37 |
| corvus | i don't think you'll need a limit | 15:38 |
| corvus | should just be able to do 'ansible-playbook playbooks/service-eavesdrop.yaml' | 15:38 |
| clarkb | and ya limit may not be necessary since its a service specific playbook that should have the appropriate hosts set | 15:38 |
| fungi | got it, thanks | 15:38 |
| fungi | #status log manually ran `ansible-playbook playbooks/service-eavesdrop.yaml` as root from /home/zuul/src/opendev.org/opendev/system-config on bridge.o.o to deploy a ptgbot fix faster than the daily periodic pipeline run would have | 15:42 |
| openstackstatus | fungi: finished logging | 15:43 |
| fungi | that seems to have done the trick | 15:43 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Make .sh browsable on swift logs https://review.opendev.org/731795 | 15:56 |
| *** ykarel is now known as ykarel|away | 15:58 | |
| clarkb | corvus: more re https://review.opendev.org/#/c/730999/7/testinfra/test_base.py that is a good point. I think maybe what we should do is test the base rules in the base case (snmp and ssh). Then zuul can check things like 443 and 19885 ? | 16:03 |
| clarkb | s/more// | 16:03 |
| openstackgerrit | Jeremy Stanley proposed openstack/project-config master: Deploy ptgbot and puppet-ptgbot changes https://review.opendev.org/731796 | 16:04 |
| corvus | clarkb: well, 19885 is there to verify that we haven't botched the streaming port on our test node; it's not actually something handled by iptables. but we could check 4730 on the scheduler | 16:04 |
| clarkb | ++ | 16:05 |
| openstackgerrit | Merged opendev/irc-meetings master: Remove the congress team meeting https://review.opendev.org/721744 | 16:11 |
| clarkb | I've started the jvb assembly line | 16:13 |
| fungi | on a related ptg note, reviews of 731796 would be appreciated... i assume it's safe, but others likely know better | 16:14 |
| *** dpawlik has quit IRC | 16:15 | |
| clarkb | fungi: they are in the same tenant so it should function. (we don't do similar with zuul/nodepool due to different tenancy) | 16:16 |
| corvus | fungi: it's worth noting that when we move system-config into the opendev tenant, we won't be able to do the deploy on 'openstack/ptgbot' changes any more | 16:16 |
| clarkb | what we do with zuul and nodepool is to run them hourly instead | 16:17 |
| clarkb | maybe that is a better future proof option? | 16:17 |
| corvus | yeah, we might want to go ahead and do that while we've got this all in our heads, so we don't have to untangle it later | 16:17 |
| fungi | yeah, i think going back to running service-eavesdrop hourly is probably safe now that we're limiting run-accessbot to only fire for changes to the config | 16:18 |
| fungi | cool, i'll adjust | 16:18 |
| *** rpittau is now known as rpittau|afk | 16:21 | |
| openstackgerrit | Jeremy Stanley proposed opendev/system-config master: Deploy eavesdrop hourly https://review.opendev.org/731800 | 16:23 |
| fungi | clarkb: corvus: ^ | 16:24 |
| openstackgerrit | Sorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Make .sh browsable on swift logs https://review.opendev.org/731795 | 16:25 |
| *** dtantsur is now known as dtantsur|afk | 16:29 | |
| openstackgerrit | Clark Boylan proposed opendev/zone-opendev.org master: Add three more jvbs to DNS https://review.opendev.org/731802 | 16:29 |
| openstackgerrit | Clark Boylan proposed opendev/system-config master: Add three more JVB servers https://review.opendev.org/731803 | 16:30 |
| clarkb | infra-root ^ that should be ready. I've accepted ssh host keys. I do need to set up reverse dns though but that is quick | 16:31 |
| clarkb | and now reverse dns is done | 16:32 |
| clarkb | that gives us 5 running jvb processes at ~40 users each (based on not a ton of of testing) | 16:33 |
| clarkb | mordred: ^ I think those changes will conflict with your reorgs, but corvus' comment on the iptabltes change likely means ar ebase anyway | 16:50 |
| *** roman_g has quit IRC | 16:59 | |
| *** roman_g has joined #opendev | 17:00 | |
| clarkb | fungi: have a moment for topic:more-jvbs ? both changes have passed the check queue at this point | 17:02 |
| fungi | yeah, gimme a sec to switch gears | 17:02 |
| openstackgerrit | Merged opendev/system-config master: Deploy eavesdrop hourly https://review.opendev.org/731800 | 17:12 |
| clarkb | fungi: thanks! | 17:17 |
| fungi | you bet. it was a nice (albeit brief) distraction ;) | 17:19 |
| openstackgerrit | Merged opendev/zone-opendev.org master: Add three more jvbs to DNS https://review.opendev.org/731802 | 17:19 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Do not lint loopvar in test-playbooks https://review.opendev.org/731814 | 17:24 |
| openstackgerrit | Merged openstack/project-config master: Restrict wheel publish jobs https://review.opendev.org/731632 | 17:25 |
| openstackgerrit | Merged opendev/system-config master: Add three more JVB servers https://review.opendev.org/731803 | 17:36 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Do not lint loopvar in test-playbooks https://review.opendev.org/731814 | 17:42 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Do not lint loopvar in test-playbooks https://review.opendev.org/731814 | 17:51 |
| mordred | clarkb: nod. and yes - the corvus iptables comment is, it turns out, important :) | 18:08 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Split inventory into multiple dirs and move hostvars https://review.opendev.org/730991 | 18:09 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Override bridge hostvars directly https://review.opendev.org/731258 | 18:09 |
| mordred | blast. one sec | 18:09 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run iptables in service playbooks instead of base https://review.opendev.org/730999 | 18:09 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Rename service-letsencrypt to just letsencrypt https://review.opendev.org/731617 | 18:10 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Split inventory into multiple dirs and move hostvars https://review.opendev.org/730991 | 18:10 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Override bridge hostvars directly https://review.opendev.org/731258 | 18:10 |
| *** elod has quit IRC | 18:10 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run iptables in service playbooks instead of base https://review.opendev.org/730999 | 18:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Rename service-letsencrypt to just letsencrypt https://review.opendev.org/731617 | 18:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Split inventory into multiple dirs and move hostvars https://review.opendev.org/730991 | 18:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Override bridge hostvars directly https://review.opendev.org/731258 | 18:11 |
| mordred | clarkb: ok- that should be fixed and then rebased | 18:11 |
| clarkb | meetpad updates should happen soon. The deploy buildset is jugging through its list | 18:12 |
| *** elod has joined #opendev | 18:17 | |
| AJaeger | config-core, here's a new airship repo creation request for review: https://review.opendev.org/731236 | 18:20 |
| clarkb | new jvb's look good at first glance. I'm going to find some lunch but we should probably do a quick conference with someone that has a wecam today. (and then start like 5 of them to ensure all 5 jvb's are doing work) | 18:40 |
| clarkb | fungi: corvus ^ I think you have webcams | 18:40 |
| clarkb | oh and I'll double check cacti after lunch too | 18:41 |
| *** elod has quit IRC | 18:45 | |
| *** elod has joined #opendev | 18:45 | |
| corvus | clarkb: yeah, i'll join you after lunch | 18:46 |
| fungi | yeah, i have my studio all set up for next week, so can help test more easily than in previous weeks | 18:51 |
| *** elod has quit IRC | 19:20 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run iptables in service playbooks instead of base https://review.opendev.org/730999 | 19:23 |
| openstackgerrit | Jeremy Stanley proposed opendev/infra-specs master: Central Authentication Service https://review.opendev.org/731838 | 19:28 |
| fungi | clarkb: corvus: mordred: ^ | 19:28 |
| fungi | my attempt at collation and distillation | 19:28 |
| fungi | it's not as distilled as i would like, but at least it flows somewhat smoothly | 19:29 |
| fungi | still may be a bit disjoint in places | 19:30 |
| fungi | not exactly light reading | 19:30 |
| fungi | and took far longer to knock out than i imagined | 19:31 |
| corvus | clarkb: back and ready whenever you are | 19:31 |
| clarkb | corvus: just a few more minutes for me | 19:32 |
| clarkb | ok here now | 19:36 |
| clarkb | I jumped into https://meetpad.opendev.org/meetpadtest1 | 19:37 |
| fungi | do we need to do separate pads each to exercise the different jvbs? | 19:39 |
| clarkb | fungi: ya | 19:42 |
| clarkb | we are on meetpadtest1 through meetpadtest5 | 19:43 |
| clarkb | 5 is where we are unmuted | 19:43 |
| clarkb | things look good | 19:44 |
| fungi | ahh, so i can open multiple chromium tabs to different pads? | 19:45 |
| clarkb | fungi: ya just be sure to mute in all but one | 19:45 |
| clarkb | fungi: however I think we are dropping now | 19:45 |
| clarkb | I can stay on if you want to test too | 19:45 |
| corvus | oh i just dropped | 19:46 |
| clarkb | fungi: I'm unmuted on 05 | 19:46 |
| clarkb | er I guess its just 5 | 19:46 |
| fungi | no worries | 19:46 |
| clarkb | I see you on 1 | 19:46 |
| clarkb | `sudo docker logs jitsi-meet-docker_jicofo_1 2>&1 | grep 'Added participant'` on meetpad01 to see where people are ending up | 19:48 |
| *** elod has joined #opendev | 19:51 | |
| corvus | based on the js console logs, i think jitsi meet may only use lowercase room names | 19:54 |
| corvus | 2020-05-29T19:51:14.607Z [modules/xmpp/ChatRoom.js] Joined MUC as apparentantelopesrejoicewonderfully@muc.localhost/2d2b0daf | 19:55 |
| corvus | that's for https://meetpad.opendev.org/ApparentAntelopesRejoiceWonderfully | 19:55 |
| clarkb | ah an expectation mismatch between meetpad and etherpad I guess then | 19:55 |
| clarkb | er jitsi meet and etherpad | 19:55 |
| fungi | sorry i missed the load test, but thanks clarkb for helping me double-check my input levels and streaming | 19:56 |
| fungi | if folks want to rename existing etherpads to their lower-case versions, i'm happy to do that for them. it's all of one api call | 19:57 |
| clarkb | etherpads auto generated names use upper case characters too. Those might be clunky to deal with | 19:58 |
| fungi | i think i also have a long-standing todo list entry to add documentation on how to use the admin api for etherpad | 19:58 |
| corvus | https://github.com/jitsi/jicofo/blob/master/src/main/java/org/jitsi/impl/protocol/xmpp/OperationSetMultiUserChatImpl.java#L136 | 19:58 |
| clarkb | but for the PTG we can probably get away with having people use lower case human generated names | 19:58 |
| fungi | i sort of expected ptg sessions to use their pregenerated etherpads as meetpad rooms anyway | 19:59 |
| corvus | https://github.com/jitsi/jicofo/commit/4ba9529087aa1398362c94e4acfe83e77f6b41d4 | 19:59 |
| corvus | fungi: are the pre-generated pads lc? | 19:59 |
| clarkb | corvus: oh is that a limitation of xmpp multi user chat? | 19:59 |
| clarkb | http://ptg.openstack.org/etherpads.html most but not all of them are lc | 20:00 |
| fungi | corvus: not sure if they're truly consistently pregenerated or if teams are just precreating them individually. i assume the latter | 20:00 |
| clarkb | ironic, octavia, and sdk-cli appear to use mixed case etherpads | 20:01 |
| clarkb | mordred: ^ is sdk-cli something you are involved with? maybe we can use that as our guinea pig for renaming if so? | 20:01 |
| clarkb | johnsom: ^ not sure if you plan to use meetpad but if you do we may need to rename your etherpad (whcih we can do via the etherpad api to keep the content and hsitory) | 20:04 |
| clarkb | rm_work: is here too cool ^ | 20:04 |
| johnsom | Yeah, we are planning to use jitsi. The bummer is we all have book marked it as is | 20:04 |
| clarkb | johnsom: bookmarked the meetpad url or etherpad url? | 20:06 |
| johnsom | etherpad | 20:06 |
| johnsom | I'm not 100% sure it is important to have the etherpad part of the session though, we all can have a separate etherpad window. | 20:07 |
| clarkb | ok if it was meetpad it would just work. But with etherpad you'll end up with two distinct sets of content | 20:07 |
| johnsom | Right, I just tested that. | 20:07 |
| mordred | clarkb: yah. I think it's fine to change the url there aren't that many of us | 20:09 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Rename service-letsencrypt to just letsencrypt https://review.opendev.org/731617 | 20:10 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Split inventory into multiple dirs and move hostvars https://review.opendev.org/730991 | 20:10 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Override bridge hostvars directly https://review.opendev.org/731258 | 20:10 |
| corvus | a lot of that code has changed since then; i wonder if that's still necessary | 20:19 |
| clarkb | I'm digging through specs to figure out what room JID requirements are fwiw | 20:22 |
| corvus | the docker images for the java stack basically just install debs, so building a local patch for that isn't as straightforward as it was for the JS | 20:23 |
| corvus | maybe i can dpkg-build a new jicofo, then patch it into the jicofo container image | 20:24 |
| clarkb | from what I can tell xmpp allows for unicode under this rfc https://tools.ietf.org/html/rfc3454 | 20:26 |
| clarkb | that does include some normalization but its of codepoints not changing meaning aiui | 20:26 |
| clarkb | (all that to say I don't think xmpp itself requires room JIDs be lower case) | 20:26 |
| corvus | i've removed the tolower line and am running this in jicofo: "dpkg-buildpackage -A -rfakeroot -us -uc" | 20:27 |
| corvus | does that sound like a plausible process? | 20:27 |
| clarkb | fungi: mordred ^ (I am the wrong person to ask about deb builds) | 20:28 |
| corvus | fungi, mordred: this is the repo; i'm running that command in its root: https://github.com/jitsi/jicofo/ | 20:28 |
| corvus | so you can see the debian dir there | 20:28 |
| fungi | tending the wok any second, but will see what i can see | 20:33 |
| corvus | seems to have built something | 20:33 |
| mordred | corvus: your dpkg-buildpackage line lgtm | 20:34 |
| corvus | cool, i'll try installing it and see what happens | 20:34 |
| mordred | corvus: worst case it'll cause the universe to implode, but probably fine | 20:35 |
| fungi | yeah, your command there looks reasonable. i generally just call it with debuild -b -uc -us | 20:35 |
| corvus | hrm, the .deb made on my bionic machine doesn't seem very compatible with the stretch image; i'll rebuild it in a docker container | 20:42 |
| fungi | or you can use something like bionic's pbuilder package to create a stretch chroot | 20:43 |
| fungi | or even just mdebootstrap/cdebootstrap | 20:43 |
| corvus | hrm, i'm not making very good progress | 20:58 |
| corvus | so the original error when i tried to install my bionic build in the image was from dpkg: invoke-rc.d: syntax error: unknown option "--skip-systemd-native" | 20:59 |
| corvus | so i tried building on debian stretch and gotthis from dpkg-buildpackage: dh: Sorry, but 11 is the highest compatibility level supported by this debhelper. | 20:59 |
| corvus | so i tried debian buster, and it has no openjdk-8-jre-headless | 21:00 |
| fungi | newer debhelper is likely available in stretch-backports | 21:05 |
| clarkb | corvus: the control file seems to say java 11 is acceptable too? | 21:05 |
| clarkb | Build-Depends: debhelper, openjdk-8-jre-headless | openjdk-11-jre-headless, maven | 21:05 |
| corvus | clarkb: yeah, but that's not what the docker images run | 21:05 |
| fungi | but in general, this does not sound like a package which would pass debian's (admittedly rigorous) qa standards | 21:05 |
| corvus | maybe that's how the debs are built though? | 21:05 |
| clarkb | corvus: oh I see | 21:05 |
| corvus | there's no info on their upstream build env | 21:05 |
| corvus | fungi: so maybe try building on stretch with backports added...? | 21:06 |
| corvus | clarkb: or build on buster with 11? | 21:06 |
| fungi | yeah, stretch-backports has debhelper 12 | 21:06 |
| mordred | I'd try that | 21:06 |
| fungi | so compat level 11 should work with newer debhelper | 21:06 |
| corvus | yeah, my gut says they're using stretch across the board | 21:06 |
| mordred | fungi: their compat file is actually looking for 12 - so that's good | 21:07 |
| mordred | https://github.com/jitsi/jicofo/blob/master/debian/compat | 21:07 |
| fungi | yeah, openjdk-11-jre-headless is also available in stretch-backports | 21:07 |
| fungi | i bet they built this on stretch with stretch-backports enabled | 21:08 |
| mordred | if only there was somehting that showed how they did that | 21:08 |
| corvus | oh yeah, they add stretch-backports to the run-time docker images, so that seems likely they'd have it for the build | 21:08 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Split inventory into multiple dirs and move hostvars https://review.opendev.org/730991 | 21:12 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Override bridge hostvars directly https://review.opendev.org/731258 | 21:12 |
| *** paladox has quit IRC | 21:27 | |
| *** paladox has joined #opendev | 21:29 | |
| corvus | welp, that was a long path back to the same spot: invoke-rc.d: syntax error: unknown option "--skip-systemd-native" | 21:31 |
| corvus | i don't know what that means, but i'm really pleased to see that systemd is involved | 21:32 |
| clarkb | corvus: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=907836 | 21:33 |
| openstack | Debian bug 907836 in lintian "lintian: add check for usage of invoke-rc.d --skip-systemd-native without a suitable Pre-Depends" [Wishlist,Fixed] | 21:33 |
| clarkb | that has a pre depends thing we might be able to add to the control file to fix it? | 21:33 |
| corvus | i'm not sure what we should add there though | 21:35 |
| corvus | oh looks like since we're using debhelper we can use that $misc thing | 21:35 |
| corvus | i'll give that a shot | 21:35 |
| fungi | the version of init-system-helpers in stretch is older than when that option was added, and there's no newer version in stretch-backports, so i have limited hope that will help | 21:36 |
| fungi | if i'm interpreting that correctly, debhelper 12 from backports is adding --skip-systemd-native to the invoke-rc.d command line, assuming it will have a new enough version of the init-system-helpers package to support that option | 21:38 |
| fungi | no, wait, i misread | 21:39 |
| fungi | there is a new enough version in stretch-backports | 21:39 |
| fungi | so the versioned pre-depends may help | 21:39 |
| fungi | or preinstalling init-system-helpers/stretch-backports | 21:40 |
| corvus | jicofo pre-depends on init-system-helpers (>= 1.54~) init-system-helpers is installed, but is version 1.48. | 21:40 |
| fungi | huh | 21:40 |
| clarkb | does the ~ means something important there? | 21:40 |
| fungi | corvus: oh, likely needs to be upgraded in the build environment | 21:41 |
| corvus | that's in the runtime env | 21:41 |
| corvus | so we'd need to upgrade it there? | 21:41 |
| fungi | clarkb: ~ is a "sort before" operator used for prereleases | 21:41 |
| clarkb | fungi: ah | 21:41 |
| fungi | corvus: oh, yeah may need to manually upgrade init-system-helpers to the version from stretch-backports in the runtime environment first then since it's a pre-dependency and not a normal dependency | 21:43 |
| corvus | okay, i have done that and it is now installed | 21:44 |
| fungi | basically meaning it has to be installed before the pre-install maintscripts for jicofo are run by apt | 21:44 |
| corvus | i have no idea how this really works | 21:44 |
| fungi | under normal circumstances it's not something you would encounter | 21:44 |
| fungi | but mixing low-level packaging bits from backports gets hairy like that | 21:44 |
| corvus | well, i don't understand how the jicofo deb from upstream gets installed | 21:45 |
| fungi | ahh | 21:45 |
| corvus | we should be building the deb pretty close to how they are, and we're installing it pretty close to how they are | 21:45 |
| corvus | so something isn't adding up | 21:45 |
| fungi | are you still getting the same error? | 21:45 |
| corvus | nope: okay, i have done that and it is now installed | 21:45 |
| corvus | i understand what we have done to fix the errors; i don't understand why they happened in the first place | 21:46 |
| corvus | these packages are updated upstream almost continuously | 21:46 |
| corvus | basically, they appear to be the result of an impossible combination of factors | 21:47 |
| corvus | anyway, i have a runtime image; i'll try it out locally and if it works, push it up to dockerhub | 21:48 |
| fungi | it seems like they have a build environment which is stretch with everything upgraded to stretch-backports (which is not really how stretch-backports is meant to be used) | 21:49 |
| fungi | i *think* that would account for all the hurdles you encountered anyway | 21:49 |
| corvus | jicofo master didn't seem to work with the rest of the system; i'm rebuilding at stable-4627 | 22:10 |
| corvus | good news and bad news: i have a local system up and running now with the new locally-built image, but removing that lowercase call is insufficient. | 22:19 |
| corvus | it looks like jitsi-meet itself has some lowercasing | 22:20 |
| fungi | gnarly | 22:24 |
| corvus | i'll try changing that too | 22:25 |
| fungi | sorry i'm not more help, rapidly approaching evening braindeath | 22:26 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul-jobs master: ensure-tox: fix tox_executable fact when tox is preinstalled https://review.opendev.org/731853 | 22:36 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul-jobs master: ensure-twine: Update using same format as ensure-tox https://review.opendev.org/731854 | 22:36 |
| *** DSpider has quit IRC | 22:57 | |
| *** mlavalle has quit IRC | 22:58 | |
| corvus | all right, that's got us to: 2020-05-29T22:59:25.613Z [JitsiConference.js] <oe>: Invalid conference name (no conference name passed or it contains invalid characters like capital letters)! | 23:00 |
| corvus | that's in lib-jitsi-meet | 23:03 |
| clarkb | it actually says "like capital letters"? | 23:05 |
| corvus | https://github.com/jitsi/lib-jitsi-meet/blob/master/JitsiConference.js#L121 | 23:06 |
| corvus | clarkb: which is funny because that's the only thing it checks. | 23:07 |
| corvus | okay, i've removed that and am rebuilding | 23:14 |
| corvus | "thank goodness" it just installes from a github sha instead of via an npm release | 23:15 |
| ianw | AJaeger: ianw: https://review.opendev.org/731637 is required on other branches? -- it seems that the jobs have been removed from the other branches, i'd say queens was just overlooked | 23:38 |
| *** tosky has quit IRC | 23:38 | |
| corvus | okay, that starts but does not work | 23:38 |
| corvus | clarkb, fungi, mordred: i think i've gone far enough down the rabbit hole to say that it's not a trivial matter to remove the case restriction -- there are assumptions lurking somewhere in some components, so i don't think this is something we can expect to change over the weekend. | 23:40 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!