Monday, 2020-12-07

*** zzzeek has quit IRC00:12
*** zzzeek has joined #openstack-infra00:16
*** tosky has quit IRC00:20
*** ociuhandu has joined #openstack-infra00:22
*** ociuhandu has quit IRC00:26
*** jamesdenton has quit IRC00:48
*** jamesden_ has joined #openstack-infra00:49
*** ociuhandu has joined #openstack-infra02:10
*** ociuhandu has quit IRC02:15
*** ramishra has joined #openstack-infra03:01
*** ramishra has quit IRC03:08
*** ramishra has joined #openstack-infra03:11
*** ociuhandu has joined #openstack-infra03:24
*** ociuhandu has quit IRC03:29
*** vesper11 has quit IRC03:34
*** vesper11 has joined #openstack-infra03:35
*** zxiiro has quit IRC03:46
*** d34dh0r53 has joined #openstack-infra03:59
*** jamesden_ has quit IRC04:34
*** jamesdenton has joined #openstack-infra04:35
*** vesper has joined #openstack-infra04:36
*** vesper11 has quit IRC04:37
*** ociuhandu has joined #openstack-infra05:12
*** ociuhandu has quit IRC05:17
*** ramishra has quit IRC05:31
*** evrardjp has quit IRC05:33
*** evrardjp has joined #openstack-infra05:33
*** ricolin has joined #openstack-infra05:37
*** ramishra has joined #openstack-infra05:57
*** vishalmanchanda has joined #openstack-infra06:15
rakhmerovfungi: yes, I understand all that. I'm not familiar with how all these gates work well enough I guess. It seems like it's out of our control what mirrors need to be used in devstack gates, I don't see that it's configured either in .zuul.yaml or plugin.sh files06:30
rakhmerovfungi: for what it's worth, it started working on its own over the weekened06:30
rakhmerovso it's now solved06:30
*** lmiccini has joined #openstack-infra06:53
*** matt_kosut has joined #openstack-infra06:56
*** fnordahl has quit IRC06:58
*** ociuhandu has joined #openstack-infra07:01
*** ociuhandu has quit IRC07:07
*** ociuhandu has joined #openstack-infra07:21
*** ociuhandu has quit IRC07:28
openstackgerritMerged openstack/project-config master: Add 'Abandon' permission for retired repo ACL  https://review.opendev.org/c/openstack/project-config/+/76558307:34
*** sboyron_ has joined #openstack-infra07:54
*** rcernin has quit IRC07:55
*** jcapitao has joined #openstack-infra08:06
*** savihou has quit IRC08:23
*** rcernin has joined #openstack-infra08:25
*** andrewbonney has joined #openstack-infra08:28
*** rcernin has quit IRC08:30
*** fdegir has quit IRC08:34
*** fdegir has joined #openstack-infra08:34
*** dchen has quit IRC08:42
*** lpetrut has joined #openstack-infra08:44
*** rpittau|afk is now known as rpittau08:46
*** ociuhandu has joined #openstack-infra08:53
*** ociuhandu has quit IRC08:53
*** lucasagomes has joined #openstack-infra08:54
*** hashar has joined #openstack-infra09:02
*** derekh has joined #openstack-infra09:04
*** ociuhandu has joined #openstack-infra09:15
*** xek_ has joined #openstack-infra09:17
*** zzzeek has quit IRC09:19
*** zzzeek has joined #openstack-infra09:20
*** mgoddard has joined #openstack-infra09:20
*** rcernin has joined #openstack-infra09:25
*** ociuhandu has quit IRC09:28
*** rcernin has quit IRC09:30
*** dtantsur|afk is now known as dtantsur10:07
*** rcernin has joined #openstack-infra10:21
*** rcernin has quit IRC10:21
openstackgerritDmitriy Rabotyagov proposed openstack/project-config master: Deprecate openstack-ansible-galera_client role  https://review.opendev.org/c/openstack/project-config/+/76577710:28
*** ociuhandu has joined #openstack-infra10:30
*** mgoddard has quit IRC10:33
*** jcapitao has quit IRC10:39
*** jcapitao has joined #openstack-infra10:45
*** mgoddard has joined #openstack-infra10:49
*** mgoddard has quit IRC10:52
*** mgoddard has joined #openstack-infra10:53
openstackgerritDmitriy Rabotyagov proposed openstack/project-config master: Remove noop jobs for galera_client deprecation  https://review.opendev.org/c/openstack/project-config/+/76578310:56
openstackgerritDmitriy Rabotyagov proposed openstack/project-config master: Deprecate openstack-ansible-galera_client role  https://review.opendev.org/c/openstack/project-config/+/76577711:02
openstackgerritDmitriy Rabotyagov proposed openstack/project-config master: Remove noop jobs for galera_client deprecation  https://review.opendev.org/c/openstack/project-config/+/76578311:02
openstackgerritDmitriy Rabotyagov proposed openstack/project-config master: Deprecate openstack-ansible-galera_client role  https://review.opendev.org/c/openstack/project-config/+/76577711:04
openstackgerritDmitriy Rabotyagov proposed openstack/project-config master: Deprecate openstack-ansible-galera_client role  https://review.opendev.org/c/openstack/project-config/+/76577711:09
openstackgerritDmitriy Rabotyagov proposed openstack/project-config master: Remove noop jobs for galera_client deprecation  https://review.opendev.org/c/openstack/project-config/+/76578311:11
openstackgerritDmitriy Rabotyagov proposed openstack/project-config master: Remove noop jobs for galera_client deprecation  https://review.opendev.org/c/openstack/project-config/+/76578311:11
openstackgerritMartin Kopec proposed openstack/project-config master: Move ansible-role-refstack-client from x/ to osf/  https://review.opendev.org/c/openstack/project-config/+/76578711:18
*** tosky has joined #openstack-infra11:22
*** DinaBelova has quit IRC11:46
*** DinaBelova has joined #openstack-infra11:51
*** rfolco has joined #openstack-infra11:54
*** zzzeek has quit IRC12:03
*** zzzeek has joined #openstack-infra12:05
*** jcapitao is now known as jcapitao_lunch12:09
*** ociuhandu has quit IRC12:09
openstackgerritDmitriy Rabotyagov proposed openstack/project-config master: Revive os_monasca  https://review.opendev.org/c/openstack/project-config/+/76579912:30
openstackgerritDmitriy Rabotyagov proposed openstack/project-config master: Revive os_monasca  https://review.opendev.org/c/openstack/project-config/+/76579912:33
*** zzzeek has quit IRC12:38
*** zzzeek has joined #openstack-infra12:39
*** hashar has quit IRC12:40
*** hashar has joined #openstack-infra12:40
*** priteau has quit IRC12:40
*** ociuhandu has joined #openstack-infra12:40
*** hashar has quit IRC12:42
*** ociuhandu has quit IRC12:49
*** raissa has joined #openstack-infra12:51
*** rlandy has joined #openstack-infra13:00
fungirakhmerov: a quick codesearch for the image name shows that's being requested in heat's in-repository job configuration and was fixed over the weekend by https://review.opendev.org/765325 and related backport to stable/victoria13:03
*** hashar has joined #openstack-infra13:05
*** ociuhandu has joined #openstack-infra13:12
*** jcapitao_lunch is now known as jcapitao13:17
openstackgerritDmitriy Rabotyagov proposed openstack/project-config master: Revive os_monasca  https://review.opendev.org/c/openstack/project-config/+/76579913:19
*** ociuhandu has quit IRC13:20
openstackgerritDmitriy Rabotyagov proposed openstack/project-config master: Re-add os_monasca roles to zuul  https://review.opendev.org/c/openstack/project-config/+/76580613:21
*** priteau has joined #openstack-infra13:27
*** ysandeep is now known as ysandeep|afk13:42
*** lyarwood has joined #openstack-infra13:46
lyarwoodhttps://289ffbb376a782338ce3-61fc75f3947f990826cd6aab56655a57.ssl.cf2.rackcdn.com/765769/1/check/openstack-tox-lower-constraints/04abea3/job-output.txt - `ERROR: Could not find a version that satisfies the requirement packaging>=20.4 (from oslo-utils)` - I'm not sure if someone is already working on this but is there a way to check if these mirrors are stale etc?13:47
*** Goneri has joined #openstack-infra13:48
lyarwoodhttps://pypi.org/project/packaging/#history - I assume this has something to do with 20.6 being yanked?13:48
fungilyarwood: i'll see if i can reproduce that locally13:51
*** __ministry1 has joined #openstack-infra13:53
*** __ministry1 has quit IRC13:53
*** ociuhandu has joined #openstack-infra13:53
lyarwoodfungi: thanks13:53
*** ociuhandu has quit IRC13:53
*** ociuhandu has joined #openstack-infra13:54
*** ociuhandu has quit IRC14:04
*** ociuhandu_ has joined #openstack-infra14:04
fungimmm, gonna be hard to reproduce locally without installing some extra system packages to be able to build those older sdists14:07
*** dayou has joined #openstack-infra14:07
fungithough if the problem were packaging 20.6 getting yanked you'd think 20.4, 20.5 or 20.7 would suffice14:07
fungioh, unless it's python-requires coming into play14:08
fungiRequires: Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*14:08
funginope, that doesn't seem to be the problem either14:08
fungiand that's with 20.714:08
*** dayou_ has quit IRC14:09
lyarwoodfungi: right, I'm assuming that 20.4, 20.5 or 20.7 aren't present in that mirror but I've got no way of confirming that14:14
fungii need to go pick up groceries but will continue trying to unravel a local reproduction of this when i get back (hopefully less than an hour from now)14:14
lyarwoodack np thanks :)14:14
fungithe "mirror" is just a caching http proxy to pypi14:14
fungiso there's nothing "in" it14:14
lyarwoodah14:14
fungianyway, back shortly14:15
*** ociuhandu_ has quit IRC14:19
*** owalsh has quit IRC14:19
*** raissa has quit IRC14:21
*** ysandeep|afk is now known as ysandeep14:33
*** ociuhandu has joined #openstack-infra14:34
*** artom has joined #openstack-infra14:37
*** owalsh has joined #openstack-infra14:42
*** ociuhandu has quit IRC14:46
*** owalsh has quit IRC14:47
*** ociuhandu has joined #openstack-infra14:49
*** zxiiro has joined #openstack-infra14:56
*** owalsh has joined #openstack-infra14:58
*** ociuhandu_ has joined #openstack-infra14:59
*** xek__ has joined #openstack-infra15:01
*** lpetrut has quit IRC15:03
*** ociuhandu has quit IRC15:03
*** xek_ has quit IRC15:04
fungimmm, yeah looks like i probably also need an older platform to reproduce this on since gcc can't build that version of greenlet against python3.915:06
fungihuh, `tox -e lower-constraints` in nova seems to want to create /usr/lib/systemd/system on my workstation, which of course it can't do without root permissions. that seems odd15:11
fungianyway, i think that means i got past the error indicated in the gate, leading me to suspect something related to the dep solver in latest pip. will try again with newer pip in the venv15:12
fungiaha, it's zVMCloudConnector which i can't install because of that15:15
fungilyarwood: ^ any idea how most devs deal with that?15:15
fungii have to assume you're not running tox as root anyway15:16
lyarwoodfungi: no, not as root15:18
lyarwoodfungi: I wonder if that's a qwirk of the lower-constraints env15:18
* lyarwood checks15:18
fungii'm trying with zVMCloudConnector removed from the local requirments/constraints copy for now under the assumption it's not involved in the problem you're encountering15:18
*** ociuhandu_ has quit IRC15:19
fungibasically just testing with the install phase of the lower-constraints testenv using a python3.6 venv as a basic reproducer15:19
fungiokay, with the pip version included in latest python 3.6 upstream it works. now to test with newest pip15:20
* lyarwood is waiting for his 3.7 tox env to build still15:23
fungilyarwood: bingo, reproduced locally, no special config15:24
fungihere's my steps:15:24
fungicheckout latest stable/victoria of nova (not sure if this also reproduces on master, haven't checked)15:25
fungicreate a python3.6 venv named "foo" (later python may also work, i was having trouble building greenlet against 3.9)15:25
fungiupgrade it to latest version of pip: `foo/bin/pip install -U pip`15:26
funginow try to install the constrainted dependency set: `foo/bin/pip install -c lower-constraints.txt -r test-requirements.txt -r requirements.txt`15:26
fungioh, optional step, edit out the zVMCloudConnector line in requirements.txt since at least for me it won't work without root perms15:27
fungilyarwood: so current theory is that this is the effect of the new dependency solver in pip 20.315:27
lyarwoodhmm 20.3.1 did seem to work with py3.7 for me15:28
lyarwoodso maybe a py3.6 bug with the newer pip?15:28
* lyarwood tests15:28
fungii'm going to test next with `foo/bin/pip install -U 'pip<20.3'`15:28
fungipython3.6 venv with pip 20.2.4 (last pre-20.3 release) succeeds15:29
lyarwoodfungi: and just to be clear you get the same packaging requirement failure with py3.6 and pip >=20.315:30
fungiyes, precisely the same error messages15:31
fungithe problem does not appear with pip 20.2.4 and python3.6, so now testing with pip 20.2.4 and adding --use-feature=2020-resolver15:32
fungiwhich should turn on the new dep solver from 20.3 under 20.215:32
fungiERROR: Could not find a version that satisfies the requirement packaging>=20.4 (from oslo-utils)15:32
fungiyep, reproduces15:32
lyarwoodkk, I can't reproduce this with py3.6 and 20.3.1 on master or stable/victoria15:32
fungiso i'm betting you can correlate the start of these failures with the date of the pip 20.3 release15:33
fungifor me it's reproducing on an up to date debian/unstable workstation with python versions compiled from upstream python source, so not really similar to our job nodes but reproducing the same exact error15:34
lyarwoodthese should all be focal15:35
lyarwoodmy env is f33 fully updated15:35
*** ociuhandu has joined #openstack-infra15:36
*** hberaud has quit IRC15:36
fungii'm seeing if i can repro with python3.8 now15:36
fungiyep, works for me on 3.815:37
fungii should say, reproduces exactly15:37
fungipython3.8 and pip 20.3.115:37
fungitrying master next15:37
fungiyep, i can make it happen with python3.8 and pip 20.3.1 on nova master as well15:39
fungier, actually was python3.9 that time, so apparently i don't hit the greenlet compilation problem on nova master15:40
openstackgerritSorin Sbârnea proposed openstack/project-config master: Enable tripleo core members to change WIP flag  https://review.opendev.org/c/openstack/project-config/+/76582115:41
fungialso the zvmcloudconnector problem doesn't surface for me on master either15:41
fungiso simplified reproducer:15:42
*** ysandeep is now known as ysandeep|away15:42
fungigit checkout origin/master15:42
openstackgerritSorin Sbârnea proposed openstack/project-config master: Enable tripleo core members to change WIP flag  https://review.opendev.org/c/openstack/project-config/+/76582115:42
fungigit clean -dfx15:42
fungipython3 -m venv foo15:42
*** hberaud has joined #openstack-infra15:42
fungifoo/bin/pip install -U pip15:42
fungifoo/bin/pip install -c lower-constraints.txt -r test-requirements.txt -r requirements.txt15:42
fungilyarwood: ^15:42
lyarwoodfungi: ack thanks, I wasn't upgrading the version of pip within the venv15:43
lyarwoodfungi: trying again now15:43
*** armax has joined #openstack-infra15:45
fungilyarwood: digging deeper, you can also reproduce far more quickly by replacing that last line with `foo/bin/pip install -c lower-constraints.txt oslo.utils`15:45
lyarwoodthanks let me write up a bug so we can at least track this now15:48
fungiyou're welcome15:50
fungiprobably the next step is to whittle down the constraints list to figure out exactly which one is preventing installing newer packaging15:51
fungiwithout the lower-constraints.txt it installs packaging-20.7 as expected15:51
fungioh, in fact this is almost certainly the packaging==17.1 line in lower-constraints.txt15:51
fungiso the error message makes sense15:52
lyarwoodah so it's a Nova bug15:54
lyarwoodthe resolver is actually catching it now15:54
fungilyarwood: oslo.utils added a dependency on packaging>=20.4 in https://review.opendev.org/739370 which appears as far back as 4.3.015:54
*** dklyle has joined #openstack-infra15:54
fungiso you'll either need to bump the packaging constraint to at least 20.4 or pin oslo.utils<4.315:55
lyarwoodyeah https://review.opendev.org/c/openstack/nova/+/748059 moved oslo.utils to 4.5 in LC15:56
lyarwoodnice15:56
fungii guess the naive install mechanism used in pip<20.3 was able to "wing it" but the new dep solver (correctly) rejects that contradiction15:56
lyarwoodindeed, cool I'll write this up and get packaging bumped in Nova's LC now15:56
lyarwoodmany thanks again for your help fungi15:57
fungisure, and good luck! let us know if you hit any other confusion there15:57
* fungi heads into next meeting tunnel, should reemerge in ~2 hours15:57
*** jamesdenton has quit IRC15:58
*** jamesdenton has joined #openstack-infra15:59
openstackgerritSorin Sbârnea proposed openstack/project-config master: Enable tripleo core members to change WIP flag  https://review.opendev.org/c/openstack/project-config/+/76582115:59
*** sboyron_ is now known as sboyron16:12
*** dhill has quit IRC16:14
*** lmiccini has quit IRC16:20
*** ociuhandu has quit IRC16:27
*** ociuhandu has joined #openstack-infra16:32
*** fresta has quit IRC16:36
*** fresta has joined #openstack-infra16:37
*** fresta has quit IRC16:45
*** fresta has joined #openstack-infra16:47
*** fresta has quit IRC16:47
*** hashar has quit IRC16:53
*** rpittau is now known as rpittau|afk16:58
openstackgerritMatt McEuen proposed openstack/project-config master: New Project Request: airship/vino  https://review.opendev.org/c/openstack/project-config/+/76388917:03
openstackgerritMatt McEuen proposed openstack/project-config master: New Project Request: airship/sip  https://review.opendev.org/c/openstack/project-config/+/76388817:03
*** fresta has joined #openstack-infra17:03
*** slaweq has quit IRC17:05
*** slaweq has joined #openstack-infra17:07
lyarwoodfungi: https://review.opendev.org/c/openstack/nova/+/765824 - FWIW this is working now, thanks again!17:16
fungilyarwood: great news!17:18
fungiand you're welcome, of course, it was my pleasure17:18
*** dhill has joined #openstack-infra17:30
*** sboyron has quit IRC17:32
*** sboyron has joined #openstack-infra17:33
*** lucasagomes has quit IRC17:33
*** ociuhandu_ has joined #openstack-infra17:38
*** ociuhandu has quit IRC17:42
*** ociuhandu_ has quit IRC17:43
*** jcapitao has quit IRC17:49
*** ociuhandu has joined #openstack-infra17:51
*** gyee has joined #openstack-infra17:54
*** ociuhandu has quit IRC17:55
*** derekh has quit IRC17:58
*** andrewbonney has quit IRC18:17
*** hamalq has joined #openstack-infra18:19
*** hamalq_ has joined #openstack-infra18:21
*** hamalq has quit IRC18:25
*** dtantsur is now known as dtantsur|afk18:28
*** hashar has joined #openstack-infra19:25
*** dciabrin_ has joined #openstack-infra20:02
*** dciabrin has quit IRC20:05
*** raissa has joined #openstack-infra20:05
*** jamesdenton has quit IRC20:15
*** elod is now known as elod_pto20:16
*** jamesdenton has joined #openstack-infra20:16
*** elod_pto has quit IRC21:13
*** hashar has quit IRC21:15
*** jamesdenton has quit IRC21:18
*** jamesdenton has joined #openstack-infra21:18
*** matt_kosut has quit IRC21:19
*** matt_kosut has joined #openstack-infra21:19
*** matt_kosut has quit IRC21:24
*** raissa has quit IRC21:43
*** sshnaidm has quit IRC21:49
*** slaweq has quit IRC21:58
*** rcernin has joined #openstack-infra22:00
*** sboyron has quit IRC22:01
*** hberaud has quit IRC22:02
*** mordred has quit IRC22:02
*** lyarwood has quit IRC22:02
*** odyssey4me has quit IRC22:02
*** johnsom has quit IRC22:02
*** sorrison has quit IRC22:02
*** jhesketh has quit IRC22:06
*** jhesketh has joined #openstack-infra22:08
*** sshnaidm has joined #openstack-infra22:08
*** fdegir has quit IRC22:14
*** tobberydberg has quit IRC22:14
*** fdegir has joined #openstack-infra22:15
*** tobberydberg has joined #openstack-infra22:19
*** xek__ has quit IRC22:28
*** lyarwood has joined #openstack-infra22:30
*** odyssey4me has joined #openstack-infra22:30
*** johnsom has joined #openstack-infra22:30
*** hberaud has joined #openstack-infra22:30
*** mordred has joined #openstack-infra22:30
*** rfolco has quit IRC22:34
*** rfolco has joined #openstack-infra22:43
*** rfolco has quit IRC22:48
*** dviroel has quit IRC22:55
*** auristor has quit IRC22:57
*** vishalmanchanda has quit IRC23:24
*** dchen has joined #openstack-infra23:36
*** stevebaker has quit IRC23:38
*** auristor has joined #openstack-infra23:45

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!