| *** bdodd has quit IRC | 00:18 | |
| *** bdodd has joined #openstack-infra | 00:19 | |
| *** ociuhandu has joined #openstack-infra | 00:20 | |
| *** ociuhandu has quit IRC | 00:24 | |
| *** dayou has quit IRC | 00:39 | |
| *** dayou has joined #openstack-infra | 00:55 | |
| *** jamesmcarthur has joined #openstack-infra | 00:58 | |
| *** yamamoto has joined #openstack-infra | 01:10 | |
| *** jamesmcarthur has quit IRC | 01:12 | |
| *** jamesmcarthur has joined #openstack-infra | 01:12 | |
| *** jamesmcarthur has quit IRC | 01:19 | |
| *** armax has quit IRC | 01:20 | |
| *** jamesmcarthur has joined #openstack-infra | 01:48 | |
| *** KeithMnemonic has quit IRC | 02:02 | |
| *** yamamoto has quit IRC | 02:03 | |
| *** __ministry has joined #openstack-infra | 02:06 | |
| *** benj_ has quit IRC | 02:07 | |
| *** apetrich has quit IRC | 02:09 | |
| *** benj_ has joined #openstack-infra | 02:16 | |
| *** yamamoto has joined #openstack-infra | 02:19 | |
| *** jamesmcarthur has quit IRC | 02:44 | |
| *** jamesmcarthur has joined #openstack-infra | 02:44 | |
| *** yamamoto has quit IRC | 02:51 | |
| *** ociuhandu has joined #openstack-infra | 03:03 | |
| *** ociuhandu has quit IRC | 03:08 | |
| *** Goneri has quit IRC | 03:19 | |
| *** roman_g has quit IRC | 03:37 | |
| *** cdearborn has quit IRC | 03:50 | |
| *** yamamoto has joined #openstack-infra | 04:33 | |
| *** evrardjp has quit IRC | 04:37 | |
| *** evrardjp has joined #openstack-infra | 04:37 | |
| *** yamamoto has quit IRC | 04:42 | |
| *** yamamoto has joined #openstack-infra | 04:48 | |
| *** yamamoto has quit IRC | 04:48 | |
| *** yamamoto has joined #openstack-infra | 04:53 | |
| *** jamesmcarthur has quit IRC | 05:06 | |
| *** benj_ has quit IRC | 05:17 | |
| *** benj_ has joined #openstack-infra | 05:17 | |
| *** yamamoto has quit IRC | 05:29 | |
| *** yamamoto has joined #openstack-infra | 05:40 | |
| *** ramishra has quit IRC | 05:45 | |
| *** ociuhandu has joined #openstack-infra | 05:46 | |
| *** yamamoto has quit IRC | 05:48 | |
| *** ociuhandu has quit IRC | 05:50 | |
| *** dklyle has quit IRC | 05:58 | |
| *** yamamoto has joined #openstack-infra | 06:04 | |
| *** yamamoto has quit IRC | 06:10 | |
| *** yamamoto has joined #openstack-infra | 06:24 | |
| *** yamamoto has quit IRC | 06:32 | |
| *** yamamoto has joined #openstack-infra | 06:35 | |
| *** igordc has joined #openstack-infra | 06:39 | |
| *** yamamoto has quit IRC | 06:40 | |
| *** yamamoto has joined #openstack-infra | 06:54 | |
| *** ociuhandu has joined #openstack-infra | 07:00 | |
| *** slaweq has quit IRC | 07:18 | |
| *** xek has joined #openstack-infra | 07:41 | |
| *** ociuhandu has quit IRC | 07:41 | |
| *** slaweq has joined #openstack-infra | 08:03 | |
| *** igordc has quit IRC | 08:06 | |
| *** slaweq has quit IRC | 08:08 | |
| *** slaweq has joined #openstack-infra | 08:22 | |
| *** __ministry has quit IRC | 08:30 | |
| *** ociuhandu has joined #openstack-infra | 08:31 | |
| *** ociuhandu has quit IRC | 08:45 | |
| *** ociuhandu has joined #openstack-infra | 08:46 | |
| *** ralonsoh has joined #openstack-infra | 09:19 | |
| *** slaweq has quit IRC | 09:29 | |
| *** ociuhandu has quit IRC | 09:36 | |
| *** slaweq has joined #openstack-infra | 09:40 | |
| *** ociuhandu has joined #openstack-infra | 09:41 | |
| *** slaweq has quit IRC | 09:45 | |
| *** ociuhandu has quit IRC | 09:46 | |
| *** ociuhandu has joined #openstack-infra | 10:11 | |
| *** ociuhandu has quit IRC | 10:44 | |
| *** ralonsoh has quit IRC | 11:27 | |
| *** ociuhandu has joined #openstack-infra | 11:31 | |
| *** ociuhandu has quit IRC | 11:36 | |
| *** yamamoto has quit IRC | 12:32 | |
| *** dayou has quit IRC | 12:41 | |
| *** ociuhandu has joined #openstack-infra | 12:48 | |
| *** yamamoto has joined #openstack-infra | 12:49 | |
| *** yamamoto has quit IRC | 12:52 | |
| *** __ministry has joined #openstack-infra | 13:13 | |
| *** yamamoto has joined #openstack-infra | 13:15 | |
| *** slaweq has joined #openstack-infra | 13:25 | |
| *** slaweq has quit IRC | 13:28 | |
| *** slaweq has joined #openstack-infra | 13:30 | |
| *** ociuhandu has quit IRC | 13:35 | |
| *** __ministry has quit IRC | 13:44 | |
| *** slaweq has quit IRC | 13:55 | |
| *** ociuhandu has joined #openstack-infra | 14:09 | |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Use legacy infra puppet jobs from system-config https://review.opendev.org/720889 | 14:14 |
|---|---|---|
| *** yamamoto has quit IRC | 14:16 | |
| openstackgerrit | Monty Taylor proposed openstack/openstack-zuul-jobs master: Remove legacy infra puppet apply jobs https://review.opendev.org/720890 | 14:20 |
| mordred | AJaeger: ^^ some cleanup | 14:24 |
| AJaeger | yeah ;) | 14:24 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Use legacy infra puppet jobs from system-config https://review.opendev.org/720889 | 15:31 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Stop running jobs on unused puppet repos https://review.opendev.org/720900 | 15:31 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Retire unused puppet modules https://review.opendev.org/720901 | 15:31 |
| *** yamamoto has joined #openstack-infra | 15:31 | |
| *** ociuhandu has quit IRC | 15:34 | |
| *** ociuhandu has joined #openstack-infra | 15:35 | |
| *** ociuhandu has quit IRC | 15:40 | |
| *** yamamoto has quit IRC | 15:41 | |
| *** armax has joined #openstack-infra | 15:50 | |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Use legacy infra puppet jobs from system-config https://review.opendev.org/720889 | 15:55 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Stop running jobs on unused puppet repos https://review.opendev.org/720900 | 15:55 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Retire unused puppet modules https://review.opendev.org/720901 | 15:55 |
| *** ociuhandu has joined #openstack-infra | 15:59 | |
| AJaeger | fungi, if you want to look into lower-constraints, see https://review.opendev.org/#/c/720877 | 16:13 |
| *** ociuhandu has quit IRC | 16:14 | |
| AJaeger | removing install_cmd leads to a failure that looks correct - so, I'm surprised why it works with install_cmd. | 16:15 |
| fungi | here's a successful build for comparison: https://zuul.opendev.org/t/openstack/build/cbac188c8bcf4125b38c91fc229a4c3b | 16:22 |
| fungi | and the failing build from the change you linked: https://zuul.opendev.org/t/openstack/build/83104f33d2c24ac5bab700583066e2a7 | 16:23 |
| fungi | tox/lower-constraints-1.log for both of them show -c/home/zuul/src/opendev.org/openstack/networking-hyperv/lower-constraints.txt is passed to the pip install command | 16:24 |
| fungi | which is good, because it's in the deps field in tox.ini | 16:24 |
| fungi | the problem seems to arise in tox/lower-constraints-2.log | 16:25 |
| AJaeger | the first (passing) has ,ryu==4.32 installed, the second fails on ryu 4.14 | 16:25 |
| fungi | this may be a timing coincidence? | 16:26 |
| fungi | the passing build has this: cmd: /home/zuul/src/opendev.org/openstack/networking-hyperv/.tox/lower-constraints/bin/pip install --exists-action w -e . | 16:26 |
| AJaeger | hyperv lower-constraints has: | 16:26 |
| AJaeger | lower-constraints.txt:ryu==4.14 | 16:26 |
| fungi | the failing one instead does: cmd: /home/zuul/src/opendev.org/openstack/networking-hyperv/.tox/lower-constraints/bin/python -m pip install --exists-action w -e . | 16:26 |
| fungi | we may need a -- in there? | 16:27 |
| AJaeger | yes, pip install vs python -m pip install | 16:27 |
| fungi | ahh, not a timing coincidence per se, but we did recently switch up the tox command to be python -m pip right? | 16:28 |
| * AJaeger now reads https://snarky.ca/why-you-should-use-python-m-pip/ | 16:28 | |
| AJaeger | fungi: no, we did not - my change *removes* it to use the default | 16:28 |
| AJaeger | fungi: similar to https://review.opendev.org/#/c/714489/ | 16:29 |
| fungi | anyway, i expect part of the problem here is that pip install is being run twice and the second time it runs the constraints option is not supplied, which is likely to lead to nondeterministic behavior | 16:29 |
| fungi | though --exists-action w should mitigate that hopefully | 16:29 |
| AJaeger | why run twice? I only see pip install run once | 16:31 |
| AJaeger | I don't see why pip vs python -m pip makes a difference since we're in a tox environment. there must be other subtile differences here ;( | 16:33 |
| fungi | Requirement.parse('ryu>=4.24'), {'neutron'}) | 16:33 |
| fungi | so i think that's saying a version of neutron is being installed which claims to require ryu>=4.24 but ryu==4.14 was found installed and pip was told not to upgrade it | 16:34 |
| AJaeger | agreed | 16:34 |
| AJaeger | but that's the same invocatoin as in the other case that works, isn't it? | 16:35 |
| fungi | so here's what i suspect to be the errant behavior in the passing job: https://zuul.opendev.org/t/openstack/build/cbac188c8bcf4125b38c91fc229a4c3b/log/tox/lower-constraints-2.log#148-151 | 16:36 |
| *** evrardjp has quit IRC | 16:37 | |
| AJaeger | check line 133 - that one complained about oslo-db and did not fail ;( | 16:37 |
| *** evrardjp has joined #openstack-infra | 16:37 | |
| fungi | yeah... i wonder if they've adjusted a lower bound on one of their deps and not regenerated the lower-constraints.txt file properly | 16:38 |
| fungi | because constraining to an oslo-db which needs a different alembic than is in the constraints list shouldn't happen | 16:39 |
| AJaeger | exactly | 16:39 |
| AJaeger | But the error behaviour here is different, pip install ignores the error while pypthon -m pip signals it. | 16:40 |
| fungi | yeah, that seems to be the case | 16:40 |
| * AJaeger thinks that the OpenStack community has not understood lower-constraints really. Nice idea - but only now with using "pip -m" (removing install_cmd) working... | 16:41 | |
| fungi | may be worth someone regenerating the lower-constraints.txt from their actual {test,})requirements.txt files and trying that with and without the tox.ini change | 16:41 |
| fungi | i wouldn't be surprised if teams have simply forgotten that changing requirements lower bounds means you need to regen your lower constraints set | 16:42 |
| fungi | and not just edit specific entries | 16:42 |
| AJaeger | smcginnis explained it should include even indirect dependencies, so needs more than {test,}requiremntst | 16:43 |
| fungi | or that borrowing a lower constraints list from another project which uses different lower bounds on their reqs won't work as expected | 16:43 |
| fungi | right, that's why i said "regenerate" | 16:43 |
| AJaeger | At least I did only chane single entries instead of regenerating... | 16:43 |
| fungi | there's a tool in the requirements repo for correctly generating constraints lists | 16:43 |
| AJaeger | whenever I touched | 16:43 |
| AJaeger | seems too few used it | 16:44 |
| AJaeger | this had me puzzling in the last weeks - but I'm not going to fix it. I started that thread to point out the problem, let's see whether anybody takes it on | 16:44 |
| fungi | looks like more evidence of ignored mismatches in the previous pip install run as well: https://zuul.opendev.org/t/openstack/build/cbac188c8bcf4125b38c91fc229a4c3b/log/tox/lower-constraints-1.log#420-424 | 16:51 |
| *** ociuhandu has joined #openstack-infra | 16:52 | |
| *** __ministry has joined #openstack-infra | 16:55 | |
| *** ociuhandu has quit IRC | 16:57 | |
| AJaeger | fungi, thanks that helps. Want to reply to my last email with those ERRORs and point out that removin the install_cmd is the right thing to get the reports? | 17:00 |
| fungi | AJaeger: yeah, i'm composing a small novel | 17:11 |
| AJaeger | thanks! | 17:11 |
| fungi | i'm discovering that https://docs.openstack.org/project-team-guide/dependency-management.html doesn't include good advice about how to update lower-constraints.txt, and the tool i thought the requirements repo had to do this isn't entirely discoverable (assuming it exists at all) | 17:11 |
| AJaeger | that's sad - yes, it left me puzzled as well yesterday | 17:12 |
| yoctozepto | fungi, AJaeger: that was precisely my point regarding docs; hence why smcginnis's -1 puzzled me ;/ | 17:45 |
| yoctozepto | so community had higher hopes in lower-constraints than what was documented | 17:46 |
| *** hashar has joined #openstack-infra | 17:49 | |
| *** hashar has quit IRC | 17:49 | |
| fungi | yoctozepto: yep, though the project-teams guide does also say lower-constraints testing is optional | 17:50 |
| fungi | so... not a tc member (at the moment) but it seems to me that any project that wants to decide to drop those jobs is free to | 17:51 |
| yoctozepto | fungi: optional and highly subjective :D | 17:51 |
| fungi | yeah, when people suggested adding it as an option in the past i pointed out these basic problems | 17:52 |
| fungi | i thought maybe they had solved them, but it doesn't seem so | 17:53 |
| fungi | (mainly the fundamental math problem of finding a coherent lower constraints set from a mildly complex list of requirements lower bounds) | 17:53 |
| yoctozepto | true, well, maybe we get some heroes to fix this issue | 17:54 |
| fungi | and unfortunately there are still python packages which you can't even discover the dependencies for until you download and unpack them | 17:55 |
| yoctozepto | AJaeger: I replied on ml - both jobs have the same output at stage 1, only 2 differs | 17:55 |
| fungi | (and especially so when you're looking at old versions of them, which is precisely the challenge posed) | 17:55 |
| fungi | yoctozepto: yeah, as for why it's breaking when install_cmd gets removed, that'll need someone who wants to spend more time playing with it but my suspicion is it boils down to behavior differences in option parsing between calling pip's entrypoint vs python3 -m pip | 17:56 |
| yoctozepto | yeah, it looks like a quirk | 17:57 |
| yoctozepto | pip is happy to reinstall, while py -m pip is not | 17:57 |
| yoctozepto | pip is a wrapper, maybe it has some weird defaults in it | 17:58 |
| yoctozepto | lemme check that | 17:58 |
| yoctozepto | http://paste.openstack.org/show/792371/ | 17:59 |
| yoctozepto | weirdly it should not differ | 17:59 |
| fungi | agreed, i expect there's some (likely unintentional) nuance to why they behave differently | 18:00 |
| *** armax has quit IRC | 18:01 | |
| fungi | the up side is it should be easy to test and compare, because the tox logs show exactly what commands are being called in those venvs | 18:01 |
| *** __ministry has quit IRC | 18:01 | |
| yoctozepto | cmd: /home/zuul/src/opendev.org/openstack/networking-hyperv/.tox/lower-constraints/bin/python -m pip install --exists-action w -e . | 18:01 |
| yoctozepto | cmd: /home/zuul/src/opendev.org/openstack/networking-hyperv/.tox/lower-constraints/bin/pip install --exists-action w -e . | 18:01 |
| yoctozepto | it differs only in going via pip wrapper | 18:01 |
| fungi | and you can always shim the python interpreter with a debugger and analyze how the command-line options are being handled | 18:02 |
| yoctozepto | and pip allows reinstall, while -m does not | 18:02 |
| fungi | also --exists-action is not what i incorrectly remembered it being | 18:03 |
| yoctozepto | w is for wipe | 18:05 |
| yoctozepto | so it looks like pip is actually doing it right and -m pip not ? | 18:06 |
| yoctozepto | https://pip.pypa.io/en/stable/reference/pip/#exists-action-option | 18:06 |
| yoctozepto | hmm | 18:06 |
| AJaeger | yoctozepto: -m pip fails the install - that is correct for me ;) | 18:06 |
| fungi | normally pip install should need --upgrade to consider upgrading an already installed package version | 18:07 |
| yoctozepto | yeah, I'm pondering how to interpret wipe | 18:08 |
| fungi | the --help for it is fairly limited, may need to dig into the pip docs | 18:08 |
| yoctozepto | yeah, I linked them just above | 18:08 |
| yoctozepto | Possible cases: downloading files or checking out repositories for installation, creating archives | 18:09 |
| fungi | oh, yep, i missed that, thanks | 18:09 |
| yoctozepto | so it should not allow downgrades anyway, eh | 18:09 |
| fungi | so this is more about making sure that fetching the source always replaces existing source for the project, i think | 18:10 |
| yoctozepto | yeah, looks like it | 18:10 |
| yoctozepto | or whatever along these lines | 18:10 |
| *** ociuhandu has joined #openstack-infra | 18:11 | |
| *** armax has joined #openstack-infra | 18:14 | |
| *** tobiash has quit IRC | 18:17 | |
| *** tobiash has joined #openstack-infra | 18:18 | |
| yoctozepto | what is more interesting, is that https://zuul.opendev.org/t/openstack/build/cbac188c8bcf4125b38c91fc229a4c3b/log/tox/lower-constraints-2.log#420-424 actually does both downgrades and upgrades | 18:20 |
| yoctozepto | so it's happy to satisfy any new constraint | 18:21 |
| yoctozepto | I tried to find a minimal PoC but I ran out of ideas, seems either solution is happy to satisfy constraints of installed package and what we observe is linked to particular state of venv... | 18:24 |
| yoctozepto | really puzzling... | 18:24 |
| AJaeger | isn't there a pip solver coming soon? Would using that help us? | 18:25 |
| * AJaeger signs off for today, wishing everybody a great weekend | 18:25 | |
| fungi | the coming pip dep solver will address much of what we invented upper constraints for | 18:34 |
| fungi | maybe what we need to do is ask the folks implementing the dep solver to also add an inversion option to make pip select the lowest versions of everything which will satisfy requirements | 18:35 |
| *** smcginnis has quit IRC | 18:36 | |
| yoctozepto | makes sense | 18:42 |
| *** ociuhandu has quit IRC | 18:43 | |
| *** Lucas_Gray has joined #openstack-infra | 19:38 | |
| *** Lucas_Gray has quit IRC | 20:17 | |
| *** Lucas_Gray has joined #openstack-infra | 20:18 | |
| *** Lucas_Gray has quit IRC | 20:18 | |
| *** Lucas_Gray has joined #openstack-infra | 20:19 | |
| *** Lucas_Gray has quit IRC | 20:22 | |
| *** ociuhandu has joined #openstack-infra | 20:23 | |
| *** ociuhandu has quit IRC | 20:29 | |
| *** Lucas_Gray has joined #openstack-infra | 20:32 | |
| *** jamesmcarthur has joined #openstack-infra | 20:41 | |
| *** smcginnis has joined #openstack-infra | 20:43 | |
| *** jamesmcarthur has quit IRC | 20:51 | |
| *** andreaf has quit IRC | 21:03 | |
| *** andreaf has joined #openstack-infra | 21:04 | |
| *** noonedeadpunk has quit IRC | 21:06 | |
| *** noonedeadpunk has joined #openstack-infra | 21:07 | |
| *** amorin has quit IRC | 21:16 | |
| *** amorin has joined #openstack-infra | 21:17 | |
| *** yamamoto has joined #openstack-infra | 21:40 | |
| *** yamamoto has quit IRC | 21:44 | |
| *** xek has quit IRC | 21:47 | |
| *** Goneri has joined #openstack-infra | 22:18 | |
| *** Lucas_Gray has quit IRC | 22:31 | |
| *** paladox has quit IRC | 22:37 | |
| *** paladox has joined #openstack-infra | 22:42 | |
| *** armax has quit IRC | 22:58 | |
| *** dkehn has joined #openstack-infra | 23:39 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!