| *** hoangcx has joined #openstack-requirements | 00:40 | |
| *** udesale has joined #openstack-requirements | 05:06 | |
| *** kornicameister has joined #openstack-requirements | 06:16 | |
| *** florianf has joined #openstack-requirements | 06:26 | |
| *** hoangcx_ has joined #openstack-requirements | 06:56 | |
| *** hoangcx has quit IRC | 06:59 | |
| *** jpich has joined #openstack-requirements | 07:00 | |
| *** jhesketh has quit IRC | 07:14 | |
| *** jhesketh has joined #openstack-requirements | 07:44 | |
| *** florianf has quit IRC | 08:27 | |
| *** florianf has joined #openstack-requirements | 08:29 | |
| *** hoangcx has joined #openstack-requirements | 08:30 | |
| *** hoangcx_ has quit IRC | 08:31 | |
| openstackgerrit | Nicola Peditto proposed openstack/requirements master: Added the IoTronic projects: - iotronic - iotronic-lightning-rod - python-iotronicclient https://review.openstack.org/455385 | 09:12 |
|---|---|---|
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements master: update constraint for cliff to new release 2.6.0 https://review.openstack.org/458422 | 10:02 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements master: update constraint for ovsdbapp to new release 0.3.0 https://review.openstack.org/458423 | 10:03 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements master: update constraint for osc-lib to new release 1.4.0 https://review.openstack.org/458424 | 10:03 |
| *** sdague has joined #openstack-requirements | 10:21 | |
| *** hoangcx has quit IRC | 10:55 | |
| *** openstackgerrit has quit IRC | 11:32 | |
| *** openstackgerrit has joined #openstack-requirements | 13:11 | |
| openstackgerrit | Eric Fried proposed openstack/requirements master: Update pypowervm to 1.1.2 https://review.openstack.org/458259 | 13:11 |
| *** jpich has quit IRC | 13:57 | |
| *** jpich has joined #openstack-requirements | 14:09 | |
| *** jpich has quit IRC | 14:18 | |
| *** Jeffrey4l has quit IRC | 14:19 | |
| *** jpich has joined #openstack-requirements | 14:19 | |
| openstackgerrit | Eric Fried proposed openstack/requirements master: Allow pypowervm 1.1.2 https://review.openstack.org/458259 | 14:44 |
| openstackgerrit | Lee Yarwood proposed openstack/requirements master: Require os-brick>=1.11.0 https://review.openstack.org/458545 | 15:09 |
| *** udesale has quit IRC | 15:11 | |
| *** hongbin has joined #openstack-requirements | 15:13 | |
| *** Jeffrey4l has joined #openstack-requirements | 15:31 | |
| openstackgerrit | Stephen Finucane proposed openstack/requirements master: Bump cliff to 2.6.0 https://review.openstack.org/458558 | 15:42 |
| fungi | dhellmann: so the idea you mentioned in #openstack-infra about problems arising from pinning pbr and some other setup_requires was to add them to http://git.openstack.org/cgit/openstack/requirements/tree/blacklist.txt | 15:51 |
| dhellmann | yeah, I'm not 100% clear of exactly what that file does (maybe more than we want in this case?) | 15:52 |
| fungi | one thing it's used for is to avoid complaining when people pin specific versions of those things in their project-specific requirements files | 15:53 |
| dhellmann | that sounds like something we wouldn't want here. we don't want the pins, and should complain if people use them | 15:53 |
| fungi | it's also used to skip entries when generating upper-constraints.txt, looks like | 15:54 |
| dhellmann | that part we do want | 15:54 |
| dhellmann | I think? | 15:54 |
| dhellmann | dirk, prometheanfire : ^^ | 15:54 |
| fungi | maybe... i mean constraints is a pip-specific concept and we're talking about things that will get installed by setuptools outside of pip's control | 15:55 |
| dhellmann | hmm, probably true | 15:55 |
| fungi | looking at relevant hits from http://git.openstack.org/cgit/openstack/requirements/tree/blacklist.txt it looks like we only use it for skipping entries during constraints validation and omitting entries when generating constraints lists | 15:56 |
| fungi | er, i mean http://codesearch.openstack.org/?q=blacklist.txt&i=nope&files=&repos= | 15:57 |
| dhellmann | so this probably needs to be a new file, to apply new rules | 15:57 |
| mordred | yah - I think we mostly want something that fails when anyone proposes changes to global-requirements.txt that touch setuptools, pbr or the other ones | 15:57 |
| mordred | yah | 15:57 |
| dhellmann | does someone have time to work on that? | 15:58 |
| fungi | or do we only care to check for addition of <, <= and !=? | 15:58 |
| fungi | and <~ is a thing too, i think? | 15:58 |
| dhellmann | minimums seem ok, but nothing else? | 15:58 |
| dhellmann | > and >= are good, everything else should fail? | 15:58 |
| fungi | i guess minimums are a problem only if they're higher than the available released versions | 15:58 |
| dhellmann | that would fail for another reason, already being checked | 15:59 |
| fungi | and that's probably a pathological situation anyway, right | 15:59 |
| dhellmann | we do try to install everything | 15:59 |
| dhellmann | so we want to allow the item in global-requirements.txt, but the rules there should only include > and >= operators | 15:59 |
| fungi | noting we already have blacklisted a ton of setuptools releases in global-requirements.txt because devstack at least can take advantage of those when explicitly calling pip install setuptools | 16:00 |
| dhellmann | notes going into https://etherpad.openstack.org/p/handling-setup-requires-in-gr | 16:00 |
| fungi | appdirs and six only have minimums declared (in master, and more recently in stable branches too i think) | 16:01 |
| fungi | what was the other setuptools setup_requires? | 16:01 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements master: update constraint for pbr to new release 3.0.0 https://review.openstack.org/458572 | 16:02 |
| fungi | aha, right, packaging | 16:02 |
| fungi | https://github.com/pypa/setuptools/blob/master/setup.py#L166 | 16:02 |
| fungi | and we similarly only declare a minimum for that in global-reqs | 16:02 |
| dirk | mordred: fungi : I don't think we want a separate file | 16:02 |
| *** newmember has joined #openstack-requirements | 16:03 | |
| dirk | is there a way to check if the setup.py template and the global requirements are in some conflict? | 16:03 |
| dirk | e.g. trigger the actual error we currently have? that might be more useful | 16:03 |
| fungi | dirk: for the pbr case, you actually need to try importing a consuming module which has the exclusion in its install_requires after setup_requires ahve already been satisfied | 16:04 |
| mordred | dirk: well, it's not just being in conflict, it's that because of how pbr and setuptools are installed _any_ constraint on them actually both doens't work and can lead to errors | 16:04 |
| dhellmann | we need to apply a validation rule to avoid allowing any of our packages to introduce that conflict | 16:05 |
| fungi | i mean, one slightly less draconian option is to check whether we're excluding the most recent available release for any of the problem packages | 16:05 |
| mordred | so it's one of those cases where it can seem like a good idea to express a constraint, but edge-cases in python packaging make it more of a footgun than anything else | 16:05 |
| mordred | fungi: ++ | 16:05 |
| fungi | oh, except in the case of maximum caps, we can't predict that future state | 16:05 |
| dhellmann | that seems more complicated to code | 16:05 |
| dhellmann | also that | 16:05 |
| mordred | I don't think we can put maximum caps into our things either - because hte install tools can't fulfull them - so as constraints they're lies | 16:06 |
| dirk | mordred: fungi : so basically https://git.openstack.org/cgit/openstack/requirements/tree/openstack_requirements/cmds/update.py#n66 and global-requirements.txt need to be identical? | 16:06 |
| mordred | like, setuptools and pbr could break us in some way in the future ... but we cannot protect ourselves from it with requirements maximum caps | 16:06 |
| dirk | or do we also need to ensure that it is not capping? | 16:06 |
| mordred | dirk: I would argue that we need to ensure it's not capping | 16:07 |
| fungi | dirk: that's far from sufficient because transitive dependencies are satisfied by setuptools/easy_install as well | 16:07 |
| dirk | right but we want that neither the one place nor the other one is capping | 16:07 |
| dirk | we want them to be identical | 16:07 |
| dirk | so yes, we should develop a check for that. | 16:08 |
| mordred | yes. we want to ensure in both places that neither cap | 16:08 |
| fungi | so anything listed in the package's setup_requires, but also anything those setup_requires declare setup_requires on (as well as setuptools and the same for its setup_requires, both direct and transitive) | 16:08 |
| mordred | I _started_ to look in to developing a check for it, but then I wasn't fully sure where the right place would be | 16:08 |
| mordred | fungi: I thinkn to start with just a list of packages that should be validated to not have caps would get us somewhere, right? | 16:08 |
| dhellmann | mordred : https://etherpad.openstack.org/p/handling-setup-requires-in-gr | 16:09 |
| fungi | yeah, a static list is probably far easier than needing to recursively parse the ast for setup.py on an unguessable number of packages | 16:09 |
| dhellmann | oh, yes, please let's not do that | 16:09 |
| fungi | it will need to evolve over time, but i'm worried that automating the thorough solution would be more complex and buggy than occasionally having to add or remove something from the list | 16:10 |
| dhellmann | right | 16:10 |
| openstackgerrit | Merged openstack/requirements master: update constraint for ovsdbapp to new release 0.3.0 https://review.openstack.org/458423 | 16:11 |
| dhellmann | so, do one of you want to work on that change or should I ask on the ML for some help? | 16:11 |
| fungi | should we consider adding setuptools to the list? it's sort of a grey area (though its setup_requires are obviously a problem) | 16:12 |
| fungi | i'm on the fence about setuptools mainly because devstack does actually do something like `pip -c upper-constraints.txt install setuptools` as part of its environment setup | 16:13 |
| mordred | fungi: it does- but that's for the gate - listing that constrained version in the actual requirements.txt can hose consumers who install things from pip | 16:13 |
| fungi | so excluding latest setuptools versions in global-requirements.txt does solve a class of setuptools-induced job breakages | 16:14 |
| mordred | it does - but software consumers can't count on the tools providing them with that version if we express it in g-r | 16:14 |
| fungi | but it's far from a complete solution in those cases and i'm always uneasy when i see that used to fix devstack while leaving the larger issue unsolved | 16:14 |
| mordred | yah | 16:14 |
| mordred | if a current setuptools is broken, us pinning it in the gate is merely a temporary thing so that the gate can move while we work with upstream to fix it | 16:15 |
| mordred | but during that time our consumers are effectively broken and there is no mechanism we can use to unbreak them automatically | 16:15 |
| mordred | which is why fixing it upstream in setuptools is essential at that point | 16:15 |
| fungi | wondering if we just need a different solution for that which won't propagate to individual projects | 16:15 |
| openstackgerrit | Merged openstack/requirements master: update constraint for osc-lib to new release 1.4.0 https://review.openstack.org/458424 | 16:16 |
| mordred | well - I think it's fine to cap it in upper-constraints | 16:16 |
| mordred | since that does not get written to metadata in the package install and therefore subsequently read by pkg_resources | 16:16 |
| mordred | it's the stuff winding up in the packages's install_requires in the egg_info that borks us | 16:16 |
| fungi | good point, we can propose/aprove a hand-edited upper-constraints.txt change which lowers teh setuptools version and attempt to avoid approving bumps back to the known broken version (for those cases where the issue doesn't cause requirements changes to fail jobs anyway) | 16:17 |
| mordred | yah | 16:17 |
| mordred | now - iirc, we actually have to do extra work in pbr to add the things from requirements into the egg-info metadata | 16:17 |
| mordred | so we _could_ also consider just removing that from pbr and not having out things express those constraints in the metadata so that pkg_resources doesn't ever try to validate it | 16:18 |
| dhellmann | fungi, mordred : maybe instead of building this, we should build the thing I proposed to stop syncing requirements | 16:18 |
| dhellmann | I don't think it's *that* much more complicated | 16:18 |
| dhellmann | https://etherpad.openstack.org/p/ocata-requirements-notes | 16:18 |
| mordred | dhellmann: I am not against that idea | 16:19 |
| *** jpich has quit IRC | 16:20 | |
| mordred | oh- ok - my idea about removing the injection of install_requires from pbr will not work | 16:21 |
| mordred | we _used_ to not inject into install_requires, but that was also when we had pbr itself call out to pip during installation to install the contents of requirements.txt | 16:21 |
| mordred | we moved from doing that to adding them to install_requires so that pip could just handle the whole thing itself | 16:21 |
| mordred | which overall is better | 16:21 |
| mordred | so I withdraw my suggestion about removing that bit from pbr | 16:22 |
| *** newmember has quit IRC | 16:22 | |
| openstackgerrit | Terry Wilson proposed openstack/requirements master: Update ovsdbapp requirement to 0.3.0 https://review.openstack.org/458581 | 16:23 |
| *** newmember has joined #openstack-requirements | 16:23 | |
| mordred | dhellmann: I believe I could work on a check to block constraints on a list of packages - I do not believe I have bandwidth this week or next to work on your sync-less proposal, although I'd be happy to _help_ on that | 16:23 |
| fungi | yeah, i'm far too underwater myself to write it right now, but as long as we track it as a thing we want hopefully we'll get around to it before the next time this happens (and also ask requirements reviewers in the meantime to be on the lookout for changes adding exclusions or caps to any of those 5 known problem packages?) | 16:26 |
| mordred | fungi: ++ | 16:27 |
| fungi | if we do move forward with an automated check, we'll need to remove the exclusions lists from pbr and setuptools in global-requirements.txt too | 16:28 |
| fungi | (not only in master, but also in stable branches i suppose) | 16:31 |
| openstackgerrit | Merged openstack/requirements master: Add Nemesis to requirements projects list. https://review.openstack.org/458212 | 16:53 |
| openstackgerrit | Merged openstack/requirements master: update constraint for cliff to new release 2.6.0 https://review.openstack.org/458422 | 16:53 |
| openstackgerrit | Stephen Finucane proposed openstack/requirements master: Bump cliff to 2.6.0 https://review.openstack.org/458558 | 16:55 |
| *** newmember has quit IRC | 16:56 | |
| *** IgorYozhikov has quit IRC | 17:14 | |
| *** IgorYozhikov has joined #openstack-requirements | 17:19 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements master: update constraint for os-brick to new release 1.12.0 https://review.openstack.org/458611 | 18:41 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements master: update constraint for os-win to new release 2.0.1 https://review.openstack.org/458614 | 18:46 |
| openstackgerrit | Merged openstack/requirements master: update constraint for monasca-common to new release 2.0.0 https://review.openstack.org/457986 | 18:50 |
| *** newmember has joined #openstack-requirements | 19:23 | |
| *** newmember has quit IRC | 19:27 | |
| *** newmember has joined #openstack-requirements | 19:28 | |
| *** newmember has quit IRC | 19:34 | |
| *** newmember has joined #openstack-requirements | 19:35 | |
| *** florianf has quit IRC | 20:21 | |
| *** sdague has quit IRC | 21:23 | |
| openstackgerrit | Merged openstack/requirements master: Allow pypowervm 1.1.2 https://review.openstack.org/458259 | 22:43 |
| *** sdague has joined #openstack-requirements | 23:06 | |
| openstackgerrit | christopher.uhler proposed openstack/requirements master: Update os-brick version for Veritas volume driver https://review.openstack.org/458688 | 23:52 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!