20:30:09 <prometheanfire> #startmeeting requirements
20:30:10 <openstack> Meeting started Wed May 15 20:30:09 2019 UTC and is due to finish in 60 minutes.  The chair is prometheanfire. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:30:11 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:30:13 <openstack> The meeting name has been set to 'requirements'
20:30:19 <prometheanfire> #topic rollcall
20:30:23 <prometheanfire> tonyb, prometheanfire, number80, dirk, coolsvap, toabctl, smcginnis, dhellmann ping
20:30:26 <prometheanfire> o/
20:30:35 <smcginnis> o/
20:34:04 <prometheanfire> #topic Any controversies in the Queue?
20:35:06 <prometheanfire> not too bad, just need to email horizon for django (email list)
20:35:09 <prometheanfire> added to todo
20:37:10 <smcginnis> Does that django one fall under the same category as the requests ones?
20:37:33 <prometheanfire> ya, it really does
20:37:35 <prometheanfire> dirk: ping
20:37:45 <prometheanfire> less impact though
20:38:02 <tonyb[m]> Yeah same idea
20:38:23 <tonyb[m]> We need to decide how proactive we're going to be there
20:38:45 <prometheanfire> https://review.opendev.org/658636
20:38:54 <prometheanfire> ignore that
20:39:27 <prometheanfire> ya, I feel like it's all or nothing.  as soon as we start accepting some out of tree stuff we become 'secure' and all that bagage
20:40:54 <tonyb> Yeah there is a little of that
20:41:09 <tonyb> Still there is scope to 'play' with it and then write up a policy
20:42:22 <tonyb> I agree with fungi though that it's a vendor thing for the most part *but* I'm pretty sure vendors aren't doign it as well as they could *and* that us doing it is a weaker signal than we'd like :(
20:43:36 <fungi> i really worry that us attempting to do it poorly ends up being a strong but dangerously misleading signal
20:44:25 <fungi> honestly, if you want some semblance of a "secure" deployment from source and pypi packages, continuously deploy master and drink from the firehose
20:45:19 <prometheanfire> what if we accept updates but do not do them ourselves?
20:45:23 <smcginnis> I wouldn't say continuous from master would be secure, but updating to latest release would be better than expecting a secure stable older branch.
20:45:30 <prometheanfire> by bot
20:45:46 <prometheanfire> heh, everyone should be on master at all times
20:46:00 <tonyb> prometheanfire: that's a pretty weak line to draw
20:46:13 <tonyb> "it wasn't us" ... just a bot we wrote and ran
20:46:17 <prometheanfire> proposed updates allowed?
20:46:27 <prometheanfire> no, I mean not us as in not bot driven
20:46:33 <fungi> i'll be the first to admit that security isn't an all-or-nothing game, and selectively applying fixes for high-risk vulnerabilities can be useful, but the proposal isn't about doing that it's about upgrading the things which are easy to upgrade rather than the things which are critical to upgrade
20:46:42 <prometheanfire> if someone wants to come by and propose something....
20:47:50 <prometheanfire> as dirk is currently doing, we'd allow that
20:47:53 <smcginnis> If we did allow it, it has potential for vendors to rally around to watch for security issues. But that's also the argument for extended maintenance, and stable in general hasn't attracted much vendor support.
20:47:55 <tonyb> fungi: to be fair I don
20:48:04 <fungi> and when the thing they want to propose requires patching projects in non-backward-compatible ways on stable branches to be able to use newer version whatever of some dependency... we tell them (and users who are relying on this as their only means of securnig python dependencies for openstack source deployments) "tough luck"?
20:48:04 <tonyb> 't think we;ve formalised the proposal yet
20:48:14 <smcginnis> Even though many of the vendors are shipping products based on those older releases.
20:48:32 <tonyb> I *think* it's use publically available data to inform and upgrade pypi packages with known CVEs
20:48:34 <prometheanfire> fungi: that's the only option we have right now
20:48:38 <tonyb> I don't know that we have moer than that
20:48:58 <prometheanfire> tonyb: I think that's the direction dirk wants to head in
20:49:00 <fungi> we have the option of telling users *clearly* not to rely on this as a secure deployment model, right?
20:49:03 <tonyb> I *suspect* that the larger chnages or more core chnages will get resistance
20:49:16 <prometheanfire> fungi: true, that is always an option
20:49:17 <tonyb> fungi: Yes we can do that
20:49:23 <smcginnis> Well, maybe for now we should just hold on any of these, then wait for someone to come up with a proposal that's at least marginally sane and maintainable.
20:49:36 <tonyb> okay
20:49:53 <tonyb> I don't want to add to dirk's workload but that sounds sane
20:50:29 <tonyb> dirk: Can you hash out a policy we'll discuss the *policy* on the list with wider input and then implement whatever that outcome is
20:50:33 <smcginnis> Stable security pop up team? :)
20:50:40 <fungi> i think whether or not we laggingly apply and test a handful of arbitrary dependency bumps for some external python deps, we have to tell users this isn't a safe solution
20:50:52 <prometheanfire> what sounds sane? can you add your proposal to https://etherpad.openstack.org/p/non-openstack-stable-security-constraints-updates ?
20:51:57 <tonyb> fungi: That
20:51:57 <fungi> because 1. we won't even know about them as far in advance as most of the distros, 2. we can't guarantee we'll be able to apply them since we don't carry forks of our deps where we apply selective backports of fixes
20:52:01 <tonyb> s fair
20:52:06 <smcginnis> Is there somewhere we should add a doc warning saying "These were the tested requirements at the time of release but there may be newer security fixes available that have not been thoroughly tested upstream"? Something like that to set the right expectation?
20:52:27 <smcginnis> Or rather, have somewhere to point to when someone comes along with an issue.
20:52:34 <prometheanfire> fungi: that's why I had two lists
20:52:46 <prometheanfire> smcginnis: at min, yes
20:53:50 <tonyb> +1
20:54:19 <fungi> yep, i think both not conflating this solution with the (intentionally frozen) upper constraints list and making it clear to users that this isn't really a significant improvement security-wise are both necessary features of any proposal, i just still question the utility of it at all
20:55:14 <prometheanfire> I'd rather not do it to be honest
20:55:17 <fungi> we're basically saying "hey, we sometimes update this other list over here with some more secure dependencies, but you're still going to want to solve the harder problem of updating the ones we can't work out a safe way to update ourselves"
20:55:21 <fungi> so... why bother?
20:56:10 <prometheanfire> exactly
20:56:19 <fungi> the thing we'd be solvnig is the easier and less useful thing to be solved, and users still have to deal with the harder thing they needed solved that we couldn't
20:56:29 <prometheanfire> I guess I'll point dirk to the meeting notes, but he's really the one pushing it I think
20:56:44 <fungi> and solving the harder problem solves the easy one anyway
20:57:22 <smcginnis> The only ones I am concerned about are the LOCI and others like that. But that's kind of a general issue with container images.
20:58:32 <fungi> at some point they're either going to need to carry patches of dependencies to secure them while keeping them working with stable branches of our software, or carry patches to stable branches of our software to work with newer dependencies
20:59:01 <prometheanfire> ya, given the way we branch it's not going to support a source based install any other way
20:59:04 <smcginnis> Containers might be an easier argument to just upgrade to latest anyway.
20:59:19 <prometheanfire> I know that's what's told in OSA
21:01:52 <prometheanfire> anything else?  I'd recommend adding stuff to https://etherpad.openstack.org/p/non-openstack-stable-security-constraints-updates for this
21:04:37 <tonyb> I
21:04:51 <tonyb> ll read it later when I have more coffee onboard
21:05:06 <prometheanfire> ok
21:05:08 <tonyb> ... actually looking at my todo list for today it wont happen today
21:05:10 <prometheanfire> #topic open floor
21:06:14 <smcginnis> I'd like to understand why we can't do the python_version capping just in upper-constraints. I don't think I understand that yet, if there actually is a reason.
21:06:35 <smcginnis> And seeing a lot of grambling in other projects about requirements breaking them now with needing to update their sphinx entries.
21:07:39 <tonyb> So with my understanding of how pip works, it's just fundamentally needed
21:08:00 <prometheanfire> do we have a good test case for it?]
21:08:02 <tonyb> what I don't get is how generate-constraints seemed to do the right thing
21:08:35 <tonyb> we all saw how the integration job broke without those python_version markers
21:08:40 <smcginnis> If pip is passed the upper constraints though, it shouldn't have an issue figuring out the max version it can use.
21:08:58 <smcginnis> I think the job broke because we didn't have the python_version flags in upper-constraints.
21:09:05 <tonyb> smcginnis: right *if* it's passed one we totally don
21:09:07 <smcginnis> We just happened to fix that by putting them in g-r instead.
21:09:09 <prometheanfire> it shouldn't be even trying to figure versions out
21:09:15 <tonyb> t need the markers in g-r
21:09:50 <smcginnis> tonyb: But shouldn't projects always be using upper constraints? I would consider that an issue for that project, not u-c vs g-r.
21:10:07 <tonyb> but *we* should need them to make our tools do the right thing
21:10:27 <tonyb> smcginnis: in the gate yes
21:10:51 <tonyb> smcginnis: but nothing actually makes them install with them
21:11:10 <tonyb> smcginnis: some vendors do a better job than others of staying on top of versions
21:11:26 <tonyb> and for most vendors they only have one version of python to deal with
21:11:48 <smcginnis> Kind of makes our debate about raising upper constraints on stable branches moot if we're now saying they aren't used anyway.
21:11:55 <tonyb> smcginnis: I agree that it's be nice to drop them but without a bunch of understanding of pip I don't see we can (yet)
21:12:08 <tonyb> smcginnis: A little yes
21:12:17 <prometheanfire> pip still doesn't dep solve :D
21:12:23 <smcginnis> OK, at least I understand a little better now. Thanks for explaining it.
21:12:48 <smcginnis> Hopefully we don't hit too many of these before we can drop py2.
21:13:26 <prometheanfire> agreed
21:13:31 <prometheanfire> anything else?
21:13:32 <tonyb> prometheanfire: that's an over simlification
21:13:57 <tonyb> I have it on my todo list to look into this but it'll be at least a week
21:14:07 <prometheanfire> tonyb: I know, was just looking at https://github.com/pypa/pip/issues/988 is all
21:14:18 <prometheanfire> was looking at it earlier today actually
21:14:31 <tonyb> Yeah
21:14:43 <prometheanfire> was mentioned in https://github.com/kennethreitz/requests/issues/5067
21:14:55 <prometheanfire> looks like they are targeting 2.22.0 for urllib3/requests
21:15:16 <prometheanfire> ok, going to close the meeing unless someone speaks up
21:15:47 <smcginnis> I'm good
21:15:55 <tonyb> I think it's probably worth writing up something for the list
21:16:03 <tonyb> to describe what happened last week
21:16:18 <tonyb> prometheanfire: do you have time to draft something on an etherpad?
21:17:01 <prometheanfire> tonyb: maybe, not tonight though
21:17:09 <tonyb> prometheanfire: cool
21:17:22 <tonyb> prometheanfire: that's still sooner than I could do it ;P
21:17:45 <prometheanfire> tomorrow at the soonest
21:19:11 <tonyb> cool beans
21:20:09 <prometheanfire> #endmeeting