Thursday, 2016-09-15

*** mordred has quit IRC04:08
*** mordred has joined #openstack-requirements04:10
*** coolsvap has joined #openstack-requirements04:34
*** ELIEWARS_03 has joined #openstack-requirements06:04
ELIEWARS_03HOLA06:06
ELIEWARS_03*EN VISTO*06:07
*** ELIEWARS_03 has left #openstack-requirements06:09
*** ELIEWARS_03 has joined #openstack-requirements06:09
*** ELIEWARS_03 has left #openstack-requirements06:09
*** ELIEWARS_03 has joined #openstack-requirements07:15
*** ELIEWARS_03 has left #openstack-requirements07:16
*** ELIEWARS_03 has joined #openstack-requirements07:16
*** ELIEWARS_03 has left #openstack-requirements07:16
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints  https://review.openstack.org/36476207:18
*** jpena|off is now known as jpena07:24
*** jpena is now known as jpena|lunch12:02
*** jpena|lunch is now known as jpena13:02
openstackgerritSylvain Bauza proposed openstack/requirements: Raise netaddr to a minimum of 0.7.13  https://review.openstack.org/37083313:39
*** mriedem has joined #openstack-requirements14:16
mriedemhuzzah!14:16
mriedemhttps://review.openstack.org/#/c/370833/14:16
mriedemnova would like ^ for our rc114:16
mriedembasically 0 impact to our ci system14:16
dimsmriedem : +2, will wait for tonyb14:21
mriedemthanks14:23
prometheanfireI -1 because of it's huge impact14:28
mriedemprometheanfire: we don't have to re-release those other things14:31
mriedemor i'm not sure why we would14:31
prometheanfireI'm not sure that's true14:31
prometheanfireI'd like to hear that the release team wouldn't re-release those things14:31
prometheanfiredhellmann: ^14:31
prometheanfireand tonyb lol ^14:32
mriedemthe only issue i could see happening, is if this lands and then that goes into stable/newton for requirements,14:32
mriedemand then that syncs to projects on their stable/newton branches14:32
mriedembecause that would be a red flag for stable/newton releases as we don't want min dep bumps in a stable branch14:33
mriedemand i see that stable/newton doesn't exist for the requirements repo yet....14:34
mriedemso that would likely happen14:34
mriedembecause the libs have stable/newton branches14:34
ttxI think we'd have to re-release to have synced constraints14:37
ttxif if we re-release oslo.config then it snowballs14:38
ttxWill sync with dhellmann when he is done commuting14:39
*** sdague has joined #openstack-requirements14:40
*** bauzas has joined #openstack-requirements14:41
sdagueprometheanfire: why do you believe this is a huge impact?14:41
ttxsdague: we usually ensure common requirements in releases14:41
sdaguewe did usually ensure that14:42
sdaguehowever, those values used to mean something for testing14:42
sdaguewhich they don't any more14:42
sdagueand we've told everyone to use upper-constraints14:42
mriedemif we cared about syncing when the server projects release, we shouldn't cut the stable branch for the libs/clients 2 weeks before the servers14:43
mriedemimo14:43
prometheanfiremy understanding is that we'd have to sync the min bump out14:43
sdaguemriedem: right, exactly that14:43
sdagueprometheanfire: no, we really wouldn't14:43
ttxprometheanfire: sdague argues we don't really need to do that14:43
mriedemprometheanfire: it will sync out naturally14:43
prometheanfirethis is why people want divergent requirements (which we are working on14:43
sdagueprometheanfire: we already have that14:43
prometheanfirettx: need is a strong word :P14:43
sdaguewe have libraries out there with different values14:44
sdaguebecause they froze 2 weeks ago14:44
prometheanfiresdague: do we?14:44
mriedemthe only issue i have is the sync is going to go into stable/newton for the libs, which means a min version bump in a stable release14:44
mriedembut as noted in the ML,14:44
sdaguemriedem: sure, but it's still basically a noop14:44
ttxmriedem: we'll have to pass it as a .z14:44
mriedempeople should be shipping from u-c upper bound for newton, now mins14:44
mriedemttx: yeah it will have to be a known thing14:44
mriedems/now/not/14:45
prometheanfireshould be14:45
prometheanfireshould is a strong word14:45
sdagueI feel like we had a process based on heuristics14:45
sdaguebut the world really changed14:45
sdagueand now the argument is the process is right, regardless of the actual risk assessment we are talking about14:46
ttxsdague: we generally cargo-cult a number of things in that area, yes, because when we change and we can't remember all the problem space, we can get burnt14:46
sdaguettx: right, and what I'm saying is I believe if you do the actual risk assessment, this is the least risky way forward14:46
ttxLast time we innovated two weeks before release there was this explosion that took us 10 days to fix as you may remember. Turns yourself forver into a careful person14:47
ttxNow I agree that this /looks/ less risky14:47
sdaguettx: ok, propose to me a scenario where this causes a problem14:47
ttxBut I'd rather think twice rather than jump in14:48
* bauzas looking at the gap between 0.7.12 and 0.7.13 to evaluate the risk14:48
dimsthis specific thing does look less risky, going back through time trying to think about the scenarios14:48
mriedemin the bug report zigo said this wouldn't be an issue for xenial since they're already using a new enough netaddr, but i'm wondering about other distros and what version of netaddr they might be shipping for newton14:48
ttxbauzas: it's not the gap that is the problem, it's the fact that we'd ship components with different minimums14:48
sdaguettx: which we already do because of when we freeze14:48
prometheanfireya, 0.7.18 is what's stable here14:48
sdagueplus all the 3rd party libraries that have different minimums14:49
ttxsdague: what do you mean by that14:49
zigomriedem: For Debian, no pb with netaddr 0.7.1814:49
sdaguettx: we have 400 packages you need to install to get openstack14:49
prometheanfirezigo: ltns14:49
dimsttx : that's what rc2 is for right? we can things if needed and match them up again14:49
dimss/can things/can cut things/14:49
sdaguesaying that something will go wrong if the 100 we control don't have exact minimums, when the 300 we don't are all over the place, is non sensical14:49
zigottx: I never understood why having components with different minimums is a problem, as long as we have something co-installable.14:50
bauzasttx: mmm, I see14:50
ttxsdague: I guess in this case the worst case scenario is we have to re-release everything14:50
zigottx: It is in fact a reality in this case that other components may not care about having an older netaddr.14:50
mriedemwe have raised mins on some other deps in the last 2 weeks, but they look like things that wouldn't be deps of other already frozen libs14:50
bauzasFWIW, 0.7.12 was released 2 years ago14:51
bauzasso I'm seriously wondering which OS could continue to deliver it as a bare minimum14:51
zigoFYI, I would have seen this earlier in other releases, but I'm late for Newton because building everything on infra for the first time.14:51
dimsbauzas : which means that we should revisit the minimums really early in the cycle14:52
zigoOh, and when we're at it, I'm always amazed to see we're still supposed to use sphinx < 1.3...14:52
zigoI really hope this will be fixed for Ocata.14:52
dimsprometheanfire : that's probably an action item for early ocata14:52
zigoThat one, and psutil ...14:52
ttxsdague: My concern is more around installing several components with different requirements files14:52
prometheanfirezigo: there's a patch to raise that that's been held back14:53
dimsaction is survey the minimums across distros and versions we would like to support and fix g-r minimums14:53
prometheanfireand mins were suposed to be looked at this cycle too14:53
sdaguettx: right, and I'm saying that it's not a real concern14:53
prometheanfirebut now we are spun up as a team so hopefully it helps14:53
zigoprometheanfire: Raise which one? psutil or sphinx?14:53
dimsprometheanfire : right, agree. we failed to do that14:53
sdagueas someone who has debugged these interactions a lot, this is a very low risk change14:53
prometheanfiresphinx14:53
prometheanfireand psutil I think14:53
zigoOk.14:53
prometheanfireI agree that it looks low risk14:54
prometheanfireif the 'intrested parties' are willing to take the risk then ok14:54
ttxsdague: ok, I'll discuss it with dhellmann14:54
prometheanfirethat's why I -1 it, not -214:54
prometheanfireso if requirements and release teams are willing to take it...14:54
dimswe should probably pause for tonyb to weigh in14:54
prometheanfireyes14:54
prometheanfireshould be 8ish hours14:55
zigopsutil 2.0.0 is from mars 2014 ... :)14:55
prometheanfireI know14:55
ttxI definitely agree it sounds low-risk and I can't make an example of how it would fail. My only reason foir resisting it is that I don't like to change the rules in the last minutes of the game14:55
prometheanfirezigo: https://review.openstack.org/33371714:55
prometheanfirettx: exactly14:55
dimszigo : when requirements ocata opens up, can you please help us fix things in g-r minimums?14:56
prometheanfirezigo: I had a review as well, but abandoned it since tony's is older14:56
zigottx: It has always been my view that the rules are broken to begin with, if the end result is to keep things broken.14:56
sdaguettx: ok, I get frustrated by holding to outdated rules instead of actually doing scenario analysis on actual risks14:56
dimssdague : totally hear your frustration14:56
prometheanfireagain, formally supporting divergent reqs is something we are working on14:57
prometheanfirehttps://bugs.launchpad.net/openstack-requirements/+bug/1623571 https://etherpad.openstack.org/p/ocata-requirements-notes14:57
openstackLaunchpad bug 1623571 in OpenStack Global Requirements "lower-constraints blueprint idea dumping ground" [Undecided,New]14:57
prometheanfireanyway, lets wait for releases team feedback and tony's feedback14:57
zigodims: For Ocata, I'd really like us to use sphinx 1.4, but it's going to be painful, as many projects are erroring when there's only warnings.14:58
ttxsdague: This system is complex and has failed in unforeseen ways in the past. Or are you saying you understand it so well at this stage it can't fail in this scenario ?14:58
ttxI'm fine deferring to your expertise -- you certainly practice this system more often than I do14:58
sdaguethe only failure scenarios on min requirement mismatch are as follows14:58
sdaguepackage a requires b >=1.114:59
sdaguepackage c requires b >=1.214:59
sdaguepackage a is installed and chooses 1.1 for some reason14:59
sdaguec is installed later, and that is moved up because of the mismatch14:59
sdagueif in that window package a has involved a started service, it will start with 1.1, and on service restart will shift forward15:00
sdaguewhich is probably not actually an issue, but may see a behavior change15:00
sdagueor if there is something really tricky with cffi in package b, the compiled version for 1.1 may not match what's in 1.215:00
ttxOnly example I could make up would be of some weird deployment system that would expect requirements to be the same across everything (since it always was) and then would deploy the wrong version for nova. But that's a pretty fucked-up scenario I'll admit15:00
prometheanfireya, I'm not aware of a dep system THAT broken15:01
sdaguettx: that means it is deploying without using pip15:01
prometheanfirewell, ya, then there's pip...15:01
sdaguethe big issues we had before was different max versions15:02
ttxsdague: Also I think we only have identical requirements at release time. I think they are not 100% synced in day +115:02
mriedemyeah caps were always the big issue before15:02
sdaguettx: we don't though15:02
sdaguebecause you can't just account for our packages15:02
sdagueyou also have to account for all the libraries and other python code we depend on15:03
sdaguemaking our packages the same wasn't a 100% solution15:03
ttxI don't understand that point. We have global-requirements. That resolves to a set of dependencies and versions.15:03
sdagueit was risk mitigation15:03
zigosdague: Your case is with things using pip. Using packages, we don't have this problem, as only version 1.2 will be used everywhere.15:03
ttxNot two or three15:03
sdaguettx: right, but I think you incorrectly assume that it solved a bigger issue than it did15:03
sdagueit didn't solve coinstallability15:04
sdagueit made is so we didn't completely f- it up in code we controlled15:04
sdagueand upper-constraints made it so we resolved to something that actually could coinstall, with g-r never actually guarunteed15:04
ttxNo, all I'm saying is that there was a reason why we wanted requirements to be exactly synced by release time, and we cargo-culted it to this day15:04
sdaguettx: as risk mitagation15:05
ttxYou're saying that the reason is no longer present15:05
ttxsdague: totally15:05
ttxrisk management, to be more precise15:05
sdagueand I'm saying this doesn't add any really incremental risk15:05
prometheanfiregr is just the minimums, in reality it's not that useful (except to add exclusions)15:06
sdagueprometheanfire: right, it is the input file to set bounds on upper-constraints resolving15:06
ttxOK, as I said, I'll make the case to dhellmann and expect him to approve it15:06
sdaguethis change would not change that answer15:06
prometheanfiresdague: UC being a defacto cap :P15:06
sdagueprometheanfire: sure15:06
prometheanfireya, I'd expect this to go in, just highly irregular15:07
ttxMinor issue is that it forces a bump on stable/newton that should in theory be a .y bump when we only have space for .z, so we'd have to have a (new) rule to consider those .z as well15:07
prometheanfireand wanted more discussion before it was workflowed15:07
sdagueok, I'm good with conversation to make sure folks are comfortable with it. Just wanted to make sure objections were based on actual risk assessment and not just cargo cult.15:08
ttxI think that was the standing reasons why we did sync reqs pre-release15:08
mriedemttx: i'd think the stable team would make an exception for this as a known thing15:08
mriedemi don't think we need new process for this in stable15:08
mriedemwe've had exceptional minimum bumps in stable before15:08
mriedemmoving forward, we should probably branch the libs/clients *after* the server projects15:09
ttxmriedem: actually we need to have some rule there (requirements change triggering .y or .z) because we are already inconsistent15:09
mriedemsince we're doing it in the wrong direction for deps15:09
prometheanfireanyway, I still haven't had caffeine, so...15:09
sdaguemriedem: that assumes deps are more important than stable testing15:09
mriedemttx: in stable it's pretty cut and dried,15:09
mriedemyou can't bump .y if that's used in the next series15:09
sdagueI really think that min dep drift is not an issue15:09
mriedemand .y is already used15:09
sdaguenot with upper-constraints15:09
ttxsdague: should we just stop syncing them then ?15:11
* ttx is 11 min late to a meeting15:11
sdaguewe still need to sync the exclusions, and we still want to move the ball forward over time. But I think eventual consistency is fine15:11
prometheanfireminimums set exclusionary criteria15:12
sdagueespecially when we're talking about a boundary that is 2 years old15:12
prometheanfirealsong with != ofc15:12
sdagueprometheanfire: sure, but it isn't the kind of change that impacts upper-constraints15:12
sdagueif this did change uc, then we'd be talking about a radically different issue15:12
prometheanfireupperconstraints is just a snapshot15:13
prometheanfireuc changes are also easier for us to merge, we did so a couple of times in the freeze15:13
prometheanfiresimpler because they don't change exclusionary criteria15:13
prometheanfireanyway, getting my caffeine now15:14
sdagueuc changes are actually the dangerous ones those, as those change / invalidate our testing15:14
sdaguethis is so much less risky than a uc change15:15
dhellmannttx, sdague, prometheanfire, mriedem : it's fine with me if we want to try to get the requirements update in place. I strongly suspect this is not the only min we have that's out of date, so I'm not sure why we need to rush it, but either way it's fine. I suggested a release note fix because that seemed easier to ensure would happen today, not because I thought there was risk in changing the min for netaddr.15:43
dhellmannI hope we can prioritize fixing the min settings across the board for ocata15:44
dimsdhellmann ++ that was my suggestion as well15:46
mriedemdhellmann: did you say that in a review or the ML?15:47
sdagueit just seems odd when we can actually do the real fix to not and do the reno15:47
dhellmannmriedem : on irc somewhere15:47
mriedemsdague wanted to avoid a release note in nova saying basically we know this is wrong, but we aren't fixing it for newton15:47
dhellmannsdague : it's fine. I didn't want nova to miss its deadline today because of this.15:48
sdagueyeh15:48
mriedempersonally i'm fine with just not fixing it15:48
mriedemand not release noting it15:48
mriedemand assuming people are at a high enough version that it won't impact them15:48
sdaguewell https://review.openstack.org/#/c/370833/ is jenkins +115:48
mriedemand if it does, oops15:48
sdagueyeh, I could honestly live with that as well15:49
mriedemdhellmann: prometheanfire: dims: ttx: ^15:51
mriedemif you guys would rather just not push, i'm cool with that and we can leave it for ocata15:51
dhellmannmriedem : that works for me, too15:51
dhellmannI like not routing around our procedures in the heat of the moment15:52
ttxit's just weird that we can't fix things that require requirements bump pre-release, but we can post-release15:52
sdaguettx: right, that ^^^15:52
sdagueit's like, we have this fix15:52
prometheanfirehonestly I'm fine going either way15:52
sdagueand we know it's fine15:52
sdagueand it would be fine on the stable branch15:52
ttxi.e. we relaxed the post-release rules a lot15:53
sdaguebut in rc phase it's not15:53
dhellmannttx: I'm not sure what you mean15:53
ttxdhellmann: I mean we are basically saying that there is a class of bugs we can't fix in RCs15:54
sdaguettx: more importantly, there is a class of really low risk bugs we can't fix in RCs15:54
ttxwhile I agree that we need to make tradeoffs in the last phases of the release, we are not there yet15:54
sdaguethere are definitely lots of bugs we can't fix in RCs for very solid reasons15:54
ttxso I'm also fine with fixing it in RC1 the way we'd fix it in 14.0.115:55
ttxbut then I think it's mostly a non-issue, too :)15:55
ttxso we could "discover" it later15:56
dhellmannwe've become very conservative about requirements changes during the rc period because of the amount of trouble we've had in the past. I think some of the things that caused that trouble have been addressed, so I'm OK with going ahead with this change (bump netaddr, do not re-release everything that uses it, update nova before its rc1). but I'm also comfortable saying that this isn't a big deal and not fixing it.15:56
dhellmannI guess I don't have a strong opinion one way or the other.15:56
ttxdhellmann: same. Basically this issue points to the need to change the rules. I don't like to change the rules while in RC phase15:57
dhellmannttx: right, that's what I meant by "heat of the moment" earlier15:57
ttxso I'm 50/5015:57
dhellmannprometheanfire : thoughts?15:58
ttxFix it now + Change the rule now vs. Fix it post-release and change the rule in Ocata15:58
sdagueI'm 60/40 in favor of fixing it, just because we can, and it might help 1 or 2 actual users15:58
prometheanfiresorry, meeting, reading now15:58
ttx(but I think we need to change the rule in Ocata, don't want to have the same discussion next time)15:59
dhellmannttx: yes, we should talk about the rule for ocata either way15:59
prometheanfireI'm on the fence too, I don't think this will cause problems and will be fine to merge15:59
prometheanfirebut I'm fine with not merging too15:59
prometheanfireand yes, we've already started small talks related to this for ocata16:00
ttxdhellmann: how likely is it going to be the only one such case ?16:00
ttxIf we expect several, I'd rather change the rule now... next issue might not be so... ignorable16:00
prometheanfirewe have a couple of other changes that are waiting til after we branch16:00
ttxIf we think it's likely the only such one, let's wait16:01
prometheanfirethere are 3-4 changes to gr that were delated16:01
prometheanfiredelayed16:01
ttxFeels like an oslo.db bump will make all that shallow16:02
prometheanfireya, that's one (though people still seem to be arguing about it)16:02
ttxdhellmann: I'm 51% on approving it now and let it be eventually consistent16:03
prometheanfireoslo.log is one16:03
prometheanfirezigo: https://review.openstack.org/36525316:03
prometheanfirekeystonemiddleware is another16:04
prometheanfireanyway, I'm fine with the merge, but believe that tony should have a say16:04
prometheanfiremriedem: can you wait til he's in?16:04
dhellmannttx: I don't have any other cases in mind, I just know we've seen a few this week and I think that means we've not been good about keeping the mins up to date at all.16:04
mriedemprometheanfire: i'll be around when tonyb is if he's going to be 'the decider'16:07
dhellmannI have to give up my lunch table for the rush, so I'm going to move across the street to the coffee shop. brb.16:07
prometheanfiremriedem: he has a new title now :D16:07
ttxThat all-requirements-synced constraint is painful to handle and if additions are kept under control, very probably useless16:07
mriedemhttp://www.azquotes.com/picture-quotes/quote-i-m-the-decider-and-i-decide-what-is-best-george-w-bush-64-47-33.jpg16:08
prometheanfirettx: as far as keeping lower-requirements (what gr basicly is) in sync and up to date goes, we have a plan16:08
prometheanfiremriedem: exactly16:08
prometheanfirettx: https://bugs.launchpad.net/openstack-requirements/+bug/162357116:08
openstackLaunchpad bug 1623571 in OpenStack Global Requirements "lower-constraints blueprint idea dumping ground" [Undecided,New]16:08
prometheanfirettx: basically, each project would keep it's own lower-constraints test, global requirements would just be the lowest lower requirement + any exclusions16:11
prometheanfirean upper-lower-constraints file could be useful to packagers for providing a globale min as well16:11
prometheanfiregr would mostly be recordkeeping at that point16:12
ttxprometheanfire, mriedem: I think we should just make the call now, waiting makes me want to defer it anyway16:19
prometheanfireok16:19
dhellmannyeah, I concur16:19
prometheanfiremerge++16:19
ttxrequirements are frozen to accomodate the release team and the release team says ok16:19
dhellmannit seems we're all either on the fence or softly in favor of going ahead with the update?16:19
prometheanfireseems so16:20
dhellmannyeah, I'll go approve that requriements change16:20
ttxI'm slightly in favor of approving if we can approve it now16:20
dhellmannprometheanfire : do you want to +2 it as well?16:20
prometheanfireI will16:20
ttxIf we wait tomorrow, then I'm in 51% in favor of pretending we never spotted the bug16:20
prometheanfirelol16:20
prometheanfire+W'd16:21
ttx% increase with every day until release day16:21
ttxI can get pretty blind by the time we reach release16:21
ttxa bug ? Nooooo16:21
dhellmannmriedem, sdague : we're going ahead with the netaddr change. I'll leave it to you to watch the g-r update and handle nova.16:23
mriedemdhellmann: we already have the nova change lined up with depends-on16:23
mriedemso ok16:23
prometheanfiresosayweall :D16:26
dhellmannwould everyone please look over my email summary to the list and ensure I didn't get anything wrong or miss anything?16:27
prometheanfirewhen it comes through16:27
ttxlooking16:27
ttxwaiting16:28
dhellmannhttp://lists.openstack.org/pipermail/openstack-dev/2016-September/103784.html16:29
dhellmannsorry, forgot to post the link16:29
prometheanfirelgtm16:30
*** jpena is now known as jpena|off17:10
sdaguedhellmann: sounds good, thank you17:31
sdagueand thanks everyone here17:31
*** david-lyle has quit IRC19:40
*** david-lyle has joined #openstack-requirements19:40
openstackgerritMerged openstack/requirements: Raise netaddr to a minimum of 0.7.13  https://review.openstack.org/37083320:13
dimsdhellmann : +1 from me20:20
dhellmanndims : thanks20:28
prometheanfireit's about time for tonyb to wake up :P20:29
* dims sends tonyb some strong coffee to help him catchup20:29
amrithdims, I'd like some too20:32
* dims hands amrith a big carafe20:52
amriththank you dims, that hit the spot.21:00
* tonyb doesn't like the looks of that scrollback :(21:13
amrithdid you drink your coffee?21:15
prometheanfirelol21:24
prometheanfiredon't think that'll help much21:25
prometheanfiretonyb: the mailing list is a better summary21:26
tonybamrith: No :( in a nova meeting then school run21:27
amrithhmm, a problem not solved by coffee. Need a stronger solution but it isn't that time (yet).21:29
prometheanfireespresso21:33
tonybprometheanfire: I only drinnk double espressos at home21:33
amrithsolution, don't be at home :)21:34
tonybmriedem: Are you goign to be around in a about 90mins? to talk thought the netarr/os-vif (and maybe oslo.db) reviews?21:34
mriedemtonyb: i'll be on tonight....but might be heading home soon21:35
tonybmriedem: okay.  that'll give me time to catch up21:35
dhellmanntonyb : I'm around for a while, too. our meetup should be starting in ~1 hr21:45
*** amit213 has quit IRC21:47
*** jhesketh has quit IRC21:47
prometheanfireI trink too much coffee21:48
*** jhesketh has joined #openstack-requirements21:49
*** amit213 has joined #openstack-requirements21:51
*** mriedem has quit IRC21:58
tonybdhellmann: Thanks.22:17
*** number80 has quit IRC22:50
*** sdague has quit IRC22:52
*** mriedem has joined #openstack-requirements23:10
mriedemtonyb: i'm around23:10
mriedemsort of23:10
*** number80 has joined #openstack-requirements23:12
prometheanfireas much as any of us can be 'around'?23:24
*** armax has quit IRC23:37
*** armax has joined #openstack-requirements23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!