Tuesday, 2018-07-03

« Monday, 2018-07-02 Index Wednesday, 2018-07-04 »
*** rpioso is now known as rpioso|afk00:12
*** edmondsw has joined #openstack-requirements00:23
*** edmondsw has quit IRC00:28
*** hongbin has joined #openstack-requirements00:57
prometheanfirejohnsom: this needs to be reverted at the same time or before https://github.com/openstack/requirements/commit/abafb89c7b5fdc8a93c7aeb731503c1e53cfdafd01:15
prometheanfirejohnsom: https://review.openstack.org/#/c/576988/ should be reverted too01:16
prometheanfireall the same subject01:16
johnsomWhat craziness is this? lol I was sure adding requirements to stable branches was a huge no-no....01:16
prometheanfirejohnsom: adding happens sometimes, rarely, but sometimes01:17
johnsomprometheanfire Just the description on https://review.openstack.org/#/c/576988/ should have been a huge red flag that something horrible had been done to the stable branch01:19
prometheanfireya, it should have been caught01:20
openstackgerritMichael Johnson proposed openstack/requirements stable/queens: Revert "add neutron-lbaas to global-requirements"  https://review.openstack.org/57972501:21
openstackgerritMichael Johnson proposed openstack/requirements stable/queens: Revert "add neutron/networking projects to blacklist for queens"  https://review.openstack.org/57972601:22
openstackgerritMichael Johnson proposed openstack/requirements stable/queens: Revert "add neutron/networking projects to blacklist for queens"  https://review.openstack.org/57972601:22
openstackgerritMichael Johnson proposed openstack/requirements stable/queens: Revert "Add constraints for the neutron-lbaas"  https://review.openstack.org/57971701:23
prometheanfireare they dependant?01:23
johnsomI rebased then in the order they need to back out.01:23
prometheanfirecool01:23
johnsomAt least I think01:23
prometheanfiretests will sort it out :P01:24
prometheanfirein the worst case01:24
johnsomYep01:24
prometheanfirethere we go, gertty took a second to show it in tree view01:25
tonybreverting 576988 will break a whole heap of things01:27
tonybprometheanfire: why do you think we need to revert the addition to the blacklist?01:28
tonybprometheanfire: where's the failure that implies that?01:28
prometheanfireyou are right, maybe I'm being overzealous, the blacklist doesn't hurt, but the original cause for adding it is being reverted too01:30
tonybprometheanfire: the original cause was those packages being on pypi ... have they been removed?01:37
prometheanfirepoint taken01:38
tonybprometheanfire: I think unblocking the neutron-lbass queens gate by merging 579725 is right, that will in turn break OSA, which will need to find a better way (perhaps local it it) to fix the problem it found on queens01:42
prometheanfireya01:44
prometheanfirejohnsom: we need to revert the constraint?01:45
johnsomThe upper-constraint is what is breaking the stable/queens gates01:45
johnsomYou can't pull from local filesystem (zuul) or from git with an upper-constraint defined.01:46
johnsomYeah, frankly I'm not sure how OSA pulled a master neutron-lbaas but a stable/queens neutron.  Something when sideways in that play. Those are all supposed to be reasonably pinned.01:47
johnsomConsidering I haven't heard others screaming about it, it might be a local change issue01:47
prometheanfirejohnsom: ok, so, mind rebasing without the blacklist patch then?01:49
johnsomSure01:50
tonybI think we're going to need to do a horrible squashy fix here rather then a step-by-step removal01:51
openstackgerritMichael Johnson proposed openstack/requirements stable/queens: Revert "Add constraints for the neutron-lbaas"  https://review.openstack.org/57971701:52
tonybI *think* it things need to be in both g-r and blacklist on queens01:52
tonyband *not* in u-c01:52
tonybjohnsom: what do you mean by 'You can't pull from local filesystem (zuul)' do you have a job log I can look at01:52
johnsomIt wasn't in G-R before this...01:53
tonybjohnsom: wheer "this" == "about 2 months befoer the queens release?01:53
johnsomtonyb let me see if I can find a patch.01:53
tonybjohnsom:01:54
johnsomtonyb Ummm, no, this was all done June 25, like a week ago01:54
tonybjohnsom: Okay01:54
tonybjohnsom: I'll need to go back and reconstruct the review history01:55
johnsomhttps://review.openstack.org/#/c/577825 June 25th, 2018 and June 19th, 2018 for https://review.openstack.org/#/c/576526/01:55
johnsomThose two, are the ones no one asked the PTL about and added requirements to the stable branch01:55
johnsomhttp://logs.openstack.org/07/579707/1/check/openstack-tox-pep8/de64ec6/job-output.txt.gz#_2018-07-02_23_07_37_10016901:56
johnsomIs an example.01:56
johnsomI am already two hours past when I should have been helping to make dinner. I will try to check back later.01:58
tonybjohnsom: Okay I see, confusing terminalogy.01:58
tonybjohnsom: Yeah okay the 2 reverst that are up are the right thing to do.  I shouldn't have +2 the addition to constraints and when I saw the review I shoudl have reverted the addition to g-r02:02
prometheanfireya, I missed it too :|02:03
prometheanfireatm I'm trying to deal with OSA using our constraints when they shouldn't be02:03
prometheanfirehttps://storyboard.openstack.org/#!/story/2002800 https://review.openstack.org/57969902:04
prometheanfiredirty hack is dirty02:04
tonybprometheanfire: what does 'PIP_OPTS' look like before you munge it?02:09
*** hwoarang has quit IRC02:10
prometheanfire--index-url ${NODEPOOL_PYPI_MIRROR} --trusted-host ${NODEPOOL_MIRROR_HOST} --extra-index-url ${NODEPOOL_WHEEL_MIRROR} --constraint global-requirement-pins.txt02:10
prometheanfiresomething like that02:11
prometheanfirepossibly02:11
prometheanfireit's in the bootstrap-ansible file02:11
tonybprometheanfire: Okay see my comment on the review02:14
tonybPIP_OPTS='--index-url ${NODEPOOL_PYPI_MIRROR} --trusted-host ${NODEPOOL_MIRROR_HOST} --extra-index-url ${NODEPOOL_WHEEL_MIRROR} --constraint global-requirement-pins.txt'02:14
tonybecho ${PIP_OPTS}02:14
tonybecho ${PIP_OPTS%%--constraint *}02:14
tonybI think that does what you need02:14
tonybit's still a hack but at least it's a tidy hack02:14
prometheanfireya, bash substring stuff, should have thought about that (I actaully like that :P02:15
*** cjloader has quit IRC03:12
*** cjloader has joined #openstack-requirements03:12
*** udesale has joined #openstack-requirements03:42
johnsomChecking in, looks like the reverts are still failing gates03:43
johnsomBoth are failing with the same legacy-requirements-integration-dsvm error, cannot import name parse,  http://logs.openstack.org/25/579725/1/check/legacy-requirements-integration-dsvm/c63c8cb/job-output.txt.gz#_2018-07-03_01_58_23_68888903:46
johnsomKnown issue with the requirements gates?03:46
prometheanfireno03:55
*** hongbin has quit IRC04:07
*** cjloader has quit IRC04:32
*** andreas_s has joined #openstack-requirements04:33
*** andreas_s has quit IRC04:38
*** hwoarang has joined #openstack-requirements05:14
*** edmondsw has joined #openstack-requirements05:48
*** edmondsw has quit IRC05:53
*** andreas_s has joined #openstack-requirements06:01
*** hwoarang has quit IRC06:23
*** hwoarang has joined #openstack-requirements06:23
*** hwoarang has quit IRC06:23
*** hwoarang has joined #openstack-requirements06:23
openstackgerritOpenStack Proposal Bot proposed openstack/requirements master: Updated from generate-constraints  https://review.openstack.org/57977406:35
*** openstackgerrit has quit IRC06:49
*** andreas_s has quit IRC06:49
*** andreas_s has joined #openstack-requirements06:49
*** andreas_s has quit IRC06:54
*** amoralej|off is now known as amoralej07:19
*** andreas_s has joined #openstack-requirements07:29
*** andreas_s_ has joined #openstack-requirements07:31
*** andreas__ has joined #openstack-requirements07:33
*** andreas_s has quit IRC07:34
*** andreas_s_ has quit IRC07:35
*** edmondsw has joined #openstack-requirements07:36
*** florianf has joined #openstack-requirements07:37
*** oanson has quit IRC07:39
*** oanson has joined #openstack-requirements07:41
*** edmondsw has quit IRC07:41
*** jpich has joined #openstack-requirements08:03
*** ccamacho has joined #openstack-requirements08:08
*** ralonsoh has joined #openstack-requirements08:30
*** edmondsw has joined #openstack-requirements09:25
*** edmondsw has quit IRC09:29
*** hwoarang has quit IRC09:53
*** hwoarang has joined #openstack-requirements09:54
*** hwoarang has quit IRC09:54
*** hwoarang has joined #openstack-requirements09:54
*** hwoarang has quit IRC09:56
*** hwoarang has joined #openstack-requirements09:57
*** hwoarang has quit IRC09:57
*** hwoarang has joined #openstack-requirements09:57
*** dtantsur|afk is now known as dtantsur10:14
*** vpickard_ is now known as vpickard11:10
*** udesale has quit IRC11:36
*** amoralej is now known as amoralej|lunch11:46
*** olivierb- has joined #openstack-requirements12:25
olivierb-Hello everyone, I have a huge issue with https://review.openstack.org/#/c/579322/1/upper-constraints.txt in which the replacement of PyMySQL===0.8.1 by 0.9. completely breaks my Ubuntu xenial system. In fact the pip installation (without venv) of this version of the MySQL driver drags cryptography 2.2.2 which completely breaks pip and any network related python function12:28
*** dmellado has joined #openstack-requirements12:28
olivierb-can not use pip freeze, pip install or anything afterwards12:28
olivierb-"/usr/lib/python2.7/dist-packages/OpenSSL/SSL.py\", line 118, in <module>\n    SSL_ST_INIT = _lib.SSL_ST_INIT\nAttributeError: 'module' object has no attribute 'SSL_ST_INIT'\n"}12:29
olivierb-I found about this because it breaks bifrost deployment12:29
*** r-mibu has joined #openstack-requirements13:00
*** edmondsw has joined #openstack-requirements13:01
*** andreas__ has quit IRC13:03
*** andreas_s has joined #openstack-requirements13:03
*** edmondsw has quit IRC13:06
*** openstackgerrit has joined #openstack-requirements13:21
openstackgerritDoug Hellmann proposed openstack/requirements master: switch project-requirements-change to use python3  https://review.openstack.org/57894413:21
*** andreas_s has quit IRC13:22
*** amoralej|lunch is now known as amoralej13:25
*** andreas_s has joined #openstack-requirements13:26
*** andreas_s has quit IRC13:36
*** andreas_s has joined #openstack-requirements13:44
*** andreas_s has quit IRC13:47
*** andreas_s has joined #openstack-requirements13:51
*** andreas_s has quit IRC13:54
*** andreas_s has joined #openstack-requirements13:55
*** cjloader has joined #openstack-requirements13:57
*** andreas_s has quit IRC13:59
*** andreas_s has joined #openstack-requirements14:01
*** andreas_s has quit IRC14:01
*** andreas_s has joined #openstack-requirements14:02
*** andreas_s has quit IRC14:02
openstackgerritOpenStack Proposal Bot proposed openstack/requirements stable/pike: update constraint for cliff to new release 2.8.2  https://review.openstack.org/57987614:03
*** andreas_s has joined #openstack-requirements14:03
openstackgerritOpenStack Proposal Bot proposed openstack/requirements stable/queens: update constraint for cliff to new release 2.11.1  https://review.openstack.org/57987814:06
openstackgerritOpenStack Proposal Bot proposed openstack/requirements master: update constraint for cliff to new release 2.13.0  https://review.openstack.org/57987914:07
*** rpioso|afk is now known as rpioso14:21
*** andreas_s has quit IRC14:21
*** andreas_s has joined #openstack-requirements14:26
*** andreas_s has quit IRC14:26
*** andreas_s has joined #openstack-requirements14:27
*** andreas_s has quit IRC14:49
*** andreas_s has joined #openstack-requirements14:50
*** edmondsw has joined #openstack-requirements14:51
*** edmondsw has quit IRC14:55
*** cjloader has quit IRC14:58
*** cjloader has joined #openstack-requirements14:58
*** vpickard has quit IRC15:03
*** vpickard has joined #openstack-requirements15:03
*** qwebirc63850 has joined #openstack-requirements15:08
*** qwebirc63850 has quit IRC15:09
olivierb-anybody any clue about upper_constraints issue mentioned ^^15:36
prometheanfireolivierb-: are you using pip system wide?15:44
prometheanfireaka, not in a venv15:44
olivierb-yes15:44
prometheanfirethat's well known to break systems15:45
olivierb-but I think that even in venv it should fail, I'll have to try a little bit later when I can restore an env15:45
olivierb-the problem is that this was working before15:45
olivierb-and if I revert 0.9.0 to 0.8.1 it works again15:45
olivierb-so for me it's a big bug15:45
prometheanfireI don't think we can or should guarantee that installing constraints system wide won't break the system15:46
olivierb-again it was not breaking system before15:46
prometheanfirewithout the cause of the issue I can't help much15:46
olivierb-the cause is PyMySQL upgraded to 0.9.0 which drags cryptography 2.2.2 which breaks everything15:47
prometheanfirewe updated to that cryptography 3 months ago15:48
prometheanfirehttps://github.com/openstack/requirements/blame/master/upper-constraints.txt#L16315:48
olivierb-may be be this was not impacting bifrost+keystone deployement until PyMySQL was updated to 0.9.915:52
olivierb-0.9.015:52
olivierb-3 days ago15:53
olivierb-merged today15:53
olivierb-AFAIK of course15:53
prometheanfiresure, that happened around the same time, but until we know how it happens we won't know the 'correct' fix15:54
*** ccamacho has quit IRC15:54
*** jpich has quit IRC16:06
olivierb-prometheanfire I understand and I think I have found an extra condition for this to happen, currently digging more into this ...16:12
prometheanfirethanks16:13
*** andreas_s has quit IRC16:22
*** andreas_s has joined #openstack-requirements16:22
*** andreas_s has quit IRC16:22
*** r-mibu has quit IRC16:29
*** ralonsoh has quit IRC16:35
openstackgerritMatthew Thode proposed openstack/requirements master: Updated from generate-constraints  https://review.openstack.org/57977416:37
johnsomprometheanfire Any idea on this "ImportError: cannot import name parse" error?16:43
*** amoralej is now known as amoralej|off16:45
prometheanfirejohnsom: nope16:49
johnsomOk, then I will just make those non-voting to get this revert moving17:00
*** dtantsur is now known as dtantsur|afk17:09
prometheanfirejohnsom: it's a problem with spec_cleaner17:11
prometheanfirehttps://github.com/openSUSE/spec-cleaner/blob/d7e49579e7c523340ef4944d84dcd0890b4e8e6f/spec_cleaner/rpmpreamble.py#L5 is called without urllib being installed17:11
johnsomSo should you be excluding that version of "spec_cleaner"?17:21
prometheanfirewell, queens upper-constraint is set to 1.0.117:22
prometheanfireI think devstack is broken it's not using constraints17:23
prometheanfirehttp://logs.openstack.org/25/579725/1/check/legacy-requirements-integration-dsvm/e3f2e50/job-output.txt.gz#_2018-07-03_04_23_00_61193017:23
prometheanfirels17:25
openstackgerritMatthew Thode proposed openstack/requirements stable/queens: update spec-cleaner global-requirement for py3 only versions  https://review.openstack.org/57993617:47
prometheanfirejohnsom: https://review.openstack.org/579936 should help I think17:47
johnsomprometheanfire Cool, thanks17:47
prometheanfirefor the level of breakage, I believe it's allowed https://docs.openstack.org/project-team-guide/dependency-management.html#global-requirements17:49
openstackgerritDirk Mueller proposed openstack/requirements stable/queens: Remove spec-cleaner from upper-constraints/gr tracking  https://review.openstack.org/57993817:53
openstackgerritDirk Mueller proposed openstack/requirements stable/queens: Revert "add neutron-lbaas to global-requirements"  https://review.openstack.org/57972517:54
prometheanfiredirk: I'm not sure we can do that to stable17:54
prometheanfiredirk: we did remove the project (and only the projects list in master is valid iirc) but I am still not sure if we can remove it policy wise17:55
prometheanfirewe have the same project in pike17:56
dirkAh, I see there are at least 4 threads about the same issue17:58
dirkGreat waste of time :-)17:58
dirkprometheanfire: spec-cleaner isn't used by any project that follows stable policy17:59
prometheanfiredirk: yep, but we still don't have policy about removing stuff from stable branches17:59
prometheanfiredirk: for now I think https://review.openstack.org/579936 is the fix18:00
dirkWell, it is the same like adding a constraint18:00
dirkThe integration job should not use it imho18:01
prometheanfirekinda, ya, I think we should bring it up in the meeting I don't trust just two of us to decide18:02
prometheanfireI do remember that it wasn't used by anything else in openstack (I removed it :P)18:02
johnsomprometheanfire Now that we have a proposed usage, would you consider upgrading to a +2: https://review.openstack.org/57898318:03
openstackgerritMichael Johnson proposed openstack/requirements stable/queens: Revert "Add constraints for the neutron-lbaas"  https://review.openstack.org/57971718:04
prometheanfireoh ya, meant to do that18:04
johnsomThanks for the rebase dirk18:04
dirkprometheanfire: well, if it helps I can remove the spec cleaner usage ;-)18:06
dirkprometheanfire: are you concerned that there's a real usage somewhere or that we're just deviating from documentated policy?18:19
olivierb-prometheanfire nailed down to installation of virtinst package on Ubuntu xenial which trigger some python dependencies which in turn produce the error if installed outside virtinst context18:20
olivierb-list is python-libvirt python-openssl python-gi python-ipaddr python-ndg-httpsclient python-requests python-urllib318:20
olivierb-the 1st 2 are already eliminated, going on with remaining items18:20
olivierb-even if verly long and slow process, VM snapshot help a lot ;-)18:21
prometheanfiredirk: policy18:23
prometheanfiredirk: just want another voice, we can update the docs to say add/remove18:26
prometheanfirejust have the be very careful about stable branches18:26
*** edmondsw has joined #openstack-requirements18:27
*** edmondsw has quit IRC18:32
olivierb-prometheanfire horray, nailed it down18:47
dirkprometheanfire: makes sense18:47
dirkprometheanfire: it looks like I will not be able to join the meeting going forward though18:48
dirkSo please bring it up there18:48
olivierb-python-ndg-httpsclient is the ubuntu xenial package which if installed cause "incompatibility" with cryptography 2.2.218:49
olivierb-I do not know why this packages is dragged by one of my mandatory packages (virtinst) but this is why bifrost deployment is failing18:49
*** florianf has quit IRC18:58
olivierb-prometheanfire and I confirm that it works perfectly well under the EXACT same conditions but using venv :-(18:58
prometheanfiredirk: :(19:05
olivierb-prometheanfire HTH19:23
prometheanfireolivierb-: ?19:25
olivierb-see ^^19:25
olivierb-prometheanfire do you want me to write short summary ?19:32
prometheanfireolivierb-: I think it'd help, I'm not familiar with your use case19:33
olivierb-ok so if you use venvs there are absolutely no issue (point 1)19:33
olivierb-if you do not use vens and package python-ndg-httpsclient is installed on your ubuntu xenial system then installation of PyMySQL===0.9.0 makes the python system unusable (pip freeze -> error, pip install -> error, any http based stuff -> unusable [like ansible package install], ...) (point 2)19:35
olivierb-this package is installed on my system because it is a dependency of virtinstall which I need for other purpose19:36
olivierb-same system with virtinstall+python-ndg-httpsclient installed and PyMySQL===0.8.1 works like  a charm19:37
olivierb-does this make sense ?19:37
olivierb-prometheanfire ^^19:38
prometheanfireolivierb-: it looks like all that changed was that 0.9.1 requires cryptography https://github.com/PyMySQL/PyMySQL/compare/v0.8.1...v0.9.1#diff-2eeaed663bd0d25b7e608891384b729819:39
olivierb-prometheanfire guess so yes19:39
olivierb-make sense with what I digged out today19:40
prometheanfireso... what is your pip install line? are you using constraints?19:40
olivierb-anyways not an issue for me as I moved to venv but may be need to be adressed somehow19:41
olivierb-prometheanfire yes here is what comes out of bifrost deployement in -vvv mode:19:41
olivierb- /usr/bin/pip2 install -c /opt/stack/requirements/upper-constraints.txt pymysql"19:41
prometheanfireok, maybe xenial doesn't like the new cryptography19:42
prometheanfireor maybe the new cryptography has a dep that's unsatisfied or something19:42
olivierb-prometheanfire your are most probably right, I logged a bug some time ago about incompatibility between dnsmasq and squid which is ONLY relevant with xenial (works with trusty and all others) and which nobody else found strangely enough ;-) https://bugs.launchpad.net/bugs/176109619:46
openstackLaunchpad bug 1761096 in squid (Ubuntu Xenial) "dnsmasq starts with error on Ubuntu Xenial amd64 when squid installed" [High,Confirmed]19:46
openstackgerritMerged openstack/requirements master: update constraint for cliff to new release 2.13.0  https://review.openstack.org/57987919:47
openstackgerritMerged openstack/requirements master: add build-lower-constraints command  https://review.openstack.org/57926820:06
*** edmondsw has joined #openstack-requirements20:16
*** edmondsw has quit IRC20:21
*** cjloader has quit IRC21:57
*** lbragstad is now known as lbragstad_50322:02
openstackgerritMerged openstack/requirements master: Proposing distro for global-requirements  https://review.openstack.org/57898322:03
*** edmondsw has joined #openstack-requirements22:05
*** edmondsw has quit IRC22:09
*** rpioso is now known as rpioso|afk22:46
prometheanfiretonyb: if you are around, dirk and I have competing reviews to fix stable gate23:41
prometheanfirehttps://review.openstack.org/579936 and https://review.openstack.org/57993823:41
*** edmondsw has joined #openstack-requirements23:53
*** edmondsw has quit IRC23:57
« Monday, 2018-07-02 Index Wednesday, 2018-07-04 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!