Tuesday, 2017-07-11

*** bkopilov has joined #openstack-glance00:00
*** jessegler has quit IRC00:04
*** gcb has quit IRC00:21
*** bkopilov has quit IRC00:54
*** bkopilov has joined #openstack-glance00:57
*** zzzeek_ has quit IRC01:09
*** zzzeek_ has joined #openstack-glance01:11
*** bkopilov has quit IRC01:16
*** bkopilov has joined #openstack-glance01:16
*** zzzeek_ has quit IRC01:33
*** zzzeek_ has joined #openstack-glance01:34
*** zzzeek_ has quit IRC01:36
*** smatzek has joined #openstack-glance01:40
*** smatzek has quit IRC01:41
*** smatzek has joined #openstack-glance01:41
*** zzzeek_ has joined #openstack-glance01:47
*** smatzek has quit IRC01:50
*** Sukhdev has quit IRC01:57
*** bkopilov has quit IRC01:59
*** gcb has joined #openstack-glance02:03
*** zhurong has joined #openstack-glance02:03
*** namnh has joined #openstack-glance03:07
openstackgerritPengju Jiao proposed openstack/glance master: Fix glance image-download error  https://review.openstack.org/46028003:17
*** bkopilov has joined #openstack-glance03:18
*** gyee has quit IRC03:23
*** links has joined #openstack-glance03:41
*** udesale has joined #openstack-glance03:46
*** mabrams has joined #openstack-glance05:02
*** aavraham has joined #openstack-glance05:02
*** aavraham has quit IRC05:03
*** aavraham has joined #openstack-glance05:04
*** ratailor has joined #openstack-glance05:10
*** d0ugal has joined #openstack-glance05:22
*** groen692 has joined #openstack-glance05:40
*** rcernin has joined #openstack-glance05:41
*** d0ugal has quit IRC05:52
*** nicolasbock has joined #openstack-glance05:55
*** pcaruana has joined #openstack-glance06:04
*** tshefi has joined #openstack-glance06:31
*** groen692 has quit IRC06:32
*** groen692 has joined #openstack-glance06:33
*** zhurong has quit IRC06:40
*** zhurong has joined #openstack-glance06:44
*** belmoreira has joined #openstack-glance06:46
*** namnh has quit IRC06:52
openstackgerritZhaokun Fu proposed openstack/glance master: replace http with https  https://review.openstack.org/45715106:55
*** namnh has joined #openstack-glance07:01
*** tesseract has joined #openstack-glance07:02
*** zhurong has quit IRC07:30
*** openstackgerrit has quit IRC07:33
*** andreykurilin has joined #openstack-glance07:44
andreykurilinhi folks!07:44
andreykurilinFor the last month we have too much failures related to glance07:44
*** aarefiev_pto is now known as aarefiev07:44
andreykurilinin rally gates07:44
andreykurilinI suppose it had happen after moving under uwsgi07:45
*** namnh has quit IRC07:45
andreykurilinthe last failure `502 Bad Gateway: Bad Gateway: The proxy server received an invalid: response from an upstream server.: Apache/2.4.18 (Ubuntu) Server at 10.13.185.120 Port 80 (HTTP 502) `07:45
andreykurilinCan anyone help me to investigate an issue, possible just proper configuration of devsack plugin is required07:46
*** markvoelker_ has quit IRC07:47
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-glance08:00
*** namnh has joined #openstack-glance08:01
*** e0ne has joined #openstack-glance08:19
*** mvk has quit IRC08:26
*** zhurong has joined #openstack-glance08:27
*** mvk has joined #openstack-glance08:55
*** zhurong has quit IRC09:27
*** e0ne has quit IRC09:31
*** e0ne has joined #openstack-glance09:34
*** openstackgerrit has joined #openstack-glance09:36
openstackgerritErno Kuvaja proposed openstack/glance master: Add Discovery stub for Image Import  https://review.openstack.org/48218209:36
*** mvk has quit IRC09:37
*** links has quit IRC09:37
*** links has joined #openstack-glance09:39
*** links has quit IRC09:44
*** dalgaaf has quit IRC09:47
*** dalgaaf has joined #openstack-glance09:47
*** markvoelker has joined #openstack-glance09:48
*** links has joined #openstack-glance09:49
*** mvk has joined #openstack-glance09:51
*** d0ugal has joined #openstack-glance09:52
*** d0ugal has quit IRC09:59
jokke_andreykurilin: I can't remember when last time the NV rally job would have succeeded in glance runs10:01
andreykurilinjokke_: we launchs scenarios of glance in our regular gates10:02
andreykurilineverything was quite stable and I did not see any failures before moving to the uwsgi10:02
andreykurilinthe concurrency bigger than 1 kills glance with a big probability10:03
jokke_hmm-m ... racecondition somewhere?10:04
andreykurilinyes10:04
andreykurilinjokke_: I can share logs of jobs, if needed10:05
*** groen692 has quit IRC10:06
andreykurilinbut I couldn't find any usefull there10:06
andreykurilinlogs.openstack.org/15/481915/4/gate/gate-rally-dsvm-rally-ubuntu-xenial/ffbf74c/logs10:07
andreykurilinan error- http://logs.openstack.org/15/481915/4/gate/gate-rally-dsvm-rally-ubuntu-xenial/ffbf74c/rally-plot/results.html.gz#/GlanceImages.create_and_get_image/failures10:07
*** namnh has quit IRC10:08
*** gcb has quit IRC10:13
*** links has quit IRC10:13
jokke_Hmm-m10:16
jokke_I think we were fighting with the bad gateway responses somewhere else while ago, just can't remember where/what was it10:16
*** markvoelker has quit IRC10:22
*** groen692 has joined #openstack-glance10:22
andreykurilinjokke_: previousy there was another issue11:11
*** bkopilov has quit IRC11:11
andreykurilinhttps://review.openstack.org/#/c/479366/11:11
andreykurilinit was 500 InternalError as far as I remember11:12
andreykurilinmtreinish: hi! are you around?11:18
*** markvoelker has joined #openstack-glance11:20
*** smatzek has joined #openstack-glance11:23
*** smatzek has quit IRC11:24
*** smatzek has joined #openstack-glance11:24
*** links has joined #openstack-glance11:46
*** ratailor has quit IRC11:51
*** markvoelker has quit IRC11:53
*** links has quit IRC11:59
*** markvoelker has joined #openstack-glance12:19
*** MVenesio has joined #openstack-glance12:32
*** bkopilov has joined #openstack-glance12:33
*** openstackgerrit has quit IRC12:47
*** udesale has quit IRC12:58
*** udesale__ has joined #openstack-glance12:58
*** burgerk has joined #openstack-glance13:01
*** gcb has joined #openstack-glance13:32
*** burgerk has quit IRC13:47
*** d0ugal has joined #openstack-glance13:54
*** jistr is now known as jistr|call13:58
*** d0ugal has quit IRC14:00
*** zzzeek_ has quit IRC14:18
*** zzzeek_ has joined #openstack-glance14:21
*** d0ugal has joined #openstack-glance14:25
*** gyee has joined #openstack-glance14:27
*** d0ugal has quit IRC14:31
*** efried has joined #openstack-glance14:34
efriedjokke_ yt?14:34
*** rcernin has quit IRC14:38
*** rcernin has joined #openstack-glance14:40
jokke_efried: yes14:55
efriedjokke_ https://review.openstack.org/#/c/460280/14 is busted FYI14:55
efriedjokke_ I don't claim to be a glance expert, mind you.14:55
efriedjokke_ I was going to ask if it would be okay for me to propose a new patch set.14:56
jokke_efried: ohh ... hmm-m ... are you sure that it's busted ... it looks right for the v1 path14:59
efriedjokke_ For the v1 path I think it's good.14:59
efriedjokke_ It's the v2 path - the specific thing the bug was trying to fix - that's broken.14:59
jokke_yeah and that's the whole point for that if..else in the cache there14:59
efriedjokke_ AFAICT, nothing ever refers to image_meta.size - only image_meta['size'] (which goes through the __getitem__ shenanigans in ImageTarget).15:00
efriedjokke_ That __getitem__ stuff actually resolves the attr on the `target` member, not on the ImageTarget itself.15:00
efriedSo setting image_meta.size won't result in image_meta['size'] having that value.15:01
jokke_efried: so the if..else in the cache code does the recognition. If it's ImageTarget we are dealing with v2 path and that handles it. If not then we know that we're dealing with v1 path and handle it correctly again15:01
jokke_or did I totally misread that15:02
efriedjokke_ Right, I'm talking specifically about the v2 path.  I don't think setting image_meta.size is the right thing to do.  You would need to set image_meta.target.size instead.15:02
jokke_oh no ... correct if I'm wrong, but it's that 'if _not_' part that flips the logic wrong way around15:03
jokke_it should work just fine without that not there15:03
jokke_I think I really misread it15:03
efriedjokke_ I'm not debating the conditionals.  They're set up correctly.  (Though I personally would have done them the other way to avoid the `not`.)15:04
efriedjokke_ I'm saying L79 is setting the wrong thing on the ImageTarget.15:05
*** mabrams has quit IRC15:11
*** zzzeek_ has quit IRC15:12
*** tshefi has quit IRC15:18
*** rcernin has quit IRC15:18
*** jistr|call is now known as jistr15:22
jokke_efried: I'm still missing something here ... let me have another look with full focus what's going on15:26
efriedjokke_ Sure thing.  Perhaps easiest to explain in the context of the test case...15:27
efriedjokke_ https://review.openstack.org/#/c/460280/14/glance/tests/unit/test_cache_middleware.py@364 is asserting on image_meta.size15:28
efriedjokke_ But nothing anywhere else in the code base ever refers to image_meta.size - it is only ever used as image_meta['size']15:28
*** catintheroof has joined #openstack-glance15:28
efriedAnd if you replace that part of the assertion with image_meta['size'], the test fails.15:28
efriedWhich means the bug ain't fixed.15:28
*** zzzeek_ has joined #openstack-glance15:30
*** aavraham has left #openstack-glance15:33
*** e0ne has quit IRC15:34
jokke_efried: I hear you ... and by the looks if it if it's ImageTarget that class has only ever target set15:37
jokke_there is nothing else as object attribute on that object15:38
*** belmoreira has quit IRC15:38
*** udesale__ has quit IRC15:40
jokke_efried: but setting it as image_meta['size'] should not work on that later case either as ImageTarget does not have __setitem__ defined so that will not be set on the target either, no?15:42
efriedjokke_ Correct - assigning to image_meta['size'] doesn't work.15:43
efriedjokke_ The way to do it is to assign to image_meta.target.size15:43
jokke_efried: yes, now we're on same page .... thanks for bearing with me on this15:43
efriedjokke_ The change as it stands will "resolve" the bug - in the sense that we no longer see the exception trace, because we're no longer trying to assign with dict syntax.15:44
efriedBut15:44
jokke_and only reason why that test does not fail is that it's referring to that size we set on ImageTarget earlier, not to the target objects size attribute15:44
efriedIt will regress the fix for bug 900959 (mentioned in the comment) because it's not actually setting the size in the right place.15:45
openstackbug 900959 in Glance "Adding remote image when cache enabled results in image having size of 0" [Critical,Fix released] https://launchpad.net/bugs/900959 - Assigned to Eoghan Glynn (eglynn)15:45
efriedjokke_ Yes, exactly.15:45
jokke_yes and that was the part I overlooked ... Thank you! Amazing catch!15:45
efriedjokke_ Sure thing.  I have the fix in my sandbox - would it be appropriate for me to propose it, or should I let the authors handle it?15:47
efriedLooks like Pengju may still be active.  I'll follow up in the code comments.15:48
jokke_efried: I think as Pengju has as much if not even more than me problem to understand what you actually found there, it would be amazing if you post fix and make sure co-authors are in the commit correctly ;)15:48
efriedjokke_ Okay.  Surely I'd want to make sure the authors are on the same page either way.15:49
jokke_Maybe the fix for the issue you pointed out helps him understand as well what went wrong in that patch15:49
efriedRoger wilco.15:50
*** aarefiev is now known as aarefiev_afk15:56
*** openstackgerrit has joined #openstack-glance16:01
openstackgerritEric Fried proposed openstack/glance master: Fix glance image-download error  https://review.openstack.org/46028016:01
efriedjokke_ ^^16:01
openstackgerritMike Fedosin proposed openstack/glance master: Satisfy API Reference documentation deleting tags  https://review.openstack.org/42003816:36
*** groen692 has quit IRC16:52
*** efried is now known as efried_lunch16:58
*** groen692 has joined #openstack-glance17:44
*** zzzeek_ has quit IRC17:46
*** zzzeek_ has joined #openstack-glance17:49
*** tesseract has quit IRC18:25
*** groen692 has quit IRC18:26
*** nicolasbock has quit IRC18:50
*** efried_lunch is now known as efried18:56
*** mvk has quit IRC19:11
*** MVenesio has quit IRC20:25
*** adisky__ has quit IRC20:36
*** zzzeek_ has quit IRC20:49
*** catintheroof has quit IRC21:04
*** zzzeek_ has joined #openstack-glance21:11
*** smatzek has quit IRC21:17
*** pcaruana has quit IRC21:23
andreykurilinoh...our gates are blocked, can anyone help with investigation about glance+uwsgi configuration? previously everything was ok22:12
*** markvoelker has quit IRC22:16
*** markvoelker has joined #openstack-glance22:17
*** efried is now known as efried_zzz22:19
*** d0ugal has joined #openstack-glance23:36
*** masber has joined #openstack-glance23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!