Monday, 2020-09-14

*** tetsuro has joined #openstack-infra00:04
*** armax has quit IRC00:29
*** xavpaice has quit IRC00:31
*** xavpaice has joined #openstack-infra00:34
*** ramishra has joined #openstack-infra02:04
*** zzzeek has quit IRC02:20
*** zzzeek has joined #openstack-infra02:22
*** tetsuro has quit IRC02:48
*** psachin has joined #openstack-infra03:37
*** zzzeek has quit IRC03:50
*** zzzeek has joined #openstack-infra03:51
*** ykarel has joined #openstack-infra04:32
*** evrardjp has quit IRC04:33
*** evrardjp has joined #openstack-infra04:33
*** ramishra has quit IRC04:41
*** vishalmanchanda has joined #openstack-infra04:45
*** lmiccini has joined #openstack-infra05:51
*** matt_kosut has joined #openstack-infra05:57
*** lpetrut has joined #openstack-infra05:57
*** ralonsoh has joined #openstack-infra06:05
*** ysandeep|away is now known as ysandeep06:10
*** fresta has joined #openstack-infra06:12
*** cgoncalves has joined #openstack-infra06:13
*** ramishra has joined #openstack-infra06:23
*** eolivare has joined #openstack-infra06:31
openstackgerritCarlos Goncalves proposed openstack/project-config master: Update branch checkout for octavia-lib DIB element  https://review.opendev.org/74587706:33
*** hashar has joined #openstack-infra06:40
*** slaweq_ has joined #openstack-infra06:50
*** slaweq_ is now known as slaweq06:56
*** Ajohn has quit IRC06:56
*** jtomasek has joined #openstack-infra06:58
*** slaweq_ has joined #openstack-infra07:14
*** slaweq has quit IRC07:15
*** jpena|off is now known as jpena07:22
*** jcapitao has joined #openstack-infra07:29
*** jtomasek has quit IRC07:39
*** jtomasek has joined #openstack-infra07:41
*** andrewbonney has joined #openstack-infra07:42
*** gfidente has joined #openstack-infra07:42
*** ysandeep is now known as ysandeep|lunch07:46
*** tosky has joined #openstack-infra08:27
*** derekh has joined #openstack-infra08:42
*** lucasagomes has joined #openstack-infra08:55
*** ysandeep|lunch is now known as ysandeep09:10
*** sshnaidm|pto is now known as sshnaidm09:10
*** xavpaice has quit IRC09:45
*** howell has quit IRC09:45
*** xavpaice has joined #openstack-infra09:46
*** howell has joined #openstack-infra09:46
*** xek has joined #openstack-infra09:47
*** xek has quit IRC09:54
*** smarcet has joined #openstack-infra10:04
*** slaweq_ is now known as slaweq10:12
*** e0ne has joined #openstack-infra10:24
*** dtantsur|afk is now known as dtantsur10:27
*** Lucas_Gray has joined #openstack-infra10:30
*** jcapitao is now known as jcapitao_lunch10:34
e0nehi. could anybody please help with vitrage gates troubleshooting? we've got issues: controller | xz: (stdin): Cannot allocate memory10:42
*** Lucas_Gray has quit IRC10:42
e0nehttps://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_631/751657/2/check/vitrage-tempest-plugin-datasources/6316f3f/job-output.txt10:42
*** Lucas_Gray has joined #openstack-infra10:47
*** Lucas_Gray has quit IRC10:51
*** Wryhder has joined #openstack-infra10:51
*** Wryhder is now known as Lucas_Gray10:51
*** jpena is now known as jpena|lunch11:31
*** dave-mccowan has joined #openstack-infra11:48
*** ramishra has quit IRC11:48
*** Lucas_Gray has quit IRC11:50
*** ramishra has joined #openstack-infra11:52
*** Lucas_Gray has joined #openstack-infra11:53
*** rfolco has joined #openstack-infra11:55
*** rfolco is now known as rfolco|ruck11:55
*** Lucas_Gray has quit IRC11:58
*** jcapitao_lunch is now known as jcapitao11:59
*** lpetrut has quit IRC12:00
*** lpetrut has joined #openstack-infra12:01
zbrinteresting thread on pbr: https://github.com/realpython/python-guide/issues/494#event-376276312812:03
*** ramishra_ has joined #openstack-infra12:05
*** smarcet has quit IRC12:08
*** ramishra has quit IRC12:08
*** eharney has joined #openstack-infra12:10
*** smarcet has joined #openstack-infra12:10
*** Goneri has joined #openstack-infra12:11
*** rlandy has joined #openstack-infra12:14
*** priteau has joined #openstack-infra12:14
*** jpena|lunch is now known as jpena12:28
*** vishalmanchanda has quit IRC12:28
EmilienMhey folks, codesearch seems down to me (fail to load)12:34
*** slaweq_ has joined #openstack-infra12:37
*** xek has joined #openstack-infra13:00
*** smarcet has quit IRC13:03
zbrclarkb: ianw: please look at https://review.opendev.org/#/c/751608/1 about install-from-bindep13:03
*** Lucas_Gray has joined #openstack-infra13:04
openstackgerritThierry Carrez proposed openstack/ptgbot master: Make clickable badges more button-like  https://review.opendev.org/75178113:05
openstackgerritThierry Carrez proposed openstack/ptgbot master: Mark current time on the schedule  https://review.opendev.org/75178213:05
*** zxiiro has joined #openstack-infra13:05
*** irclogbot_3 has quit IRC13:19
*** irclogbot_2 has joined #openstack-infra13:25
*** kaisers has joined #openstack-infra13:30
*** artom has joined #openstack-infra13:33
*** tkajinam has quit IRC13:37
*** slaweq_ has quit IRC13:38
*** jtomasek has quit IRC13:39
*** ricolin has quit IRC13:52
*** d34dh0r53 has joined #openstack-infra13:54
*** smarcet has joined #openstack-infra14:01
fungiEmilienM: thanks for the heads up, it should be returning to service shortly (it had crashed with an odd json encoding panic)14:01
openstackgerritMerged openstack/ptgbot master: Make clickable badges more button-like  https://review.opendev.org/75178114:02
EmilienMfungi: thanks!14:02
*** auristor has quit IRC14:05
openstackgerritMerged openstack/ptgbot master: Mark current time on the schedule  https://review.opendev.org/75178214:05
*** sshnaidm is now known as sshnaidm|afk14:07
*** ricolin has joined #openstack-infra14:11
*** hashar has quit IRC14:14
*** auristor has joined #openstack-infra14:18
*** smarcet has quit IRC14:20
*** smarcet has joined #openstack-infra14:23
*** ykarel_ has joined #openstack-infra14:25
*** ykarel has quit IRC14:28
*** dklyle has joined #openstack-infra14:30
*** ykarel__ has joined #openstack-infra14:32
*** ykarel_ has quit IRC14:35
*** ykarel__ is now known as ykarel14:35
openstackgerritCarlos Goncalves proposed openstack/project-config master: Add 'check arm64' trigger to check-arm64 pipeline  https://review.opendev.org/75182914:36
*** slaweq_ has joined #openstack-infra14:41
*** icey has quit IRC14:48
*** icey has joined #openstack-infra14:49
*** __ministry1 has joined #openstack-infra14:54
*** slaweq_ has quit IRC14:55
*** ykarel is now known as ykarel|away15:00
*** lpetrut has quit IRC15:06
*** armax has joined #openstack-infra15:12
*** ykarel|away has quit IRC15:20
noonedeadpunk\o/15:22
noonedeadpunkhave pretty nasty thing here and not sure whom I can ask for some help... We have issue with releasenotes, that they got merged for train and ussuri. I mean that reno for train is ok, but U ones have both train and ussuir15:23
noonedeadpunkI'm not sure how reno distinguish them by releases....15:23
*** Topner has joined #openstack-infra15:23
clarkbnoonedeadpunk: I feel like I keep getting this wrong but it is my understanding that reno should run from a single context view that sees all branches and constructs the release notes. For everything in one document15:24
AJaegernoonedeadpunk: best ask on openstack-release channel15:24
*** ianychoi has joined #openstack-infra15:24
noonedeadpunkok, thanks!15:24
fungiyeah, the release team has the best handle on nuances of reno15:24
noonedeadpunkyeah, seems like this for me as well....15:24
clarkbzbr: can you explain why that change is necessary? I think the current expectation is that builds use the /output dir15:25
clarkbzbr: I'm not sure I understand the problem that is trying to solve, you can write the image builds to use that dir15:25
*** lpetrut has joined #openstack-infra15:28
*** xek has quit IRC15:35
e0nehi. could anybody please help with vitrage gates troubleshooting? we've got issues: controller | xz: (stdin): Cannot allocate memory15:36
e0nehttps://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_631/751657/2/check/vitrage-tempest-plugin-datasources/6316f3f/job-output.txt15:36
clarkbe0ne: that isn't the error causing the job to fail15:37
*** lmiccini has quit IRC15:38
clarkbe0ne: it looks like tox has 13 failed tests which is why the job fails. Then when it tries tocollect logs xz is unable to compress the journal due to a lack of memory (perhaps the journal is too big or there is very little memory, both may be related to the tox failure)15:38
clarkbe0ne: it looks like we aren't getting other service logs due to the xz error, you may want to push a change up to devstack to make that do its best but if it fails the play should continue15:40
clarkb(then you should get logs for the other services assuming the memory issue isn't critical)15:40
e0neclarkb: sorry, I didn't  what are you proposing to do with a devstack15:43
clarkbe0ne: make the export tasks in https://opendev.org/openstack/devstack/src/branch/master/roles/export-devstack-journal/tasks/main.yaml not fail on error15:44
clarkbI think you can use ansible's failed_when: false for that15:44
e0neclarkb: will try it. thanks'15:45
clarkbthe idea is it should do its absolute best to get those logs but if there are problems (as in your case) then just continue and get whatever logs it can get15:45
e0negot it15:47
*** e0ne has quit IRC15:53
*** zzzeek has quit IRC15:54
*** zzzeek has joined #openstack-infra15:55
*** lucasagomes has quit IRC16:01
*** zzzeek has quit IRC16:02
*** zzzeek has joined #openstack-infra16:04
*** priteau has quit IRC16:04
zbrclarkb: re /output dir, at least for the e-r this folder contains only the files to be installed, there is no use for putting the wheels inside the builded containers16:09
zbris like including rpm/dep files inside container16:09
*** zzzeek has quit IRC16:09
*** psachin has quit IRC16:09
zbryou want to install them, but you do not want to copy them inside the container, not into a layer.16:09
*** zzzeek has joined #openstack-infra16:10
zbrif you use dive tool to look inside the builder container you realise that output is likely not something we want.16:10
zbrone trick that can be used to avoit this is to use /tmp instead, as that one is not stored inside.16:11
clarkbzbr: the builder container is not part of the final product16:11
zbrstill, the current install-from-bindep is hardcoded to use /output16:11
*** jcapitao has quit IRC16:11
clarkbzbr: the builder container is a temporary throwaway scratch space so that we detroy all the unneeded build artifacts16:11
clarkbzbr: it is meant to be used in conjunction with the python-base image which does form the basis of the final published images16:12
zbrtake a look at https://review.opendev.org/#/c/750958/8/Dockerfile16:12
zbrlike 25 does copy output from builder to final one, something we do not want.16:12
zbrwe do not need to store wheels on target one, we want to install them16:13
clarkbyes please see how other images do this: https://opendev.org/zuul/zuul/src/branch/master/Dockerfile#L53-L55 I don't believe this is an issue16:13
zbri used dive tool to investigate containers and I seen layers with /output in them, not small at all.16:14
clarkbzbr: did you look at projects like zuul which seem to do the right thing?16:15
*** zzzeek has quit IRC16:15
zbri do not see a difference16:15
zbrbut i can try to investigate zuul image, to see if there is something different or not16:15
clarkbZuul intentionally rm's the /output dir when it is done with it16:15
clarkbother images may not, but I think what zuul does is the expected use of the interface16:16
clarkbif zuul looks good then my hunch is the problem is in the other consumers of python-builder not in the api presented by python-builder16:17
clarkb(and we can fix that up/document it as necessary)16:17
* zbr looking at zuul....16:18
zbrbut as far as I know, the less layers the better.16:18
clarkbright that should be one layer though because its all part of the single directive16:19
clarkbbasicaly it shouldn't add any more layers or data aiui16:19
zbrwith docker one can so something like: RUN --mount=type=bind,target=/output,from=elastic-recheck-builder,source=/output,rw command16:19
*** zzzeek has joined #openstack-infra16:19
clarkbya that might be a cleaner way to express what is going on there16:20
zbrthis basically mounts the source, avoiding the copy operation. bad part, podman chokes on that, also old docker probably too.16:20
clarkbbut that still doesn't require a change to python-builder, just better use of the existing interfaces16:20
clarkbah not sure how important that is for projects like zuul but other places it is probably ok (like opendev images since we don't use podman)16:21
zbrif the installer would allow being run with arbitrary folders, we could avoid this, w/o having to rely on features that are not available everywhere.16:22
zbr"/output" was designed to be temporary, why it was not "/tmp/output"?16:22
*** derekh has quit IRC16:22
clarkbzbr: but why does it need to be arbitrary? I think that is my biggest concern here. Lots of code assumes /output and your change doesn't address any of that. If the goal is explicitly to use /tmp then maybe we should just use that instead (but I'm not sure how using /tmp works in this context if it is temporary since hte python-builder container is not running when python-base copies from it right?)16:23
*** derekh has joined #openstack-infra16:23
clarkbusing arbitrary dirs and ensuring minimal image sizes are different goals16:23
zbrthe issue is the installer, if you try to run it from onether folder it chokes as it has hardcoded path inside.16:24
clarkbthe issue isn't hardcoded paths though right? the actual issue is you want to minimize image sizes? I'm saying we can hardcode /tmp instead and achieve the same goal16:25
clarkbbasically I want to understand the actual issue and address it rather thn just adding features to the tool that then allow people to continue to get things wrong if indeed we are doing things wrong in the zuul case16:25
clarkbmaking thing sarbitrary makes that more difficult not easier16:25
zbrdepends how you look at it, IMHO the bug is the hardcoded /output in installer. If that would not be hardcoded, I could even use the install from multiple builders.16:26
clarkbthat is a different issue though. WHich is why I keep coming back to lets identify the actual issue and work from there16:26
clarkbI think adding more flexibility here will make it easier for people to get minimal image sizes wrong16:27
clarkbif the goal is to make minimal images then lets try to avoid adding too much flexibility16:27
*** __ministry1 has quit IRC16:27
*** ralonsoh has quit IRC16:27
zbrbut can we do this without breaking current consumers of the builder?16:28
clarkbI don't know, I think that would be the next thing to figure out if indeed we're failing at the minimalness goal16:28
zbrprobably it would be easier to make the lookups relative to the executable location, this should be backwards compatible and allow others to copy files to different locations than /output16:30
clarkbyes, I think that solves the backward compatibility issue. I do still think it may make it hard for people to do the right thing, but maybe we can address that with documentation in the short term16:30
clarkb(then flip a default later or something)16:31
*** jpena is now known as jpena|off16:31
zbri will update it. zuul is still building, that so slow to build.16:31
*** ysandeep is now known as ysandeep|away16:31
clarkbmostly my concern here is that we identify the problem to be solved and then articulate how the current thing helps. Rathe rthan just adding new features that may help a single user but leave everyone else building suboptimal images16:31
zbrsure, avoiding the cost of options.16:32
zbrit makes sense.16:32
zbrbtw, the newer buildkit option in docker does bring some huge improvements in build speed.16:33
clarkbon the multiarch builds I think we're using the latest buildkit but maybe not on the single arch builds16:39
clarkb(I think because buildx for multiarch implies buildkit16:39
*** hamalq has joined #openstack-infra16:53
*** ykarel|away has joined #openstack-infra16:56
*** lpetrut has quit IRC16:57
*** Gyuseok_Jung has quit IRC17:00
*** derekh has quit IRC17:00
*** vishalmanchanda has joined #openstack-infra17:04
*** ykarel|away has quit IRC17:05
*** dklyle has quit IRC17:21
*** david-lyle has joined #openstack-infra17:21
zbrii checked, not big diff. if i have time i will rework it with relative dir.17:22
*** andrewbonney has quit IRC17:31
*** gfidente is now known as gfidente|afk17:34
*** dtantsur is now known as dtantsur|afk17:35
*** Lucas_Gray has quit IRC17:41
*** __ministry has joined #openstack-infra17:43
*** __ministry has quit IRC17:43
*** hamalq has quit IRC17:53
*** hamalq has joined #openstack-infra17:53
*** Jeffrey4l has quit IRC18:08
AJaegerconfig-core, please review https://review.opendev.org/745877 https://review.opendev.org/749688 https://review.opendev.org/716298 https://review.opendev.org/#/c/742732/ https://review.opendev.org/#/c/751130/18:30
AJaegerfungi, could you review https://review.opendev.org/#/c/749777/ - revert setuptool pin, please?18:30
fungiapproved. fingers crossed 50.3.0 hasn't reintroduced the problem18:31
AJaeger;)18:31
*** eolivare has quit IRC18:32
clarkbI'm pretty sure I rechecekd the system-config change on 50.3.0 and it was fine (it has the same issues as our image venvs)18:34
*** mgoddard has quit IRC18:34
openstackgerritMerged openstack/project-config master: Revert "Pin setuptools<50 in our image venvs"  https://review.opendev.org/74977718:37
*** mgoddard has joined #openstack-infra18:41
*** Jeffrey4l has joined #openstack-infra18:48
*** zxiiro has quit IRC18:54
*** Jeffrey4l has quit IRC18:55
*** jtomasek has joined #openstack-infra18:55
*** Jeffrey4l has joined #openstack-infra18:57
*** jtomasek has quit IRC19:08
*** tbachman has joined #openstack-infra19:12
*** david-lyle is now known as dklyle19:14
*** zzzeek has quit IRC19:14
*** zzzeek has joined #openstack-infra19:16
*** zzzeek has quit IRC19:20
*** zzzeek has joined #openstack-infra19:23
*** zzzeek has quit IRC19:28
*** zzzeek has joined #openstack-infra19:29
*** hamalq has quit IRC19:32
*** hamalq has joined #openstack-infra19:33
*** zzzeek has quit IRC19:33
*** zzzeek has joined #openstack-infra19:35
*** Topner has quit IRC19:39
*** vishalmanchanda has quit IRC19:44
*** slaweq_ has joined #openstack-infra19:53
*** zzzeek has quit IRC19:54
*** zzzeek has joined #openstack-infra19:57
tbachmanwas py27 taken out of the instances used for gate jobs recently? Like in the last week?20:02
clarkbtbachman: I think that may have been part of gmann's focal work since focal doesn't have a python27 by default?20:02
clarkbtbachman: but ya the switch to ubuntu focal for jobs removes python2.7. I think it may have an optionally installable 2.7 or you can keep running jobs on bionic that need 2.720:02
tbachmank - I seem to recall being able to specify the instance OS type - will try that then20:03
tbachmanthanks clarkb!20:03
*** Topner has joined #openstack-infra20:29
*** Topner has quit IRC20:34
*** matt_kosut has quit IRC20:38
*** slaweq__ has joined #openstack-infra20:55
*** slaweq has quit IRC20:55
*** Topner has joined #openstack-infra21:06
*** slaweq__ has quit IRC21:08
*** Topner has quit IRC21:11
*** rfolco|ruck has quit IRC21:12
*** kaisers2 has joined #openstack-infra21:37
*** kaisers1 has quit IRC21:37
*** slaweq_ has quit IRC21:38
*** slaweq_ has joined #openstack-infra21:40
*** zxiiro has joined #openstack-infra21:51
*** slaweq_ has quit IRC21:53
*** slaweq has joined #openstack-infra21:54
*** slaweq has quit IRC22:05
*** slaweq has joined #openstack-infra22:09
*** slaweq has quit IRC22:21
*** smarcet has quit IRC22:30
*** tosky has quit IRC22:33
*** tkajinam has joined #openstack-infra22:52
*** Topner has joined #openstack-infra23:07
*** Topner has quit IRC23:12
*** portdirect has quit IRC23:13
*** portdirect has joined #openstack-infra23:13
*** artom has quit IRC23:46
*** Goneri has quit IRC23:53

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!