Monday, 2017-07-24

*** EricGonczer_ has quit IRC00:02
*** yamamoto has joined #openstack-infra00:08
*** jascott1 has joined #openstack-infra00:09
*** jascott1_ has joined #openstack-infra00:10
*** jascott1 has quit IRC00:10
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Move setfiles to outside chroot with runcon  https://review.openstack.org/48589900:14
*** gmann has quit IRC00:14
*** gmann has joined #openstack-infra00:14
*** jascott1_ has quit IRC00:19
*** jascott1 has joined #openstack-infra00:19
*** jascott1 has quit IRC00:20
*** jascott1_ has joined #openstack-infra00:20
*** dingyichen has joined #openstack-infra00:23
*** lbragsta_ has quit IRC00:28
*** thorst has joined #openstack-infra00:31
*** aeng has quit IRC00:35
*** hichihara has joined #openstack-infra00:35
*** gildub has joined #openstack-infra00:35
*** thorst has quit IRC00:36
*** aeng has joined #openstack-infra00:52
*** EricGonczer_ has joined #openstack-infra00:58
*** gouthamr has joined #openstack-infra01:01
*** LindaWang has joined #openstack-infra01:02
*** mriedem has joined #openstack-infra01:03
*** thorst has joined #openstack-infra01:03
*** thorst has quit IRC01:03
*** cuongnv has joined #openstack-infra01:08
*** r-daneel has joined #openstack-infra01:18
*** Daisy_ has joined #openstack-infra01:23
*** EricGonczer_ has quit IRC01:23
*** Daisy_ has quit IRC01:30
*** Daisy_ has joined #openstack-infra01:33
*** Daisy_ has quit IRC01:40
*** aeng has quit IRC01:43
*** aeng has joined #openstack-infra01:44
*** MatthewLi has joined #openstack-infra01:44
*** cody-somerville has joined #openstack-infra01:45
*** cody-somerville has joined #openstack-infra01:45
*** Daisy_ has joined #openstack-infra01:45
*** stakeda has joined #openstack-infra01:49
*** EricGonczer_ has joined #openstack-infra02:01
openstackgerritMerged openstack/diskimage-builder master: Move setfiles to outside chroot with runcon  https://review.openstack.org/48589902:04
*** r-daneel has quit IRC02:06
*** armax has joined #openstack-infra02:07
*** funzo has joined #openstack-infra02:10
*** funzo has quit IRC02:15
*** r-daneel has joined #openstack-infra02:23
*** EricGonczer_ has quit IRC02:39
*** mriedem has quit IRC02:43
*** sbezverk has quit IRC02:44
*** lbragsta_ has joined #openstack-infra02:48
*** gouthamr has quit IRC02:49
*** ykarel has joined #openstack-infra02:58
*** pahuang has quit IRC03:00
*** Daisy_ has quit IRC03:03
*** thorst has joined #openstack-infra03:04
*** cody-somerville has quit IRC03:07
*** thorst has quit IRC03:10
*** funzo has joined #openstack-infra03:11
*** pahuang has joined #openstack-infra03:13
*** mtreinish has quit IRC03:14
*** funzo has quit IRC03:15
*** mtreinish has joined #openstack-infra03:20
*** ramishra has joined #openstack-infra03:39
*** aeng has quit IRC03:40
*** jesusaurum has quit IRC03:41
*** aeng has joined #openstack-infra03:41
*** links has joined #openstack-infra03:46
*** rodrigopaiva has joined #openstack-infra03:50
*** lbragsta_ has quit IRC04:02
*** lbragsta_ has joined #openstack-infra04:03
*** funzo has joined #openstack-infra04:12
*** funzo has quit IRC04:16
*** adisky__ has joined #openstack-infra04:28
*** lbragsta_ has quit IRC04:35
*** bhavik1 has joined #openstack-infra04:37
*** stakeda has quit IRC04:39
*** shu-mutou-AWAY is now known as shu-mutou04:40
openstackgerritMerged openstack-infra/project-config master: networking-odl: make fullstack with carbon voting  https://review.openstack.org/48634704:48
*** prateek has joined #openstack-infra04:48
*** sbezverk has joined #openstack-infra04:51
openstackgerritMerged openstack-infra/project-config master: setup correct release jobs for networking-bigswitch  https://review.openstack.org/42729104:51
*** dassistant is now known as chandankumar04:52
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Bump fedora-minimal DIB_RELEASE 26  https://review.openstack.org/48257004:54
*** Dinesh_Bhor has joined #openstack-infra04:56
*** thorst has joined #openstack-infra05:05
*** rodrigopaiva has quit IRC05:06
*** rodrigopaiva has joined #openstack-infra05:06
*** thorst has quit IRC05:10
*** jascott1_ has quit IRC05:12
*** funzo has joined #openstack-infra05:13
*** jascott1 has joined #openstack-infra05:13
*** pgadiya has joined #openstack-infra05:14
*** rodrigopaiva has quit IRC05:14
*** rodrigopaiva has joined #openstack-infra05:15
*** Julien-zte has quit IRC05:16
*** rodrigopaiva has quit IRC05:16
*** funzo has quit IRC05:17
*** Julien-zte has joined #openstack-infra05:17
*** jascott1 has quit IRC05:20
*** stakeda has joined #openstack-infra05:23
*** arturb has joined #openstack-infra05:26
*** psachin has joined #openstack-infra05:30
*** boris-42____ has joined #openstack-infra05:30
*** aeng has quit IRC05:34
*** sree_ has joined #openstack-infra05:36
*** sree_ is now known as Guest5591505:36
*** dhajare has joined #openstack-infra05:45
*** wolverineav has joined #openstack-infra05:45
*** aeng has joined #openstack-infra05:47
*** wolverineav has quit IRC05:50
*** jascott1 has joined #openstack-infra05:57
*** bhavik1 has quit IRC05:57
*** aeng has quit IRC05:58
*** aeng has joined #openstack-infra05:59
*** cshastri has joined #openstack-infra06:00
*** jascott1 has quit IRC06:02
*** jbadiapa has joined #openstack-infra06:08
AJaegerfungi, interesting - the number of defined jobs as output by JJB has decreased from 17170 to 12503.06:09
*** funzo has joined #openstack-infra06:13
*** kong has quit IRC06:17
*** isaacb has joined #openstack-infra06:17
*** funzo has quit IRC06:18
*** coolsvap has joined #openstack-infra06:19
*** rcernin has joined #openstack-infra06:23
*** makowals has joined #openstack-infra06:23
*** thorst has joined #openstack-infra06:27
*** andreas_s has joined #openstack-infra06:27
*** hashar has joined #openstack-infra06:29
*** thorst has quit IRC06:31
*** tuanluong has joined #openstack-infra06:32
*** yamamoto has quit IRC06:34
*** jascott1 has joined #openstack-infra06:39
*** jascott1 has quit IRC06:43
*** hichihara has quit IRC06:59
openstackgerritSimon Kuhnle proposed openstack-infra/git-review master: Fix listing changes over SSH for 2.14  https://review.openstack.org/47919207:00
*** AJaeger has quit IRC07:00
*** AJaeger has joined #openstack-infra07:01
*** ccamacho has joined #openstack-infra07:02
*** nicolasbock has joined #openstack-infra07:03
*** yolanda_ has joined #openstack-infra07:05
*** yolanda_ has quit IRC07:06
*** 7ITABD5MB has joined #openstack-infra07:06
*** 07IAALFJ9 has joined #openstack-infra07:06
openstackgerritThomas Herve proposed openstack-infra/project-config master: Filter out services in designate jobs  https://review.openstack.org/48622107:07
*** 07IAALFJ9 has quit IRC07:07
*** 7ITABD5MB has quit IRC07:08
*** yolanda_ has joined #openstack-infra07:08
*** yolanda_ is now known as yolanda07:11
*** dizquierdo has joined #openstack-infra07:11
*** migi_ is now known as migi07:12
*** ralonsoh has joined #openstack-infra07:13
*** vtapia has quit IRC07:13
*** xarses has quit IRC07:14
*** vtapia has joined #openstack-infra07:14
*** xarses has joined #openstack-infra07:15
*** xarses has quit IRC07:15
*** xarses has joined #openstack-infra07:15
*** isaacb has quit IRC07:15
*** lennyb has quit IRC07:19
*** jascott1 has joined #openstack-infra07:20
*** ralonsoh_ has joined #openstack-infra07:22
*** ralonsoh has quit IRC07:22
*** isaacb has joined #openstack-infra07:23
*** Hal has joined #openstack-infra07:25
*** Hal is now known as Guest3671607:25
*** jascott1 has quit IRC07:27
*** flepied has quit IRC07:28
*** lennyb has joined #openstack-infra07:32
*** iyamahat has joined #openstack-infra07:33
*** gildub has quit IRC07:35
*** iyamahat has quit IRC07:38
openstackgerritMerged openstack-infra/project-config master: Filter out services in designate jobs  https://review.openstack.org/48622107:41
*** yamamoto has joined #openstack-infra07:45
*** egonzalez has joined #openstack-infra07:47
hwoarangdirk: good morning do you have a second?07:49
dirkhwoarang: sure, good morning07:50
hwoarangdirk: hi in regards to your keystone comment on this https://review.openstack.org/#/c/485985/07:50
hwoarangis it this https://bugs.launchpad.net/openstack-ansible/+bug/1705521 btw ?07:50
openstackLaunchpad bug 1705521 in openstack-ansible "[os_keystone] uwsgi process segfaults on openSUSE" [Undecided,New] - Assigned to Markos Chandras (hwoarang)07:50
*** david-lyle has quit IRC07:50
*** dklyle has joined #openstack-infra07:50
hwoarangbecause i am trying to debug this failure which happens only on opensuse with cryptography-2.007:51
*** alexchadin has joined #openstack-infra07:54
dirkhwoarang: yes07:54
dirkhwoarang: yes, because cryptography 2.0 added a universal wheel with an embedded openssl 1.107:55
dirkwhich has symbol clashes with the opensuse provided openssl 1.007:55
dirkit doesn't happen on any other distro in infra because infra is rebuilding all wheels and provides a wheel mirror07:55
dirkhttps://review.openstack.org/#/c/486305/ is going to fix that07:56
dirkbut its actually a terrible workaround07:56
dirkhwoarang: the --no-binary is imho more correct07:56
hwoarangaha07:57
hwoarangi also have an issue opened with upstream07:57
hwoarangmaybe worth telling them all that and fix their wheel...07:58
hwoarangthank you dirk07:59
*** rossella__ has joined #openstack-infra08:01
dirkhwoarang: its like the 3rd or 4th time the cryptography upstream adds a manylinux1 wheel08:01
dirkeven though it is clearly very much outside the defined abi of manylinux1. so I'm afraid they just don't care08:01
*** flepied has joined #openstack-infra08:02
*** jascott1 has joined #openstack-infra08:04
hwoaranghmm i see. i will them know and then see how to fix it for openstack-ansible in the meantime :/08:06
hwoarangthank you for all the infor!08:06
*** markus_z has joined #openstack-infra08:06
dirkhwoarang: does openstack-ansible use devstack scripts?08:08
hwoarangno08:08
AJaegerdirk: for 485305, best work with pabelanger later today...08:08
dirkhwoarang: is there actually any openstack-ansible script that passes on opensuse? I was doing some experimental checks a few days ago and couldn't ifnd one that passed (before the cryptography 2.0 messup)08:08
*** jascott1 has quit IRC08:09
hwoarangthe openstack-ansible-repo_server should pass08:09
dirkhwoarang: once the wheel mirror is there it should work again then08:09
hwoarangdirk: do all projects consume the stuff from the wheel mirrors?08:10
hwoarangor do they need to do something special to get them instead of the pypi ones08:10
dirkhwoarang: iirc all nodepool images have a WHEEL_MIRROR set08:10
dirkbut I'm not exactly sure how that all works08:11
*** dizquierdo has quit IRC08:11
hwoaranghmm08:11
dirksee nodepool/scripts/configure_mirror.sh in project-config which is setting the wheel_mirror based on the AFS_SLUG08:11
dirkI believe that script is run on image instantiation inside a particular cloud08:12
hwoaranghmm i see08:12
*** funzo has joined #openstack-infra08:15
*** yamahata has quit IRC08:15
*** markus_z has quit IRC08:16
*** danpawlik has quit IRC08:17
*** danpawlik has joined #openstack-infra08:17
*** funzo has quit IRC08:19
*** derekh has joined #openstack-infra08:23
*** lucas-afk is now known as lucasagomes08:26
*** e0ne has joined #openstack-infra08:27
*** thorst has joined #openstack-infra08:27
mordreddirk, hwoarang it gets run once a node is booted but before jobs are run on the node08:31
*** s-shiono_ has quit IRC08:32
mordredand it makes sure pip and friends are configured to use the right mirror based on the region08:32
*** thorst has quit IRC08:32
openstackgerritDaniel Mellado proposed openstack-infra/project-config master: Add nova service to kuryr gate  https://review.openstack.org/48652508:32
*** dklyle has quit IRC08:32
mordredthere's also some docs about the metadata that is put on images: https://docs.openstack.org/infra/nodepool/scripts.html08:33
*** dklyle has joined #openstack-infra08:33
mordredwhich you can use to configure mirror if for some reason you can't directly rely on the pre-configured mirror information08:33
*** dtantsur|afk is now known as dtantsur08:34
mordredfor instance, loci does this: http://git.openstack.org/cgit/openstack/loci/tree/scripts/run-tests.sh#n11508:34
mordredsince they need mirrors set inside of the containers they're building08:35
mordred(which obviously the base images don't know anything about08:35
*** dklyle has quit IRC08:36
*** vponomaryov has joined #openstack-infra08:37
openstackgerritDaniel Mellado proposed openstack-infra/project-config master: Add nova service to kuryr gate  https://review.openstack.org/48652508:38
*** jangutter has joined #openstack-infra08:42
*** jascott1 has joined #openstack-infra08:42
*** markus_z has joined #openstack-infra08:42
*** jascott1 has quit IRC08:47
*** david-lyle has joined #openstack-infra08:47
*** david-lyle has quit IRC08:49
*** david-lyle has joined #openstack-infra08:50
*** dingyichen has quit IRC08:52
*** dklyle has joined #openstack-infra08:53
*** david-lyle has quit IRC08:53
openstackgerritGael Chamoulaud proposed openstack-infra/tripleo-ci master: WIP: Enable tripleo-validations tests  https://review.openstack.org/48108008:53
openstackgerritDeepak Mourya proposed openstack-infra/os-loganalyze master: Stop ignoring H904 hacking rule in tox  https://review.openstack.org/46876108:57
*** yamamoto has quit IRC09:02
*** jaosorior has joined #openstack-infra09:02
*** rlandy has joined #openstack-infra09:04
*** florianf has joined #openstack-infra09:05
hwoarangmordred: thank you for the information09:14
*** isaacb has quit IRC09:15
*** yamamoto has joined #openstack-infra09:15
*** funzo has joined #openstack-infra09:16
*** sambetts|afk is now known as sambetts09:16
*** isaacb has joined #openstack-infra09:16
*** samP has joined #openstack-infra09:16
samPHi o/.. Need help to create 3rd party CI group for project Masakari09:19
*** boris-42____ has quit IRC09:19
*** funzo has quit IRC09:20
*** pblaho has joined #openstack-infra09:21
*** dizquierdo has joined #openstack-infra09:23
openstackgerritKien Nguyen proposed openstack-infra/project-config master: Add topology flag to Heat job template  https://review.openstack.org/48204809:23
*** electrofelix has joined #openstack-infra09:24
*** shu-mutou is now known as shu-mutou-AWAY09:35
*** claudiub has joined #openstack-infra09:38
*** jascott1 has joined #openstack-infra09:39
*** katkapilatova has joined #openstack-infra09:39
*** katkapilatova has left #openstack-infra09:40
*** jascott1 has quit IRC09:44
*** yamamoto has quit IRC09:47
*** wolverineav has joined #openstack-infra09:51
*** cuongnv has quit IRC09:58
*** tuanluong has quit IRC10:02
*** sdague has joined #openstack-infra10:02
*** yamamoto has joined #openstack-infra10:02
*** yamamoto has quit IRC10:02
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add settings for OVB in RDO Cloud  https://review.openstack.org/48095010:06
*** stakeda has quit IRC10:08
*** LindaWang has quit IRC10:13
*** kong has joined #openstack-infra10:16
*** funzo has joined #openstack-infra10:16
*** jascott1 has joined #openstack-infra10:21
openstackgerritLucas Alvares Gomes proposed openstack-infra/project-config master: Make tripleo-ci-centos-7-scenario007 non-voting for networking-ovn  https://review.openstack.org/48655710:21
*** Guest55915 has quit IRC10:21
*** funzo has quit IRC10:21
*** sree_ has joined #openstack-infra10:22
*** sree_ is now known as Guest423410:22
*** jascott1 has quit IRC10:25
*** Guest4234 has quit IRC10:26
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: [WIP] LVM support for dib-block-device  https://review.openstack.org/47206510:29
*** thorst has joined #openstack-infra10:29
*** rlandy has quit IRC10:31
*** LindaWang has joined #openstack-infra10:32
*** rlandy has joined #openstack-infra10:32
*** yamamoto has joined #openstack-infra10:33
*** thorst has quit IRC10:33
*** electrofelix has quit IRC10:38
*** electrofelix has joined #openstack-infra10:43
*** jkilpatr has quit IRC10:45
*** lucasagomes is now known as lucas-hungry10:52
*** r-daneel has quit IRC10:55
*** jkilpatr has joined #openstack-infra11:02
*** Dinesh_Bhor has quit IRC11:07
*** hashar is now known as hasharLunch11:13
*** jascott1 has joined #openstack-infra11:23
*** jascott1 has quit IRC11:27
*** flepied has quit IRC11:28
*** Guest36716 has quit IRC11:29
*** Hal has joined #openstack-infra11:36
*** Hal is now known as Guest2341511:36
*** ldnunes has joined #openstack-infra11:41
*** HeOS has quit IRC11:48
*** sree has joined #openstack-infra11:52
*** thorst has joined #openstack-infra11:54
*** thorst_ has joined #openstack-infra11:56
*** thorst_ has quit IRC11:56
*** thorst_ has joined #openstack-infra11:57
*** thorst has quit IRC11:58
*** EricGonczer_ has joined #openstack-infra12:03
*** lucas-hungry is now known as lucasagomes12:03
*** flepied has joined #openstack-infra12:03
*** jascott1 has joined #openstack-infra12:05
*** Julien-zte has quit IRC12:09
*** jascott1 has quit IRC12:09
*** Julien-zte has joined #openstack-infra12:10
openstackgerritIfat Afek proposed openstack-infra/project-config master: Run tempest tests on python35 devstack (non-voting for now)  https://review.openstack.org/48659112:15
*** stevebaker has quit IRC12:16
*** stevebaker has joined #openstack-infra12:16
*** pgadiya has quit IRC12:17
*** funzo has joined #openstack-infra12:18
*** khappone has quit IRC12:19
*** yamamoto has quit IRC12:19
*** khappone has joined #openstack-infra12:20
*** funzo has quit IRC12:22
*** baoli has joined #openstack-infra12:24
*** askb has quit IRC12:24
*** jascott1 has joined #openstack-infra12:27
openstackgerritIfat Afek proposed openstack-infra/project-config master: Run tempest tests on python35 devstack (non-voting for now)  https://review.openstack.org/48659112:29
*** rhallisey has joined #openstack-infra12:30
*** jcoufal has joined #openstack-infra12:31
*** jascott1 has quit IRC12:32
*** baoli has quit IRC12:34
*** baoli has joined #openstack-infra12:34
*** yamamoto has joined #openstack-infra12:36
*** yamamoto has quit IRC12:36
*** trown|outtypewww is now known as trown12:38
*** gordc has joined #openstack-infra12:39
electrofelixjamielennox: have you had a chance to look over https://review.openstack.org/475474 since I updated it?12:40
*** ldnunes has quit IRC12:44
electrofelixanyone around to review an elastic-recheck change: https://review.openstack.org/32098812:49
*** chlong has joined #openstack-infra12:53
*** kgiusti has joined #openstack-infra12:53
*** ralonsoh_ is now known as ralonsoh12:54
*** bobh has joined #openstack-infra12:54
*** ldnunes has joined #openstack-infra12:56
openstackgerritMarkos Chandras (hwoarang) proposed openstack-infra/project-config master: jenkins: openstack-ansible: Add openSUSE 42.3 jobs  https://review.openstack.org/48660212:56
*** mriedem has joined #openstack-infra12:59
*** hasharLunch is now known as hashar13:01
*** jaypipes has joined #openstack-infra13:04
*** xarses has quit IRC13:05
*** esberglu has joined #openstack-infra13:07
*** r-daneel has joined #openstack-infra13:09
*** jascott1 has joined #openstack-infra13:09
*** ihrachys has joined #openstack-infra13:09
*** bobh has quit IRC13:10
*** EricGonczer_ has quit IRC13:10
*** eharney has joined #openstack-infra13:11
*** r-daneel has quit IRC13:13
*** jascott1 has quit IRC13:13
*** gagehugo has quit IRC13:15
*** ykarel has quit IRC13:17
*** jamesdenton has joined #openstack-infra13:18
*** links has quit IRC13:20
*** gagehugo has joined #openstack-infra13:21
*** prateek has quit IRC13:24
*** jamesdenton has quit IRC13:25
*** jamesdenton has joined #openstack-infra13:25
*** xarses has joined #openstack-infra13:31
*** yamamoto has joined #openstack-infra13:36
*** bobh has joined #openstack-infra13:39
*** Dinesh_Bhor has joined #openstack-infra13:40
*** gouthamr has joined #openstack-infra13:41
*** gordc has quit IRC13:42
*** yamamoto has quit IRC13:44
*** xyang1 has joined #openstack-infra13:44
*** ykarel has joined #openstack-infra13:50
*** Goneri has joined #openstack-infra13:50
*** eharney has quit IRC13:50
*** camunoz has joined #openstack-infra13:52
*** trown is now known as trown|brb13:52
*** eharney has joined #openstack-infra13:55
*** trown|brb is now known as trown13:57
*** isaacb has quit IRC13:58
*** efoley has joined #openstack-infra14:04
*** alexchadin has quit IRC14:06
*** iyamahat has joined #openstack-infra14:10
*** isaacb has joined #openstack-infra14:12
*** rbrndt has joined #openstack-infra14:16
*** HeOS has joined #openstack-infra14:18
pabelangerAJaeger: dirk: for 485305 we should consider waiting until after PTG. It might be a good thing to do in zuulv3 too so we don't need to stand up a opensuse static node14:19
*** funzo has joined #openstack-infra14:19
*** dizquierdo has quit IRC14:20
*** annegentle has joined #openstack-infra14:20
*** gordc has joined #openstack-infra14:23
*** funzo has quit IRC14:24
*** kgiusti has quit IRC14:25
*** sree has quit IRC14:26
*** jcoufal has quit IRC14:27
*** jcoufal has joined #openstack-infra14:27
*** yamahata has joined #openstack-infra14:29
*** felipemonteiro has joined #openstack-infra14:30
*** baoli has quit IRC14:31
*** felipemonteiro_ has joined #openstack-infra14:31
*** felipemonteiro has quit IRC14:34
openstackgerritSagi Shnaidman proposed openstack-infra/project-config master: Transit Heat convergence job to use quickstart  https://review.openstack.org/48329214:35
*** cshastri has quit IRC14:39
*** marst has quit IRC14:39
openstackgerritGael Chamoulaud proposed openstack-infra/tripleo-ci master: Enable tripleo-validations tests  https://review.openstack.org/48108014:40
AJaegerpabelanger: there's a problem with one of the wheels that hwoarang and dirk discussed this morning (cryptography-2.0), see backscroll.14:41
AJaegerpabelanger: is there a short time solution for these?14:42
hwoarangAJaeger: for openstack-ansible i am proposing --no-binary in https://review.openstack.org/#/c/486580/14:42
hwoarangthe upstream bug report is rather informative btw https://github.com/pyca/cryptography/issues/380414:42
pabelangerAJaeger: hwoarang: Ya, 486580 is best approach right now, adding opensuse wheel-mirror will require some work / planning14:43
hwoarangyeah14:43
dirkAJaeger: pabelanger : hwoarang : any review on https://review.openstack.org/#/c/485985/ appreciated14:45
hwoarangdirk: i didn't vote because of Ian's comment so I was expecting a new patchset to be honest14:45
dirkI just don't want to have things bitrot for months, as it takes time to recover the job into a green state when it keeps failing for extended period of time (and this sneaked in while the previous patch to unbreak it was under review)14:45
*** EricGonczer_ has joined #openstack-infra14:45
dirkhwoarang: right, but the code doesn't hurt once it is in dib14:46
dirkhwoarang: it will just become a noop then14:46
dirkpabelanger: how does zuulv3 help with that? I'm confused.14:47
dirkpabelanger: would there not be a need for wheels then anymore? or does zuulv3 spin up dynamic nodes?14:47
*** marst has joined #openstack-infra14:47
pabelangerdirk: right, we'd move wheel builds into nodepool and not on static nodes14:47
pabelangerI mean, we should be able to do that today too. Maybe we should talk about it at our weekly meeting14:48
pabelangerjust need to deal with vos release (AFS)14:48
pabelangerya, wheel-release can still be trusted14:49
pabelangerbut wheel-build job, could use an untrusted node14:50
*** jamesdenton has quit IRC14:51
*** makowals has quit IRC14:51
*** jamesdenton has joined #openstack-infra14:52
*** baoli has joined #openstack-infra14:52
*** sree has joined #openstack-infra14:53
clarkbregardless of zuulv3 or zuul v2.5 you need working afs on suse right? which last I checked wasn't a thing?14:53
*** makowals has joined #openstack-infra14:53
pabelangerYa, I haven't looked into that either14:54
clarkbpabelanger: fungi: will you be able to respond to citycloud? I am less familiar with the current situation (though they are looking into background info which I guess any infraer should be able to provide)14:54
pabelangerpreformance aside, opensuse could upload to xenial and have xenial publish to afs14:55
fungiAJaeger: yeah, i believe it's because zuul-launcher registers both build:<jobname> and build:<jobname>:<nodetype> into gearman, so the number of build registrations will always be >2x the number of job definitions as long as all jobs get registered14:55
pabelangerclarkb: Ya, was holding off for fungi this morning14:55
pabelangerbut I can reply now14:55
clarkbfungi: AJaeger yup that sounds correct14:57
*** jamesdenton has quit IRC14:59
*** baoli has quit IRC14:59
fungidirk: hwoarang: AJaeger: if it matters, the wheel mirror we build isn't so much to work around problems with individual wheels as it is to prevent jobs from wasting a lot of time building platform-specific wheels. it provides a pretty massive speedup for devstack, for example14:59
dirkclarkb: there should be working afs I think (see the link I pasted)15:00
*** dizquierdo has joined #openstack-infra15:00
*** annegentle has quit IRC15:00
fungione thing we're still missing is a step to exclude wheels which are already present on pypi. the goal with the wheel mirrors is to fill in the gaps for wheels pypi doesn't already carry, not rebuild and replace them with our own, which is unfortunately the end result15:00
dirkfungi: sure, but its one thing that is different for the opensuse job than for othrs, and others don't notice the problems with upstream wheels because they get rebuilt anyway15:01
fungiyep, i completely understand the issue15:01
*** sree has quit IRC15:01
clarkbdirk: are you sure that pip will work that way? we add our wheel mirror as an extra index whcih I think means it is not preferred15:01
pabelangerfungi: clarkb: replied to citycloud15:02
fungithanks pabelanger!15:02
*** EricGonczer_ has quit IRC15:02
*** sree has joined #openstack-infra15:02
*** jamesdenton has joined #openstack-infra15:03
clarkbdirk: but that would certainly explain why the other distros don't have a problem in testing with cryptography15:03
fungithe issue is that these are non-pure-python wheels15:03
clarkbfungi: right but the pypi hosted wheel built against the simple linux from land before time should be preferred by pip because pypi is a regular index url, but maybe I misunderstand the behavipr there15:04
fungiso i believe what happens is that while pip is willing to use the manylinux1 wheels it prefers, the platform-specific ones even if they're found in the extra-index15:04
fungier, what happens is that while pip is willing to use the manylinux1 wheels, it prefers the platform-specific ones even if they're found in the extra-index15:05
fungibad comma placement ;)15:05
clarkbdirk: https://software.opensuse.org/package/openafs looks like all available packages are "unstable"?15:05
*** sree has quit IRC15:06
clarkbbut I guess they are there in filesystems so maybe we don't care15:06
*** markus_z has quit IRC15:07
*** isaacb has quit IRC15:08
*** baoli has joined #openstack-infra15:10
*** dhajare has quit IRC15:13
*** dklyle is now known as david-lyle15:14
fungiinterestingly, the ones built by our wheel builders are still named like cryptography-2.0-cp27-cp27mu-manylinux1_x86_64.whl so pip isn't finding a more specific wheel architecture after all?15:15
*** jascott1 has joined #openstack-infra15:16
inc0good morning, qq, RAX jobs didnt seem to get volume on /opt15:18
inc0and we ran out of space for ceph15:18
inc0(with 30Gig disk)15:18
inc0is this planned (and should figure out how to use less disk) or something went wrong?15:19
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove ZUUL_PROJECT  https://review.openstack.org/48625115:19
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove ZUUL_UUID  https://review.openstack.org/48625215:19
*** annegentle has joined #openstack-infra15:20
jeblairinc0: yes, rax doesn't have ephemeral disks, it just has large root disks15:20
*** jascott1 has quit IRC15:20
jeblairinc0: i thought there was more free space than that though15:20
*** Apoorva has joined #openstack-infra15:21
pabelanger40GB is what I am seeing in rax-ord ATM15:21
*** chlong_ has joined #openstack-infra15:21
inc0right, just partitioned15:22
inc0http://logs.openstack.org/40/456140/15/check/gate-kolla-ansible-dsvm-deploy-ceph-ubuntu-source-ubuntu-xenial-2-node-nv/9516ad4/console.html#_2017-07-24_14_13_10_16930115:23
pabelangerinc0: have you mounted /dev/xvde2?15:23
inc0no15:23
jeblairon the node i looked at, we apparently have 38g total, 28g free15:23
inc0it's mointed on /15:23
*** chlong has quit IRC15:24
inc0anyway, I'll remove one OSD since it's not unplanned15:24
pabelangerIIRC, devstack-gate handles this right? And sets up /mnt on rackspace?15:25
dtantsuryolanda: hi! if you're still here, mind checking https://review.openstack.org/#/c/480332/ please? should be a quick one15:25
fungipabelanger: /opt15:25
jeblairfungi, pabelanger: does rax have an ephemeral volume now?15:26
fungithat's what i'm checking, though i thought it always did unless you told it to boot without15:26
dmsimardMaybe I'm just impatient but I subscribed to a ml on lists.o.o and haven't gotten anything back to confirm the subscription in like 15 minutes ? :/ is legal-discuss moderated ?15:26
pabelangerfungi: jeblair: preformance1-8 is reporting 80GB ephemeral15:27
pabelanger| performance1-8          | 8 GB Performance                  |   8192 |   40 |        80 |     8 | N/A       |15:27
fungihttp://paste.openstack.org/show/616317/ is in the middle of running a devstack-gate based job in rax-ord15:28
fungi72gb /opt mounted15:28
fungi38gb rootfs15:28
pabelangerright, I think kolla just needs to do that15:29
jeblairoh, yeah, that's handled by devstack-gate15:29
pabelangerIsn't fix_disk_layout in d-g where we do that now?15:29
fungid-g partitions /dev/xvde into two partitions there, one for swap and one for /opt15:29
fungiwhich is why it's not 80gb15:29
fungidmsimard: i can check the mailman logs, just a sec15:29
jeblairinc0: ^ devstack-gate will set it up for you15:30
*** Julien-zte has quit IRC15:30
inc0that's my point, df didn't seem to show that it did. But on second glance that also doesn't seem to be issue for me15:31
*** Julien-zte has joined #openstack-infra15:31
dmsimardfungi: got it just now, not sure if you did anything15:32
fungidmsimard: nope, maybe your mta was greylisting us15:32
inc0how about osic?15:33
fungidmsimard: bingo... 2017-07-24 15:15:44 1dZf5D-0002m5-1L SMTP error from remote mail server after RCPT TO:<REDACTED>: host mx1.redhat.com [209.132.183.28]: 451 4.7.1 Greylisting in action, please come back in 00:05:0015:34
jeblairinc0: in http://logs.openstack.org/40/456140/15/check/gate-kolla-ansible-dsvm-deploy-ceph-ubuntu-source-ubuntu-xenial-2-node-nv/9516ad4/logs/  where are the devstack-gate logs?15:34
*** rcernin has quit IRC15:34
dmsimardfungi: ah, there you go. At least I wasn't crazy :)15:34
jeblairinc0: should have names like workspace-setup or something15:35
inc0hold on, scrach that, sorry guys I haven't get my caffeine flowing. Looked at wrong job again15:35
clarkbfungi: re unbound and opendns and google, I don't think order matters in unbound config, eg ther is no way to say use foo then fall back on bar. It will use foo and bar evenly15:35
inc0http://logs.openstack.org/40/456140/15/check/gate-kolla-ansible-dsvm-deploy-ceph-ubuntu-binary-ubuntu-xenial-2-node-nv/89dd4e6/console.html <- that node had disk issues15:36
*** LindaWang has quit IRC15:36
inc0http://logs.openstack.org/40/456140/15/check/gate-kolla-ansible-dsvm-deploy-ceph-centos-source-centos-7-2-node-nv/1930a51/console.html and that15:37
inc0so rax-ord15:37
openstackgerritDima Kuznetsov proposed openstack-infra/project-config master: Dragonflow: generate HTML results in tempest job  https://review.openstack.org/48666415:38
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: testing  https://review.openstack.org/48666515:38
jeblairinc0: i still can't find devstack-gate logs.  do they get copied in those jobs?15:38
fungiinc0: jeblair: i think the "dsvm" names on those jobs are misleading. they don't seem to use devstack-gate at all15:38
pabelangerright15:39
*** coolsvap has quit IRC15:39
inc0hmm...possible15:39
jeblaircan someone explain like i'm 5 what we're debugging?15:39
pabelangerkolla needs to add d-g to there jobs, or setup ephemeral mounts themself15:40
openstackgerritMerged openstack-infra/zuul-jobs master: Remove nodepool DIB specific logic  https://review.openstack.org/48582415:40
fungiwell, before jumping to solutions, a clear outline of what the expectations for the jobs are would help me as well15:40
inc0so if we're not devstack gates, which would make sense I guess15:40
inc0we need to change name of jobs;)15:41
jeblairinc0: no, please use devstack-gate15:41
*** ccamacho has left #openstack-infra15:41
inc0ok...15:41
jeblairit does *lots* of stuff to set up a node for testing15:41
jeblairlike it took us 6 years to make that script15:41
jeblairsave yourself 6 years and use it :)15:41
jeblairit doesn't have to run devstack15:41
jeblairi'm really sorry it has devstack in the name15:41
inc0ok15:41
inc0where do I define that?15:42
jeblairand we'll fix that in about a month and a half, but we can't change it right now :)15:42
inc0since it's not just job name:)15:42
jeblairinc0: basically, we should set up a job like most of the other devstack-based jobs, and then override the 'gate hook' to do kolla things instead of devstack things15:42
jeblairlemme dig around for an example15:42
inc0thank you jeblair15:43
*** zara_the_lemur__ is now known as Zara15:43
*** kong has quit IRC15:45
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Remove .txt suffix from tox logs  https://review.openstack.org/48666515:46
*** Apoorva has quit IRC15:46
*** armax has quit IRC15:47
jeblairinc0, clarkb, pabelanger: do we need the devstack-gate multinode networking setup for these jobs?15:48
openstackgerritJeremy Stanley proposed openstack-infra/infra-manual master: Extra disk space is not necessarily mounted  https://review.openstack.org/48667215:49
fungijeblair: inc0: ^ clarification. our document was unfortunately misleading and made unmentioned assumptions15:49
*** prateek has joined #openstack-infra15:50
openstackgerritClark Boylan proposed openstack-infra/elastic-recheck master: Add query for bug 1706103  https://review.openstack.org/48667315:51
openstackbug 1706103 in OpenStack-Gate "mkswap fails on cannot open /dev/xvde1" [Undecided,New] https://launchpad.net/bugs/170610315:51
*** armax has joined #openstack-infra15:52
*** hashar is now known as hasharMeeting15:52
clarkbjeblair: it depends on what the job is doing. Its possible they don't need it because their container stuff ships its own overlays (k8s will do this for example)15:52
jeblairinc0: this job looks like a good skeleton to work from: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n115:53
jeblairinc0: gate-dg-hooks-dsvm  -- it's a job that just exercises the hooks without doing anything, so it's easy to see the structure15:53
jeblairinc0: i'd start with that, delete the pre_ and post_ test hooks, and then replace gate_hook with some shell script to execute your actual job15:54
inc0jeblair: so in our gate I should run everything as hooks instead of big shell like we do today?15:54
jeblairinc0: that will do all of the host setup (filesystems, mirrors, git repos, etc)  and none of the devstack stuff.  (the devstack/tempest stuff is what's normally in the gate_hook, so you'll be overriding that stuff)15:54
jeblairinc0: i would write just a tiny bit of shell to put in the gate hook which then invokes what you're doing now15:55
jeblairinc0: so like:  function gate_hook {{ cd /somewhere/kolla; ./run-kolla-test.sh }}15:55
inc0I'll look at it thanks15:55
jeblairkeep that part simple, just get from devstack-gate to kolla asap :)15:56
*** jamesden_ has joined #openstack-infra15:56
openstackgerritClark Boylan proposed openstack-infra/devstack-gate master: Sync and partprobe after mkpart on ephemeral disk  https://review.openstack.org/48667615:56
*** vhosakot has joined #openstack-infra15:56
inc0probably gonna do mounts before that to fix immediate issue, but I'll look at devstack gate as well15:56
clarkbthis d-g change is semi related (noticed due to failres in e-r uncategorized queue)15:56
*** nicolasbock has quit IRC15:57
openstackgerritJeremy Stanley proposed openstack-infra/infra-manual master: Extra disk space is not necessarily mounted  https://review.openstack.org/48667215:57
*** funzo has joined #openstack-infra15:57
*** funzo has quit IRC15:57
*** funzo has joined #openstack-infra15:57
*** jamesdenton has quit IRC15:57
jeblairinc0: the only other sort of general-purpose thing that devstack-gate does but this *won't* do is setup a multinode network overlay.  if you find you need that, we can look deeper into that.  but that's tied up with the more devstacky parts of this, so it's not as straightforward.15:57
fungianother thing d-g does is try and set up a reasonable amount of swap memory (not sure if that got mentioned already)15:58
inc0multinode network overlay as in tunnels15:58
inc0?15:58
*** rcernin has joined #openstack-infra15:58
clarkbinc0: yes so that multinode test nodes can have a shared layer 2 regardless of how they were actually booted in $cloud15:58
fungiinc0: yes, a layer 2 "tunnel" between nodes in the multi-node group so that they can be guaranteed layer 2 connectivity and the ability to use whatever ip addresses they want there15:58
clarkbinc0: this makes it easy to be able to ssh from a VM on one instance to a VM on another instance for example15:59
inc0alternatively we can wait for zuulv3 and rework our gates to ansible bottoms-up right?15:59
clarkbsince we can give openstack control of this virtual networking15:59
inc0yeah we could use that to for vm network, but that's lesser priority16:00
inc0question, will zuulv3 jobs look differently than jobs today?16:00
*** Douhet has quit IRC16:01
inc0I mean, I know zuulv3 will be backwards compatible, but will zuulv3 native jobs be different that what's there today?16:01
*** Douhet has joined #openstack-infra16:01
clarkbinc0: there will be differences, distribution of encrypted secrets, nodepool node reservation api, ability to use ansible directly in job, etc16:02
inc0what I'm asking is, should we work on d-g today or wait for zuulv3 and do d-g+zuulv3 changes16:02
*** baoli has quit IRC16:02
fungiinc0: https://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/16:02
pabelangerwe still need to re-write devstack-gate into ansible however. Still working on refactoring tox jobs16:02
*** baoli has joined #openstack-infra16:02
*** lucasagomes is now known as lucas-afk16:04
clarkbinc0: I think you'll want to use d-g either way. Only the manner of consumption will change. So probably makes sense to just go ahead and use d-g now if it will make your jobs more reliabel16:04
*** armax has quit IRC16:04
fungiinc0: our goal with the migration and maintaining some backward-compatibility is that nobody should wait to write jobs just so they can make then zuulv3-native (though they may be waiting to write jobs which depend on v3-specific features)16:05
*** ijw has joined #openstack-infra16:05
fungianything that's done by executing literal shell instructions, which is nearly everything in our jjb config, can still be done by putting the literal shell into playbooks/roles16:06
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: do not merge  https://review.openstack.org/48667916:06
openstackgerritMerged openstack-infra/elastic-recheck master: Add query for bug 1706103  https://review.openstack.org/48667316:06
openstackbug 1706103 in OpenStack-Gate "mkswap fails on cannot open /dev/xvde1" [Undecided,New] https://launchpad.net/bugs/170610316:06
fungibut for a lot of the common job patterns and macros we'll likely have them rewritten in equivalent ansible instead of literal shell script16:07
*** baoli has quit IRC16:07
*** andreas_s has quit IRC16:08
*** egonzalez has quit IRC16:08
inc0clarkb fungi thing is, I don't want to make small revolution to our gates this close to release16:09
*** ijw has quit IRC16:10
*** bhavik1 has joined #openstack-infra16:10
*** kgiusti has joined #openstack-infra16:10
inc0so I'll wait with d-g to after release anyway, and that, assuming you'll manage to land zuulv3 before PTG, will be pretty close16:10
*** jascott1 has joined #openstack-infra16:10
inc0so might as well do it all since I'll need zuulv3 features anyway16:10
fungiinc0: makes sense. i'm saying don't wait to do it because of zuul v3. if you have other reasons to wait, then those are other reasons16:11
inc0yeah, I agree16:11
fungiand at least now you know where the issue lies, and what the "right" solution is, and can plan accordingly16:11
inc0I just wanted to make sure if there is anything in zuulv3 that would justify changes to gate, and I understand that is the case16:12
inc0yeah16:12
*** chlong__ has joined #openstack-infra16:13
*** chlong_ has quit IRC16:13
*** EricGonczer_ has joined #openstack-infra16:13
fungibut i share jeblair's opinion as far as devstack-gate goes there. if you try to make an openstack integration testing framework from scratch, you get to make many of the same mistakes and reexperience most of the problems we solved over the years with devstack-gate16:14
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: do not merge  https://review.openstack.org/48667916:14
fungiand you're unlikely to get a lot of sympathy or help in re-solving problems we already solved once16:15
inc0oh I don't want to make this from scrach16:15
inc0I'll just fix immediate issue16:15
inc0and work on d-g as long term16:15
pabelangerfungi: ++16:15
inc0or rather d-g + zuulv316:15
inc0probably worth a goal for Q16:15
*** claudiub has quit IRC16:15
inc0I was sure we're using dvsm today;)16:16
fungithe mini-project pabelanger mentioned around ansiblification of devstack-gate will hopefully mean that we can provide a more decomposed set of modular solutions broken out of d-g in the future, so you can mix and match in your own jobs without needing the entire framework16:17
fungibut as demonstrated, you can hollow out most of the logic in it right now by substituting your own hooks16:18
jeblairpabelanger: re 486665 -- does our logserver handle serving ".log" files as text now?16:19
pabelangerjeblair: it worked with chrome browser, let me find the apache rewrite code16:19
*** bhavik1 has quit IRC16:20
*** jascott1 has quit IRC16:20
pabelangerjeblair: Oh, looks like you added it: https://review.openstack.org/#/c/330119/16:21
*** jascott1 has joined #openstack-infra16:21
jeblairneat.  glad someone's on the ball.16:21
pabelanger:)16:21
fungisend a thank-you beer back to your past self!16:21
*** jamesden_ has quit IRC16:21
openstackgerritMerged openstack-infra/zuul-jobs master: Remove .txt suffix from tox logs  https://review.openstack.org/48666516:22
jeblair"future me is going to thank me for this someday"16:22
*** ykarel has quit IRC16:23
clarkbfwiw most of the setupy type stuff in d-g is in ansible playbooks now. So it is also theoretically possible to consume it that way today. The big missing piece is the network overlay stuff16:23
*** jamesdenton has joined #openstack-infra16:23
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Create openstack-py35 job with upper-constraints  https://review.openstack.org/48398716:23
clarkb(we don't have any examples of how something consuming d-g setup via ansible would look today that I know of though)16:23
jeblairclarkb: true.  i'd caution against that because i don't think we want to expose that as a stable api for jobs yet.16:23
jeblairclarkb: ie, i'd like us to be able to forklift that into the openstack-zuul-jobs repo without having to have a deprecation period.  :)16:24
fungiright, will get ugly if/when we reshuffle those out of the d-g repo16:24
*** vhosakot has quit IRC16:24
pabelangeradding the disk fixup stuff as a role could be a log hanging fruit for somebody16:24
*** jascott1_ has joined #openstack-infra16:24
*** jascott1 has quit IRC16:24
*** jascott1_ has quit IRC16:25
*** jascott1 has joined #openstack-infra16:26
openstackgerritSam Betts proposed openstack-infra/os-loganalyze master: Repair htmlify-log.py command  https://review.openstack.org/38381216:27
openstackgerritMerged openstack-infra/infra-manual master: Extra disk space is not necessarily mounted  https://review.openstack.org/48667216:27
clarkbah but fix_disk_layout is in setup_workspace not part of setup_host so it isn't ansible either16:28
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Allow loading additional variables file for site config  https://review.openstack.org/44773416:30
*** ykarel has joined #openstack-infra16:31
*** e0ne has quit IRC16:31
*** jascott1 has quit IRC16:31
jeblairclarkb: setup_<meaningless word> followed by setup_<other meaningless word>  :)16:31
*** jascott1 has joined #openstack-infra16:32
jeblairi'm sure we could probably come up with a "reason" mounting disks is setting up a workspace rather than a host.  but i'm guessing those words are just not sufficient to describe what those functions are actually doing after so much evolution.  :)16:32
*** jascott1 has quit IRC16:33
*** baoli has joined #openstack-infra16:33
*** jascott1 has joined #openstack-infra16:35
*** EricGonczer_ has quit IRC16:39
*** chlong_ has joined #openstack-infra16:40
openstackgerritClark Boylan proposed openstack-infra/system-config master: Update the keystone files in logstash worker config  https://review.openstack.org/48669116:41
*** chlong__ has quit IRC16:41
clarkbsdague: mtreinish lbragstad ^ review on that would be great to make sure I didn't miss something re keystone log file indexing16:41
*** Apoorva has joined #openstack-infra16:42
lbragstadclarkb: looks ok to me16:42
*** ykarel has quit IRC16:42
*** EricGonczer_ has joined #openstack-infra16:44
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: do not merge  https://review.openstack.org/48667916:44
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Implement autohold  https://review.openstack.org/48669216:45
clarkbmtreinish: good point I'll push a new patchset16:45
clarkbmtreinish: actually16:46
clarkbmtreinish: I'm not sure how systemd/journald + grenade will work out, eg will we have split logs at all?16:46
clarkbmtreinish: maybe its best to wait on that just so that we don't add in things that will never pan out16:46
mtreinishclarkb: ok, that's fair. I don't have an answer for what the grenade log story with systemd either16:47
mtreinishclarkb: then I'm +116:47
mtreinishsdague: ^^^ will we need to change something for grenade when we branch pike?16:47
clarkbmtreinish: sdague if we don't change anything I think we get all the logs in a single file16:48
clarkbwhich is probably fine?16:48
*** jascott1 has quit IRC16:49
*** ykarel has joined #openstack-infra16:49
mtreinishclarkb: well it might make it kinda confusing to see the upgrade boundary16:49
sdaguehonestly, a single set of logs would probably be easier to handle, the important thing is going to be making the boundary clear16:50
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Update tox.ini to use linters envlist  https://review.openstack.org/48669516:52
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Create openstack-py35 job with upper-constraints  https://review.openstack.org/48398716:52
*** armax has joined #openstack-infra16:52
*** ykarel has quit IRC16:53
*** wolverineav has quit IRC16:53
*** ykarel has joined #openstack-infra16:54
clarkbsdague: mtreinish should be simple to ensure each service logs a "I just started" message (or similar) to make the boundary clear16:54
clarkbI think most services already do that in the form of loading their config?16:54
sdagueclarkb: they do16:55
sdaguethe oslo service startup is pretty distinctive16:55
*** vhosakot has joined #openstack-infra16:56
*** derekh has quit IRC16:56
dirkclarkb: fungi : sorry, lost contact on the pip /wheels discussion.. I'm certainly fine to postpone the wheels topic if we could get a workaround in place to have the opensuse job no longer bitrotting .. e.g. https://review.openstack.org/#/c/485985 should be doing that16:57
*** ykarel has quit IRC16:58
*** felipemonteiro_ has quit IRC16:59
*** EricGonczer_ has quit IRC17:02
*** ralonsoh has quit IRC17:02
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Log an extra blank line to get space after each skip  https://review.openstack.org/48669817:04
*** chlong_ has quit IRC17:08
*** ricardoas has joined #openstack-infra17:08
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: do not merge  https://review.openstack.org/48667917:09
*** harlowja has joined #openstack-infra17:09
*** chlong_ has joined #openstack-infra17:10
*** ricardoas has quit IRC17:14
jeblairclarkb: do you need any gerrit 2.13 testing help from me?17:14
*** ricardoas has joined #openstack-infra17:14
*** efoley has quit IRC17:15
*** ramishra has quit IRC17:16
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Create tox_environment_defaults variable for tox based jobs  https://review.openstack.org/48667917:17
*** flepied has quit IRC17:17
*** trown is now known as trown|lunch17:18
*** jascott1 has joined #openstack-infra17:20
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Create tox_environment_defaults variable for tox based jobs  https://review.openstack.org/48667917:20
*** baoli has quit IRC17:20
*** baoli has joined #openstack-infra17:23
openstackgerritAshwin Agate proposed openstack-infra/project-config master: Fix devstack py35 functional test for monasca-transform  https://review.openstack.org/48575617:25
*** electrofelix has quit IRC17:27
clarkbjeblair: I don't think I do right now other than individuals just checking that new gerrit doesn't break personal workflows. I'm back home now so will be working on getting noopy zuuls deployed (v2.5 and v3)17:27
*** rbrndt has quit IRC17:28
clarkbmy plan for that is to just run them out of virtualenvs in my homedir on review-dev to start17:29
clarkbinfra-root reviews on https://review.openstack.org/#/c/486691/ will help us be able to better track keystone related bugs with e-r17:29
fungijeblair: actually, if you want to lend some mysql expertise, we seem to have experienced a regression in support for extended codepoints in review comments on review-dev17:31
jeblairclarkb: ok.  let me know if you want help with any of that.  you might consider using the 'git' zuulv3 driver for that, so you can put some review-dev zuulv3 config in a local git repo rather than bootstrapping it in gerrit itself.17:31
jeblairfungi: oh dear.  my understanding of that is minimal, but it's not zero.  maybe i can help push it slightly forward while we stall for time until mordred's back.  :)17:32
clarkbI'll likely dig into it in force tomorrow. Thanks for tip on git driver. Today is catch up on email and all that17:32
*** sambetts is now known as sambetts|afk17:33
openstackgerritAshwin Agate proposed openstack-infra/project-config master: Fix devstack py35 functional test for monasca-transform  https://review.openstack.org/48575617:33
funginot sure if something happened with the upgrade-related db migrations or if it's a behavior change in gerrit but it now seems to store a lot of characters (including some i tested which are 3-byte in utf-8) as a literal "?" in the database backend (i double-checked with mysqlclient)17:33
clarkbfungi: you said you observed that behavior on 2.11 and were checking if 2.13 was different? or is that new to 2.13?17:34
*** eharney has quit IRC17:35
*** baoli has quit IRC17:37
*** thorst_ has quit IRC17:38
*** felipemonteiro has joined #openstack-infra17:43
*** felipemonteiro_ has joined #openstack-infra17:44
*** thorst has joined #openstack-infra17:44
*** baoli has joined #openstack-infra17:45
*** felipemonteiro has quit IRC17:47
*** HeOS has quit IRC17:49
*** thorst has quit IRC17:49
*** baoli has quit IRC17:50
*** eharney has joined #openstack-infra17:50
fungiclarkb: the behavior waynr reported was a 500 error trying to insert a "see no evil monkey" glyph in a review comment17:50
*** yamahata has quit IRC17:50
fungion production review.o.o17:50
*** iyamahat has quit IRC17:51
clarkbfungi: and now it isn't an error but replaces with "?" ?17:51
fungiso we were trying to see if it could be replicated on review-dev but it's doing substitutions there17:51
clarkbsee no evil monkey is 4 byte so that makes sense on production17:51
fungiwhen it clearly wasn't at one point because we have other reviews from the 2.11 timeframe with utf-8 (snowman et cetera)17:51
*** flepied has joined #openstack-infra17:52
fungi500 error on 4-byte utf-8 codepoints is a known behavior for our production deployment? for some reason i thought we had it working there17:52
fungibut i could be conflating that with etherpad17:52
clarkbfungi: neither of our mysql deployments support 4 byte utf8 because the mysql version behind those servers is too old for the utf8mb4 encoding17:53
fungiaha, okay so the only surprising bit is the (recent?) behavior change on review-dev then17:54
fungimaybe we should plan to incorporate a db upgrade into the gerrit upgrade plan while we're at it? (or as a separate maintenance?)17:54
clarkbif/when we upgrade to newer mysql we can update the tablse to utf8mb4 fairly safely since its a strict superset of current data17:54
*** felipemonteiro_ has quit IRC17:54
*** felipemonteiro has joined #openstack-infra17:54
clarkbI think a separate maintenance would be best17:54
clarkbfungi: what characters did you try on review-dev?17:55
fungiseems like the maintenance for that would be pretty quick. mostly just the mysqldump/source time17:55
fungiclarkb: snowman, some kanji and some (confirmed 3-byte) kana17:55
*** wolverineav has joined #openstack-infra17:56
clarkbya just confirmed snowman becomes ?17:56
clarkbhttps://bugs.chromium.org/p/gerrit/issues/detail?id=1499 perhaps misconfigured?17:56
fungialso some non-ascii which fits in the latin-1 codepage and that came through fine17:56
clarkbbut I wouldn't expect my.cnf to affect us that way17:57
clarkbhrm maybe review-dev didn't get its db tables updated ?17:57
clarkbor the new account review db thing is not utf8?17:57
clarkbthat would be annoying17:58
*** claudiub has joined #openstack-infra17:58
AJaegerproject-config cores, here's a one line cleanup review for https://review.openstack.org/485110 - could I get a second +2, please?17:59
*** wolverineav has quit IRC17:59
clarkbAJaeger: will that update the subnodes config too?18:01
clarkbyup confirmed it should affect all nodes local.conf18:04
clarkbsince it is applied before we do the split between sub and primary in the localrc generation logic18:04
*** e0ne has joined #openstack-infra18:04
fungiclarkb: apparently i was able to snowman a review-dev change comment as recently as december, so something definitely changed on upgrade: https://review-dev.openstack.org/533518:05
*** dtantsur is now known as dtantsur|afk18:05
AJaegerthanks, clarkb18:05
clarkbfungi: that likely points at the new database on 2.13, we should inspect the tables and see if gerrit is overriding defaults there or something18:05
clarkbfungi: etherpad indicates I definitely altered the db to be utf8 by default, but gerrit could be creating the tables with an override maybe?18:06
*** jbernard has quit IRC18:06
clarkbfungi: or maybe we restored a database at some point and lost a latin1 to utf8 transition? zaro do you know?18:06
fungiclarkb: more interesting is that it's a write-related behavior change, because review-dev is still serving that snowman intact18:07
clarkbhuh18:07
fungibut it translates newer ones to "?" when writing to the same table18:08
*** gtrxcb has quit IRC18:08
clarkbmaybe we lack some new gerrit side config that says  "it is safe to utf8"18:08
bswartzIs lib/neutron-legacy required by devstack or is there a way to avoid using it?18:08
clarkbbswartz: you'll likely get better answer to that question in #openstack-qa (devstack is qa project) or in #openstack-neutron (I think neutron devs did a bunch of the work to get off of the legacy scripts)18:09
fungibswartz: i think that's related to the lib/neutron rewrite, but probably better to ask in #openstack-qa. it could be cruft at this stage, i don't know18:09
bswartzk ty18:09
*** ricardoas has left #openstack-infra18:11
*** iyamahat has joined #openstack-infra18:11
*** jbernard has joined #openstack-infra18:11
*** thorst has joined #openstack-infra18:12
openstackgerritMerged openstack-infra/project-config master: networking-ovn: Remove DEVSTACK_SUBNODE_CONFIG  https://review.openstack.org/48511018:14
*** psachin has quit IRC18:16
*** rcernin has quit IRC18:16
*** prateek has quit IRC18:17
*** trown|lunch is now known as trown18:17
*** dizquierdo has quit IRC18:18
*** rbrndt has joined #openstack-infra18:20
*** florianf has quit IRC18:22
*** felipemonteiro_ has joined #openstack-infra18:24
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: Move subunit processing  https://review.openstack.org/48584018:25
*** felipemonteiro has quit IRC18:27
*** EricGonczer_ has joined #openstack-infra18:29
*** dimak has joined #openstack-infra18:29
openstackgerritJohn Fulton proposed openstack-infra/tripleo-ci master: Create a loop device in multinode subnodes  https://review.openstack.org/48496318:30
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: Move subunit processing  https://review.openstack.org/48584018:30
*** yamahata has joined #openstack-infra18:30
*** gtrxcb has joined #openstack-infra18:31
*** baoli has joined #openstack-infra18:32
*** cody-somerville has joined #openstack-infra18:33
*** watersoul has quit IRC18:34
*** baoli has quit IRC18:37
*** EricGonczer_ has quit IRC18:41
openstackgerritMerged openstack-infra/tripleo-ci master: Nic config for containerized undercloud  https://review.openstack.org/44209818:41
*** EricGonczer_ has joined #openstack-infra18:43
*** dougwig has joined #openstack-infra18:44
*** watersoul has joined #openstack-infra18:45
*** jamesden_ has joined #openstack-infra18:49
*** jamesdenton has quit IRC18:49
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: Move subunit processing  https://review.openstack.org/48584018:51
*** camunoz has quit IRC18:54
openstackgerritEmilien Macchi proposed openstack-infra/project-config master: Introduce experimental TripleO scenario008  https://review.openstack.org/48672618:54
*** claudiub has quit IRC18:56
*** hasharMeeting is now known as hasharDinner18:56
*** EricGonczer_ has quit IRC18:57
*** dhajare has joined #openstack-infra18:59
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: Move subunit processing  https://review.openstack.org/48584019:00
*** annegentle has quit IRC19:00
*** EricGonczer_ has joined #openstack-infra19:00
*** dhajare has quit IRC19:03
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: Move subunit processing  https://review.openstack.org/48584019:09
openstackgerritJohn Fulton proposed openstack-infra/tripleo-ci master: Create a loop device in multinode subnodes  https://review.openstack.org/48496319:14
*** thorst is now known as thorst_afk19:15
*** rossella__ has quit IRC19:16
*** rossella__ has joined #openstack-infra19:17
*** EricGonczer_ has quit IRC19:19
*** e0ne has quit IRC19:21
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Implement autohold  https://review.openstack.org/48669219:21
*** baoli has joined #openstack-infra19:22
*** hasharDinner has quit IRC19:23
*** wolverineav has joined #openstack-infra19:28
*** reed has quit IRC19:31
*** reed has joined #openstack-infra19:32
*** camunoz has joined #openstack-infra19:33
*** csomerville has joined #openstack-infra19:37
*** cody-somerville has quit IRC19:40
*** gtrxcb has quit IRC19:41
openstackgerritDirk Mueller proposed openstack-infra/project-config master: Adding more openSUSE 42.3 variants to existing 42.2 jobs  https://review.openstack.org/48449919:44
openstackgerritDirk Mueller proposed openstack/os-client-config master: Manually sync with g-r  https://review.openstack.org/48416619:46
*** danpawlik has quit IRC19:48
*** danpawlik has joined #openstack-infra19:51
*** jcoufal has quit IRC19:54
*** HeOS has joined #openstack-infra19:56
*** ldnunes has quit IRC19:58
openstackgerritJulia Kreger proposed openstack-infra/project-config master: Remove non-ironic governed gerrit annoucements  https://review.openstack.org/48674119:59
pabelangermordred: mtreinish: clarkb: jeblair: fungi: Any reason that subunit2html (os-testr) is not a test-requirements.txt dependency in jobs that want to create testr_results.html ?20:00
pabelangertoday we install os-testr into a virutualenv at DIB creation20:01
*** EricGonczer_ has joined #openstack-infra20:01
*** e0ne has joined #openstack-infra20:01
*** ekhugen has quit IRC20:03
mtreinishpabelanger: because it's not used by every project that runs tests20:03
*** zz_ja has quit IRC20:03
mtreinishit's a postprocessing step step like stackviz20:03
mtreinishit just happens to share a python package with a test runner and an output filter used by most projects20:05
*** patriciadomin has quit IRC20:05
*** mdrabe has quit IRC20:05
*** ihrachys_ has joined #openstack-infra20:07
*** rossella__ has quit IRC20:07
*** Apoorva_ has joined #openstack-infra20:08
*** Apoorva_ has quit IRC20:09
*** ihrachys has quit IRC20:09
*** Apoorva_ has joined #openstack-infra20:09
*** mamitchl has joined #openstack-infra20:09
*** mdrabe has joined #openstack-infra20:09
*** patriciadomin has joined #openstack-infra20:09
*** ekhugen has joined #openstack-infra20:09
*** Apoorva_ has quit IRC20:10
*** zz_ja has joined #openstack-infra20:10
*** rossella__ has joined #openstack-infra20:10
*** Apoorva_ has joined #openstack-infra20:10
*** ldnunes has joined #openstack-infra20:10
*** Apoorva has quit IRC20:11
clarkbpabelanger: fungi if https://review.openstack.org/#/c/486691/ could get reviews that would be helpful. For indexing keystone logs20:12
*** EricGonczer_ has quit IRC20:14
*** rossella__ has quit IRC20:16
clarkbfungi: I'm going to start poking at the new db gerrit made now20:17
fungithanks clarkb!20:17
*** rossella__ has joined #openstack-infra20:18
pabelangermtreinish: well, any job that has a .testrepository file today we run subunit2html. It seems like we could add os-testr as a dependency for jobs using .testrepository20:20
clarkbfungi: the new db appears to be utf8 as expected based on show table status output20:21
fungiclarkb: so presumably a gerrit behavior change then20:21
pabelangermtreinish: I'm in the process of converting run-tox.sh into ansible playbooks, and trying to break out subunit2html into its own role20:21
clarkbfungi: ya that is what I am thinking20:21
fungiclarkb: the latest keystone service log path is logs/screen-keystone.txt? what does "screen" mean in this context? i thought latest devstack wasn't using screen by default any longer?20:21
clarkbfungi: the reviewdb tables are all utf as well20:21
clarkbfungi: we kept screen in the filenames for backward compat so that we wouldn't have to rewrite all the log parsing tools and configs20:22
fungiahh, okay20:22
jeblairpabelanger: can you elaborate on what you mean by ""add os-testr as a dependency for jobs using .testrepository20:22
jeblair20:21 < clarkb> fungi: the new db appears to be utf8 as expected based on show table status output20:22
jeblairpaste fail20:22
jeblairbut the first line was still correct :)20:22
jeblairpabelanger: i don't know what "dependency for jobs" means in this context20:23
mtreinishjeblair: I was assuming he meant add os-testr to the test-requirements for every project. But I could be mistaken in how I was reading it20:23
pabelangerjeblair: sure, adding os-testr to test-requirements.txt20:23
clarkbfungi: eventually when we stop having screen on old branches I think we will want to tackle addressing that. But right now with mixed content it seems easiest to pretend we still have screen for log processing20:23
jeblairpabelanger: okay, that's adding as a dependency to a project -- are you suggesting adding as a dependency to every project in our system?20:24
pabelangerjeblair: yes?  What I am trying to determine is, when we should be running subunit2html for our tox role: http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/tox/tasks/main.yaml#n28 because, it is possible we could have tox jobs that don't want to gerenate it20:25
fungito put it in context, that would be like adding zuul to test-requirements.txt of most projects because their jobs use zuul-cloner to checkout repos20:26
jeblairfungi: apt analogy20:26
pabelangerfungi: sure, this would be non-openstack projects using zuul-jobs tox role20:26
jeblairpabelanger: we made subunit2html so that we could stop using the jenkins xunit plugin because it had global locks that were preventing us from scaling.  in other words, we did it to achieve feature parity with jenkins.20:27
clarkbdigging in gerrit git history I'm not really seeing anything (at least immediately) that would impact unicode handling. Perhaps it is in the db driver itself?20:27
fungimight make sense to move the subunit2html filter calls to a separate playbook? or whatever the new equivalent of a jjb macro is (i'm still a little fuzzy on the difference between ansible roles, playbooks and plays)20:27
pabelangerjeblair: okay. Should we be using it for zuulv3?20:28
jeblairpabelanger: it's a feature of the ci system that we chose to provide to openstack projects.  i don't know that all the projects would agree that generating html output from their tests is something that should happen in all cases (even if they do agree that it's nice to have in the ci system).20:28
jeblairpabelanger: absolutely!  the experience is terrible without it.  :)20:28
jeblair(to be fair, i don't think anyone has ever asked projects if they would like to opt-in to automatic html generation when they run their tests locally)20:29
jeblair(but i'm not optimistic the answer would be 100% yes)20:29
fungipabelanger: it's what creates pages like http://logs.openstack.org/43/484443/1/gate/gate-cinder-python35/259237d/testr_results.html.gz which seems to me like a useful feature20:30
jeblairpabelanger: to directly answer the question at hand -- i think we should run subunit2html for all tox jobs in the world20:30
jeblairpabelanger: it is a gift from the openstack community to the world of zuul users running tox jobs.  :)20:30
jeblairpabelanger: er, tox+testr i should say20:31
jeblairpabelanger: so basically, i think our opinion should be: if you are using zuul to run tox+testr, the default python unit test job in zuul-jobs will give you nice html output20:31
*** baoli has quit IRC20:31
jeblairi don't think there is a good reason to consider that an openstack-only thing20:32
pabelangerjeblair: right, it would make sense for test-requirements.txt to have os-testr. And if we find it, generate html20:33
jeblairpabelanger: no i disagree.  we should not ask folks to add os-testr to test-requirements.txt20:33
pabelangerk20:34
jeblairi realize that's inconvenient for us because we now have to figure out how to install this thing.  but it's not that much different than bindep in that regard.20:35
clarkbfungi: I am beginning to wonder if the old mysql install on review-dev for the on host database server is crossing some stream between jdbc and trove hosted mysql when determining connection details20:35
*** wolverineav has quit IRC20:35
jeblairclarkb, fungi: do they have the same trove profile thingy?20:36
*** wolverineav has joined #openstack-infra20:36
jeblairclarkb, fungi: dev / prod20:36
clarkbjeblair: yes I think they do, in order to address the connection time outs, but I will double check20:36
jeblairoh, right, 'worked in december' probably takes care of that.  but still worth a sanity check.20:36
clarkbthe jdbc connection apparently tries to automatically determine this stuff if not explicity set, and system my.cnf values can be used to do that (we have an /etc/mysql/my.cnf from when we ran a server there)20:37
openstackgerritMerged openstack-infra/system-config master: Update the keystone files in logstash worker config  https://review.openstack.org/48669120:37
fungiargh, that's a terrible side effect behavior20:37
*** wolverineav has quit IRC20:40
clarkbjeblair: confirmed they both have the "sanity" configuration applied to them20:41
*** rockyg has joined #openstack-infra20:41
jeblairclarkb: review-dev and review have the same my.cnf20:42
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: Move subunit processing into fetch-testr-output  https://review.openstack.org/48584020:43
clarkbjeblair: that makes gerrit 2.13 seem more a likely culprit then20:44
*** baoli has joined #openstack-infra20:44
clarkbalso database appears to be storing a literal '?' so it isn't some presentation layer encoding problem I don't think20:44
*** baoli has quit IRC20:45
openstackgerritBoris Pavlovic proposed openstack-infra/project-config master: Actually Disable Swift Service in Rally DSVM jobs (when needed)  https://review.openstack.org/48674920:45
*** baoli has joined #openstack-infra20:45
clarkb`select message from reviewdb.change_messages where change_id = 107957 and uuid = 'df33271e_c13ae2cb';` will show a literal ? that was intended to be a snowman20:46
*** eharney has quit IRC20:47
*** e0ne has quit IRC20:50
clarkbone thought is we provide an explicit jdbc url now for the accountpatchreviewdb perhaps that is affecting some global jdbc state? (accountpatchreviewdb doesn't appear to have anything to do with the actual comment content)20:51
*** Goneri has quit IRC20:52
*** sileht has quit IRC20:52
*** rlandy has quit IRC20:53
clarkbjdbc:mysql://localhost:3306/dbname?useUnicode=yes&characterEncoding=UTF-8 is apparently how you force it to utf8 via the connection url20:53
*** annegentle has joined #openstack-infra20:53
clarkbI could try updating that url really quickly and see if it fixes the problem. Any objections to trying that?20:54
*** trown is now known as trown|outtypewww20:54
jeblairclarkb: ++20:59
clarkbok stopping gerrit on review-dev now21:00
jeblairclarkb: note: both servers have "character_set_server = utf8" in /etc/mysql/conf.d/client.conf21:01
jeblairclarkb: so if this works, maybe that file is now being ignored21:02
clarkbjeblair: ya I'm wondering if the explicit url overrides that behavior21:02
jeblairclarkb: (otoh, i don't know if jdbc used it before)21:02
*** rhallisey has quit IRC21:02
clarkbI don't think that affected anything, still getting ? on snowman21:03
clarkbwhich is sane considering the db for which we have the url explicitly set doesn't contain comment data21:03
*** jkilpatr has quit IRC21:04
*** armax has quit IRC21:05
*** EricGonczer_ has joined #openstack-infra21:06
clarkbtobiash: you are running gerrit 2.13 as well. Any chance you use mysql as the database server? do you observe this behavior as well (gerrit storing unicode characters as literal ? in the database)21:07
*** yamamoto_ has joined #openstack-infra21:09
*** hichihara has joined #openstack-infra21:10
*** baoli has quit IRC21:15
fungiclarkb: have you tried tweaking (or removing) the /etc/mysql/my.cnf on the server yet?21:15
*** ldnunes has quit IRC21:15
clarkbfungi: I have not21:15
fungisince it sounds like gerrit may (surprisingly) inspect that21:15
*** baoli has joined #openstack-infra21:16
clarkbthe other possibility is locale? I'm going to double check that really quick21:16
jeblairfungi: i think that's a standard mysql client lib behavior.  the file is intended to be applicable to both21:16
fungiokay21:16
clarkbLANG=en_US.UTF-8 is in the gerrit process environment according to procfs21:17
fungiso it's not so much that gerrit assumes a local /etc/mysql/my.cnf means any database gerrit is using is the locally-configured one, but rather that a local /etc/mysql/my.cnf is intended as a way of configuring client applications to know what to expect from remote databases they might access?21:18
clarkbfungi: ya it has global client config in /etc/mysql as well as server config21:18
fungiand there's no ~gerrit2/.my.cnf either at least21:20
jeblairfungi: right (and the ~ would override that i believe)21:20
*** yamamoto_ has quit IRC21:20
*** baoli has quit IRC21:20
fungiclarkb: so apparently you can set database.url in the gerrit config too, not just accountPatchReviewDb.url21:24
fungithis looks like what wmf did with their gerrit last december21:25
clarkbfungi: rather than feeding it individual pecies of info for hostname and username and password and db name?21:25
clarkbalso I double checked that error_log doesn't record an error when converting snowman to ?21:26
clarkberror_log looks really clean on review-dev nothing there21:26
fungiwell, they seemed to already be using database.url in their config, but yes21:27
jeblairfungi: if you do that, does password still come from secret file?21:27
jeblairsecure.conf21:28
*** baoli has joined #openstack-infra21:28
clarkbjeblair: I think you drop the password key value entirely and put the url in secure.conf like we did with accountpatchreviewdb21:29
fungiwe set all those independent values in secure.conf right now, and the way i understand that works whatever you put in secure.conf is merged with what's in gerrit.conf21:29
fungimainly i was pointing out that database.url is an option on modern gerrits rather than relying on all those separate values as individual settings21:29
fungieasy enough to test on review-dev regardless21:30
clarkbright so we could attempt to force utf8 via the jdbc url if we wanted21:30
clarkblike I tried with accountpatchreviewdb21:30
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: WIP: Move subunit processing into fetch-testr-output  https://review.openstack.org/48584021:32
fungiyeah, same format. can even try setting it in secure.conf21:32
*** Apoorva_ has quit IRC21:34
*** Apoorva has joined #openstack-infra21:34
fungilooks like they actually set both? https://phabricator.wikimedia.org/source/operations-puppet/browse/production/modules/gerrit/templates/gerrit.config.erb;53545623f46161413f10dcb1256c457d2604c64f$7421:36
fungiaha, and yeah they leave out the password so probably still setting that separately in secure.conf21:37
fungithat seems like a safe enough way to go21:37
clarkbfungi: do we want to try stopping puppet on review-dev and manually editing config to make that change?21:41
openstackgerritMerged openstack-infra/tripleo-ci master: Fix stackviz collect data  https://review.openstack.org/47455921:41
clarkbalso I thought I read java wants UTF-8 ?21:41
clarkbI suppose we can try both21:41
*** baoli has quit IRC21:41
*** sileht has joined #openstack-infra21:43
clarkbI've got a sandwich with my name on it but I can work on ^ after21:45
*** esberglu has quit IRC21:46
*** Apoorva has quit IRC21:47
*** Apoorva has joined #openstack-infra21:48
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Allow loading additional variables file for site config  https://review.openstack.org/44773421:50
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove state_dir from setMountsMap  https://review.openstack.org/48676621:50
*** marst has quit IRC21:52
*** jascott1 has quit IRC21:52
openstackgerritMerged openstack-infra/openstackid-resources master: Updated Eventbrite permission model  https://review.openstack.org/48273021:52
*** jascott1 has joined #openstack-infra21:53
*** Goneri has joined #openstack-infra21:53
fungiclarkb: or just ignore puppet? i think gerrit only rereads that file on service start anyway21:54
*** xyang1 has quit IRC21:54
fungibut yeah, i'm in the midst of chores/food as well (plus, zuul meeting starts in 5 minutes)21:55
*** jascott1 has quit IRC21:55
*** kong has joined #openstack-infra21:55
*** armax has joined #openstack-infra21:56
clarkbfungi: I think puppet may take that as a trigger to reinit gerrit (when the config changes)21:57
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Add query for nova functional placement test bug 1706207  https://review.openstack.org/48677321:57
openstackbug 1706207 in OpenStack Compute (nova) "resource-class-in-use_create_a_resource_class fails with "MismatchError: '204' not in ['201']" since 7/22" [High,Confirmed] https://launchpad.net/bugs/170620721:57
clarkbfungi: which is probably something we want to avoid21:57
*** jascott1 has joined #openstack-infra21:58
clarkbI'll go ahead and put it in the emergency list just to avoid any unexpected problems21:59
*** felipemonteiro_ has quit IRC22:00
fungioh, i thought we adjusted gerrit config semi-regularly in production and it didn't reinit, but i have no objection to extra caution22:00
jeblairit's zuul meeting time in #openstack-meeting-alt22:00
*** jascott1 has quit IRC22:01
*** yamamoto has joined #openstack-infra22:01
*** esberglu has joined #openstack-infra22:05
openstackgerritMerged openstack-infra/elastic-recheck master: Add query for nova functional placement test bug 1706207  https://review.openstack.org/48677322:06
openstackbug 1706207 in OpenStack Compute (nova) "resource-class-in-use_create_a_resource_class fails with "MismatchError: '204' not in ['201']" since 7/22" [High,Confirmed] https://launchpad.net/bugs/170620722:06
*** EricGonczer_ has quit IRC22:06
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Monitor job root and kill over limit jobs  https://review.openstack.org/48590222:07
*** bobh has quit IRC22:08
*** kgiusti has left #openstack-infra22:09
*** esberglu has quit IRC22:10
*** jgriffith has quit IRC22:10
*** jgriffith has joined #openstack-infra22:10
*** jgriffith has quit IRC22:12
*** jgriffith has joined #openstack-infra22:15
*** jgriffith has quit IRC22:16
openstackgerritdane-fichter proposed openstack-infra/project-config master: Add barbican-tempest-plugin tests to Glance  https://review.openstack.org/48678122:16
*** jgriffith has joined #openstack-infra22:19
*** jgriffith has quit IRC22:19
*** jkilpatr has joined #openstack-infra22:21
*** jgriffith has joined #openstack-infra22:23
*** rockyg has quit IRC22:26
*** gouthamr has quit IRC22:27
*** jaypipes has quit IRC22:32
*** EricGonczer_ has joined #openstack-infra22:32
*** ihrachys_ has quit IRC22:40
*** ihrachys_ has joined #openstack-infra22:40
*** gouthamr has joined #openstack-infra22:41
*** vhosakot has quit IRC22:50
*** Goneri has quit IRC22:50
*** rbrndt has quit IRC22:53
*** gtrxcb has joined #openstack-infra22:57
*** gordc has quit IRC22:58
openstackgerritSteve Baker proposed openstack-infra/tripleo-ci master: Use prepare command to build container images env file  https://review.openstack.org/48317823:00
*** aeng has quit IRC23:03
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Create tox_environment_defaults variable for tox based jobs  https://review.openstack.org/48667923:05
*** danpawlik has quit IRC23:06
*** rossella__ has quit IRC23:14
*** rossella__ has joined #openstack-infra23:16
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Create tox_environment_defaults variable for tox based jobs  https://review.openstack.org/48667923:17
*** aeng has joined #openstack-infra23:20
clarkbfungi: jeblair I'm going to hack in a url for the gerrit db now. Will paste what I end up with before restarting gerrit on review-dev23:22
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: DNM  https://review.openstack.org/48679223:23
fungithanks clarkb. and as i said, looks like you can leave out username and password and continue to define them in separate options (if wmf's production config is to be believed anyway)23:23
clarkbfungi: yup going to try that first23:24
*** EricGonczer_ has quit IRC23:24
clarkbactually why don't you just look at it on disk?23:24
fungilooking23:24
clarkbthat way I don't ahve to copy the db host hostname here whcih is semi secret23:24
fungilgtm, though do we want useunicode?23:25
fungidoes that make gerrit store unicode instead of utf-8 in the fields?23:25
fungii guess i could look up config docs for those options23:26
clarkbfungi: that comes from my earlier investigation of how to force a connection to use utf823:26
clarkbfungi: it may not be necessary though my research seemed to imply you may need it in some cases23:26
fungiahh, certainly confusing23:27
fungii mean, there is a defined mapping between unicode and utf-8 but they're entirely different binary representations after all23:27
clarkbya I think in this case it means we are going to do unicode using a utf8 representation23:28
*** aeng has quit IRC23:28
*** HeOS has quit IRC23:28
clarkbfungi: should I go ahead and restart gerrit with that in place as is?23:28
fungimakes sense, and sure go for it23:29
clarkbok here goes23:29
fungiftr, for those following along the options being tested in the dburl are: ?characterSetResults=utf8&characterEncoding=utf8&connectionCollation=utf8_bin&useUnicode=yes23:29
openstackgerritPaul Belanger proposed openstack-infra/shade master: WIP: Initial commit of zuulv3 jobs  https://review.openstack.org/48402723:29
clarkbit works \o/23:30
*** gildub has joined #openstack-infra23:30
clarkbfungi: I changed the collation method compared to wmf too since _bin is what we have set on our tables23:30
fungiyep, i figured that was why23:31
fungiglad i noticed they were doing it that way23:31
fungialso it looks like the old extended characters are still read correctly: https://review-dev.openstack.org/533523:31
fungiobviously those written to the db a ? though are still ?: https://review-dev.openstack.org/10795823:32
clarkbyup the previous ? are still ? because they went into the db that way23:32
fungiexcellent23:32
fungimy concern was that we would end up telling it to read a different encoding than we'd written earlier, but the old comment in https://review-dev.openstack.org/5335 demonstrates that it didn't23:33
clarkbok, I'll work on a patch to get that into our gerrit configs via puppet23:33
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Create tox_environment_defaults variable for tox based jobs  https://review.openstack.org/48667923:34
fungiclarkb: presumably we want to go ahead and roll that to production and just carefully monitor it after a restart23:34
openstackgerritPaul Belanger proposed openstack-infra/shade master: Initial commit of zuulv3 jobs  https://review.openstack.org/48402723:34
fungiindependent of any upgrade plans to 2.1323:34
clarkbfungi: ya23:34
clarkband we'll want to update the accountpatchreviewdb to use the same set of args for consistency23:35
*** chlong__ has joined #openstack-infra23:36
*** chlong_ has quit IRC23:37
openstackgerritMerged openstack-infra/system-config master: Remove pholio.openstack.org  https://review.openstack.org/47738623:39
clarkbprobably worth pushing up a patch to gerrit docs too23:41
clarkbmordred: speaking of gerrit patches did you ever manage to push your accountpatchreviewdb change up?23:41
*** aeng has joined #openstack-infra23:41
openstackgerritIan Wienand proposed openstack-infra/infra-specs master: Abandon pholio spec  https://review.openstack.org/48679423:45
clarkbianw: yay for black magic disk partitioning I guess23:45
fungiclarkb: oh, right, we keep a copy of the config in our system-config docs. we should definitely update it accordingly23:45
clarkbianw: except I think I need to use sudo in my change23:46
clarkbianw: ya just double checked locally so I'll push up a new patchset23:46
openstackgerritClark Boylan proposed openstack-infra/devstack-gate master: Sync and partprobe after mkpart on ephemeral disk  https://review.openstack.org/48667623:47
clarkbianw: ^ there we go23:47
ianwclarkb: ahh, good catch.  yeah ... it's still not clear to me if a sleep() effectively does the same thing, but it doesn't hurt23:47
ianwalthough, the partprobe was getting us confused in dib, as we hard partprobe makeing /dev/loopXpY devices, then kpartx calls making the same /dev/mapper/ ... devices23:48

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!