Thursday, 2016-09-15

*** jamielennox|away is now known as jamielennox00:00
*** gyee has quit IRC00:01
*** wfoster has quit IRC00:02
mordredclarkb: cool. I'll let it sit there for a bit then00:02
mordredclarkb: just making sure it wasn't oversight00:02
amrithanybody having issues with git?00:02
amrithunable to git review ... hangs00:03
*** slagle has joined #openstack-infra00:03
clarkbamrith: daily db backups00:03
odyssey4mehmm, it seems like I'm getting no response from POST operations on gerrit00:04
amriththx clarkb. oddly if I ctrl-C it leaves my client side out of whack with the server and it won't allow me to retry00:04
amrithit just hangs00:04
openstackgerritMerged openstack-infra/glean: Append to interface definition for multi-address  https://review.openstack.org/36686200:04
odyssey4mepretty much just a hang00:04
amrithyup00:04
clarkbyes daily db backups00:04
smcginnisodyssey4me: Seeing the same thing.00:04
odyssey4meah, just got a 50000:04
amrithhmm, let's wait. if it is the backup, it should be a shor tthing00:04
*** rook has quit IRC00:04
amrithi'll wait, back off, drink another glass of wine. really honey, have to have another glass of wine, gerrit is getting a backup :)00:05
smcginnisodyssey4me: Ah, better now.00:05
odyssey4meyep, it does seem better now00:05
*** gyee has joined #openstack-infra00:05
smcginnisamrith: Er, I mean, no, it's still having trouble. Really should be OK in about 10-15 minutes. ;)00:05
*** rook has joined #openstack-infra00:06
*** rook is now known as Guest895200:06
*** wfoster has joined #openstack-infra00:07
openstackgerritClark Boylan proposed openstack-infra/devstack-gate: DNM NOOP testing  https://review.openstack.org/37048800:09
amrithworked for me now, thanks00:09
amrithback to wine :)00:09
*** sai has quit IRC00:10
openstackgerritVarun Gadiraju proposed openstack-infra/project-config: Migrating vars that devstack_gate uses to vars in DEVSTACK_LOCAL_CONFIG  https://review.openstack.org/35434400:10
*** lucas-dinner has quit IRC00:10
*** adrian_otto has quit IRC00:11
*** wfoster has quit IRC00:12
*** Guest8952 has quit IRC00:12
pabelangerclarkb: mordred: it was likely an oversight on my behalf. I can update based on clarks comments, but that won't be for tonight00:12
openstackgerritTristan Cacqueray proposed openstack-infra/system-config: Add an option to limit owner statistics to a single project  https://review.openstack.org/37049000:12
clarkbpabelanger: well  Ithink the old code neglects it too00:12
clarkbbut I am pretty sure we want that so that rax works on first boot00:12
pabelangerI can dive more into it in a follow up patch if needed. I didn't really dive into what alot of the code did, just appened ipv6 bits when possible00:13
*** adrian_otto has joined #openstack-infra00:13
*** kjorgensen has quit IRC00:14
clarkbpabelanger: kk feel free to self approve that as is then (its why I +2d rather than -1)00:14
pabelangerclarkb: Right, i didn't test gentoo in rack, maybe prometheanfire could confirm00:15
clarkbya I think it will work elsewhere just not if relying on config drive network info like in rax00:15
clarkbbut I also think it was always that way so didn't -100:15
clarkbin other news grenade just failed in the gate on the same issue my change did so now I can worry about it less00:16
*** lucasagomes has joined #openstack-infra00:16
*** rook_ has joined #openstack-infra00:16
pabelangerclarkb: ack00:16
*** wfoster has joined #openstack-infra00:16
*** sai has joined #openstack-infra00:16
*** thorst has joined #openstack-infra00:17
*** kjorgensen has joined #openstack-infra00:17
*** Swami__ has joined #openstack-infra00:19
*** tqtran has quit IRC00:19
*** kjorgens_ has joined #openstack-infra00:20
*** fguillot is now known as fguillot_afk00:20
*** fguillot_afk has quit IRC00:20
*** kjorgensen has quit IRC00:21
*** Swami_ has quit IRC00:22
*** thorst has quit IRC00:24
*** kjorgens_ has quit IRC00:24
*** gyee has quit IRC00:25
*** _sarob has quit IRC00:25
*** sarob has joined #openstack-infra00:25
*** kaisers has joined #openstack-infra00:31
*** kaisers3 has joined #openstack-infra00:32
*** annegentle has joined #openstack-infra00:33
*** ddieterly has joined #openstack-infra00:33
*** ddieterly is now known as ddieterly[away]00:33
*** ddieterly[away] is now known as ddieterly00:33
*** kaisers2 has quit IRC00:34
*** kaisers1 has quit IRC00:34
*** Swami__ has quit IRC00:36
*** Swami__ has joined #openstack-infra00:37
*** jlvillal has quit IRC00:41
*** fguillot has joined #openstack-infra00:46
*** jlvillal has joined #openstack-infra00:46
*** jlvillal has quit IRC00:47
*** jlvillal has joined #openstack-infra00:47
*** jlvillal has quit IRC00:47
*** jlvillal has joined #openstack-infra00:47
*** akshai has joined #openstack-infra00:51
*** akshai has quit IRC00:53
*** gongysh has joined #openstack-infra00:55
*** pahuang has quit IRC01:04
*** tuanla_ has joined #openstack-infra01:07
*** itsuugo has quit IRC01:08
*** roxanaghe has quit IRC01:09
*** itsuugo has joined #openstack-infra01:10
*** roxanaghe has joined #openstack-infra01:12
*** mmedvede has joined #openstack-infra01:13
*** kzaitsev_mb has quit IRC01:14
*** itsuugo has quit IRC01:15
*** sarob has quit IRC01:15
*** itsuugo has joined #openstack-infra01:16
*** ijw has quit IRC01:17
*** adrian_otto has quit IRC01:17
*** pahuang has joined #openstack-infra01:19
*** ddieterly has quit IRC01:21
*** thorst has joined #openstack-infra01:21
*** itsuugo has quit IRC01:22
*** vinaypotluri has quit IRC01:22
*** itsuugo has joined #openstack-infra01:22
*** ijw has joined #openstack-infra01:25
*** itsuugo has quit IRC01:27
*** itsuugo has joined #openstack-infra01:28
*** thorst has quit IRC01:29
*** ddieterly has joined #openstack-infra01:31
openstackgerritamrith proposed openstack-infra/project-config: enable the trove pylint job  https://review.openstack.org/37050601:33
openstackgerritamrith proposed openstack-infra/project-config: enable the trove pylint job  https://review.openstack.org/37050601:35
*** thorst has joined #openstack-infra01:35
*** Sukhdev has quit IRC01:37
*** hongbin has joined #openstack-infra01:39
*** baoli has joined #openstack-infra01:39
*** baoli has quit IRC01:41
*** salv-orlando has joined #openstack-infra01:41
*** baoli has joined #openstack-infra01:41
*** roxanaghe has quit IRC01:42
openstackgerritPete Zaitcev proposed openstack-infra/git-review: Fix b'12345' in -l under py3  https://review.openstack.org/37050701:42
*** salv-orl_ has quit IRC01:43
*** rbergero1 has joined #openstack-infra01:44
*** thorst has quit IRC01:45
*** baoli has quit IRC01:46
*** gongysh has quit IRC01:47
*** roxanaghe has joined #openstack-infra01:48
*** itsuugo has quit IRC01:48
*** woodster_ has quit IRC01:50
*** itsuugo has joined #openstack-infra01:50
*** itsuugo has quit IRC01:55
*** ijw has quit IRC01:56
*** itsuugo has joined #openstack-infra01:57
*** Apoorva_ has joined #openstack-infra02:01
*** Apoorva has quit IRC02:04
*** ijw has joined #openstack-infra02:05
*** Apoorva_ has quit IRC02:06
*** itsuugo has quit IRC02:08
*** itsuugo has joined #openstack-infra02:09
*** kzaitsev_mb has joined #openstack-infra02:10
*** roxanaghe has quit IRC02:10
*** ddieterly has quit IRC02:11
*** 32NABYP9K has joined #openstack-infra02:14
*** itsuugo has quit IRC02:16
*** itsuugo has joined #openstack-infra02:17
*** fguillot is now known as fguillot_afk02:19
*** fguillot_afk has quit IRC02:19
*** itsuugo has quit IRC02:22
openstackgerritamrith proposed openstack-infra/project-config: enable the trove pylint job  https://review.openstack.org/37050602:22
*** itsuugo has joined #openstack-infra02:23
*** fguillot has joined #openstack-infra02:23
*** ddieterly has joined #openstack-infra02:24
openstackgerritamrith proposed openstack-infra/project-config: enable the trove pylint job  https://review.openstack.org/37050602:24
*** itsuugo has quit IRC02:29
*** itsuugo has joined #openstack-infra02:30
*** itsuugo has quit IRC02:35
*** itsuugo has joined #openstack-infra02:36
*** annegentle has quit IRC02:38
*** itsuugo has quit IRC02:43
*** inc0 has joined #openstack-infra02:43
*** itsuugo has joined #openstack-infra02:44
*** rlandy has quit IRC02:44
*** thorst has joined #openstack-infra02:47
*** yuanying has quit IRC02:47
*** itsuugo has quit IRC02:48
*** mriedem has quit IRC02:49
*** itsuugo has joined #openstack-infra02:50
*** gongysh has joined #openstack-infra02:50
*** sdake has joined #openstack-infra02:53
*** ddieterly has quit IRC02:53
*** thorst has quit IRC02:56
*** sdake has quit IRC02:58
*** rbergero1 is now known as rbergeron03:00
*** rbergeron has joined #openstack-infra03:00
*** itsuugo has quit IRC03:01
*** sdake has joined #openstack-infra03:01
*** megm has quit IRC03:02
*** itsuugo has joined #openstack-infra03:03
*** megm has joined #openstack-infra03:04
*** aeng has quit IRC03:04
*** ijw has quit IRC03:04
*** ijw has joined #openstack-infra03:05
openstackgerritArmando Migliaccio proposed openstack-infra/project-config: Grafana Neutron: fix Grenade dashboard  https://review.openstack.org/37052103:07
*** itsuugo has quit IRC03:08
*** itsuugo has joined #openstack-infra03:10
*** yamahata has quit IRC03:11
*** ijw has quit IRC03:13
*** itsuugo has quit IRC03:17
*** itsuugo has joined #openstack-infra03:18
*** gouthamr has quit IRC03:19
*** aeng has joined #openstack-infra03:20
*** gongysh has quit IRC03:21
*** itsuugo has quit IRC03:23
*** itsuugo has joined #openstack-infra03:24
*** gongysh has joined #openstack-infra03:25
*** roxanaghe has joined #openstack-infra03:26
prometheanfirehi?03:28
prometheanfireclarkb: pabelanger what's up?03:28
*** kjorgensen has joined #openstack-infra03:29
prometheanfirepabelanger: ipv4 netmask is prefered to be cidr as well for gentoo03:30
prometheanfireclarkb: pabelanger commented03:32
*** kjorgensen has quit IRC03:33
*** fguillot has quit IRC03:33
*** itsuugo has quit IRC03:33
openstackgerritArmando Migliaccio proposed openstack-infra/project-config: Grafana Neutron: fix Grenade dashboard  https://review.openstack.org/37052103:34
*** itsuugo has joined #openstack-infra03:36
*** roxanaghe has quit IRC03:37
*** itsuugo has quit IRC03:41
*** itsuugo has joined #openstack-infra03:42
*** vikrant has joined #openstack-infra03:44
*** thorst has joined #openstack-infra03:44
*** inc0 has quit IRC03:45
*** vinaypotluri has joined #openstack-infra03:45
*** thorst has quit IRC03:46
*** ijw_ has joined #openstack-infra03:49
*** jkilpatr has quit IRC03:49
*** yuanying has joined #openstack-infra03:51
*** tsg has joined #openstack-infra03:51
*** itsuugo has quit IRC03:51
*** itsuugo has joined #openstack-infra03:52
*** ijw_ has quit IRC03:54
*** itsuugo has quit IRC03:59
*** itsuugo has joined #openstack-infra04:00
*** Shrews has quit IRC04:07
*** phschwartz has quit IRC04:07
*** kzaitsev_mb has quit IRC04:08
*** mordred has quit IRC04:08
*** mordred has joined #openstack-infra04:10
*** itsuugo has quit IRC04:10
*** phschwartz has joined #openstack-infra04:11
*** Shrews has joined #openstack-infra04:11
*** itsuugo has joined #openstack-infra04:12
*** hongbin has quit IRC04:14
*** kjorgensen has joined #openstack-infra04:14
*** itsuugo has quit IRC04:16
*** itsuugo has joined #openstack-infra04:18
*** kjorgensen has quit IRC04:18
*** psachin has joined #openstack-infra04:21
*** Swami__ has quit IRC04:21
*** itsuugo has quit IRC04:23
*** yamamoto_ has joined #openstack-infra04:24
*** itsuugo has joined #openstack-infra04:24
*** ociuhandu has joined #openstack-infra04:24
*** camunoz has quit IRC04:26
*** megm has quit IRC04:27
*** itsuugo has quit IRC04:29
*** thorst has joined #openstack-infra04:29
*** itsuugo has joined #openstack-infra04:29
*** megm has joined #openstack-infra04:29
*** Genek has quit IRC04:30
*** itsuugo has quit IRC04:34
*** coolsvap has joined #openstack-infra04:34
*** itsuugo has joined #openstack-infra04:35
*** abregman has joined #openstack-infra04:38
*** thorst has quit IRC04:39
*** itsuugo has quit IRC04:40
*** adriant has quit IRC04:40
*** itsuugo has joined #openstack-infra04:41
*** tsg has quit IRC04:43
*** tsg has joined #openstack-infra04:43
*** aeng has quit IRC04:43
*** thorst has joined #openstack-infra04:45
*** jaosorior has joined #openstack-infra04:47
*** asselin_ has quit IRC04:52
*** kzaitsev_mb has joined #openstack-infra04:52
*** itsuugo has quit IRC04:53
*** itsuugo has joined #openstack-infra04:54
*** aeng has joined #openstack-infra04:56
*** thorst has quit IRC04:58
openstackgerritSridhar Ramaswamy proposed openstack-infra/project-config: Make tacker python3 jobs voting  https://review.openstack.org/37038804:58
*** thorst has joined #openstack-infra04:58
*** Jeffrey4l__ has joined #openstack-infra05:00
*** jaosorior has quit IRC05:01
*** jaosorior has joined #openstack-infra05:02
*** sdake has quit IRC05:03
*** itsuugo has quit IRC05:03
*** Jeffrey4l_ has quit IRC05:04
*** itsuugo has joined #openstack-infra05:04
*** pahuang has quit IRC05:06
*** thorst has quit IRC05:07
*** yamamoto_ has quit IRC05:07
*** yamamoto_ has joined #openstack-infra05:09
*** thorst has joined #openstack-infra05:09
*** kzaitsev_mb has quit IRC05:09
*** itsuugo has quit IRC05:10
*** itsuugo has joined #openstack-infra05:10
*** thorst has quit IRC05:13
*** roxanaghe has joined #openstack-infra05:14
*** florianf has joined #openstack-infra05:15
*** thorst has joined #openstack-infra05:15
*** pgadiya has joined #openstack-infra05:16
*** tqtran has joined #openstack-infra05:17
*** rcernin has joined #openstack-infra05:17
*** yamamoto_ has quit IRC05:19
*** thorst has quit IRC05:19
*** itsuugo has quit IRC05:20
*** itsuugo has joined #openstack-infra05:21
*** sdake has joined #openstack-infra05:22
*** tqtran has quit IRC05:22
*** roxanaghe has quit IRC05:22
*** itsuugo has quit IRC05:26
*** itsuugo has joined #openstack-infra05:28
*** adriant has joined #openstack-infra05:29
*** dahoo has joined #openstack-infra05:29
*** dimtruck is now known as zz_dimtruck05:31
*** gouthamr has joined #openstack-infra05:31
*** thorst has joined #openstack-infra05:34
*** itsuugo has quit IRC05:37
*** itsuugo has joined #openstack-infra05:38
*** markvoelker has quit IRC05:41
*** thorst has quit IRC05:42
*** itsuugo has quit IRC05:43
*** itsuugo has joined #openstack-infra05:43
*** thorst has joined #openstack-infra05:44
openstackgerritSridhar Ramaswamy proposed openstack-infra/project-config: Make tacker python3 jobs voting  https://review.openstack.org/37038805:46
*** thorst has quit IRC05:49
*** tsg has quit IRC05:49
*** thorst has joined #openstack-infra05:50
*** itsuugo has quit IRC05:51
*** itsuugo has joined #openstack-infra05:51
*** zz_dimtruck is now known as dimtruck05:52
*** thorst has quit IRC05:55
*** itsuugo has quit IRC05:56
*** oanson has joined #openstack-infra05:56
*** thorst has joined #openstack-infra05:56
*** yuval has joined #openstack-infra05:57
*** itsuugo has joined #openstack-infra05:58
*** thorst has quit IRC06:01
*** aeng has quit IRC06:01
*** nherciu has joined #openstack-infra06:02
*** thorst has joined #openstack-infra06:02
*** itsuugo has quit IRC06:03
*** dimtruck is now known as zz_dimtruck06:03
*** yamamoto_ has joined #openstack-infra06:03
*** itsuugo has joined #openstack-infra06:03
*** pcaruana has joined #openstack-infra06:04
*** kzaitsev_mb has joined #openstack-infra06:06
*** thorst has quit IRC06:07
*** thorst has joined #openstack-infra06:08
*** whoops has quit IRC06:10
*** whoops has joined #openstack-infra06:10
*** jtomasek has joined #openstack-infra06:12
*** pcaruana is now known as pcaruana|afk|06:13
*** claudiub has joined #openstack-infra06:15
*** thorst has quit IRC06:17
*** itsuugo has quit IRC06:18
*** thorst has joined #openstack-infra06:19
*** itsuugo has joined #openstack-infra06:19
*** dtardivel has joined #openstack-infra06:21
*** vinaypotluri has quit IRC06:22
*** zz_dimtruck is now known as dimtruck06:22
*** roxanaghe has joined #openstack-infra06:23
*** isaacb has joined #openstack-infra06:24
*** thorst has quit IRC06:24
*** thorst has joined #openstack-infra06:25
*** kjorgensen has joined #openstack-infra06:26
*** roxanaghe has quit IRC06:28
*** itsuugo has quit IRC06:28
AJaegerzigo, read the eavesdrop archive on what needs to be done for novnc. And then either propose a new change that fixes it properly - or we cleanup the broken state we have right now.06:28
*** asettle has joined #openstack-infra06:29
*** thorst has quit IRC06:30
*** andreas_s has joined #openstack-infra06:30
*** kjorgensen has quit IRC06:30
*** itsuugo has joined #openstack-infra06:30
*** adriant has quit IRC06:32
*** thorst has joined #openstack-infra06:32
*** dimtruck is now known as zz_dimtruck06:32
*** asettle has quit IRC06:34
*** sdake has quit IRC06:36
*** thorst has quit IRC06:36
*** itsuugo has quit IRC06:37
*** itsuugo has joined #openstack-infra06:38
*** mikelk has joined #openstack-infra06:39
*** sshnaidm|afk is now known as sshnaidm06:40
*** pcaruana|afk| is now known as pcaruana06:40
*** gouthamr has quit IRC06:41
*** itsuugo has quit IRC06:44
*** thorst has joined #openstack-infra06:44
*** rwsu has joined #openstack-infra06:44
*** jamielennox is now known as jamielennox|away06:45
*** itsuugo has joined #openstack-infra06:45
*** berendt has joined #openstack-infra06:45
*** berendt has quit IRC06:45
*** berendt has joined #openstack-infra06:46
*** jascott1_ has joined #openstack-infra06:47
*** jascott1 has quit IRC06:49
*** dahoo has quit IRC06:49
openstackgerritDong Ma proposed openstack-infra/jenkins-job-builder: Update xunit plugin  https://review.openstack.org/37056006:50
odyssey4memorning all06:50
*** salv-orlando has quit IRC06:50
odyssey4meyolanda when you're in, a review of https://review.openstack.org/369405 would be appreciated06:51
*** salv-orlando has joined #openstack-infra06:51
*** thorst has quit IRC06:52
*** thorst has joined #openstack-infra06:54
yolandahi odyssey4me , review time now06:57
AJaegermorning, yolanda and odyssey4me !06:57
yolandagood morning06:58
*** thorst has quit IRC06:59
*** itsuugo has quit IRC07:00
*** thorst has joined #openstack-infra07:00
*** esikachev has joined #openstack-infra07:02
*** kzaitsev_mb has quit IRC07:02
*** itsuugo has joined #openstack-infra07:02
*** Na3iL has joined #openstack-infra07:03
*** thorst has quit IRC07:05
*** florianf has quit IRC07:05
*** drifterza has joined #openstack-infra07:07
*** tqtran has joined #openstack-infra07:08
openstackgerritMerged openstack-infra/project-config: Promote OSA os_swift multi-region tests to voting  https://review.openstack.org/36940507:08
*** berendt has quit IRC07:08
*** larainema has quit IRC07:08
openstackgerritVasyl Saienko proposed openstack-infra/project-config: Switch ironic-pxe_ipa-postgres-nv to Xenial  https://review.openstack.org/37057007:08
*** berendt has joined #openstack-infra07:09
*** zul has joined #openstack-infra07:10
*** larainema has joined #openstack-infra07:10
*** thorst has joined #openstack-infra07:11
AJaegerdtantsur|afk, clarkb : Please review https://review.openstack.org/#/c/370570 for an Ironic Xenial change - this looks strange to me.07:13
*** Ravikiran_K has joined #openstack-infra07:13
*** florianf has joined #openstack-infra07:14
*** itsuugo has quit IRC07:17
*** thorst has quit IRC07:17
*** thorst has joined #openstack-infra07:18
*** itsuugo has joined #openstack-infra07:18
vsaienkomorning AJaeger: why it looks strange?07:19
*** jed56 has joined #openstack-infra07:19
*** shardy has joined #openstack-infra07:20
*** thorst_ has joined #openstack-infra07:21
*** Hal1 has quit IRC07:21
*** zhurong has joined #openstack-infra07:21
AJaegervsaienko: we follow a different model of converting to xenial. And that job has been run on stable releases and now you change the OS below it - that's something we normally don't do.07:22
AJaegervsaienko: I let dtantsur|afk and clarkb look at it, they're working on the big picture here.07:22
*** samuelBartel has joined #openstack-infra07:22
*** nmagnezi_ has joined #openstack-infra07:22
*** nmagnezi_ has quit IRC07:22
*** thorst has quit IRC07:23
*** itsuugo has quit IRC07:23
vsaienkoAJaeger: I see, I will contact dtantsur to get more details, thanks!07:23
*** nmagnezi has joined #openstack-infra07:24
*** matrohon has joined #openstack-infra07:24
*** itsuugo has joined #openstack-infra07:24
*** thorst_ has quit IRC07:25
AJaegerthanks, vsaienko07:27
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE: Test oooq with osinfra  https://review.openstack.org/37043707:27
openstackgerritMerged openstack-infra/puppet-infracloud: fix typo in bridge config for trusty  https://review.openstack.org/37019907:28
*** abregman_ has joined #openstack-infra07:28
*** itsuugo has quit IRC07:29
*** jlanoux has joined #openstack-infra07:29
openstackgerritMerged openstack-infra/project-config: puppet-tripleo: remove puppet-manager-core core permissions  https://review.openstack.org/37026007:30
*** abregman has quit IRC07:30
*** itsuugo has joined #openstack-infra07:30
*** abregman_ has quit IRC07:30
openstackgerritMerged openstack-infra/project-config: Add gate-tripleo-buildimage-overcloud-full-centos-7-nv to check  https://review.openstack.org/37041407:32
*** abregman has joined #openstack-infra07:32
openstackgerritMerged openstack-infra/project-config: Skip magnum identity v3 test job on docs changes  https://review.openstack.org/37040807:32
*** moravec has quit IRC07:33
*** amoralej|off is now known as amoralej07:34
*** itsuugo has quit IRC07:35
*** itsuugo has joined #openstack-infra07:35
*** jpich has joined #openstack-infra07:36
*** chem has joined #openstack-infra07:36
*** hichihara has joined #openstack-infra07:36
*** moravec has joined #openstack-infra07:37
*** asettle has joined #openstack-infra07:37
*** cody-somerville has quit IRC07:37
*** cody-somerville has joined #openstack-infra07:38
*** Hal has joined #openstack-infra07:38
*** abregman is now known as abregman|mtg07:39
*** asettle has quit IRC07:39
*** gongysh has quit IRC07:40
*** salv-orl_ has joined #openstack-infra07:40
*** markvoelker has joined #openstack-infra07:42
*** salv-orlando has quit IRC07:43
*** e0ne has joined #openstack-infra07:44
*** Rockyg has quit IRC07:45
*** jlanoux has quit IRC07:46
*** jlanoux_ has joined #openstack-infra07:46
*** itsuugo has quit IRC07:47
*** markvoelker has quit IRC07:48
*** itsuugo has joined #openstack-infra07:49
openstackgerritTomas Sedovic proposed openstack-infra/project-config: Publish docs for tripleo-validations  https://review.openstack.org/36948707:53
*** dtantsur|afk is now known as dtantsur07:55
*** zul has quit IRC07:57
*** jamielennox|away is now known as jamielennox07:57
openstackgerritDong Ma proposed openstack-infra/jenkins-job-builder: Update checkstyle plugin  https://review.openstack.org/37059207:58
*** hashar has joined #openstack-infra07:58
*** kzaitsev_mb has joined #openstack-infra07:58
*** bethwhite_ has joined #openstack-infra07:59
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-infra08:00
openstackgerritShu Muto proposed openstack-infra/project-config: Change JS test jobs to 'voting' for Magnum-UI  https://review.openstack.org/37059408:01
*** jascott1_ has quit IRC08:05
*** dizquierdo has joined #openstack-infra08:05
zigoAJaeger: Where to find the eavesdrop archive?08:11
*** sree has joined #openstack-infra08:13
*** yamahata has joined #openstack-infra08:13
openstackgerritmathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton  https://review.openstack.org/32375008:14
*** amotoki has joined #openstack-infra08:14
*** sheeprine has quit IRC08:15
openstackgerritMerged openstack-infra/project-config: Grafana Neutron: fix Grenade dashboard  https://review.openstack.org/37052108:16
*** Ravikiran_K has quit IRC08:17
*** ccamacho has joined #openstack-infra08:17
*** sheeprine has joined #openstack-infra08:18
openstackgerritMerged openstack-infra/project-config: update api-ref to be a ./ path, not a symbolic name  https://review.openstack.org/37033308:19
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/37060708:22
*** zz_dimtruck is now known as dimtruck08:23
openstackgerritMerged openstack-infra/project-config: Make subunit2sql py35 jobs voting  https://review.openstack.org/37032208:25
openstackgerritMerged openstack-infra/project-config: add zuul-cloner location to $PATH  https://review.openstack.org/37021008:25
dtantsurAJaeger, hey! do we have a framework in place to conditionally switch nodes to xenial on branches?08:26
openstackgerritMaciej Relewicz proposed openstack-infra/project-config: Added new repos for Murano chef cookbook  https://review.openstack.org/36672908:26
dtantsurAJaeger, I don't really want us to end up duplicating all jobs in the same way tox jobs are duplicated right now..08:26
openstackgerritMerged openstack-infra/project-config: Move ARA integration jobs to Xenial, remove Trusty  https://review.openstack.org/36948208:27
*** kzaitsev_mb has quit IRC08:29
*** bethwhite_ has quit IRC08:29
AJaegerzigo: eavesdrop.openstack.org/irclogs/%23openstack-infra/08:31
AJaegerdtantsur: that duplication is how we do it right now - best talk with clarkb later08:31
*** itsuugo has quit IRC08:32
dtantsurAJaeger, clarkb, we can duplicate a couple of jobs, then switch them all at once, I guess. we can't afford running all jobs duplicated in parallel.08:32
*** dimtruck is now known as zz_dimtruck08:33
*** itsuugo has joined #openstack-infra08:34
AJaegerdtantsur: sorry, don't have time for now to help here - please discuss with clarkb. I'll happily review afterwards ;)08:34
dtantsurfine, thanks08:35
dtantsurvsaienko, let's start with duplicating one job and making sure it at least works. we don't have too much time to fix issues in newton..08:36
*** tkelsey has joined #openstack-infra08:36
vsaienkodtantsur: ok, I will upload a new patch08:37
*** kjorgensen has joined #openstack-infra08:38
*** kjorgensen has quit IRC08:42
openstackgerritMaciej Relewicz proposed openstack-infra/project-config: Added new repos for Murano chef cookbook  https://review.openstack.org/36672908:43
*** electrofelix has joined #openstack-infra08:43
*** itsuugo has quit IRC08:46
apuimedowhere can we get the images used for gates?08:46
*** derekh has joined #openstack-infra08:46
apuimedogate-kuryr-libnetwork-dsvm-fullstack-nv08:46
*** itsuugo has joined #openstack-infra08:46
*** shu-mutou is now known as shu-mutou-AFK08:49
*** itsuugo has quit IRC08:51
*** itsuugo has joined #openstack-infra08:52
*** asettle has joined #openstack-infra08:52
*** tuanla has joined #openstack-infra08:54
*** ihrachys has joined #openstack-infra08:55
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Also set Haproxy addresses in no_proxy  https://review.openstack.org/37062308:57
*** tuanla_ has quit IRC08:57
*** asettle has quit IRC08:57
*** asettle has joined #openstack-infra08:58
*** abregman_ has joined #openstack-infra08:59
AJaegerapuimedo: you need to build them yourself - project-config/tools/build-image.sh09:01
apuimedoAJaeger: very well, thanks!09:01
*** abregman|mtg has quit IRC09:02
*** hichihara has quit IRC09:04
*** itsuugo has quit IRC09:06
*** itsuugo has joined #openstack-infra09:08
*** jtomasek has quit IRC09:08
*** watanabe_isao has joined #openstack-infra09:08
*** jtomasek has joined #openstack-infra09:09
tobiashhi, does someone know what's the default expiration time for the server cache in shade?09:09
*** tuanla_ has joined #openstack-infra09:09
tobiashlooks like wait_for_server in my nodepool instance waits much longer than the vm takes time to boot09:10
*** jordanP has joined #openstack-infra09:11
*** tuanla has quit IRC09:11
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: tripleo.sh: log overcloud deploy command arguments before deploy  https://review.openstack.org/37063209:15
*** itsuugo has quit IRC09:18
*** rossella_ has quit IRC09:18
*** itsuugo has joined #openstack-infra09:19
openstackgerritMerged openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/37060709:19
*** rossella_ has joined #openstack-infra09:19
*** yaume has joined #openstack-infra09:21
*** adrian_otto has joined #openstack-infra09:22
*** zhurong has quit IRC09:24
*** sambetts|afk is now known as sambetts09:25
*** tqtran has quit IRC09:26
*** alex_xu has quit IRC09:26
*** gildub has quit IRC09:26
*** kzaitsev_mb has joined #openstack-infra09:26
*** alex_xu has joined #openstack-infra09:27
*** jlanoux_ has quit IRC09:30
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove horizon coverage jobs  https://review.openstack.org/37064109:30
*** jlanoux has joined #openstack-infra09:30
*** kzaitsev_mb has quit IRC09:32
openstackgerritMaciej Relewicz proposed openstack-infra/puppet-apps_site: [wip] Glare support for app-catalog  https://review.openstack.org/35902909:32
openstackgerritJesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Add experimental scenario jobs to OpenStack-Ansible  https://review.openstack.org/37064609:36
*** thorst has joined #openstack-infra09:38
*** kzaitsev_mb has joined #openstack-infra09:39
*** gnuoy` has joined #openstack-infra09:39
openstackgerritSimon McCartney proposed openstack-infra/puppet-jenkins: Enable a specific version of jenkins to be installed  https://review.openstack.org/36680309:39
*** gnuoy has quit IRC09:40
*** sree has quit IRC09:41
*** HeOS has quit IRC09:43
*** markvoelker has joined #openstack-infra09:44
*** kzaitsev_mb has quit IRC09:44
*** tosky has joined #openstack-infra09:44
*** yamahata has quit IRC09:49
*** adrian_otto has quit IRC09:49
*** markvoelker has quit IRC09:49
openstackgerritIgor Duarte Cardoso proposed openstack-infra/irc-meetings: Update Neutron Classification Framework Meeting  https://review.openstack.org/37065309:49
*** sree has joined #openstack-infra09:51
odyssey4meAJaeger yolanda when you have a moment, a review of https://review.openstack.org/370646 would be appreciated09:52
*** sree has quit IRC09:56
*** hashar has quit IRC09:58
*** roxanaghe has joined #openstack-infra09:59
*** itsuugo has quit IRC10:00
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Adding support for Manual Build Trigger  https://review.openstack.org/20254310:00
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Consolidate trigger-manual and trigger-parameterized-builds  https://review.openstack.org/31410810:00
*** salv-orl_ has quit IRC10:01
*** itsuugo has joined #openstack-infra10:01
*** gnuoy has joined #openstack-infra10:01
*** gnuoy` has quit IRC10:02
*** gnuoy has quit IRC10:03
*** gnuoy has joined #openstack-infra10:03
*** roxanaghe has quit IRC10:04
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Allow using lockfile per jenkins master  https://review.openstack.org/29363110:09
*** gongysh has joined #openstack-infra10:10
electrofelixzaro: Is my response on https://review.openstack.org/#/c/312885/2/git_review/cmd.py about difference between '--all' for 'pull' versus 'fetch' for why I removed it in that review sufficient to answer you're -1 query?10:10
*** itsuugo has quit IRC10:11
*** itsuugo has joined #openstack-infra10:12
*** hashar has joined #openstack-infra10:12
*** HeOS has joined #openstack-infra10:15
*** acoles_ is now known as acoles10:18
*** sarob has joined #openstack-infra10:18
*** sarob has quit IRC10:23
*** adrian_otto has joined #openstack-infra10:23
*** zz_dimtruck is now known as dimtruck10:24
*** itsuugo has quit IRC10:25
*** itsuugo has joined #openstack-infra10:27
*** itsuugo has quit IRC10:32
*** ramishra has quit IRC10:32
*** _degorenko|afk is now known as degorenko10:32
*** itsuugo has joined #openstack-infra10:32
*** dimtruck is now known as zz_dimtruck10:33
*** ramishra has joined #openstack-infra10:33
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Add comment about proxy env pass through  https://review.openstack.org/27124410:34
*** dtantsur is now known as dtantsur|bbl10:39
*** itsuugo has quit IRC10:39
*** itsuugo has joined #openstack-infra10:41
*** itsuugo has quit IRC10:46
*** itsuugo has joined #openstack-infra10:46
*** rtheis has joined #openstack-infra10:48
openstackgerritMaciej Relewicz proposed openstack-infra/project-config: Added new repos for Murano chef cookbook  https://review.openstack.org/36672910:48
*** asselin_ has joined #openstack-infra10:48
*** chem` has joined #openstack-infra10:49
*** asselin__ has joined #openstack-infra10:49
*** chem has quit IRC10:50
*** sree has joined #openstack-infra10:50
*** asselin_ has quit IRC10:52
*** kzaitsev_mb has joined #openstack-infra10:55
*** sree has quit IRC10:55
*** apetrich has quit IRC10:56
*** shardy has quit IRC10:57
*** pilgrimstack has quit IRC10:58
*** pilgrimstack has joined #openstack-infra11:00
*** sdague has joined #openstack-infra11:01
*** electrofelix has quit IRC11:01
*** yamamoto_ has quit IRC11:01
*** salv-orlando has joined #openstack-infra11:02
*** vsaienko has quit IRC11:03
*** lucasagomes is now known as lucas-hungry11:06
*** salv-orlando has quit IRC11:06
*** adrian_otto has quit IRC11:09
*** apetrich has joined #openstack-infra11:11
*** EricGonczer_ has joined #openstack-infra11:12
*** jlanoux has quit IRC11:14
*** jlanoux has joined #openstack-infra11:15
*** jlanoux has quit IRC11:17
*** jlanoux has joined #openstack-infra11:17
*** jkilpatr has joined #openstack-infra11:18
openstackgerritAnton Arefiev proposed openstack-infra/project-config: Fix missing '$' sign in ironic-inspector-enabled builder  https://review.openstack.org/37070511:22
*** itsuugo has quit IRC11:23
*** itsuugo has joined #openstack-infra11:25
*** thorst has quit IRC11:27
*** salv-orlando has joined #openstack-infra11:28
*** asettle has quit IRC11:30
*** vsaienk0 has joined #openstack-infra11:33
*** pgadiya has quit IRC11:35
*** yamamoto has joined #openstack-infra11:38
*** yamamoto has quit IRC11:41
*** edmondsw has joined #openstack-infra11:42
*** amitgandhinz has quit IRC11:43
*** itsuugo has quit IRC11:43
*** itsuugo has joined #openstack-infra11:44
*** markvoelker has joined #openstack-infra11:45
*** ldnunes has joined #openstack-infra11:47
*** kzaitsev_mb has quit IRC11:47
*** yamamoto has joined #openstack-infra11:47
*** rfolco has joined #openstack-infra11:49
openstackgerritJiri Stransky proposed openstack-infra/tripleo-ci: Remove forced delorean dib build  https://review.openstack.org/37072411:49
*** baoli has joined #openstack-infra11:49
*** markvoelker has quit IRC11:50
*** yamamoto has quit IRC11:51
*** yamamoto has joined #openstack-infra11:52
*** trown|outtypewww is now known as trown11:52
*** gnuoy has quit IRC11:56
*** gnuoy has joined #openstack-infra11:56
*** yamamoto has quit IRC11:57
*** kzaitsev_mb has joined #openstack-infra11:57
*** baoli_ has joined #openstack-infra11:58
*** gordc has joined #openstack-infra11:59
*** pgadiya has joined #openstack-infra11:59
*** baoli has quit IRC12:01
*** jkraj has joined #openstack-infra12:04
*** amoralej is now known as amoralej|lunch12:07
*** psilvad has joined #openstack-infra12:08
*** jlanoux has quit IRC12:09
*** sree has joined #openstack-infra12:10
*** gongysh has quit IRC12:12
*** Na3iL has quit IRC12:12
*** xyang1 has joined #openstack-infra12:12
*** pradk has joined #openstack-infra12:13
*** sree has quit IRC12:14
*** amitgandhinz has joined #openstack-infra12:15
*** claudiub has quit IRC12:16
*** lucas-hungry is now known as lucasagomes12:17
*** asselin__ has quit IRC12:17
openstackgerritMike Fedosin proposed openstack-infra/project-config: Create glare-specs project  https://review.openstack.org/37074912:19
*** sarob has joined #openstack-infra12:19
*** pgadiya has quit IRC12:19
*** itsuugo has quit IRC12:23
*** itsuugo has joined #openstack-infra12:23
openstackgerritamrith proposed openstack-infra/project-config: enable the trove pylint job  https://review.openstack.org/37050612:23
*** sarob has quit IRC12:24
amrithAJaeger, https://review.openstack.org/#/c/370506/ with changes (some of them) that you requested.12:24
*** zz_dimtruck is now known as dimtruck12:24
AJaegeramrith: LGTM12:25
openstackgerritamrith proposed openstack-infra/project-config: add newton gate jobs for trove  https://review.openstack.org/36806712:26
amrithThanks AJaeger12:26
amrithI've also pushed the commit (above) for the newton gate jobs12:27
openstackgerritJiri Stransky proposed openstack-infra/tripleo-ci: Remove forced delorean dib build  https://review.openstack.org/37072412:27
*** kgiusti has joined #openstack-infra12:27
amrithyou'd commented on it earlier; I just update commit message with a depends-on on the trove rc1 tag commit12:27
*** dizquierdo has quit IRC12:27
*** esberglu has joined #openstack-infra12:28
*** itsuugo has quit IRC12:28
*** itsuugo has joined #openstack-infra12:29
*** yamamoto has joined #openstack-infra12:30
*** asettle has joined #openstack-infra12:30
amrithhi infra, may I have a review/approval on https://review.openstack.org/#/c/368067/ and https://review.openstack.org/#/c/370506/ please. we just tagged RC1 and if there are any failures resulting, I'd like to start getting them caught early so we can fix them in an RC2. thanks!12:30
*** yamamoto has quit IRC12:31
*** yamamoto has joined #openstack-infra12:31
*** dimtruck is now known as zz_dimtruck12:34
*** ccamacho is now known as ccamacho|lunch12:35
openstackgerritDoug Hellmann proposed openstack-infra/project-config: do not prompt before submitting multiple reviews  https://review.openstack.org/37076912:35
*** esberglu_ has joined #openstack-infra12:36
*** rhallisey has joined #openstack-infra12:36
*** asettle has quit IRC12:36
openstackgerritMaciej Relewicz proposed openstack-infra/puppet-apps_site: [wip] Glare support for app-catalog  https://review.openstack.org/35902912:37
*** esberglu has quit IRC12:37
*** esberglu_ is now known as esberglu12:37
*** yamamoto has quit IRC12:37
*** sdake has joined #openstack-infra12:38
*** markvoelker has joined #openstack-infra12:38
*** tuanla_ has quit IRC12:38
openstackgerritMehdi Abaakouk (sileht) proposed openstack-infra/project-config: gnocchi: fixes stable/2.2 branch gate breakage  https://review.openstack.org/36930012:38
openstackgerritMehdi Abaakouk (sileht) proposed openstack-infra/project-config: gnocchi: add upgrade testing jobs  https://review.openstack.org/36916412:38
*** yamamoto has joined #openstack-infra12:38
openstackgerritDoug Hellmann proposed openstack-infra/project-config: do not prompt before submitting multiple reviews  https://review.openstack.org/37076912:39
silehtAJaeger, can you re-approve https://review.openstack.org/#/c/369300/ ? I removed the patch dependencies that was blocking it to be merged12:39
silehtyolanda, ^12:43
*** mriedem has joined #openstack-infra12:44
*** burgerk has joined #openstack-infra12:45
*** itsuugo has quit IRC12:45
openstackgerritMehdi Abaakouk (sileht) proposed openstack-infra/project-config: gnocchi: Remove grenade testing jobs  https://review.openstack.org/36916312:45
openstackgerritMehdi Abaakouk (sileht) proposed openstack-infra/project-config: gnocchi: add upgrade testing jobs  https://review.openstack.org/36916412:45
openstackgerritMehdi Abaakouk (sileht) proposed openstack-infra/project-config: gnocchi: fixes stable/2.2 branch gate breakage  https://review.openstack.org/36930012:45
silehtmordred, I'm stupid I have put the patches in reverse order, sorry: https://review.openstack.org/36930012:46
*** itsuugo has joined #openstack-infra12:46
openstackgerritSpyros Trigazis proposed openstack-infra/project-config: [magnum] Make dib job a template  https://review.openstack.org/37078112:47
*** jamesdenton has joined #openstack-infra12:48
*** shardy has joined #openstack-infra12:50
*** rlandy has joined #openstack-infra12:50
*** dtantsur|bbl is now known as dtantsur12:51
zigoAJaeger: I fixed the deb-novnc repository on Alioth.12:51
zigoThe way to fix it was described here: https://www.perforce.com/blog/150109/git-out-vulnerability-out-fire12:52
zigoI'm bookmarking the URL this time...12:52
*** Goneri has joined #openstack-infra12:53
*** annegentle has joined #openstack-infra12:53
*** electrofelix has joined #openstack-infra12:53
*** vikrant has quit IRC12:55
*** burgerk has quit IRC12:56
*** yamamoto has quit IRC12:56
*** fguillot has joined #openstack-infra12:57
*** yamamoto has joined #openstack-infra12:58
*** askb has quit IRC13:00
*** ociuhandu has quit IRC13:01
*** itsuugo has quit IRC13:01
*** bethwhite_ has joined #openstack-infra13:02
*** jaosorior has quit IRC13:02
*** jaosorior has joined #openstack-infra13:02
*** itsuugo has joined #openstack-infra13:02
mtreinishyolanda, fungi, pabelanger: if you get a sec I've got a simple doc patch: https://review.openstack.org/#/c/237268/13:03
*** burgerk has joined #openstack-infra13:03
mtreinishthat one kinda slipped through the cracks. Heh, it's been open for almost a year13:03
mtreinishrcarrillocruz: ^^^13:03
*** psilvad has quit IRC13:05
*** psilvad has joined #openstack-infra13:05
*** oanson has quit IRC13:06
*** cardeois has joined #openstack-infra13:07
*** yamamoto has quit IRC13:07
*** yamamoto has joined #openstack-infra13:07
*** bexelbie_ has joined #openstack-infra13:09
*** psilvad has quit IRC13:09
*** bexelbie_ has left #openstack-infra13:09
*** psilvad has joined #openstack-infra13:09
*** akshai has joined #openstack-infra13:10
*** drawsmcgraw has left #openstack-infra13:10
*** annegentle has quit IRC13:11
*** moravec has quit IRC13:11
*** moravec has joined #openstack-infra13:12
*** drawsmcgraw has joined #openstack-infra13:12
*** fguillot has quit IRC13:13
*** sdake_ has joined #openstack-infra13:14
*** asselin__ has joined #openstack-infra13:14
*** amoralej|lunch is now known as amoralej13:15
*** salv-orlando has quit IRC13:15
*** sdake has quit IRC13:16
pabelangerstill a little early for hacking on glean :)13:16
pabelangermoar coffee13:16
openstackgerritIhar Hrachyshka proposed openstack-infra/devstack-gate: DNM: validate that low network_device_mtu on Nova side breaks gate  https://review.openstack.org/37081013:17
*** sarob has joined #openstack-infra13:17
*** psilvad has quit IRC13:17
*** krtaylor has joined #openstack-infra13:18
openstackgerritIhar Hrachyshka proposed openstack-infra/devstack-gate: DNM: validate that low network_device_mtu on Nova side does not break gate  https://review.openstack.org/37081313:18
*** psilvad has joined #openstack-infra13:18
*** rossella_ has quit IRC13:18
*** rossella_ has joined #openstack-infra13:19
*** pblaho has quit IRC13:21
*** sarob has quit IRC13:21
*** mdrabe has joined #openstack-infra13:22
*** ccamacho|lunch is now known as ccamacho13:22
*** shardy has quit IRC13:22
*** matt-borland has joined #openstack-infra13:23
*** Na3iL has joined #openstack-infra13:23
*** yamahata has joined #openstack-infra13:23
*** gouthamr has joined #openstack-infra13:24
openstackgerritMerged openstack-infra/system-config: Add documentation about openstack-health deployment  https://review.openstack.org/23726813:25
vsaienk0devstack-gate: please review https://review.openstack.org/#/c/352909 the patch fixes enabled services on subnode. Needed for ironic multinode job13:25
zxiiroelectrofelix: waynr: I rebased all the remaining api2.0 changes. There's only 5 left and I don't think they should take much time. We probably should consider any other things we want to get done for the code sprint tomorrow13:26
mordredsileht: no worries!13:26
zxiiroelectrofelix: waynr: I'd really like to see views support added and am working on some new patches to get it working. I'm going to see if I can break up the current patch into something smaller.13:28
*** mriedem has quit IRC13:29
electrofelixzxiiro: cheers, both views and folders would be useful, might want to check what is in python-jenkins so far for both as well13:29
zxiiroelectrofelix: agreed, folders would be handy too.13:29
openstackgerritPete Zaitcev proposed openstack-infra/git-review: Fix the outout of -l under py3  https://review.openstack.org/37050713:31
*** asettle has joined #openstack-infra13:33
openstackgerritMerged openstack-infra/project-config: gnocchi: fixes stable/2.2 branch gate breakage  https://review.openstack.org/36930013:34
openstackgerritSpyros Trigazis proposed openstack-infra/project-config: [magnum] Make dib job a template  https://review.openstack.org/37078113:35
*** roxanaghe has joined #openstack-infra13:35
*** tsg has joined #openstack-infra13:36
asselin__mordred, quick review on this please? https://review.openstack.org/#/c/363074/13:36
*** eharney has joined #openstack-infra13:36
*** asettle has quit IRC13:37
*** yamamoto has quit IRC13:38
mordredasselin__: looks fine to me - but I think we should also get opinion from fungi and clarkb ... I have a nagging memory that we didn't do delete on purpose, but I do not remember for the life of me what it was or if I'm hallucinating13:38
openstackgerritMerged openstack-infra/project-config: gnocchi: Remove grenade testing jobs  https://review.openstack.org/36916313:39
*** roxanaghe has quit IRC13:40
*** trown is now known as trown|afk13:40
*** EricGonczer_ has quit IRC13:41
openstackgerritRyan Moats proposed openstack-infra/project-config: Fix the experimental VPNaaS rally job  https://review.openstack.org/37083813:41
openstackgerritMike Fedosin proposed openstack-infra/project-config: Create glare-specs project  https://review.openstack.org/37074913:42
*** e0ne has quit IRC13:44
*** akshai has quit IRC13:44
*** esikache1 has joined #openstack-infra13:44
*** e0ne has joined #openstack-infra13:44
openstackgerritMonty Taylor proposed openstack-infra/zuul: Add async action plugin to override upstream async  https://review.openstack.org/37030913:44
mordredjeblair: ^^ works for both normal and async tasks now13:44
*** artmr has joined #openstack-infra13:45
*** inc0 has joined #openstack-infra13:46
*** degorenko has quit IRC13:46
*** itsuugo has quit IRC13:46
*** amitgandhinz has quit IRC13:47
*** esikachev has quit IRC13:47
*** degorenko has joined #openstack-infra13:47
*** amitgandhinz has joined #openstack-infra13:48
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: Use sequence znodes for image build numbers  https://review.openstack.org/37043513:48
*** akshai has joined #openstack-infra13:48
*** itsuugo has joined #openstack-infra13:48
*** eharney_ has joined #openstack-infra13:49
*** eharney has quit IRC13:49
*** eharney_ is now known as eharney13:49
*** akshai has quit IRC13:50
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Workaround patch to bring CI back  https://review.openstack.org/37072413:50
phschwartzMorning infra13:50
*** amitgandhinz has quit IRC13:50
*** gongysh has joined #openstack-infra13:51
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: Try Mistral API with 4 workers  https://review.openstack.org/37084713:51
*** tonytan4ever has joined #openstack-infra13:51
*** amitgandhinz has joined #openstack-infra13:51
*** hongbin has joined #openstack-infra13:52
*** dizquierdo has joined #openstack-infra13:52
openstackgerritSpyros Trigazis proposed openstack-infra/project-config: [magnum] Make dib job a template  https://review.openstack.org/37078113:53
*** baoli has joined #openstack-infra13:54
*** akshai has joined #openstack-infra13:55
*** gongysh has quit IRC13:55
*** tonytan_brb has joined #openstack-infra13:56
*** baoli__ has joined #openstack-infra13:56
*** akshai has quit IRC13:57
*** baoli_ has quit IRC13:57
*** ddieterly has joined #openstack-infra13:57
*** akshai has joined #openstack-infra13:58
*** tonytan4ever has quit IRC13:58
*** yamamoto has joined #openstack-infra13:59
*** baoli has quit IRC13:59
*** ddieterly has quit IRC14:00
*** akshai has quit IRC14:01
*** ramishra has quit IRC14:01
*** ddieterly has joined #openstack-infra14:02
*** mriedem has joined #openstack-infra14:03
*** akshai has joined #openstack-infra14:03
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Delete default plan before cli deployment  https://review.openstack.org/37085714:03
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Delete default plan before cli deployment  https://review.openstack.org/37085714:04
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Add helper tag to join arbitrary lists into strings  https://review.openstack.org/26592614:05
*** gouthamr has quit IRC14:07
*** drifterza has quit IRC14:07
*** ramishra has joined #openstack-infra14:08
*** asettle has joined #openstack-infra14:09
fungimordred: asselin__: i'm not personally aware of any reason we don't delete unmanaged files from /etc/puppet but perhaps clarkb knows why we shouldn't?14:09
*** claudiub has joined #openstack-infra14:09
*** andreww has quit IRC14:10
fungiif anything, i can see where having stuff we deleted in our git repos linger on the machines getting puppet apply run might cause subtle and hard to diagnose issues?14:11
asselin__fungi, I agree...I don't remember what issue I ran into (if any) when I made that change to my local setup. But the behavior should be consistent with what the puppet master would have done.14:12
electrofelixwaynr zxiiro: I'd like to also see if we can pick up https://review.openstack.org/#/c/194497 - JJB DSL, to hammer out some of the details as I think https://review.openstack.org/#/c/179459/ might warrant trying to add a warning about future behaviour change or hiding the behaviour behind a config option for a V2 preview?14:12
*** zz_dimtruck is now known as dimtruck14:12
*** ddieterly has quit IRC14:13
asselin__fungi, it's possible I was just testing stuff and noticing files I added not get deleted.14:13
*** burgerk has quit IRC14:14
*** asettle has quit IRC14:15
fungiasselin__: yeah, i think it's probably an improvement but i'll let clarkb approve it if he agrees once around14:15
*** ddieterly has joined #openstack-infra14:16
zxiiroelectrofelix: agreed, I think those would be useful to work in considering Jenkins 2.0 has those as major features now14:16
*** nmagnezi has quit IRC14:16
*** psilvad has quit IRC14:17
*** fguillot has joined #openstack-infra14:17
zxiiroelectrofelix: waynr I'm taking care of the views feature at the moment. If someone wants to take care of the other ones. Or we can sync up tomorrow and figure out what's important14:17
*** psilvad has joined #openstack-infra14:17
cloudnullmornings14:19
*** OferBarber has quit IRC14:21
*** rajinir has joined #openstack-infra14:23
waynrzxiiro electrofelix i've been traveling for a week now and won't be home until pretty late tonight, i should be able to look into the things you are mentioning tomorrow morning though.14:24
zxiirowaynr: cool no worries14:24
sdaguethis is a new git error I've never seen before - http://logs.openstack.org/02/369502/2/gate/gate-grenade-dsvm-neutron-ubuntu-trusty/6485b8e/logs/devstack-gate-setup-workspace-old.txt.gz#_2016-09-15_04_08_14_32514:25
*** flepied1 has joined #openstack-infra14:25
*** annegentle has joined #openstack-infra14:26
*** flepied has quit IRC14:26
*** jheroux has joined #openstack-infra14:27
sdaguehttp://logs.openstack.org/02/369502/2/gate/gate-grenade-dsvm-neutron-ubuntu-trusty/6485b8e/logs/devstack-gate-setup-workspace-old.txt.gz#_2016-09-15_04_08_14_325 maybe it's the "Unexpected end of command stream"14:27
*** yamamoto has quit IRC14:28
*** yamamoto has joined #openstack-infra14:29
*** yamamoto has quit IRC14:30
*** andreww has joined #openstack-infra14:30
*** trown|afk is now known as trown14:30
*** sree has joined #openstack-infra14:31
electrofelixwaynr: lets plan for tomorrow afternoon, I'm finding the reserve 4 hours for JJB exclusively at the same time every 2 weeks to be reasonably good14:31
electrofelixzxiiro: ^^ work for you?14:31
clarkbfungi: mordred asselin__ I would have to do testing but there is all sorts of stuff un /etc/puppet we dont want deleted on the puppetmaster14:32
fungiclarkb: oh, great point14:33
asselin__clarkb, this doesn't delete on the puppet master, it deletes on the puppet slave so that it's consistent with what was already deleted on the master14:33
*** yamamoto has joined #openstack-infra14:34
clarkbasselin__: yes but we treat the master as a slave too14:34
fungiasselin__: we run puppet apply on our bastion14:34
clarkbso it runs the same sync code I think14:34
asselin__yes that's true...so rsync with itself shouldn't delete anything AFAIK14:34
*** yamamoto has quit IRC14:35
*** flepied1 is now known as flepied14:35
fungiwhy wouldn't rsync delete anything? is it not called with --delete?14:35
*** sree has quit IRC14:35
asselin__fungi, no, not by default (when called via ansible)14:36
asselin__http://docs.ansible.com/ansible/synchronize_module.html14:36
openstackgerritMerged openstack-infra/storyboard-webclient: Reinstate the page size modification control in result-set-pager  https://review.openstack.org/37031014:38
*** watanabe_isao has quit IRC14:39
fungisdague: i'm not entirely convinced the "Couldn't find remote ref refs/zuul/master/Z599be3eec71646acbabb6471d0b15c74" for neutron there is related, as the change triggering that job was for nova and may not have had any neutron changes ahead of it in the gate when it ran14:39
*** tsg has quit IRC14:41
fungisdague: yeah, confirmed, the playbook indicates the only changes for that job were the one that triggered it, so it was just tested with that applied to nova's master branch tip, and all other involved projects have no zuul refs for that run14:42
*** pradk has quit IRC14:42
sdaguefungi: ok, but why was it pulling master on old on a grenade job?14:43
fungisdague: oh, now that's an excellent question. it did try to pull mitaka first14:43
*** thiagolib has joined #openstack-infra14:44
fungirather, it tried Couldn't find remote ref refs/zuul/stable/mitaka/Z599be3eec71646acbabb6471d0b15c7414:44
openstackgerritEmily Hugenbruch proposed openstack-infra/irc-meetings: Change WOS-mentoring meetings to weekly  https://review.openstack.org/37089914:44
openstackgerritDougal Matthews proposed openstack-infra/tripleo-ci: [TESTING] Force debug on tripleo.sh  https://review.openstack.org/37090014:45
*** pradk has joined #openstack-infra14:45
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Delete default plan before cli deployment  https://review.openstack.org/37085714:45
*** jed56 has quit IRC14:45
*** ddieterly has quit IRC14:46
*** akshai has quit IRC14:49
fungisdague: strange that it's calling git fetch directly too... for some reason i thought we'd already converted devstack-gate to use zuul-cloner for this logic?14:50
fungibut i guess not http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/functions.sh#n23714:50
*** chem` is now known as chem14:50
*** isaacb has quit IRC14:51
*** burgerk has joined #openstack-infra14:51
*** akshai has joined #openstack-infra14:51
*** akshai has quit IRC14:51
*** akshai has joined #openstack-infra14:53
fungiand yeah, the "Your branch is based on 'origin/master', but the upstream is gone." seems to indicate that somehow the origin remote was unset in the local clone of neutron there (or for other reasons it thinks the remote has no branch named master)14:54
*** baoli has joined #openstack-infra14:54
fungisdague: have you looked to see whether this is widespread or just a one-off failure?14:55
*** baoli__ has quit IRC14:55
sdagueI have not yet14:56
*** Zhongjun__ has joined #openstack-infra14:56
fungii'm poking in logstash now in that case14:56
*** tonytan_brb is now known as tonytan4ever14:57
*** edtubill has joined #openstack-infra14:57
*** rbrndt has joined #openstack-infra14:57
*** edtubill has quit IRC14:57
fungisdague: weird, in the past 24 hours it hit two jobs, both at almost exactly the same time and for the same change14:58
*** edtubill has joined #openstack-infra14:58
fungithe aforementioned gate-grenade-dsvm-neutron-ubuntu-trusty run and also gate-tempest-dsvm-full-devstack-plugin-ceph-ubuntu-xenial14:59
fungimatching log entries are 40 seconds apart14:59
*** edtubill has quit IRC15:00
clarkbgate-greande-dsvm-neutron-ubuntu-trusty is super flaky recently for other reasons15:00
*** berendt has quit IRC15:03
fungisdague: yeah, looks like logstash only has those two hits over the past week, so i'm inclined to blame clouds and the internet. amusingly one ran in osic-cloud1 and the other in bluebox-sjc115:04
fungiso presumably this was an issue locally in rackspace-dfw or a hiccup on one specific server15:05
sdaguefungi: yeh15:05
*** yamahata has quit IRC15:06
openstackgerritEmilien Macchi proposed openstack-infra/project-config: tripleo: disable voting on gate-tripleo-ci-centos-7-nonha-multinode  https://review.openstack.org/37092215:06
*** yuval has quit IRC15:06
fungisdague: maybe https://status.rackspace.com/index/viewincidents?group=21&start=147382560015:06
EmilienMhi Infra, we're having a critical problem in tripleo right now, and we're working on it. Until we fix our bug, we need this job non voting asap ^ -- thanks for your review15:06
*** regXboi has joined #openstack-infra15:07
*** baoli_ has joined #openstack-infra15:07
zxiiroelectrofelix: sounds reasonable to me. let's go with that15:07
fungisdague: hrm, no unless they got the filing date wrong on that incident, it's about 24 hours too early15:07
*** mtanino has joined #openstack-infra15:08
mordredfungi: is any way to distinguish between "I talked to the remote and it did not have the ref you requested" and "I could not reach the remote"  ?15:09
mordredbecause not being able to reach the remote for a sec then falling back to master seems like a bad fallback15:09
fungimordred: not sure. needs some research15:10
*** baoli has quit IRC15:10
*** asettle has joined #openstack-infra15:11
*** zul has joined #openstack-infra15:12
*** rbrndt_ has joined #openstack-infra15:14
* regXboi wanders in15:14
jrollfungi: without trying to parse scrollback, dfw had some control plane issues overnight, if that could be related15:14
jeblairfungi, mordred: zuul-cloner does have that capability15:14
jeblairso switching d-g to use z-c would improve that condition15:15
mordredjeblair: ok. so the answer is "migrate to z-c"15:15
regXboiola - I've posted https://review.openstack.org/#/c/370838/ to start fixing the rally job for neutron-vpnaas - would like some eyes on it when people have cycles15:15
*** gouthamr has joined #openstack-infra15:15
*** csomerville has joined #openstack-infra15:16
jrollhrm, probably not control plane issues :/15:16
fungijroll: the issue we recorded was minor, fleeting and vague so could be caused almost anything15:16
*** asettle has quit IRC15:16
jrollah, ok15:16
*** gouthamr has quit IRC15:17
*** gouthamr has joined #openstack-infra15:17
*** mtanin___ has joined #openstack-infra15:17
*** spzala has joined #openstack-infra15:17
*** jaosorior has quit IRC15:18
*** mtanino has quit IRC15:18
*** cody-somerville has quit IRC15:19
*** pcaruana has quit IRC15:19
*** rbrndt has quit IRC15:19
*** salv-orlando has joined #openstack-infra15:19
*** andreas_s has quit IRC15:19
*** adrian_otto has joined #openstack-infra15:20
*** M-docaedo_vector has quit IRC15:21
*** salv-orlando has quit IRC15:22
*** yaume has quit IRC15:22
*** zul has quit IRC15:22
*** EricGonczer_ has joined #openstack-infra15:23
*** tsg has joined #openstack-infra15:25
*** kjorgensen has joined #openstack-infra15:25
*** ddieterly has joined #openstack-infra15:25
*** ddieterly has quit IRC15:26
*** ddieterly has joined #openstack-infra15:26
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: Use sequence znodes for image build numbers  https://review.openstack.org/37043515:28
openstackgerritEmilien Macchi proposed openstack-infra/project-config: tripleo: disable voting on gate-tripleo-ci-centos-7-nonha-multinode  https://review.openstack.org/37092215:29
EmilienMmordred: I had to do a small change again - thx for your +2, very helpful ^15:29
*** sree has joined #openstack-infra15:29
*** mikelk has quit IRC15:30
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Delete default plan before cli deployment  https://review.openstack.org/37085715:30
*** mdrabe has quit IRC15:32
*** sree has quit IRC15:34
*** EricGonczer_ has quit IRC15:35
*** yamamoto has joined #openstack-infra15:35
*** woodster_ has joined #openstack-infra15:36
*** mtanino has joined #openstack-infra15:36
*** mtanin___ has quit IRC15:37
*** mdrabe has joined #openstack-infra15:38
zigomordred: I have the feeling that ftp.fr.debian.org has a script to ban us or something. The sbuild setup script sometimes fails to fetch the Release file... :/15:39
openstackgerritSpyros Trigazis proposed openstack-infra/project-config: [magnum] Make dib job a template  https://review.openstack.org/37078115:39
zigomordred: Failure to download from Debian repositories is the majority of our problems. I hope we can switch to a real Debian mirror soon.15:40
mordredzigo: I think pabelanger has the patch done to add sources to our mirror15:40
mordredand yah15:40
mordredthe rsync15:40
zigoYup.15:40
zigoWe need signed repo to be able to setup sbuild.15:40
zigoI was using httpredir.debian.org, and there was too many errors, so I switched to ftp.fr.debian.org.15:40
mordredzigo: it's good to know that our work of making mirrors is not wasted effort :)15:41
mordredand is, in fact, _very_ necessary15:41
*** yamamoto has quit IRC15:42
*** vinaypotluri has joined #openstack-infra15:48
*** itsuugo has quit IRC15:48
*** salv-orlando has joined #openstack-infra15:49
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Test dib build removal and temprevert together  https://review.openstack.org/37096115:49
*** priteau has joined #openstack-infra15:49
*** Hal has quit IRC15:49
*** itsuugo has joined #openstack-infra15:49
*** jgriffith has quit IRC15:52
*** tsg has quit IRC15:54
*** andreww is now known as xarses15:56
*** dizquierdo has quit IRC15:56
*** jkraj has quit IRC15:58
*** tphummel has joined #openstack-infra15:58
*** drifterza has joined #openstack-infra15:59
*** matbu is now known as matbu|brb15:59
*** ChanServ changes topic to "Discussion of OpenStack Developer and Community Infrastructure | docs http://docs.openstack.org/infra/ | bugs https://storyboard.openstack.org/ | source https://git.openstack.org/cgit/openstack-infra/ | channel logs http://eavesdrop.openstack.org/irclogs/%23openstack-infra/"16:00
*** tsg has joined #openstack-infra16:00
pleia2thanks for bug sprint things yesterday, everyone :)16:00
*** abregman__ has joined #openstack-infra16:00
pleia2I'll write up a summary email momentarily16:00
*** abregman_ has quit IRC16:00
EmilienMcan anyone approve this patch please? this is outstanding for tripleo group https://review.openstack.org/#/c/370922/16:00
dmsimardIs it possible to filter on a particular topic in Zuul layout ? Somewhat like how we filter jobs on particular branches.16:00
*** abregman__ has quit IRC16:01
pabelangermordred: zigo: can talk about it next week at mid-cycle, but ftp-mirror script from debian will work. But, it will mirror all distros16:01
*** abregman has joined #openstack-infra16:01
zigopabelanger: You mean sid and experimental? That's a good thing, I also need those to do backports.16:02
mgagnepabelanger: so we added better caching for images. lets wait and see if it fixes the problem16:02
mgagne(at internap-mtl01)16:02
EmilienMpabelanger: can you review https://review.openstack.org/#/c/370922/ when you can please? we're trying to unblock tripleo CI and we need this patch (we found the root cause of the problem)16:03
clarkbdmsimard: the topic is included in the change attributes https://review.openstack.org/Documentation/json.html#change so it should at least be possible but not sure if zuul supports that /me reads zuul docs16:04
fungidmsimard: if you mean gerrit topic, no zuul doesn't pay attention to gerrit topics at all (and they're also highly malleable so it would probably be dangerous if it did anyway)16:04
mordredpabelanger: wow, we can't pick releases? oh - is it because of the pool structure?16:04
dmsimardfungi: ack, thanks16:04
mordredpabelanger: basically it's going to mirror the whole pool I guess?16:04
fungidmsimard: oh, though clarkb makes a good point there... it may be that zuul treats all change attributes the same in that regard16:05
dmsimardfungi: well, there /are/ slight mentions of topic in Zuul  code (i.e, https://github.com/openstack-infra/zuul/blob/c19d3d9eff8921d7a76abd58729a3acb08d3e1e5/tests/base.py#L295 ) but it's not clear16:05
*** Hal has joined #openstack-infra16:05
dmsimardI guess there is only one way to find out for sure16:05
dmsimard(This is in RDO's Zuul instance)16:06
*** adrian_otto has quit IRC16:06
clarkbbased on http://docs.openstack.org/infra/zuul/triggers.html#gerrit it doesn't look like theyare arbitrary in the zuul config16:06
*** abregman has quit IRC16:06
clarkbso you'll have to add that ot zuul if you want it16:06
*** abregman has joined #openstack-infra16:06
*** gyee has joined #openstack-infra16:06
fungidmsimard: yeah, i think the mentions of "topic" in zuul's unit test framework are there so that we have an accurate depiction of a gerrit change event16:07
* dmsimard nods16:07
pleia2EmilienM: looking now16:07
*** mdrabe has quit IRC16:08
EmilienMthanks16:08
*** mdrabe has joined #openstack-infra16:08
*** samuelBartel has quit IRC16:08
*** Apoorva has joined #openstack-infra16:09
pleia2lgtm16:10
clarkblogstash says 7 gate resets in the last 24 hours due to the neutron grenade issue16:10
*** piet has joined #openstack-infra16:10
*** jgriffith has joined #openstack-infra16:11
* clarkb puts keystone up next on the xenialification ride16:12
*** asettle has joined #openstack-infra16:12
jeblairdmsimard: what fungi says is true.  what's your use case?16:12
mtreinishclarkb: is that in e-r yet?16:13
pleia2oomichi: great email, thank you!16:13
dmsimardjeblair: disabling certain jobs when a review is on a particular topic16:13
dmsimard(or the reverse logic of that)16:14
jeblairdmsimard: what's the underlying reason you want to disable/enable those jobs?16:14
*** Hal has quit IRC16:14
dmsimardjeblair: long running irrelevant jobs on trivial reviews in a particular topic16:15
jeblairdmsimard: (i'm wearing my architect hat at the moment while i ask these questions :)16:15
dmsimardjeblair: that would otherwise be relevant if not for that topic16:15
*** yamahata has joined #openstack-infra16:15
dmsimardjeblair: It's not a huge deal, we're going to disable these particular jobs while we crunch out these reviews and re-enable them afterwards16:16
dmsimardit's a one time deal16:16
jeblairdmsimard: is the topic truly an indicator of thes (like, all changes for the "cleanup-whitespace" topic shouldn't run these jobs) or is that a topic seems like a convenient piece of metadata to put a control like "topic: dont-run-slow-jobs")16:16
*** dizquierdo has joined #openstack-infra16:17
*** asettle has quit IRC16:17
openstackgerritSudipta Biswas proposed openstack/diskimage-builder: ppc64el: fix for grub2 missing package  https://review.openstack.org/37098016:17
dmsimardjeblair: a little bit of column A and a little bit of column B ?16:18
jeblairdmsimard: heh, what's the nature of these reviews you need to crunch out?16:18
dmsimardjeblair: I think in the case of "B" we can have a basic set of jobs and then have a trigger "check slow-jobs" to trigger them16:18
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Test dib build removal and temprevert together  https://review.openstack.org/37096116:18
clarkbmtreinish: I don't know I had to run away last night to do family things16:18
*** docaedo4 has quit IRC16:18
dmsimardjeblair: not unlike "check experimental"16:18
clarkbmtreinish: build_name:"gate-grenade-dsvm-neutron-ubuntu-trusty" AND filename:"console.html" AND message:"tempest.lib.exceptions.SSHTimeout: Connection to the" AND build_queue:"gate" is my query16:18
clarkbmtreinish: I have to run off again now to run pre trip errands16:18
jeblairdmsimard: (yeah, i agree that's a good way to handle that)16:18
mtreinishmriedem, armax: ^^^ do we have a bug and/or e-r query for that issue?16:19
sdaguemtreinish: I don't know there is a particular deep signature, I was kevinbenton noticed it as well16:19
sdaguehttp://tinyurl.com/zdpx7rh16:19
armaxmtreinish: nope16:19
sdaguehe was going to dive in16:19
armaxI got a bug report though16:19
armaxqw16:19
jeblairdmsimard: would a branch be a useful construct for this series of changes?16:19
kevinbentonsdague: what's weird is that it's not very consistent in where it fails16:19
*** matrohon has quit IRC16:19
kevinbentonsdague: i had one make it through the first vm fine16:19
armaxwe’re still fishing for potential causes16:19
kevinbentonsdague: and then die16:20
kevinbentonon the second16:20
armaxbug 162373216:20
openstackbug 1623732 in neutron "failed: route add -net "0.0.0.0/0" gw "10.1.0.1" leads to test_network_basic_ops to fail with SSHTimeout" [Critical,Confirmed] https://launchpad.net/bugs/162373216:20
dmsimardjeblair: let's go to #zuul :)16:20
sdaguekevinbenton: right, the ssh connect issues tend to be like that16:20
pabelangermordred: yes, for stable, testing and unstable16:20
kevinbentonsdague: i mean it's not even necessarily ssh16:20
kevinbentonsdague: sometimes the VM can't get dhcp16:20
kevinbentonsdague: sometimes it can be reached with ssh16:20
jeblairpabelanger, mordred, fungi: can you pls review afs-docs changes https://review.openstack.org/369738 and child?16:21
openstackgerritMerged openstack-infra/project-config: tripleo: disable voting on gate-tripleo-ci-centos-7-nonha-multinode  https://review.openstack.org/37092216:21
sdaguekevinbenton: you know, randomness like that in network land makes me wonder if there is a different mtu regression16:21
sdaguekevinbenton: are we failing on old side or new side?16:21
jeblairAJaeger: i think i have the marker stuff worked out from yesterday, and i think we can proceed the way we were planning.16:21
armaxclarkb: it’d be great if we could get hold of a node that’s failing on that jpb16:21
armaxjob16:21
mriedemmtreinish: http://status.openstack.org//elastic-recheck/index.html#1355573 ?16:22
mriedemmtreinish: http://status.openstack.org//elastic-recheck/index.html#1349617 ?16:22
jeblairAJaeger: see updated code comments in https://review.openstack.org/369738 for details; though nothing substantial changed except i think it will actually work now :)16:22
*** tobiash has quit IRC16:22
kevinbentonsdague: new side16:22
fungijeblair: you read my mind. i had just pulled 369738 back up from last night16:22
armaxmriedem: those are not catching what’s been seen in the last 24 hours16:23
*** tobiash has joined #openstack-infra16:23
armaxas far as I can tell16:24
*** kzaitsev_mb has quit IRC16:24
*** lucasagomes is now known as lucas-dinner16:24
*** dtantsur is now known as dtantsur|afk16:25
*** florianf has quit IRC16:25
mriedemarmax: and bug https://bugs.launchpad.net/neutron/+bug/1623732 is the one we're talking about yes?16:26
openstackLaunchpad bug 1623732 in neutron "failed: route add -net "0.0.0.0/0" gw "10.1.0.1" leads to test_network_basic_ops to fail with SSHTimeout" [Critical,Confirmed]16:26
armaxyea16:26
armaxmriedem: at least that’s the one I have been seeing burning in the house16:26
*** rcernin has quit IRC16:26
stevemarclarkb: what's the xenialification ride?16:26
*** inc0 has quit IRC16:26
*** e0ne has quit IRC16:26
mriedemok, i'm dangerously close to lunch time16:27
*** inc0 has joined #openstack-infra16:27
*** jascott1 has joined #openstack-infra16:27
stevemarmriedem: and we won't like you when you're hangry?16:27
mriedemyou'll probably like me equally16:27
*** florianf has joined #openstack-infra16:30
*** jascott1 has quit IRC16:30
*** asettle has joined #openstack-infra16:31
*** jascott1 has joined #openstack-infra16:31
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Test dib build removal and temprevert together  https://review.openstack.org/37096116:31
*** jpich has quit IRC16:32
*** itsuugo has quit IRC16:32
*** adrian_otto has joined #openstack-infra16:33
prometheanfirepabelanger: clarkb: about the glean reveiw, I do think we should enable each interface, but because of the way interfaces in gentoo works, we only need to enable interfaces that have config info16:33
*** itsuugo has joined #openstack-infra16:34
prometheanfirepabelanger: clarkb: an example of this is a bond and two eths, the bond has config info and depends on the two eths, so it will automatically bring the two eths up before bringing up/configuring the bond16:34
prometheanfireit's similiar for vlans16:34
prometheanfirehttps://review.openstack.org/36748716:34
*** kzaitsev_mb has joined #openstack-infra16:34
*** baoli_ has quit IRC16:34
openstackgerritJames E. Blair proposed openstack-infra/system-config: Add files01.openstack.org  https://review.openstack.org/37099016:36
jeblairfungi, pabelanger, mordred: ^16:36
*** asettle has quit IRC16:36
clarkbstevemar: migrating your tests that run on newton to xenial16:36
clarkbstevemar: its mostly done except for your functional jobs many of which are non voting looks like16:36
*** adrian_otto has quit IRC16:37
*** esikache1 has quit IRC16:37
erlonhi clarkb16:37
*** drifterza has quit IRC16:38
erlonclarkb: does the gate changes the working if add this patch as dependency? https://review.openstack.org/#/c/369519/16:38
erlonclarkb: I tryed but id didn't worked16:38
*** spzala has quit IRC16:39
erlonclarkb: added a depends-on here: https://review.openstack.org/#/c/147186/16:39
erlonclarkb: but the gate seems to be sending the same job16:39
*** roxanaghe has joined #openstack-infra16:39
*** spzala has joined #openstack-infra16:39
*** adrian_otto has joined #openstack-infra16:40
*** jordanP has quit IRC16:40
*** gouthamr has quit IRC16:40
*** fguillot has quit IRC16:41
amrithhi infra, may I have a second review/approval on https://review.openstack.org/#/c/368067/ and https://review.openstack.org/#/c/370506/ please. we just tagged RC1 and if there are any failures resulting, I'd like to start getting them caught early so we can fix them in an RC2. thanks!16:42
*** tonytan4ever has quit IRC16:42
clarkberlon: changes to the jjb configs are not self testing16:43
stevemarclarkb: does this have anything to do with why my stable/newton grenade job is failin?16:43
*** gouthamr has joined #openstack-infra16:43
stevemarclarkb: https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:stable/newton+topic:create-newton16:43
erlonclarkb: hmm, as I guessed16:43
*** adrian_otto has quit IRC16:43
openstackgerritGraham Hayes proposed openstack-infra/project-config: Add Experiemental PDNS job on xenial  https://review.openstack.org/37099616:43
erlonclarkb: do you have +2 powers?16:43
stevemarclarkb: http://logs.openstack.org/77/370877/1/check/gate-grenade-dsvm-neutron-ubuntu-xenial/5d1b019/logs/grenade.sh.txt.gz#_2016-09-15_14_39_01_54616:43
stevemarWARNING: this script has not been tested on xenial :)16:43
erlonclarkb: even if not, when possible, can you check that patch?16:44
*** roxanaghe has quit IRC16:44
*** priteau has quit IRC16:44
*** tobiash has quit IRC16:44
pabelangermgagne: great, thanks for looking into it16:44
*** yamamoto has joined #openstack-infra16:44
*** regXboi has quit IRC16:44
clarkbstevemar: no thatd likely due to all thr branches not existing yet and grenad ebeing silly and failing on untested platforms16:44
*** yamamoto has quit IRC16:44
*** asselin_ has joined #openstack-infra16:44
*** docaedo4 has joined #openstack-infra16:45
*** asselin__ has quit IRC16:45
*** tobiash has joined #openstack-infra16:45
*** adrian_otto has joined #openstack-infra16:46
stevemarclarkb: advice on fixing it? :)16:46
clarkbstevemar: all ptojects need newton brnaches and grenade should stop failing if it doesnt recog ize the distro16:46
clarkbit says "warning" then exits non zero iirc16:47
pabelangerprometheanfire: okay, so we need to make a change to 367487 or we are good for now?  If you'd like to make an update, I have no issues. I don't have the bandwidth to upload a gentoo image in to rax right now. Also, I've never used it before, so going to be some learning curve there for me too16:47
clarkbhaving a warning is fine buy it shouldnt error16:47
prometheanfirepabelanger: I'll try to test tonight at home or on osic16:47
prometheanfirepabelanger: because of the way networks depend on one another it's 'odd'16:48
*** adrian_otto has joined #openstack-infra16:48
pabelangerprometheanfire: ack16:48
prometheanfirebut at least glean can be run over and over to live test16:48
*** spzala has quit IRC16:49
*** derekh has quit IRC16:49
*** vhosakot has joined #openstack-infra16:49
*** senk has joined #openstack-infra16:51
*** annegentle has quit IRC16:52
openstackgerritJerry Zhao proposed openstack-infra/project-config: remove requirements check for networking-fortinet  https://review.openstack.org/37047016:53
*** adrian_otto1 has joined #openstack-infra16:53
*** asettle has joined #openstack-infra16:55
mordredjeblair: typo-oopsie in afs patch16:55
jeblairmordred: ya just noticed that16:55
jeblairmordred: the www_root/base thing?16:55
mordredjeblair: yup16:55
openstackgerritJames E. Blair proposed openstack-infra/system-config: Add files01.openstack.org  https://review.openstack.org/37099016:56
jeblairmordred: there we go!  pabelanger ^16:56
*** tosky has quit IRC16:56
jeblair(yay apply test!)16:56
pabelangerclearly a better reviewer them me16:56
pabelangerthen*16:56
jeblairi think he had help16:56
*** adrian_otto has quit IRC16:57
*** bethwhite_ has quit IRC16:57
*** maishsk has joined #openstack-infra16:57
*** yamahata has quit IRC16:58
*** yamahata has joined #openstack-infra16:58
*** sree has joined #openstack-infra16:58
mordredssssh16:58
*** asettle has quit IRC17:01
openstackgerritMerged openstack-infra/zuul: Ansible launcher: add AFS publisher  https://review.openstack.org/36973817:02
*** sree has quit IRC17:03
*** rfolco has quit IRC17:03
*** tsg_ has joined #openstack-infra17:03
*** hashar has quit IRC17:04
*** rfolco has joined #openstack-infra17:04
*** dahoo has joined #openstack-infra17:04
*** adrian_otto1 has quit IRC17:04
*** adrian_otto has joined #openstack-infra17:05
fungioh, whoops. i guess i double-approved the k5start one17:05
*** esikache1 has joined #openstack-infra17:05
jeblairfungi: unrelated, but would you mind aprving https://review.openstack.org/370448 so we can roll it into the same restart?17:06
fungilooking17:06
fungioh, right-o, that one17:06
*** tsg has quit IRC17:07
JayFWe have a spammer in #openstack-ironic17:07
JayFcan someone with access do the needful17:07
JayFfungi: jeblair: pabelanger: ^ help?17:07
JayFoh, jroll can do it17:07
JayFfixed, jroll wins :D17:07
dstanekspammer in #openstack-keystone too17:09
*** esikache1 has quit IRC17:11
fungiyeah, i just added them to the akick list there17:11
openstackgerritMerged openstack-infra/zuul: Ansible launcher: run k5start in playbook  https://review.openstack.org/37023517:12
openstackgerritMerged openstack-infra/zuul: Ansible launcher: format ipv6 urls [correctly]  https://review.openstack.org/37044817:13
*** esikache1 has joined #openstack-infra17:13
*** dizquierdo has quit IRC17:13
openstackgerritMerged openstack-infra/tripleo-ci: remove DIB workaround  https://review.openstack.org/36979217:14
*** senk has quit IRC17:18
*** rossella_ has quit IRC17:18
*** rossella_ has joined #openstack-infra17:19
*** esikache1 has quit IRC17:22
*** bethwhite_ has joined #openstack-infra17:22
jeblairfungi, pabelanger, mordred, ianw: we currently have 3 documentation websites (docs, api, developer). i made a 'docs' volume and rooted it at /afs/o.o/docs.  i'm thinking that maybe what we want is a volume for each of those sites, and to mount them at /afs/o.o/web/<site>.  eg /afs/o.o/web/docs.17:22
fungijeblair: i believe api is just a redirect now17:22
fungito developer17:23
sc68cal]/w 2617:23
*** abregman has quit IRC17:23
sc68calwhoops, thanks irssi17:23
fungijeblair: or at least that's how it looked when i was digging in cloudsites configuration anyway17:23
jeblairfungi: hrm.  it still has a jjb site configuration and a job that publishes there.  maybe it only publishes .htaccess files or something?17:24
jeblairit certainly does redirect :)17:24
fungiyeah, attempting to browse to apps.openstack.org just redirects me to http://developer.openstack.org/ and yes i believe te redirect is done via a .htaccess file17:24
jeblairfungi: you mean api.o.o, right?17:24
fungibecause you can't set redirect-style aliases for sites in cloudsites17:24
fungigrr, yes api17:24
fungii'm also in the app catalogue meeting discussing their glare support implementation at the same time, sorry17:25
jeblairok cool, so probably we can drop a bunch of stuff by just making that an apache redirect.17:25
openstackgerritMerged openstack-infra/system-config: Add files01.openstack.org  https://review.openstack.org/37099017:26
*** fguillot has joined #openstack-infra17:26
*** megm has quit IRC17:26
jeblairhrm.  i'm less enthused by my subdir idea atm.17:26
jeblairmaybe we should just stick with /docs and make /developer-docs a sibling.17:27
jeblair(this is all fairly easy to change later, thanks to multiple mounts)17:27
*** megm has joined #openstack-infra17:28
*** trown is now known as trown|lunch17:28
*** jascott1 has quit IRC17:29
*** M-docaedo_vector has joined #openstack-infra17:29
*** jascott1 has joined #openstack-infra17:29
openstackgerritJames E. Blair proposed openstack-infra/system-config: Add afs sites to zuul-launchers  https://review.openstack.org/37101817:29
*** chem has quit IRC17:29
*** chem has joined #openstack-infra17:30
*** chem has quit IRC17:30
*** chem has joined #openstack-infra17:30
*** e0ne has joined #openstack-infra17:30
*** sambetts is now known as sambetts|afk17:30
*** electrofelix has quit IRC17:31
jeblairfungi, mordred, pabelanger: i think it's safe to land that ^ before the launcher restart; that way it will restart with the necessary configuration.  it should even be safe to do before we install the keytab.17:32
pabelangerLGTM17:33
*** jascott1 has quit IRC17:34
fungiagreed17:35
*** degorenko is now known as _degorenko|afk17:35
*** adrian_otto has quit IRC17:36
*** tonytan4ever has joined #openstack-infra17:37
*** SumitNaiksatam has joined #openstack-infra17:38
*** tosky has joined #openstack-infra17:39
*** inc0 has quit IRC17:41
*** Zhongjun__ has quit IRC17:41
*** ihrachys has quit IRC17:43
*** amoralej is now known as amoralej|off17:43
*** Na3iL has quit IRC17:43
*** tkelsey has quit IRC17:45
*** yamamoto has joined #openstack-infra17:45
openstackgerritMerged openstack-infra/system-config: Add afs sites to zuul-launchers  https://review.openstack.org/37101817:45
*** jascott1 has joined #openstack-infra17:46
*** andrewbogott has quit IRC17:47
*** andrewbogott has joined #openstack-infra17:47
*** kjorgens_ has joined #openstack-infra17:49
*** adrian_otto has joined #openstack-infra17:52
*** yamamoto has quit IRC17:52
*** esikachev has joined #openstack-infra17:53
*** kjorgensen has quit IRC17:53
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Re-enable temprevert/cherry-pick/pin functionality  https://review.openstack.org/37096117:54
*** gyee has quit IRC17:54
*** adrian_otto1 has joined #openstack-infra17:56
*** adrian_otto has quit IRC17:57
openstackgerritSpyros Trigazis proposed openstack-infra/project-config: [magnum] Make dib job a template  https://review.openstack.org/37078117:57
*** adrian_otto1 has quit IRC17:57
*** jascott1 has quit IRC17:58
*** jascott1 has joined #openstack-infra17:58
*** jascott1 has quit IRC17:59
*** jascott1 has joined #openstack-infra17:59
*** jascott1_ has joined #openstack-infra17:59
*** jascott1 has quit IRC18:01
*** eeiden has quit IRC18:03
*** eeiden has joined #openstack-infra18:06
*** rook_ is now known as rook18:07
AJaegerjeblair: great to see the progress. I won't be able to help today, will just do a few reviews now...18:09
*** tqtran has joined #openstack-infra18:10
*** tsg_ has quit IRC18:10
*** eeiden has quit IRC18:11
*** yamahata has quit IRC18:13
*** asettle has joined #openstack-infra18:13
*** EricGonczer_ has joined #openstack-infra18:13
*** tsg_ has joined #openstack-infra18:13
AJaegerzigo, then please propose a revert of fungi's change so that we can setup deb-novnc again18:14
*** hasharAway is now known as hashar18:14
AJaegerjeblair: api and developer is the same website. We renamed it. Developer is the site, api an alias.18:18
*** bethwhite_ has quit IRC18:19
*** kzaitsev_mb has quit IRC18:19
AJaegerah, fungi said it as well ;)18:19
jeblairAJaeger: yeah, we have some things in project-config we can probably clean up after this is done by just making api.o.o an apache redirect18:20
openstackgerritPaul Hummer proposed openstack-infra/project-config: Add publish-to-pypi for the nova-lxd project to zuul  https://review.openstack.org/37028618:20
*** psachin has quit IRC18:20
jeblairAJaeger: so i'm just going to ignore api.o.o for now and plan on us having done that by the time we retire the ftp sites :)18:20
*** acoles is now known as acoles_18:20
*** jascott1_ has quit IRC18:21
AJaegersileht: why are you rebasing 369164 twice? There's no need to rebase at all unless you have a merge conflict18:22
*** jascott1 has joined #openstack-infra18:22
AJaegerjeblair, let's try that ;)18:22
fungior maybe the docs team will decide that a couple years of permanent redirecting is long enough to just retire api.openstack.org entirely18:22
*** rodrigods has quit IRC18:22
*** rodrigods has joined #openstack-infra18:22
*** pcaruana has joined #openstack-infra18:22
*** EricGonczer_ has quit IRC18:23
AJaegerfungi: Let's just do that.18:23
*** EricGonczer_ has joined #openstack-infra18:24
AJaegerjeblair: where do we have stuff for api.o.o in project-config? My grep does not show anything - is yours better?18:24
*** spzala_ has joined #openstack-infra18:25
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: CI test - never merge  https://review.openstack.org/37103418:25
silehtAJaeger, I have a branch with all Gnocchi changes I haven't send patches in correct order18:25
AJaegersileht: ah...18:25
*** ddieterly is now known as ddieterly[away]18:25
silehtAJaeger, and I have moved the one that was fixing the stable branch of Gnocchi on bootom18:26
AJaegersileht: ok, understood now18:26
*** annegentle has joined #openstack-infra18:26
*** sdake_ is now known as sdake18:26
*** dtardivel has quit IRC18:27
*** spzala_ has quit IRC18:27
odyssey4meproject config cores, I'd appreciate a second review of https://review.openstack.org/37064618:27
*** spzala has joined #openstack-infra18:28
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool: Use sequence znodes for image upload numbers  https://review.openstack.org/37104518:29
*** asettle has quit IRC18:34
dhellmanndoes this pypi upload error mean anything to anyone here? http://logs.openstack.org/42/4222da0a5911845dcaeef1e695915e010190245d/pre-release/murano-agent-pypi-both-upload/6787944/console.html18:34
*** kjorgens_ has quit IRC18:35
dhellmanna 400 error on the home_page?18:35
openstackgerritElizabeth K. Joseph proposed openstack-infra/puppet-statusbot: Add Twitter support  https://review.openstack.org/36823818:35
*** sarob has joined #openstack-infra18:36
*** tongli has joined #openstack-infra18:39
*** florianf has quit IRC18:40
*** maishsk has quit IRC18:41
*** maishsk has joined #openstack-infra18:41
*** _sarob has joined #openstack-infra18:41
dhellmannnevermind, I figured out the murano-agent issue18:42
fungidhellmann: what did it wind up being? i just started to dig into it18:42
*** itsuugo has quit IRC18:42
*** xarses has quit IRC18:42
dhellmannfungi : they have a bad home-page url in their setup.cfg (htt:// or some other invalid scheme)18:42
dhellmannfungi : https://review.openstack.org/37105018:43
fungigot it18:43
fungithat's a fun one18:43
mordredthat's awesome18:43
*** trown|lunch is now known as trown18:43
*** fguillot has quit IRC18:44
fungii had some notes at one point on a check we can implement to figure out if we have invalid dist metadata from pypi's perspective, but seem to have misplaced it now18:44
*** itsuugo has joined #openstack-infra18:44
fungidstufft: ^ is the set of things pypi will reject an upload over libified in any way we could dry-run locally?18:44
mordredfungi: seems like a good candidate for a thing to add to the pep8 check :)18:45
dstufftfungi: not currently18:45
*** sarob has quit IRC18:45
fungii think my earlier notes were only for checking that trove classifiers are valid18:45
*** yamahata has joined #openstack-infra18:46
fungidstufft: ahh, i guess if warehouse behaves similarly down the road, maybe that's a good candidate for abstracting into a lib18:46
*** florianf has joined #openstack-infra18:46
dstufftfungi: you're probably already using Warehouse to upload :D18:46
dstufftbut yea that'd probably make a reasonable thing to extract into a lib, similar to readme_renderer18:46
fungidstufft: we are, though still to the old legacy url (unless my change to upload to pypi.org has merged now)18:47
*** AnarchyAo has joined #openstack-infra18:47
fungibut yeah, i more meant "if this is a behavior warehouse retains when old cheeseshop pypi is no longer relevant"18:47
*** spzala has quit IRC18:47
*** maishsk has quit IRC18:48
jeblairAJaeger: there's at least one job that publishes to the 'jenkins' api site18:48
fungiat least for our use case, it would be awesome to know before we tag a new release whether what we're tagging is in a state that it will be uploadable, to keep from burning through additional releases just to figure out what's finally in shape to upload18:48
*** florianf has quit IRC18:50
*** fguillot has joined #openstack-infra18:50
*** fguillot has quit IRC18:51
*** baoli has joined #openstack-infra18:52
*** fguillot has joined #openstack-infra18:52
*** cardeois has quit IRC18:54
openstackgerritClark Boylan proposed openstack-infra/project-config: Run keystone jobs on xenial  https://review.openstack.org/37106018:54
clarkbstevemar: ^ thats what I had in mind18:54
*** cardeois has joined #openstack-infra18:55
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: CI test - never merge  https://review.openstack.org/37103418:57
*** maishsk has joined #openstack-infra18:58
clarkbAJaeger: if you are still around reviews on 371060 would be great18:59
* clarkb fixes the topic on it18:59
*** Rockyg has joined #openstack-infra18:59
stevemarclarkb: ah, not what i had in mind19:00
*** jascott1_ has joined #openstack-infra19:00
stevemarclarkb: i better go bug the qa folks19:00
openstackgerritClark Boylan proposed openstack-infra/project-config: Run keystone jobs on xenial  https://review.openstack.org/37106019:01
*** ijw has joined #openstack-infra19:03
*** e0ne has quit IRC19:03
*** jascott1 has quit IRC19:04
*** krtaylor has quit IRC19:07
*** salv-orlando has quit IRC19:07
*** matrohon has joined #openstack-infra19:07
*** abregman has joined #openstack-infra19:08
*** akshai has quit IRC19:09
openstackgerritMatt Riedemann proposed openstack-infra/project-config: Run neutron-linuxbridge job on nova libvirt/vif.py changes  https://review.openstack.org/37106519:10
mriedemsdague: ^ can you check my regex there?19:10
sdaguemriedem: sure, let me look19:10
openstackgerritClark Boylan proposed openstack-infra/project-config: Fix grenade job selection for newton  https://review.openstack.org/37106619:10
*** EricGonczer_ has quit IRC19:10
sdaguemriedem: you need an | in there19:10
*** akshai has joined #openstack-infra19:11
openstackgerritJeremy Stanley proposed openstack-infra/project-config: Revert "Pin bundler to 1.12.5 for some jobs"  https://review.openstack.org/37106719:11
mriedemsdague: i think that makes it infinite and hit nothing19:11
mriedemit's messed up19:11
clarkbstevemar: https://review.openstack.org/371066 should make grenade things better for you. Sorry about that19:11
mriedemit's a skip-if not19:11
clarkbsdague: maybe you can also look at ^19:11
fungimriedem: more if a skip-if knot19:11
fungier, of19:11
clarkbinfra-root 371066 would be a good one to get in before more projects get newton branches19:11
mriedemfungi: i see what you were trying there19:12
sdaguemriedem: explain in english what you are trying to do19:12
sdagueclarkb: looking19:12
fungiyeah, not my day for typing accuracy in jokes19:12
pleia2I think my statusbot patches for the twitter are a bit tangled up 368238 and 368239 (the second definitely depends on the first, but the first is also unhappy), any thoughts?19:12
stevemarclarkb: that makes it all sorts of better for me :)19:13
mriedemsdague: we want to conditionally run the linuxbridge job for nova changes in nova/network/* or nova/virt/libvirt/vif.py19:13
mriedemthe ! is because of the skip-if19:13
mriedemsomething like a skip-unless would mean we'd need to OR with |19:14
sdagueclarkb: https://review.openstack.org/#/c/371066/1/zuul/layout.yaml seems fine ... though I would change the negative assertion on branch to a possitive one so that it has to change less over time for stable19:14
sdagueso instead of  branch: ^(?!stable/(liberty|ocata)).*$19:14
mriedemskipUnless(nova/network/* OR nova/virt/libvirt/vif.py)19:14
*** rkukura_ has joined #openstack-infra19:14
clarkbsd ya we explicitly list mitaka newton and master19:14
sdaguebranch: stable/(mitaka|newton)19:14
mriedemwhat i'm trying to do is: skipIf(!(nova/network/* AND nova/virt/libvirt/vif.py))19:15
mriedems/AND/OR/ there :P19:15
openstackgerritClark Boylan proposed openstack-infra/project-config: Fix grenade job selection for newton  https://review.openstack.org/37106619:15
clarkbsdague: ^ that look better?19:15
sdagueclarkb: you are missing a /19:16
clarkbbah19:16
*** rkukura has quit IRC19:16
*** rkukura_ is now known as rkukura19:16
*** r-mibu has quit IRC19:16
openstackgerritClark Boylan proposed openstack-infra/project-config: Fix grenade job selection for newton  https://review.openstack.org/37106619:16
clarkbnow with more slash19:16
*** r-mibu has joined #openstack-infra19:17
sdagueclarkb: that seems correct to me19:17
sdagueok, back to mriedem's thing19:17
*** jimbaker has quit IRC19:17
*** jimbaker has joined #openstack-infra19:17
fungiclarkb: your todo comment on 371066 is now slightly skewed from reality, but i'm happy to approve in the current state unless you really feel strongly about correcting it19:19
*** tsg_ has quit IRC19:19
openstackgerritIsaku Yamahata proposed openstack-infra/project-config: networking-odl: add job for OpenDaylight carbon  https://review.openstack.org/36335619:19
fungii think it's close enough that someone else can figure out what you mean if they take over that task19:19
clarkbfungi: I don't I think that the commit message + the commetn should make the intent clear19:19
clarkbso anyone spelunking should sort it out without much trouble19:19
fungiapprovidado then19:20
sdaguemriedem: so, honestly, what about making it a new job name and not using the skip-if but instead use the files: thing to include it in19:21
sdaguethe skip-if / not logic hurts my head19:21
fungithat goes for me too19:21
*** abregman has quit IRC19:21
fungiskip-if is a bit too magical, and too easy to screw up19:22
mriedemmeh19:22
mriedemit works in http://www.regexpal.com/19:22
mriedemwith testing nova/compute/manager.py19:22
sdaguefungi: does files: patterns have the ability to scope to project?19:22
mriedemand nova/network/os_vif_util.py19:22
mriedemsdague: i think jeblair told me awhile back that it doesn't yet19:22
mriedemwhen i was originally adding what's already there19:22
fungisdague: ahh, no but making it a separate job would let you scope that job to a different project19:22
sdaguemriedem: yeh, it doesn't look like it in looking at layout19:22
sdaguefungi: yeh, that's what I'm thinking19:23
sdaguemriedem: that would be my suggestion, because I'm sure we're going to have to change this again, and this is way too much brain power to get right :)19:23
fungiby where i mean separately named job based on the same job template with no substantive alteration other than the job name19:23
*** mhickey has joined #openstack-infra19:23
*** matt-borland has quit IRC19:23
*** kjorgensen has joined #openstack-infra19:23
mriedemthat's why i test it on ^19:24
fungii don't feel so strongly about it that i downvote changes using skip-if, i simply let others take responsibility for reviewing those19:24
*** ddieterly[away] has quit IRC19:26
fungimostly because we've seen plenty of bugs slip into projects because they tried to get fancy with skipping jobs they didn't think would be influenced by changes to certain files, only to discover that either their assumptions or their patterns were wrong19:26
*** esikachev has quit IRC19:27
*** kjorgensen has quit IRC19:28
fungiand in particular, skip-if has been a magnet for people obsessing over premature optimization about when to run their jobs19:29
AJaegerjeblair: do you know which job?19:29
*** pcaruana has quit IRC19:32
*** kzaitsev_mb has joined #openstack-infra19:33
*** tsg_ has joined #openstack-infra19:33
*** ddieterly has joined #openstack-infra19:33
*** davidlenwell has quit IRC19:34
*** asettle has joined #openstack-infra19:34
*** cardeois has quit IRC19:37
*** jimbaker is now known as jimbaker_19:38
openstackgerritMerged openstack-infra/project-config: Fix grenade job selection for newton  https://review.openstack.org/37106619:38
*** tongli has quit IRC19:38
*** jimbaker_ is now known as jimbaker`19:39
*** jimbaker` is now known as jimbaker19:39
*** asettle has quit IRC19:40
*** david-lyle has quit IRC19:40
*** david-lyle has joined #openstack-infra19:40
*** Apoorva has quit IRC19:40
*** kjorgensen has joined #openstack-infra19:41
sdagueclarkb or anyone, we're trying to wrap our heads around this - http://logs.openstack.org/32/370132/4/check/gate-tempest-dsvm-full-ubuntu-xenial/8e4da50/logs/screen-n-cpu.txt.gz#_2016-09-15_14_54_49_53919:46
sdagueit seems like something is forcing the daemons to run in an ascii mode19:46
sdagueso printing unicode goes bonkers19:46
sdaguebut it's not locally reproducable, so I'm wondering if there is something in the gate that makes it that way19:47
*** davidlenwell has joined #openstack-infra19:47
sdagueI've been playing with some locale settings, and it's not really changing much19:47
*** cardeois has joined #openstack-infra19:48
*** sdake has quit IRC19:49
*** sdake_ has joined #openstack-infra19:49
clarkbI think jeblair decided not to set locale in zuul?19:50
*** esikachev has joined #openstack-infra19:50
*** jascott1_ has quit IRC19:51
pabelangerAh one point, it was handled by bindep19:51
pabelangerI think we had some discussions about adding it to DIB directly19:51
sdagueclarkb: ok, but even when I set it in stack.sh, which spawns all this, it still doesn't seem to take19:52
*** jascott1 has joined #openstack-infra19:52
sdaguehttps://review.openstack.org/#/c/370132/ actually gets all brutal about setting it19:52
*** jascott1 has quit IRC19:52
*** jascott1 has joined #openstack-infra19:52
*** rcernin has joined #openstack-infra19:53
*** jascott1 has quit IRC19:53
*** itsuugo has quit IRC19:53
clarkbthsts not for the stack user though right?19:53
*** jascott1 has joined #openstack-infra19:54
sdaguewhy wouldn't it be?19:54
clarkbwell you only write to the file if it exists19:55
*** itsuugo has joined #openstack-infra19:55
clarkbnot sure it exists19:55
jeblairclarkb, sdague: i'm not up to speed yet, but to perhaps clarify -- the zuul launcher attempts to give you the locale that you would get if you logged into the test node directly (like, on the console).19:56
clarkband everything else is user env for current user19:56
*** baoli has quit IRC19:56
jeblair(and of course the job can change it after that if it needs to)19:56
sdagueclarkb: but I also export it19:56
clarkbsdague: does that get into the stack user env?19:56
*** baoli has joined #openstack-infra19:56
*** krtaylor has joined #openstack-infra19:56
jeblair(but it will vary otherwise -- ie, you may get a different locale on fedora24 than you would on trusty)19:57
sdaguethis is the stack user19:57
clarkbI think its jenkins19:57
clarkbthen devstack makes stack later19:57
*** jascott1 has quit IRC19:57
sdaguejeblair: sure, I get this isn't a generic fix right now, I'm just trying to figure this one out19:57
sdagueclarkb: hmmmm19:57
sdagueclarkb: https://github.com/openstack-infra/devstack-gate/blob/e8b15e7627feceab7ffe9768a01a4b5650f2ca3f/devstack-vm-gate.sh#L631-L63319:58
sdaguelooks like stack user to me19:58
jeblairsdague: yep, i don't have an opinion on any fixes; i'm just trying to backfill context to (hopefully) help you out so you know what effect zuul might be having on this [hopefully none].  :)19:58
*** asettle has joined #openstack-infra19:58
sdaguejeblair: yeh, fair20:00
*** artmr has quit IRC20:00
clarkbhrm not sure then. if its starting as stack user and runs that way the exports should be sufficient. Its possible the images don't have that locale installed and thus callback to C or something?20:01
*** maishsk has quit IRC20:02
pabelangerthat is possible, we manage language-pack-en with bindep right now20:02
sdagueclarkb: well, the locale-gen seems to think it's a thing20:03
clarkbare we using screen? screen by default is not utf8 friendly (or wasn't for forever maybe modern screen is better)20:04
sdagueclarkb: not in the gate20:04
sdaguealso, screen locally works fine20:04
clarkbyou ahve to do screen -U iirc otherwise it doesn't unicode20:05
fungilater screen packages are like running screen -U by default if memory serves20:05
sdagueright, but we stopped using screen in the gate and we just launch with bash directly20:05
sdaguebecause screen command push sometimes doesn't work20:05
fungihrm, actually manpage for screen on trusty at least seems to indicate -U is still not default behavior... but regardless as sdague says not related to this20:06
*** Apoorva has joined #openstack-infra20:06
*** mhickey has quit IRC20:06
*** Apoorva_ has joined #openstack-infra20:07
*** xarses has joined #openstack-infra20:07
fungiso... for unit test jobs and the like, our bindep fallback list installs language-pack-en explicitly http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/data/bindep-fallback.txt#n2920:07
clarkblooking at logs for ps4 I don't see where any of this is getting logged20:07
sdagueclarkb: which ?20:08
clarkbsdague: the locale-gen20:08
sdagueit's in devstack-early20:08
sdagueit's before devstack starts explicit logging20:08
sdaguehttp://logs.openstack.org/32/370132/4/check/gate-tempest-dsvm-full-ubuntu-xenial/8e4da50/logs/devstack-early.txt.gz20:08
clarkbcan we get that to be set -x ?20:08
clarkblooks like we just have the output to go by20:09
sdagueit is20:09
sdagueor it should be20:09
sdaguehttps://github.com/openstack-dev/devstack/blob/233fce71da6d58d39b40b8c073c78c0e58ea8e16/stack.sh#L2520:09
clarkblooks liek its all split up20:09
clarkbthe end of the file has the commands and the beginning has the output?20:10
*** Apoorva has quit IRC20:10
fungii don't see language-pack-en in any of the packages lists in devstack20:10
fungiso it's almost certainly not installed unless it gets dragged in as a dependency of some other package20:11
*** ijw has quit IRC20:11
*** ijw has joined #openstack-infra20:11
sdaguefungi: ok, I can try installing that. You think that will impact things?20:12
clarkbC.UTF-8 is what we set it to on our xenial instances in /etc/default/locale20:12
fungibut yeah, localegen seems to act like it is at least, so maybe it's a dep of something else20:12
sdaguefungi: it's not in the package list20:12
sdaguehttp://logs.openstack.org/32/370132/4/check/gate-tempest-dsvm-full-ubuntu-xenial/8e4da50/logs/dpkg-l.txt.gz20:12
clarkband if I su to stack it has C.UTF-820:13
fungisdague: i believe we put language-pack-en in our bindep fallback list because it was preinstalled on rackspace's base images but wasn't on our dib-based ubuntu-minimal images20:13
clarkbso it should be running in a unicoded env by default20:13
*** asselin__ has joined #openstack-infra20:13
sdagueclarkb: ok, so I don't understand why we'd get an ascii decode error20:14
clarkbsdague: where is the error?20:14
sdaguein the original link20:14
fungiand yeah, default on xenial will be C.utf-8 because that's built into the libc20:14
sdaguehttp://logs.openstack.org/32/370132/4/check/gate-tempest-dsvm-full-ubuntu-xenial/8e4da50/logs/screen-n-cpu.txt.gz#_2016-09-15_14_54_49_53920:14
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: CI test - never merge  https://review.openstack.org/37103420:14
clarkbI am going to guess beacuse something somewhere has decided to treat that as a string not a u'' string20:15
clarkbso python is doing its best then failing20:15
sdagueclarkb: ok, except, it's not locally reproducable unless you force to a non utf8 locale20:16
sdaguethis is because the instance name in a debug message is utf820:16
*** ijw has quit IRC20:16
*** rhallisey has quit IRC20:16
sdaguehttps://bugs.launchpad.net/nova/+bug/158072820:17
openstackLaunchpad bug 1580728 in OpenStack Compute (nova) "UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 386: ordinal not in range(128) in nova.virt.libvirt.vif:unplug with unicode instance.display_name" [Medium,Triaged]20:17
*** asselin_ has quit IRC20:17
*** asettle has quit IRC20:18
fungiyeah, given this is xenial which will have a utf-8 locale available even if you don't install any optional locales, i doubt it's an issue of a missing package20:19
clarkblooking at /proc/$pid/environ for a nova compute started by devstack on an infra xenail instance you have LC_ALL=C and LC_MESSAGES=C.UTF-820:20
clarkbI don't see any other lang/locale vars20:20
*** hamzy has quit IRC20:20
clarkbso it is getting C somehow for LC_ALL20:20
fungithe default stackrc has it20:21
*** kzaitsev_mb has quit IRC20:21
fungior does in my checkout20:21
*** xarses has quit IRC20:21
clarkboh right thats what sdague's chagne changes20:22
clarkbso I'd need to inspect the hosts his change is running on /me looks20:22
fungioh, it moved to stack.sh20:22
fungihttps://git.openstack.org/cgit/openstack-dev/devstack/tree/stack.sh#n3420:22
*** kgiusti has left #openstack-infra20:22
sdaguefungi: right, but I'm changing this20:22
fungiaha20:23
sdaguethat's the point of this patch20:23
clarkbthe d-g projects has LC_ALL=C.UTF-820:23
sdagueto try to change that so we don't push back to LC_ALL=C, but my approaches do not seem to be taking20:23
clarkbso it starts the way we expect it to at least20:23
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Don't use cached images in tripleoclient changes  https://review.openstack.org/37111520:24
sdagueyeh20:24
fungiyeah, it's going to be fiddly at a minimum... ubuntu-trusty, ubuntu-xenial, centos-7 and fedora-24 are all going to behave differently here i think20:24
clarkbthe stack.sh's have that too so they look good that far along20:25
clarkbno nova processes yet though20:25
fungibecause there's been a lot of upheaval in distros over the past few years on what locales to make available by default and whether to guarantee an available utf-8 locale20:25
sdaguefungi sure20:25
sdaguebut I can't seem to eliminate this in the "easy" case20:25
fungiso the default locale on one won't even be a viable locale on another, and one might require packages to make utf-8 available at all while another won't20:26
sdaguealso, I can't reproduce the issue with a base ubuntu20:26
fungitried setting LC_ALL=C.utf-8 ? i expect that to work on xenial at least20:26
sdaguefungi: right, that's fine, I'm trying to figure out what that combination of things is. Because honestly, if we aren't running in a utf8 supporting mode in our services, that's kind of problematic20:26
sdaguefungi: I did not yet, because it looks like en_US.UTF-8 is there20:27
clarkbapache is reseting keystone to LANG=C20:27
openstackgerritEmilien Macchi proposed openstack-infra/project-config: Revert "tripleo: disable voting on gate-tripleo-ci-centos-7-nonha-multinode"  https://review.openstack.org/37113320:28
*** fguillot has quit IRC20:28
clarkbstill waiting on the nova processes20:28
*** _sarob has quit IRC20:28
openstackgerritThomas Goirand proposed openstack-infra/project-config: Revert "Temporarily remove deb-novnc from projects.yaml"  https://review.openstack.org/37113420:28
sdagueclarkb: ok, that's interesting, and something we should probably fix20:28
*** abregman has joined #openstack-infra20:28
zigofungi: ^20:28
clarkbsdague: I checked that the root apache process is lang=c too and it is20:28
fungioh, that too. apache will expect very explicit locale configuration if you want anything other than c ascii behavior20:28
sdaguefungi: sure20:28
sdaguethis is n-compute though20:28
sdagueit's an evently daemon20:28
clarkbya waiting for devstack to start that one20:28
sdagueeventlet20:29
*** jascott1 has joined #openstack-infra20:29
*** fguillot has joined #openstack-infra20:29
sdaguefungi: but it is a good point to put on the list. It was also something that came up in the ops summit. Our utf8 testing is definitely pretty spotty, so there ends up being commands where people set users or projects with utf8 chars, and then api calls or clis sometimes crash20:30
sdagueso I wanted to make sure we were utf8 fully in devstack at least to make it easier to even know20:30
fungisdague: i completely agree we should be figuring out whatever is needed to test utf-8 end-to-end20:30
*** kzaitsev_mb has joined #openstack-infra20:31
*** sarob has joined #openstack-infra20:32
fungilooking at that exception, which of those has the non-ascii content? is it the format string or one of the args list?20:32
clarkbfungi: its the instance name apparently so oen of the args list I think20:32
*** ijw has joined #openstack-infra20:33
*** abregman has quit IRC20:33
fungiyeah, i never can remember which direction that particular decoding exception indicates20:33
clarkbneutron server is running with LC_ALL=en_US.UTF-820:33
sdaguefungi: https://github.com/openstack/nova/blob/bad2497b70605b66ada28960d9ccd9796a0867ee/nova/virt/libvirt/vif.py#L1009-L1012 this is what's triggering it20:33
sdaguethe instance has a unicode name when that happens20:34
clarkband nova compute too20:34
clarkbso its working fine20:34
*** sarob has quit IRC20:35
* clarkb will paste the full env stuff for nova compute in a minute20:35
*** zzzeek has quit IRC20:35
sdagueclarkb: ok, well lets see what the end of the run looks like20:35
*** esikachev has quit IRC20:35
sdaguethat iteration had the additional package install20:36
clarkbhttp://paste.openstack.org/show/577708/20:36
*** sarob has joined #openstack-infra20:37
*** xarses has joined #openstack-infra20:37
fungii wonder if making that u'vif_type=... would alter the behavior at all20:37
*** EricGonczer_ has joined #openstack-infra20:37
*** _sarob has joined #openstack-infra20:37
clarkbya thats what I was saying20:37
*** Apoorva_ has quit IRC20:37
*** ijw has quit IRC20:38
clarkbyou are trying to put unicode in a bytestring. Not sure if python will like that20:38
fungior a liberal helping of six.texttype20:38
*** Apoorva has joined #openstack-infra20:38
sdagueso if that was the case, I would think you'd see this locally as well20:38
*** ddieterly is now known as ddieterly[away]20:39
sdaguealso, that's going to suck a lot if we need to u'' every string20:39
fungihaving a format string interpolate unicode parameters totally works as written there in py3k, but not so simple in 2.720:39
sdagueI can run that experiment in parallel20:39
*** zzzeek has joined #openstack-infra20:39
fungii mean, this (unicode support by default) is one of the main reasons why py3k happened20:40
sdaguefungi: sure20:40
fungiand making this work consistently between python 2 and 3 is why six.texttype exists20:40
*** baoli has quit IRC20:41
*** sarob has quit IRC20:41
*** baoli has joined #openstack-infra20:41
*** jkilpatr has quit IRC20:41
fungia pyton mud codebase i work on, before switching to py3k-only, just basically used u'whatever' everywhere to get things working unicode-clean20:42
clarkbI just reproduced it fwiw20:42
sdagueclarkb: ok, cool20:42
sdaguehow?20:42
*** EricGonczer_ has quit IRC20:42
*** tkelsey has joined #openstack-infra20:42
clarkbhttp://paste.openstack.org/show/577709/ like that20:42
clarkboh except thats backwards20:43
clarkbencode vs decode20:43
clarkbanyways I think that shows that this may just be a python issue20:44
fungibut i agree it's strange that 370132 is working locally for sdague (unless somehow the test with the unicode instance name is somehow being skipped?)20:44
*** askb has joined #openstack-infra20:44
*** Goneri has quit IRC20:45
sdagueclarkb: ok, if it's going through format, then that's a thing20:45
sdaguebut if I used % it's not20:46
sdagueI wonder if it has to do with log defered formating20:46
*** tkelsey has quit IRC20:47
sdagueanyway, kind of end of day here, I'm going to experiment with this to smooth it over20:47
*** itsuugo has quit IRC20:47
fungialso if you don't u the snowman definition, it seems to not raise an exception20:47
sdaguefungi / clarkb thanks for all the ideas here20:47
sdaguefungi: right20:47
sdagueIn [1]: u = u'☃'20:48
sdagueIn [2]: a = "%s" % u20:48
sdagueIn [3]: print a20:48
sdague20:48
fungii think because then it's just being treated as a bytestring20:48
*** kzaitsev_mb has quit IRC20:48
*** Rockyg has quit IRC20:48
fungie.g. '\xe2\x98\x83' in my terminal at least20:48
*** trown is now known as trown|outtypewww20:48
clarkbfungi: yup and I think that because the error is on a decode not an encode the input is a bytestring that its trying to envode into ascii20:49
clarkber thats not right. Thats its triny to decode as ascii to make an internap python object20:49
mgagneinternap python object :O20:49
clarkband I am guessing that the loggers assume bytestrings are ascii20:49
clarkbmgagne: er internal20:50
*** itsuugo has joined #openstack-infra20:50
jeblairfungi, pabelanger, mordred: i just tried to launch a node.  ansible timed out with no debug info.  do you have suggestions as to how to proceed?20:50
fungiyep. basically if you have utf8 byte sequences you can interpolate them into a non-unicode format string all day long. if you have a (declared as) unicode string you can't just interpolate it into a bytestring without an explicit encoding20:50
fungijeblair: mordred tracked it down to something having to do with broken newer ansible again?20:51
jeblairfungi, pabelanger, mordred: the first thing i noticed is that we don't output anything from ansible until the end20:51
fungiwhen i ran it with an earlier ansible from a virtualenv it worked for me last week20:51
jeblairso i couldn't even tell what task it was hung on until it completely timed out20:51
jeblairfungi: should we pin ansible on puppetmaster?20:52
fungijeblair: probably. i thought mordred was reporting something upstream20:52
fungii'm looking for details on what worked for me20:52
jeblairi would like us to start treating this as something that needs to work all the time20:52
clarkbsdague: ya if I do b = b'☃' ; s = "%s" % b ; s.decode() I get that error20:53
fungijeblair: i agree. looks like it worked for me with 2.1.1.020:53
*** baoli has quit IRC20:53
clarkbso that would be my hunch, we have raw utf8 in a bytestring rather than a unicode string and its getting munged on its way through logging20:53
*** ijw has joined #openstack-infra20:53
*** raildo has quit IRC20:53
fungijeblair: my concern is whether we've grown a dependency on any post-2.1.1.0 ansible features for other things run from the puppetmaster now20:53
ianwjeblair / (future) AJaeger : i'm going to call the sync to old-docs done ... i'm running it again with "newer only" but it seems to be making odd choices about what to download.20:54
fungibut i guess we can try downgrading and find out20:54
*** akshai has quit IRC20:54
*** gouthamr has quit IRC20:54
clarkbfungi: I doubt it, we were running 2.0.1 for a long time just fine20:54
fungijeblair: oh, wait...20:54
fungii used _newer_ ansible?20:54
fungi2.0.2.0 is what is installed system-wide on the puppetmaster right now20:54
fungiso i guess it's a question of whether we can safely upgrade to 2.1.1.0 or need to downgrade to something before 2.0.2.020:55
*** xarses has quit IRC20:55
*** baoli has joined #openstack-infra20:55
clarkbmordred was supposed to figure this out a couple weeks ago iirc so that puppetmaster and zuul could use the same ansible version20:55
clarkbbut not sure if that happened20:55
fungithinking back, mordred may have said it looked like 2.0.2.0 was failing on a bug which was fixed upstream, and then when i tested with 2.1.1.0 things worked20:56
fungi(for launch-node.py)20:56
*** nherciu has quit IRC20:57
mordredyes20:57
mordredthat is my memory - we are currently pinned back, and there is a fix in newer ansible which fixes the problem we were seeing a few days ago whenever we looked in to this last20:58
fungiso the question is can we upgrade ansible on puppetmaster to 2.1.1.0? that would fix this at least20:58
*** ijw has quit IRC20:58
*** _sarob has quit IRC20:58
clarkbpabelanger's fix for types to make 2.1.1.0 work did merge but not sure if we have tested that it works as a whole20:58
*** kzaitsev_mb has joined #openstack-infra21:00
mordredcommit c6c51f90f120345aad01331837b894d00e5bff26 in puppet-ansible21:00
mordredis the one where we pinned it21:00
mordred    * Our group hiera files are not being copied over.21:00
mordred    * ansible_module_puppet_post_puppetdb.py causes a "MODULE FAILURE".21:00
mordredare the two reasons we pinned21:00
*** rfolco has quit IRC21:02
fungijeblair: oh, and the other thing which helped was that i was running with ianw's stack of fixes up to 359500 applied. i'm good with them but they could use someone approving (or i could approve them now and call his being a core reviewer good enough not to wait for another reviewer)21:02
fungijeblair: improvements for debuggability21:03
jeblairfungi: ah.  i'm making some major changes to that right now21:03
jeblairfungi: i will build upon ianw's changes21:03
fungiawesome21:03
fungianyway, i need to break away and do a pass at the yard before it gets too late21:03
jeblairfungi: if you feel like +3ing them that will be fine21:04
jeblairotherwise i can :)21:04
fungioh, looks like mordred just did21:04
fungiso, er, +6!21:04
mordredianw: I have a question about a comment in https://review.openstack.org/#/c/359499/1/launch/launch-node.py21:04
mordredianw: (I started to ask you this a few days ago, then lost the train of thought)21:04
*** eharney has quit IRC21:05
*** matrohon has quit IRC21:05
mordredianw: Remove cloud and region from the environment to work around a bug in occ ...21:05
mordredianw: I'd like to understand more about that?21:05
*** EricGonczer_ has joined #openstack-infra21:06
fungii've removed my approval on 359499 until mordred gets a 'splaination21:06
fungihe can +3 when he's ready on that one21:06
mordredfungi: oh - it's ok ... we can approve that ... I just want to make sure I don't not fix an occ bug21:06
*** kzaitsev_mb has quit IRC21:06
mordredthis patch is fine21:06
*** psilvad has quit IRC21:06
fungiokay, approval readded21:06
fungiand now, the lawn21:06
fungibbiaw21:07
*** ddieterly[away] is now known as ddieterly21:08
*** fguillot has quit IRC21:09
*** gouthamr has joined #openstack-infra21:09
*** ijw has joined #openstack-infra21:09
*** fguillot has joined #openstack-infra21:10
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: Use popen rather than check_output  https://review.openstack.org/37115821:10
kfox1111question... http://logs.openstack.org/53/371153/1/check/gate-kolla-kubernetes-python27-ubuntu-xenial/2f72774/console.html21:10
jeblairianw, fungi, mordred: ^ untested changes to launch node21:10
jeblairi will try now21:10
kfox1111its failing due to timeout.21:10
*** kzaitsev_mb has joined #openstack-infra21:10
kfox1111it normally takes 22 seconds on my machine to run.21:10
openstackgerritMerged openstack-infra/system-config: launch-node.py: set ansible log path  https://review.openstack.org/35949921:10
*** akshai has joined #openstack-infra21:10
kfox1111are things really backed up? is there some how to make the test run longer?21:11
*** salv-orlando has joined #openstack-infra21:11
*** jkilpatr has joined #openstack-infra21:11
jeblairmordred: so how should i run launch-node (what's the resolution to the ansible issue?)21:11
pleia2heading out for a bit, bbl21:12
*** hashar has quit IRC21:12
jeblairokay,21:13
clarkbkfox1111: edit the timeout for your unittests in your tests21:13
jeblairi guess the resolution is to use a virtualenv?21:13
openstackgerritMerged openstack-infra/system-config: launch-node.py: More verbose logging  https://review.openstack.org/35950021:13
jeblairi will add that documentation to launch-node21:13
mordredjeblair: to my knowledge, 2.1.1.0 is the thing that was used last time this was done21:13
kfox1111clarkb: know off hand how to do that?21:14
clarkbkfox1111: grep for TIMEOUT and see where its set21:14
mordredjeblair: yah - I think until there is more consensus on running 2.1.1.0 generally on puppetmaster21:14
mordredjeblair: virtualenv is the best bet21:14
clarkbkfox1111: its a per project thing so no I dont know how kolla does it21:14
kfox1111is it in the tox config, or in the project-config?21:14
clarkbkfox1111: but you can see it in the test runner commands21:14
clarkbkfox1111: its not in project co fig its in your project21:15
*** hashar has joined #openstack-infra21:15
*** hashar has quit IRC21:15
*** hashar has joined #openstack-infra21:15
clarkbOS_TEST_TIMEOUT is the var according to that job log21:15
mordredjeblair: your untested change makes sense to me21:15
clarkbso grep for that21:15
jeblairmordred: cool, i will test it now in a virtualenv21:15
*** hashar has quit IRC21:15
mordredjeblair: woot21:15
*** burgerk has quit IRC21:15
*** ldnunes has quit IRC21:16
*** hashar has joined #openstack-infra21:16
mordredjeblair: I eagerly await word21:16
* amrith send mordred a scrabble dictionary21:16
kfox1111ah. .testr.conf it looks like.21:17
*** ccamacho has quit IRC21:17
*** krtaylor has quit IRC21:17
kfox1111clarkb: thx.21:17
*** kzaitsev_mb has quit IRC21:18
*** salv-orlando has quit IRC21:18
*** rossella_ has quit IRC21:18
*** aeng has joined #openstack-infra21:18
*** asettle has joined #openstack-infra21:19
*** rossella_ has joined #openstack-infra21:19
*** fguillot has quit IRC21:21
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: document using a virtualenv for launch-node  https://review.openstack.org/37116221:21
*** jascott1 has quit IRC21:21
jeblairmordred: ^21:21
*** salv-orlando has joined #openstack-infra21:22
EmilienMwe think tripleo multinode job is stable again, please review this revert so our gate is voting again https://review.openstack.org/#/c/371133/ - thanks!21:22
EmilienMslagle: fyi ^21:22
mordredOH!21:22
mordredI understand from this patch what the occ bug is21:22
mordredalthough it's not really an occ bug21:22
mordredI mean, it sort of is21:23
ianwmordred: heh, that was there when i got there :)21:23
*** hashar has quit IRC21:23
ianwlet's go digging21:23
*** ddieterly is now known as ddieterly[away]21:24
*** asettle has quit IRC21:24
*** ijw has quit IRC21:25
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: Use popen rather than check_output  https://review.openstack.org/37115821:25
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: document using a virtualenv for launch-node  https://review.openstack.org/37116221:25
ianwmordred: i looks like you added it :) https://review.openstack.org/#/c/296127/21:25
ianwit21:25
*** EricGonczer_ has quit IRC21:25
mordredianw: ah - I believe we have solved that occ bug now :)21:26
mordredbut there is another bug here21:26
*** kzaitsev_mb has joined #openstack-infra21:26
*** gyee has joined #openstack-infra21:27
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: Use popen rather than check_output  https://review.openstack.org/37115821:27
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: document using a virtualenv for launch-node  https://review.openstack.org/37116221:27
*** fguillot has joined #openstack-infra21:27
mordredianw: OH HOLY CRAP that bug still exists. wtf21:27
* mordred cries21:27
*** esberglu has quit IRC21:28
* jeblair hands mordred a fluffy kitty to dry his eyes21:28
*** ddieterly[away] is now known as ddieterly21:28
*** adriant has joined #openstack-infra21:28
jeblairmind the claws21:28
*** sarob has joined #openstack-infra21:30
*** piet has quit IRC21:30
amrith:)21:31
*** tsg__ has joined #openstack-infra21:32
*** rvasilets___ has joined #openstack-infra21:33
*** tsg_ has quit IRC21:34
*** jascott1 has joined #openstack-infra21:34
pabelangerjust catching up on backscroll21:36
*** kzaitsev_mb has quit IRC21:37
pabelangerguess we don't use PYTHONUNBUFFERED = 1 for launch-node.py21:38
*** jascott1 has quit IRC21:39
*** jascott1 has joined #openstack-infra21:39
jeblairpabelanger: no it's because of subprocess.check_result21:42
jeblairer check_output21:42
jeblairpabelanger: see 37115821:42
pabelangerAh21:42
*** jascott1 has quit IRC21:42
jeblairsubprocess.CalledProcessError: Command '['ansible-playbook', '-i', '/tmp/tmpJYRagc/inventory', '-l', 'files01.openstack.org', '--private-key=/tmp/tmpJYRagc/id_rsa', "--ssh-common-args='-o StrictHostKeyChecking=no'", '-e', 'target=files01.openstack.org', './../playbooks/remote_puppet_adhoc.yaml']' returned non-zero exit status 121:43
jeblairhrm21:43
jeblairit still timed out even running within the virtualenv21:43
jeblairthis time i got to see that it was taking a long time21:43
jeblairbut of course, i don't have any output from ansible21:43
jeblairso i'm no closer to understanding the failure than i was an hour ago, except now i knew it was running puppet earlier.21:44
*** tkelsey has joined #openstack-infra21:44
*** kzaitsev_mb has joined #openstack-infra21:44
openstackgerritMonty Taylor proposed openstack/os-client-config: Don't create envvars cloud if cloud or region are set  https://review.openstack.org/37117321:44
jeblairmordred: do you have any further suggestions?21:45
*** samyewell has quit IRC21:45
*** TheJulia has quit IRC21:45
*** jraim has quit IRC21:45
*** asaprykin has quit IRC21:45
*** skath has quit IRC21:45
*** mattfarina has quit IRC21:45
*** vaidy has quit IRC21:45
*** isviridov_away has quit IRC21:45
pabelangerwon't be able to have more tonight, but I'd add --keep to then check syslog on the server. maybe something in puppet apply is failing21:45
pabelangerneed to run out for some errands21:45
jeblairpabelanger: i already tried logging into the server21:45
jeblairi was unable to21:45
pabelangerOh, missed that21:45
*** TheJulia has joined #openstack-infra21:45
*** scottda has quit IRC21:45
*** sweston has quit IRC21:45
*** ogelbukh has quit IRC21:45
*** agordeev has quit IRC21:45
*** hyakuhei has quit IRC21:45
*** adam_g has quit IRC21:45
*** knikolla has quit IRC21:45
jeblairpabelanger: oh, you mean with the temp key?21:45
jeblairi didn't try taht21:46
pabelangerjeblair: right21:46
*** kuromagi^ has joined #openstack-infra21:46
mordredjeblair: if that doesn't shed light - also adding a -vvvv to the playbook invocation parameters might get more info too21:46
*** kuromagi has quit IRC21:46
*** cargonza has quit IRC21:46
*** rm_work has quit IRC21:46
*** AJaeger has quit IRC21:46
*** bmjen has quit IRC21:46
*** mjb has quit IRC21:46
*** craige has quit IRC21:46
*** madorn has quit IRC21:46
*** cj has quit IRC21:46
*** JayF has quit IRC21:46
*** jamielennox has quit IRC21:46
*** mhayden has quit IRC21:46
*** alexm2 has quit IRC21:46
jeblairwe have to find a way to get the failed output off of the machine21:46
mordredyes. I agree21:47
*** amit213 has quit IRC21:47
*** ashtokolov has quit IRC21:47
*** jhesketh has quit IRC21:47
*** mattoliverau has quit IRC21:47
*** dkehn has quit IRC21:47
*** gus has quit IRC21:47
*** d34dh0r53 has quit IRC21:47
*** ryanpetrello has quit IRC21:47
*** mrhillsman has quit IRC21:47
*** vkmc has quit IRC21:47
*** tbarron has quit IRC21:47
mordredI'm curious if ansible is hanging trying to ssh to the machine21:47
*** matt6434 has joined #openstack-infra21:47
mordredwhich I think we'd see with -vvvv21:47
jeblaircause this is a 30 minute timeout (for you to learn you need to run --keep), and then another up to 30 minutes when you run it the second time with --keep21:47
jeblairis not cloudy21:48
mordredyes. totally agree.21:48
*** pradk has quit IRC21:48
*** tkelsey has quit IRC21:48
*** gus has joined #openstack-infra21:48
*** cj has joined #openstack-infra21:48
jeblairi just have no idea how to do that with ansible21:48
jeblairmaybe we need to have launch-node open a second ssh connection21:48
jeblairand stream syslog21:48
jeblairi will do that21:48
*** samyewell has joined #openstack-infra21:49
*** scottda has joined #openstack-infra21:49
*** sweston has joined #openstack-infra21:49
*** jhesketh has joined #openstack-infra21:49
*** jascott1 has joined #openstack-infra21:49
*** jraim has joined #openstack-infra21:49
mordredjeblair: I kind of feel like what we really want is to run ansible the way we do with zuul-launcher21:49
*** skath has joined #openstack-infra21:49
*** matt6434 is now known as mattoliverau21:49
mordredbut that's likely not going to happen in the next 15 minutes21:49
mordredso I think your second ssh connection streaming syslog is a great idea21:50
jeblairmordred: well, that's where all that code i just added came from :)21:50
mordred:)21:50
jeblairmordred: but the issue here is the 'main puppet run'21:50
*** agordeev has joined #openstack-infra21:50
jeblairmordred: where lots of black magic happens inside of one ansible task21:50
*** ogelbukh has joined #openstack-infra21:50
mordredok - so we are certain it's getting as far as running puppet21:50
mordredyah21:50
*** jascott1 has quit IRC21:50
*** ashtokolov has joined #openstack-infra21:50
jeblairmordred: oh yeah, we're past all the hier, env stuff and on to "TASK [puppet : run puppet] *****************************************************21:50
mordredkk21:51
*** asaprykin has joined #openstack-infra21:51
*** adam_g has joined #openstack-infra21:51
jeblairmordred: so it's the familiar "ansible command output" problem21:51
*** amit213 has joined #openstack-infra21:51
*** bmjen has joined #openstack-infra21:51
*** adam_g has quit IRC21:51
*** adam_g has joined #openstack-infra21:51
jeblairmordred: which we might be able to solve with an action plugin :)21:51
mordredjeblair: I would offer to shell in and help more directly than just making random suggestions, but I go to airport in about 20 minutes21:51
*** kzaitsev_mb has quit IRC21:51
*** rm_work has joined #openstack-infra21:51
*** d34dh0r53 has joined #openstack-infra21:51
*** craige has joined #openstack-infra21:52
*** ryanpetrello has joined #openstack-infra21:52
openstackgerritIan Wienand proposed openstack-infra/project-config: Revert "Revert "Switch in Fedora 24 devstack job""  https://review.openstack.org/37048321:52
*** mjb has joined #openstack-infra21:52
jeblairmordred: is it worth giving that a spin now in this context, or does it need a little more hacking first?21:52
*** isviridov_away has joined #openstack-infra21:52
mordredjeblair: but yah - that's sort of what I was thinking ... something tells me what we want when running zuul is _also_ useful in ops environments21:52
*** tbarron has joined #openstack-infra21:52
jeblairmordred: yes21:52
*** vaidy has joined #openstack-infra21:52
mordredjeblair: the action plugin I have now will not help this21:52
*** cargonza has joined #openstack-infra21:52
*** ijw has joined #openstack-infra21:52
mordredit _will_ in the future tense21:52
*** knikolla has joined #openstack-infra21:52
mordredbut it is not in shape for you to use at the moment21:52
jeblairmordred: ok.  couldn't remember if you'd done the output bits yet21:52
mordrednot yet21:52
mordredplane hacking FTW!21:52
*** hyakuhei has joined #openstack-infra21:53
jeblairmordred: gotcha.  so i will ssh/syslog so our sysadmins can launch nodes, and we will have nice things later.21:53
mordredI like having nice things21:53
*** mattfarina has joined #openstack-infra21:54
cloudnullif im not seeing our test prefix here http://status.openstack.org/openstack-health/#/tests does it mean we've done something wrong, or missed a step somewhere?21:55
* cloudnull looking for openstack-ansible21:55
clarkbI think you have to have subunit log files21:56
*** kzaitsev_mb has joined #openstack-infra21:56
*** jheroux has quit IRC21:57
openstackgerritMonty Taylor proposed openstack-infra/system-config: Use OS_REGION_NAME so that OCC ignore it properly  https://review.openstack.org/37117621:57
clarkbprobably in the same location as in the dsvm jobs21:57
mtreinishcloudnull: the only ansible job I see data for is: http://status.openstack.org/openstack-health/#/g/project/openstack~2Fansible-role-cloud-launcher21:57
mordredjeblair: ^^ change to the README to make things friendlier for when the occ change lands21:57
*** vkmc has joined #openstack-infra21:57
cloudnullclarkb:  that may be21:57
mtreinishcloudnull: clarkb is right you need to make sure your test runs put a testrepository.subunit file either in the log output either in the base dir or logs/21:58
*** mriedem has quit IRC21:58
mtreinishother wise the subunit worker won't be able to populate the subunit2sql db with the test results21:58
cloudnullodyssey4me: ^21:58
cloudnulli think we need to make some adjustments21:58
*** csomerville has quit IRC21:58
cloudnull:) thanks mtreinish clarkb21:59
*** csomerville has joined #openstack-infra21:59
*** madorn has joined #openstack-infra21:59
mtreinishcloudnull: this is where things look for the subunit file: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/logstash/jenkins-log-client.yaml#n1421:59
odyssey4meah ok, interesting - good to know how the facility works21:59
*** alexm2 has joined #openstack-infra21:59
*** 32NABYP9K is now known as sabari21:59
*** dkehn has joined #openstack-infra21:59
*** JayF has joined #openstack-infra21:59
* odyssey4me will leave cloudnull to figure that one out21:59
odyssey4me:p21:59
*** mrhillsman has joined #openstack-infra22:00
mtreinishodyssey4me, cloudnull: do you have an example log link I can look at? Just curious to see what it's doing now22:00
*** AJaeger has joined #openstack-infra22:00
*** mhayden has joined #openstack-infra22:00
cloudnullhttp://logs.openstack.org/60/371160/2/check/gate-openstack-ansible-galera_server-ansible-func-ubuntu-trusty/877e3bf/22:01
*** jamielennox has joined #openstack-infra22:01
clarkbany other project-config root want to help get keystone jobs running on xenial ? https://review.openstack.org/#/c/371060/22:01
*** jascott1 has joined #openstack-infra22:01
openstackgerritJonathan Como proposed openstack-infra/jenkins-job-builder: Implement a publisher for HockeyApp  https://review.openstack.org/37117822:02
jeblairooh, the syslog tailing is working22:02
*** yamamoto has joined #openstack-infra22:02
*** akshai has quit IRC22:02
*** yamamoto has quit IRC22:02
*** kzaitsev_mb has quit IRC22:03
mtreinishcloudnull: ah ok, thanks. That's what I guessed was going on. You guys just use ansible to do things and not a test runner (which totally makes sense)22:03
mtreinishcloudnull: I don't think ansible has native subunit support (although you could add a module for that I guess)22:03
*** rtheis has quit IRC22:03
mtreinishcloudnull: or you could use generate-subunit like devstack does22:03
mtreinishcloudnull: http://docs.openstack.org/developer/os-testr/generate_subunit.html22:03
*** krtaylor has joined #openstack-infra22:04
*** esberglu has joined #openstack-infra22:04
openstackgerritJonathan Como proposed openstack-infra/jenkins-job-builder: Fix reference repo configuration for Git SCM  https://review.openstack.org/37118122:04
mordredjeblair: yay!22:04
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: stream syslog on remote host  https://review.openstack.org/37118222:04
jeblairianw, mordred: ^22:04
mordredjeblair: that sounds like a wonderful improvement22:04
jeblairSep 15 22:03:09 files01 puppet-user[21891]: (/Stage[main]/Openafs::Client/Package[kstart]/ensure) ensure changed 'purged' to 'present'22:05
jeblairi'm wondiring if it's a dkms build or something22:05
mordredoh. yeah. that might take a second22:05
cloudnullthanks mtreinish i'll look into generating that22:05
jeblairmordred: or 1800?22:05
*** tphummel has quit IRC22:06
jeblairyeah, cc1 is running.22:07
*** xyang1 has quit IRC22:07
mordredjeblair: files01 is a small-ish vm right?22:08
jeblairmordred: the 'timeout 30' is somewher in the puppet role isn't it?22:08
*** tphummel has joined #openstack-infra22:08
mordredyah22:08
mordredone sec22:08
jeblairmordred: yeah, i'm booting it as 4g22:09
jeblairmordred: i wonder if we could parameterize that22:09
jeblairso i could pass '--timeout 60' to launch node and it could pass that to ansible-playbook22:09
*** Apoorva has quit IRC22:09
jeblairooh, it has moved on from the dkms stuff22:09
mordredjeblair: actually, I do not see a timeout setting being passed to puppet at all22:09
jeblairfatal: [files01.openstack.org]: FAILED! => {"changed": false, "failed": true, "msg": "/usr/bin/timeout -s 9 30m /usr/bin/puppet apply --detailed-exitcodes --logdest syslog --no-noop /opt/system-config/production/manifests/site.pp failed with return code: 6", "rc": 6, "stderr": "", "stdout": "", "stdout_lines": []}22:10
jeblairmordred: however, it does not look like the timeout is at issue here22:11
jeblairit looks like it did complete in time, it just failed.22:11
*** mdrabe has quit IRC22:11
mordredjeblair: awesome! (and we have the timeout in the puppet ansible module - but it is not exposed in the role ... I can work on that)22:11
jeblair(so that might be a nice thing for the future, but probably not urgent as if this hasn't hit the timeout then i think we can say we haven't seen a legit case of it)22:11
jeblairi think there's an apache error22:12
jeblairand so puppet is failing22:12
*** yamamoto has joined #openstack-infra22:13
mordredjeblair: ++22:13
mtreinishcloudnull: let me know if you need any help with it22:13
jeblairbut yay, i can see that now :)22:13
jeblairmordred: we don't have a way to use a 'local' copy of system-config for the adhoc puppet playbook do we?22:13
*** kzaitsev_mb has joined #openstack-infra22:14
*** edmondsw has quit IRC22:14
*** mdrabe has joined #openstack-infra22:14
mordredjeblair: one sec22:15
*** itsuugo has quit IRC22:15
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: stream syslog on remote host  https://review.openstack.org/37118222:15
*** fguillot has quit IRC22:16
*** itsuugo has joined #openstack-infra22:17
cloudnullmtreinish: where would I need to store a generated subunit ?22:17
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: Use popen rather than check_output  https://review.openstack.org/37115822:18
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: document using a virtualenv for launch-node  https://review.openstack.org/37116222:18
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: stream syslog on remote host  https://review.openstack.org/37118222:18
mordredjeblair: no, we do not have that facility that I can see. we DO support puppet_environment on the underlying puppet role22:18
cloudnullafter every test I would need to run "generate-subunit $(date +%s) 1 success 'name of test'"  ?22:18
*** yamamoto has quit IRC22:18
cloudnullwhere 1 is the number of seconds it took ?22:18
jeblairmordred: so we could make a variable for that playbook?22:18
mordredjeblair: but that is not exposed in the playbook. yeah22:19
jeblairmordred: you're not missing your flight are you? :)22:19
mordredjeblair: not yet :)22:19
*** jkraj has joined #openstack-infra22:19
*** cardeois has quit IRC22:19
*** ijw has quit IRC22:19
*** vhosakot has quit IRC22:20
mordredjeblair: actually  - I think if you just set puppet_environment on the command line as an extra var, it'll just work22:20
jeblair'ansible-playbook -e puppet_environment=foo' ?22:20
jeblairmordred: huh, i would have thought that would need to be plumbed through the playbook.22:21
mordredjeblair: it might work. I also think it might need to be plumbed through22:21
*** asettle has joined #openstack-infra22:21
*** kzaitsev_mb has quit IRC22:21
jeblairokay, i will try shortly.  :)22:21
mordred(I'm making a plumbing patch right now)22:21
jeblairi'm running launch-node with keep now, and i'll just run adhoc on the resulting node until it works22:21
*** ijw has joined #openstack-infra22:22
openstackgerritMonty Taylor proposed openstack-infra/system-config: Plumb puppet_environment through to the adhoc playbook  https://review.openstack.org/37119322:22
mordredjeblair: ^^22:22
mordredjeblair: obvs untested22:22
*** esberglu has quit IRC22:22
*** Apoorva has joined #openstack-infra22:22
*** mdrabe has quit IRC22:23
mtreinishcloudnull: either in the root dir of the artifacts that gets uploaded to the log server or under /logs22:23
cloudnullok.22:23
mtreinishcloudnull: the subunit worker will be looking at files here: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/logstash/jenkins-log-client.yaml#n1422:23
*** yamamoto has joined #openstack-infra22:24
*** yamamoto has quit IRC22:24
jeblairmordred: do multiple env vars each get a -e argument like: '-e foo=1 -e bar=2' ?22:24
*** yamamoto has joined #openstack-infra22:24
cloudnullso calling it "logs/testrepository.subunit" should be all thats needed to get it picked up22:24
mordredcloudnull: ^^ do you know the answer to jeblair's question?22:25
cloudnullrunning ansible on the CLI, yes22:25
mtreinishcloudnull: yep that should work fine22:25
clarkbI think I have done it as -e'foo=bar bar=foo' before22:25
cloudnullmordred: jeblair: we've found it easier to put them all in a file and do "-e @/file/name.yml"22:26
*** asettle has quit IRC22:26
mordredyah22:26
mordredjeblair: so - multiple -e work - as does putting in a file22:26
*** tonytan4ever has quit IRC22:27
mordredjeblair: I just tested with a checkout I have that prints a lot of internal ansible debugging :)22:27
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: add puppet environment option  https://review.openstack.org/37119622:27
mordredas well as a bunch of print statements with profanity22:27
jeblairmordred, cloudnull: thanks :) ^22:27
cloudnullwe have so many options that could be overriden we simply find the file to be more managable22:27
*** yamamoto has quit IRC22:28
*** gordc has quit IRC22:29
jheskethMorning22:29
cloudnullraise SystemExit('$#@$ %@@ ##@ $# @ @####') -- is generally how i add breakpoints :)22:29
jeblairclarkb, mordred, ianw, pabelanger: i'm happy with launch-node's debugability with the stack ending at https://review.openstack.org/371182 applied22:29
openstackgerritCaleb Boylan proposed openstack-infra/nodepool: [WIP] Fix subnode deletion  https://review.openstack.org/37045522:30
jeblairi think we should land that stack asap to avoid leaving infra-root in a lurch22:30
*** salv-orlando has quit IRC22:32
mordredjeblair: +2 from me on all of them22:32
*** sarob has quit IRC22:33
*** ddieterly is now known as ddieterly[away]22:34
*** itsuugo has quit IRC22:34
mtreinishjeblair: I've been seeing a lot of stack traces like this in my gertty log lately: http://paste.openstack.org/show/577737/ is that something you've seen before?22:35
mtreinishI was assuming it was just gerrit slowness or something, but I wasn't sure22:36
jeblairmtreinish: master?22:36
mtreinishyep22:36
*** itsuugo has joined #openstack-infra22:37
clarkbjeblair: re https://review.openstack.org/#/c/371182/3 do you find that the interleaving of syslog and ansible is a problem? wondering ifwe can do that cleaner like adding a puppet filter to the tail or something22:38
mtreinishjeblair: heh, I maintain the gertty arch package but I don't ever use it22:38
clarkbjeblair: but not sure if in practice it causes any issues22:38
prometheanfireclarkb: pabelanger testing the gentoo thing now22:38
jeblairmtreinish: hrm.  that looks like it could be a problem22:38
*** tsg__ has quit IRC22:39
jeblairclarkb: it's easy to mentally filter because syslog has timestamps, also, it tends to be one or the other. (like a bit of ansible, then it stops while interesting things happen in syslog, then syslog stops and more ansible)22:39
jeblairclarkb: looks like http://paste.openstack.org/show/577738/22:40
clarkbjeblair: it apparently fails linting too22:40
jeblairmtreinish: any chance you're running in debug mode?22:40
clarkboh ya thats readable so fine as is (once jenkins is happy)22:40
*** tphummel has quit IRC22:40
*** rlandy is now known as rlandy|bbl22:41
mtreinishjeblair: I'm not, but I'll restart it in debug mode22:41
openstackgerritMerged openstack-infra/system-config: launch: Use popen rather than check_output  https://review.openstack.org/37115822:41
*** itsuugo has quit IRC22:41
mtreinishof course now I didn't hit the issue, I might have to wait a while for it to do a big sync again22:42
jeblairmtreinish: k, thanks.  i'm worried there's something wrong with the batching api there.  it's possible you may be missing some changes.22:42
clarkbmordred: and your ansible puppet change apparently needs a reabse22:42
*** itsuugo has joined #openstack-infra22:42
*** esberglu has joined #openstack-infra22:43
*** mriedem has joined #openstack-infra22:44
openstackgerritMerged openstack-infra/system-config: launch: document using a virtualenv for launch-node  https://review.openstack.org/37116222:45
prometheanfirepabelanger: the code isn't writing out a new config :|22:45
prometheanfireKeyError: 'netmask'22:46
prometheanfireneat22:46
*** salv-orlando has joined #openstack-infra22:46
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: stream syslog on remote host  https://review.openstack.org/37118222:46
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: add puppet environment option  https://review.openstack.org/37119622:46
jeblairclarkb: ^22:46
*** esberglu has quit IRC22:47
*** piet has joined #openstack-infra22:47
*** baoli has quit IRC22:48
*** esberglu has joined #openstack-infra22:48
*** chem` has joined #openstack-infra22:49
*** EricGonczer_ has joined #openstack-infra22:49
*** yamahata has quit IRC22:49
openstackgerritClark Boylan proposed openstack-infra/project-config: Run horizon jobs on xenial  https://review.openstack.org/37120222:50
clarkbnow horizon can xenial22:50
clarkbjeblair: I think mordreds still needs sorting which is making your second one unhappy22:50
clarkbbut I am just going to review them as is22:50
prometheanfireah, found it22:50
*** ijw has quit IRC22:50
jeblairclarkb: yeah, that one isn't critical22:50
*** chem has quit IRC22:51
clarkbjeblair: that last one adds ansible_cmd.+=22:52
clarkbI think the . is extra and should be removed?22:52
jeblairprobably so22:52
*** sdague has quit IRC22:52
openstackgerritJames E. Blair proposed openstack-infra/system-config: launch: add puppet environment option  https://review.openstack.org/37119622:53
openstackgerritEmilien Macchi proposed openstack-infra/project-config: Create more Puppet OpenStack Gerrit groups  https://review.openstack.org/37120522:53
mgagnepabelanger: actively working on the issue in mtl0122:53
mgagnepabelanger: still not fixed :-/22:54
*** jamesdenton has quit IRC22:56
*** amitgandhinz has quit IRC22:56
*** ddieterly[away] is now known as ddieterly22:58
*** tphummel has joined #openstack-infra22:59
*** ddieterly has quit IRC23:00
openstackgerritMatthew Thode proposed openstack-infra/glean: Add ipv6 support to gentoo  https://review.openstack.org/36748723:01
prometheanfireclarkb: jeblair ^ just did the enable_interfaces bit23:02
prometheanfirebah23:02
prometheanfirepabelanger: ^23:02
*** kzaitsev_mb has joined #openstack-infra23:03
*** piet has quit IRC23:03
*** claudiub has quit IRC23:06
*** kzaitsev_mb has quit IRC23:07
*** itsuugo has quit IRC23:08
*** pahuang has joined #openstack-infra23:09
*** itsuugo has joined #openstack-infra23:09
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Deploy TripleO with Puppet 4  https://review.openstack.org/37120923:09
EmilienMcan someone review https://review.openstack.org/#/c/371133/ please?23:10
*** ijw has joined #openstack-infra23:11
openstackgerritDoug Hellmann proposed openstack-infra/project-config: skip constraints update if there is no tox.ini file  https://review.openstack.org/37121123:11
*** rbrndt_ has quit IRC23:11
*** jkraj has quit IRC23:13
*** hongbin has quit IRC23:13
*** amitgandhinz has joined #openstack-infra23:14
*** whoops has quit IRC23:14
*** kzaitsev_mb has joined #openstack-infra23:14
*** ijw has quit IRC23:16
*** asettle has joined #openstack-infra23:22
*** itsuugo has quit IRC23:23
*** itsuugo has joined #openstack-infra23:24
*** whoops has joined #openstack-infra23:24
*** Apoorva has quit IRC23:25
*** drawsmcgraw has quit IRC23:26
sdake_hey folks, having trouble adding a user to our core review team23:26
sdake_gerrit pops up 3 names for the individual23:26
sdake_they are all identical23:26
*** asettle has quit IRC23:27
sdake_electing any ofthem yields the error "Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b (old logs from #kolla  http://goo.gl/VKpPzA)"23:27
sdake_thats not the error23:28
sdake_sorry23:28
sdake_Account Not found: Christian Berendt <berendt@..>23:29
sdake_(id = berendt23:29
*** itsuugo has quit IRC23:35
*** esberglu has quit IRC23:35
*** itsuugo has joined #openstack-infra23:37
*** armax has quit IRC23:37
*** rvasilets___ has quit IRC23:37
*** markvoelker has quit IRC23:39
*** Jeffrey4l__ is now known as Jeffrey4l23:40
*** AnarchyAo has quit IRC23:41
*** tkelsey has joined #openstack-infra23:45
*** camunoz has joined #openstack-infra23:45
*** thcipriani is now known as thcipriani|afk23:45
*** chem` has quit IRC23:49
*** itsuugo has quit IRC23:49
*** tkelsey has quit IRC23:49
*** kzaitsev_mb has quit IRC23:50
*** ijw has joined #openstack-infra23:50
*** kzaitsev_mb has joined #openstack-infra23:50
*** itsuugo has joined #openstack-infra23:51
*** dahoo has quit IRC23:52
*** tqtran has quit IRC23:52
*** fguillot has joined #openstack-infra23:54
*** armax has joined #openstack-infra23:54
*** tqtran has joined #openstack-infra23:55
*** kzaitsev_mb has quit IRC23:56
*** yamamoto_ has joined #openstack-infra23:58
*** piet has joined #openstack-infra23:58
*** ijw has quit IRC23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!