Thursday, 2017-12-07

*** masayukig[m] has joined #openstack-infra00:02
*** stapler118 has left #openstack-infra00:04
*** salv-orlando has quit IRC00:06
*** salv-orlando has joined #openstack-infra00:06
*** salv-orlando has quit IRC00:11
joadavisThanks again clarkb. That fix worked to get monasca-api one step further. :)00:11
*** david-lyle has quit IRC00:13
*** jamesmcarthur has quit IRC00:14
*** jamesmcarthur has joined #openstack-infra00:18
*** jamesmcarthur has quit IRC00:24
*** ilpianista_ has joined #openstack-infra00:29
*** aspiers[m] has joined #openstack-infra00:29
*** wolverineav has quit IRC00:29
*** armaan has quit IRC00:30
*** yamamoto has joined #openstack-infra00:34
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Save oooq generated files  https://review.openstack.org/51713600:34
*** jonher has quit IRC00:35
*** jonher has joined #openstack-infra00:36
*** sree has joined #openstack-infra00:36
*** mat128 has quit IRC00:38
*** yamamoto has quit IRC00:39
*** sree has quit IRC00:41
*** bobh has quit IRC00:43
*** tosky has quit IRC00:44
*** linkmark has quit IRC00:45
*** mat128 has joined #openstack-infra00:48
*** bobh has joined #openstack-infra00:51
*** gk__ has joined #openstack-infra00:52
gk__fungi: Hi fungi, are you around?00:53
*** bobh has quit IRC00:56
openstackgerritClark Boylan proposed openstack-infra/nodepool feature/zuulv3: Clarify terminology around node request locks  https://review.openstack.org/52623301:01
openstackgerritClark Boylan proposed openstack-infra/nodepool feature/zuulv3: Handle race between handler and request cleanup  https://review.openstack.org/52623401:01
*** bobh has joined #openstack-infra01:02
*** liujiong has joined #openstack-infra01:03
clarkbgk__: its a bit late for fungi I think. Anything we can help you with?01:04
gk__clarkb: I wanted to ask fungi to remove some remaining connections from our account disco.ci01:05
clarkbgk__: in gerrit?01:06
gk__clarkb: yes, our third-party ci cannot connect to gerrit01:06
clarkbgk__: is that because you have reached the connection limit?01:07
gk__clarkb: yes01:07
*** bobh has quit IRC01:07
gk__clarkb: we have schedule a meeting next week with our IT team to figure out the root cause01:07
*** kiennt26 has joined #openstack-infra01:08
clarkbgk__: so if we remove the connections now it will just be expected to hit the limit again? or does it happen very slowly?01:08
*** rfolco has quit IRC01:08
*** cuongnv has joined #openstack-infra01:09
*** Kevin_Zheng has joined #openstack-infra01:09
gk__clarkb: it will hit the limt again around every two weeks01:10
clarkbI see do you have the account id available/ I can take a look at the existing connections for it01:10
gk__clarkb: user is disco.ci01:12
*** gyee has quit IRC01:12
*** edmondsw has quit IRC01:12
*** bobh has joined #openstack-infra01:15
*** hongbin has quit IRC01:15
*** cody-somerville has quit IRC01:15
*** liujiong has quit IRC01:16
*** huanxie has joined #openstack-infra01:16
*** zhurong has joined #openstack-infra01:19
*** bobh has quit IRC01:20
clarkbgk__: I am not seeing any connections with that user name01:20
*** rfolco has joined #openstack-infra01:20
*** sree has joined #openstack-infra01:20
gk__clarkb: ok let me check if I can connect to gerrit01:20
gk__clarkb: it is working. Thank you for your check and sorry for bothering you01:22
clarkbno problem01:22
fungiyeah, i had stepped away for dinner. back for a moment but seems all is settled01:23
gk__fungi: yes it is ok now01:24
*** RuiChen has joined #openstack-infra01:24
*** sree has quit IRC01:25
*** bobh has joined #openstack-infra01:25
*** salv-orlando has joined #openstack-infra01:27
fungiexcellent01:27
* fungi fades back into the night01:27
SamYapleheh01:28
clarkbI too need to go help with the dinner prep01:29
*** bobh has quit IRC01:29
*** psachin has joined #openstack-infra01:30
*** cshastri has joined #openstack-infra01:31
*** cody-somerville has joined #openstack-infra01:31
*** bobh has joined #openstack-infra01:33
*** salv-orlando has quit IRC01:34
*** salv-orlando has joined #openstack-infra01:35
*** gk__ has quit IRC01:35
*** yamamoto has joined #openstack-infra01:35
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: handler: fix support for handler without launch_manager  https://review.openstack.org/52477301:37
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: builder: do not cleanup image for driver not managing image  https://review.openstack.org/51692001:37
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add a plugin interface for drivers  https://review.openstack.org/52462001:37
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a static driver for Nodepool  https://review.openstack.org/46862401:37
*** bobh has quit IRC01:38
*** salv-orlando has quit IRC01:39
*** yamahata has joined #openstack-infra01:40
*** yamamoto has quit IRC01:41
*** bobh has joined #openstack-infra01:43
*** bobh has quit IRC01:47
*** david-lyle has joined #openstack-infra01:48
*** bobh has joined #openstack-infra01:56
*** caphrim007_ has quit IRC01:57
*** caphrim007 has joined #openstack-infra01:58
*** yamamoto has joined #openstack-infra01:58
*** david-lyle has quit IRC02:00
*** bobh has quit IRC02:01
*** caphrim007 has quit IRC02:03
*** bobh has joined #openstack-infra02:03
*** bobh has quit IRC02:08
*** salv-orlando has joined #openstack-infra02:08
*** rcernin has quit IRC02:12
*** Wei_Liu has joined #openstack-infra02:13
*** cody-somerville has quit IRC02:15
*** salv-orlando has quit IRC02:18
*** salv-orlando has joined #openstack-infra02:19
*** rcernin has joined #openstack-infra02:20
*** harlowja has quit IRC02:22
*** salv-orlando has quit IRC02:23
*** Apoorva_ has joined #openstack-infra02:24
*** jamesmcarthur has joined #openstack-infra02:25
*** bobh has joined #openstack-infra02:25
*** Apoorva has quit IRC02:27
*** pbourke has quit IRC02:27
*** cody-somerville has joined #openstack-infra02:28
*** Apoorva_ has quit IRC02:28
*** pbourke has joined #openstack-infra02:29
*** bobh has quit IRC02:29
*** jamesmcarthur has quit IRC02:29
*** xarses_ has quit IRC02:31
*** edmondsw has joined #openstack-infra02:33
*** bobh has joined #openstack-infra02:33
*** annp has joined #openstack-infra02:34
*** edmondsw has quit IRC02:37
*** xarses has joined #openstack-infra02:38
*** bobh has quit IRC02:38
*** dhinesh has quit IRC02:38
*** yamahata has quit IRC02:41
*** bobh has joined #openstack-infra02:43
*** bobh has quit IRC02:48
*** caphrim007 has joined #openstack-infra02:49
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Update Fedora defaults to 27  https://review.openstack.org/52168403:00
openstackgerritIan Wienand proposed openstack/diskimage-builder master: ironic-agent: don't remove make  https://review.openstack.org/52625703:00
*** bobh has joined #openstack-infra03:00
*** hongbin has joined #openstack-infra03:01
*** aeng has quit IRC03:04
*** mat128 has quit IRC03:04
*** bobh has quit IRC03:06
ianw#status log fedora 27 mirroring complete, fedora 25 removed03:08
openstackstatusianw: finished logging03:08
*** ramishra has joined #openstack-infra03:10
*** armax has joined #openstack-infra03:11
openstackgerritIan Wienand proposed openstack-infra/system-config master: fedora-mirror: skip alternative archs in atomic image mirrors  https://review.openstack.org/52625903:12
*** hemna_ has quit IRC03:13
*** coolsvap has joined #openstack-infra03:13
*** mat128 has joined #openstack-infra03:14
*** mat128 has quit IRC03:18
*** aeng has joined #openstack-infra03:21
AJaeger_jeblair: what needs to be done with mistral-tempest-plugin? Zuul still does not know about it ;( Restart of Zuul?03:23
*** mat128 has joined #openstack-infra03:26
*** csomerville has joined #openstack-infra03:28
*** dhill_ has quit IRC03:29
*** dhill_ has joined #openstack-infra03:32
*** bobh has joined #openstack-infra03:32
*** bobh has quit IRC03:37
*** aeng has quit IRC03:38
*** aeng has joined #openstack-infra03:38
*** sree has joined #openstack-infra03:42
*** bobh has joined #openstack-infra03:45
*** yamamoto has quit IRC03:46
*** aviau has joined #openstack-infra03:46
*** bobh has quit IRC03:50
*** yamamoto has joined #openstack-infra03:52
*** threestrands has joined #openstack-infra03:52
*** threestrands has quit IRC03:52
*** threestrands has joined #openstack-infra03:52
*** sree_ has joined #openstack-infra03:53
*** sree_ is now known as Guest3103:53
*** sree has quit IRC03:54
*** udesale has joined #openstack-infra04:01
*** ihrachys has quit IRC04:02
openstackgerritIan Wienand proposed openstack-infra/openstack-zuul-jobs master: [WIP] Move prepare-zanata-client to o-z-j  https://review.openstack.org/52576004:06
*** sree has joined #openstack-infra04:08
*** rlandy has quit IRC04:09
*** Guest31 has quit IRC04:10
*** Apoorva has joined #openstack-infra04:15
openstackgerritIan Wienand proposed openstack-infra/openstack-zuul-jobs master: [WIP] Move prepare-zanata-client to o-z-j  https://review.openstack.org/52576004:15
*** dave-mccowan has quit IRC04:17
*** bobh has joined #openstack-infra04:17
*** adisky_ has joined #openstack-infra04:21
*** pgadiya has joined #openstack-infra04:21
*** bobh has quit IRC04:21
*** zhurong has quit IRC04:22
*** ykarel|afk has joined #openstack-infra04:22
*** links has joined #openstack-infra04:26
*** andreas_s has joined #openstack-infra04:26
*** andreas_s has quit IRC04:31
*** harlowja has joined #openstack-infra04:39
*** bobh has joined #openstack-infra04:41
*** bobh has quit IRC04:45
*** hongbin has quit IRC04:47
*** jaosorior has joined #openstack-infra04:49
*** bobh has joined #openstack-infra04:52
*** bobh has quit IRC04:57
ianwAJaeger_: yeah, it just seemed as easy to put the various pkg names in as skip it :)05:03
*** armax has quit IRC05:06
ianwhmm, this certificate storage stuff though is painful.  i don't think it's worth working through that for it never to be used05:12
*** janki has joined #openstack-infra05:14
*** tushar has quit IRC05:21
openstackgerritIan Wienand proposed openstack-infra/openstack-zuul-jobs master: [WIP] Move prepare-zanata-client to o-z-j  https://review.openstack.org/52576005:21
*** tushar has joined #openstack-infra05:21
*** csomerville has quit IRC05:22
*** csomerville has joined #openstack-infra05:22
*** salv-orlando has joined #openstack-infra05:23
*** numans has quit IRC05:23
*** gouthamr has quit IRC05:24
*** jamesmcarthur has joined #openstack-infra05:25
*** numans has joined #openstack-infra05:26
*** yamamoto has quit IRC05:29
*** zhurong has joined #openstack-infra05:30
*** jamesmcarthur has quit IRC05:30
*** harlowja has quit IRC05:30
openstackgerritJens Harbott (frickler) proposed openstack-infra/system-config master: Add frickler to statusbot and infra-root user keys  https://review.openstack.org/52574805:30
*** salv-orlando has quit IRC05:31
*** bobh has joined #openstack-infra05:33
*** ykarel|afk has quit IRC05:35
*** ykarel|afk has joined #openstack-infra05:35
*** bobh has quit IRC05:38
*** shu-mutou-AWAY has quit IRC05:40
*** vickyc has quit IRC05:41
*** daidv has quit IRC05:42
*** daidv has joined #openstack-infra05:42
*** yamamoto has joined #openstack-infra05:47
*** ykarel|afk is now known as ykarel05:51
*** bobh has joined #openstack-infra05:52
*** Apoorva has quit IRC05:53
*** Apoorva has joined #openstack-infra05:54
*** sree has quit IRC05:56
*** yamamoto has quit IRC05:56
*** sree has joined #openstack-infra05:56
*** bobh has quit IRC05:56
*** Apoorva has quit IRC05:58
*** bobh has joined #openstack-infra06:01
*** sree_ has joined #openstack-infra06:02
*** sree_ is now known as Guest3861706:02
*** sree has quit IRC06:02
*** bobh has quit IRC06:05
*** gcb has quit IRC06:08
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/52628906:10
*** gcb has joined #openstack-infra06:10
*** mandre has quit IRC06:11
*** mandre has joined #openstack-infra06:15
*** mat128 has quit IRC06:17
*** Guest38617 has quit IRC06:18
openstackgerritChandan Kumar proposed openstack-infra/project-config master: Added initial jobs for mistral-tempest-plugin  https://review.openstack.org/52495206:19
*** bobh has joined #openstack-infra06:21
chandankumarAJaeger_: frickler https://review.openstack.org/#/c/524952/ i have no idea why it is failing  The project "openstack/mistral-tempest-plugin" was not found.06:21
*** sree has joined #openstack-infra06:23
*** bobh has quit IRC06:26
*** salv-orlando has joined #openstack-infra06:28
*** yamamoto has joined #openstack-infra06:28
*** ykarel is now known as ykarel|afk06:31
*** threestrands has quit IRC06:32
*** yamamoto has quit IRC06:33
*** rossella_s has quit IRC06:33
*** rossella_s has joined #openstack-infra06:34
*** aeng has quit IRC06:35
ianw2017-12-06 11:41:54,041 ERROR zuul.GerritConnection: Cannot get references from openstack/mistral-tempest-plugin06:35
ianwwas the project created before the zuul bits were merged?06:37
*** rosmaita has quit IRC06:39
*** e0ne has joined #openstack-infra06:41
*** bobh has joined #openstack-infra06:42
*** jascott1 has joined #openstack-infra06:42
chandankumarianw: https://review.openstack.org/#/c/524869/ this was got merged which created the project06:42
*** sree_ has joined #openstack-infra06:43
*** sree_ is now known as Guest8483406:43
*** yamamoto has joined #openstack-infra06:44
*** rwsu has joined #openstack-infra06:44
*** bobh has quit IRC06:46
*** e0ne has quit IRC06:46
*** sree has quit IRC06:46
*** sree has joined #openstack-infra06:47
*** yamamoto has quit IRC06:48
*** Guest84834 has quit IRC06:49
*** salv-orlando has quit IRC06:49
ianw[2017-12-06 12:06:48,031 +0000] 43e5a99f openstack-project-creator a/6199 gerrit.create-project.--require-change-id.openstack/mistral-tempest-plugin 1ms 7885ms 006:50
ianwwell it seems that gerrit started looking for the project about 20 minutes before it was created06:51
*** andreas_s has joined #openstack-infra06:53
chandankumarianw: i think recheck will fix it?06:54
*** andreas_s has quit IRC06:54
*** andreas_s has joined #openstack-infra06:55
chandankumarnope, still same issue06:55
ianwchandankumar: i think maybe zuul will need to be restarted06:55
ianwjeblair: ^ may have considered this case ... it's a bit more than i can dig into atm sorry06:55
chandankumarianw: thanks for the help :-)06:57
*** yamamoto has joined #openstack-infra06:58
ianwwell not much help.  maybe this is the first time we merged jobs and project creation in the same change?   or just the first we've noticed06:58
*** bobh has joined #openstack-infra07:01
fricklerianw: see backlog from yesterday, seems zuul and gerrit saw that change out of order. not sure though how to fix it, either07:05
*** bobh has quit IRC07:06
ianwmaybe, if we removed the project from zuul and re-added it, that would trigger it to look again and avoid a restart07:10
*** rcernin has quit IRC07:11
ianwthe gates is really quite busy, so i'm reticent to restart unless it's dire07:11
*** sree has quit IRC07:12
*** sree has joined #openstack-infra07:13
*** ccamacho has left #openstack-infra07:13
*** ccamacho has quit IRC07:13
frickleroh, gate backlog is at 13h now, yesterday I was hoping it would be slowly decreasing. I'll try to do some digging, too bad e-r still isn't working properly it seems07:14
frickler"Delay in Elastic Search: Indexing behind by 11 hours" isn't really helpful in that, too :(07:15
*** zhurong has quit IRC07:16
*** sree has quit IRC07:21
openstackgerritMerged openstack-dev/pbr master: Put test-requirements into an extra named 'test'  https://review.openstack.org/49261907:23
*** sree has joined #openstack-infra07:29
*** ykarel|afk is now known as ykarel07:30
*** bobh has joined #openstack-infra07:31
*** bobh has quit IRC07:37
*** kjackal has quit IRC07:37
*** sree has quit IRC07:38
*** sree has joined #openstack-infra07:39
*** sree_ has joined #openstack-infra07:43
*** jtomasek has joined #openstack-infra07:44
*** sree_ is now known as Guest4602907:44
*** slaweq_ has joined #openstack-infra07:44
*** sree has quit IRC07:47
*** alexchadin has joined #openstack-infra07:51
*** yamamoto has quit IRC07:51
*** bobh has joined #openstack-infra07:51
*** bobh has quit IRC07:55
*** yamamoto has joined #openstack-infra07:58
*** salv-orlando has joined #openstack-infra08:00
*** florianf has joined #openstack-infra08:01
*** bobh has joined #openstack-infra08:01
*** yamamoto has quit IRC08:03
*** salv-orlando has quit IRC08:04
*** bobh has quit IRC08:06
*** kjackal has joined #openstack-infra08:09
*** bobh has joined #openstack-infra08:13
*** yamamoto has joined #openstack-infra08:14
*** ccamacho has joined #openstack-infra08:15
*** tesseract has joined #openstack-infra08:15
*** bobh has quit IRC08:18
*** yamamoto has quit IRC08:18
*** yamamoto has joined #openstack-infra08:18
*** yamamoto has quit IRC08:18
*** rcernin has joined #openstack-infra08:19
*** rossella_s has quit IRC08:19
*** dhajare has joined #openstack-infra08:19
fricklerwho is the owner of #openstack-101 or who would know the original topic? seems to be only being set to different statusbot warnings for at least a year08:21
*** yamamoto has joined #openstack-infra08:22
*** rossella_s has joined #openstack-infra08:22
*** huanxie has quit IRC08:23
*** yamamoto has quit IRC08:24
*** yamamoto has joined #openstack-infra08:24
*** yamamoto has quit IRC08:24
*** armaan has joined #openstack-infra08:27
*** jpena|off is now known as jpena08:32
openstackgerritJens Harbott (frickler) proposed openstack-infra/project-config master: Temporarily disable check-reqs for designate  https://review.openstack.org/52631308:38
fricklerconfig-core: ^^ fix to allow designate to merge a fix for the new requirements check. alternative solutions welcome, but this looks easiest to me08:39
*** yamamoto has joined #openstack-infra08:41
*** e0ne has joined #openstack-infra08:42
*** Hal has joined #openstack-infra08:46
*** yamamoto has quit IRC08:56
*** jpich has joined #openstack-infra08:58
*** yamamoto has joined #openstack-infra08:58
openstackgerritPavlo Shchelokovskyy proposed openstack-infra/project-config master: Add n-g-s gerritbot to ironic IRC channel  https://review.openstack.org/52631809:00
*** salv-orlando has joined #openstack-infra09:01
*** yamamoto has quit IRC09:01
*** hashar has joined #openstack-infra09:01
*** shardy has joined #openstack-infra09:03
*** bobh has joined #openstack-infra09:03
*** Guest46029 has quit IRC09:03
*** salv-orlando has quit IRC09:06
*** andreas_s has quit IRC09:07
*** bobh has quit IRC09:07
*** andreas_s has joined #openstack-infra09:08
*** andreas_s has quit IRC09:12
*** linkmark has joined #openstack-infra09:23
*** bobh has joined #openstack-infra09:24
*** stakeda has quit IRC09:25
*** jamesmcarthur has joined #openstack-infra09:26
*** dizquierdo has joined #openstack-infra09:28
*** bobh has quit IRC09:28
*** sshnaidm|afk is now known as sshnaidm|rover09:30
*** jamesmcarthur has quit IRC09:30
*** ganso has joined #openstack-infra09:30
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: builder: do not cleanup image for driver not managing image  https://review.openstack.org/51692009:31
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: handler: fix support for handler without launch_manager  https://review.openstack.org/52477309:31
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add a plugin interface for drivers  https://review.openstack.org/52462009:31
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a static driver for Nodepool  https://review.openstack.org/46862409:31
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement an OpenContainer driver  https://review.openstack.org/46875309:31
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a Kubernetes driver  https://review.openstack.org/52135609:31
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a generic run_handler  https://review.openstack.org/52632509:31
*** lucas-afk is now known as lucasagomes09:35
*** alexchadin has quit IRC09:35
*** electrofelix has joined #openstack-infra09:36
*** derekh has joined #openstack-infra09:36
*** yee37930 has joined #openstack-infra09:37
*** ykarel is now known as ykarel|lunch09:37
*** alexchad_ has joined #openstack-infra09:38
*** martinkopec has joined #openstack-infra09:40
*** yee379 has quit IRC09:40
*** pahuang has quit IRC09:41
*** pahuang has joined #openstack-infra09:41
*** andreaf has quit IRC09:42
*** bobh has joined #openstack-infra09:42
*** andreaf has joined #openstack-infra09:42
*** bobh has quit IRC09:47
*** armaan has quit IRC09:47
*** armaan has joined #openstack-infra09:48
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Save oooq generated files  https://review.openstack.org/51713609:51
*** annp has quit IRC09:55
*** armaan has quit IRC09:55
*** armaan has joined #openstack-infra09:56
*** rossella_s has quit IRC09:56
fricklerclarkb: probably a bit late, but if you could link me into the OVH/IPv6 conversation, I'd be interested in that09:56
*** dizquierdo has quit IRC10:00
*** salv-orlando has joined #openstack-infra10:02
*** yamamoto has joined #openstack-infra10:02
*** shardy has quit IRC10:02
*** shardy has joined #openstack-infra10:04
*** gmann is now known as gmann_afk10:05
*** salv-orlando has quit IRC10:06
*** yamamoto has quit IRC10:08
*** bobh has joined #openstack-infra10:08
*** tosky has joined #openstack-infra10:11
*** cuongnv has quit IRC10:12
*** bobh has quit IRC10:13
*** dizquierdo has joined #openstack-infra10:16
*** cshastri has quit IRC10:20
*** ldnunes has joined #openstack-infra10:20
*** rcernin has quit IRC10:21
*** raissa has quit IRC10:22
*** bobh has joined #openstack-infra10:23
*** alex_xu has quit IRC10:24
*** alex_xu has joined #openstack-infra10:25
*** bobh has quit IRC10:27
*** alexchad_ has quit IRC10:27
*** dsariel has quit IRC10:28
*** alexchadin has joined #openstack-infra10:28
*** gcb has quit IRC10:29
*** dtantsur|afk is now known as dtantsur10:32
*** alexchadin has quit IRC10:39
*** yamamoto has joined #openstack-infra10:39
*** alexchadin has joined #openstack-infra10:40
*** yamamoto has quit IRC10:41
*** tobiash has quit IRC10:45
*** cody-somerville has quit IRC10:46
*** pgadiya has quit IRC10:47
*** daidv has quit IRC10:48
*** kiennt26 has quit IRC10:48
*** daidv has joined #openstack-infra10:49
*** kiennt26 has joined #openstack-infra10:49
fricklerconfig-core: infra-root: neutron is seeing a high rate of timeouts on openstack-tox-cover, default timeout is 30 minutes from base, successful runs are around 25 minutes mostly. should we increase the timeout globally for that job, do a variant with longer timeout, or can we just add a higher timeout for neutron specifically?10:49
*** alex_xu has quit IRC10:50
*** alex_xu has joined #openstack-infra10:51
*** kjackal has quit IRC10:52
*** kjackal has joined #openstack-infra10:53
*** bobh has joined #openstack-infra10:55
*** sdague has joined #openstack-infra10:56
*** ldesimone has joined #openstack-infra10:57
*** sdague has quit IRC10:57
ldesimoneHi all. I'm using tempest to perform some testing on nova. I've run nosetests --with-coverage option to understand which part of nova are covered by the test (in particular, a specific scenario, e.g., tempest.scenario.test_minimum_basic:TestMinimumBasicScenario). The problem is that the report does not show calls to nova functions...is it normal? What's wrong?11:00
*** bobh has quit IRC11:00
cmurphyfrickler: I actually saw that for keystone too, though only once so far http://logs.openstack.org/47/524747/5/check/openstack-tox-cover/9fb79e0/11:01
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: handler: fix support for handler without launch_manager  https://review.openstack.org/52477311:01
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add a plugin interface for drivers  https://review.openstack.org/52462011:01
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: builder: do not cleanup image for driver not managing image  https://review.openstack.org/51692011:01
*** bobh has joined #openstack-infra11:02
*** sambetts|afk is now known as sambetts11:02
*** salv-orlando has joined #openstack-infra11:02
*** andreas_s has joined #openstack-infra11:03
*** katkapilatova has joined #openstack-infra11:05
fricklercmurphy: yeah, but the difference is that the normal run duration for keystone seems to be around 10 minutes, while for neutron we seem to be at 2511:07
*** salv-orlando has quit IRC11:07
*** bobh has quit IRC11:07
fricklerand I observed at least two gate failures in neutron due to this issue today11:07
cmurphyah okay, if that's normal for neutron then it's a different story :)11:08
fricklersadly elastic-recheck doesn't seem to classify jobs that time out, because no subunit file is generated11:08
openstackgerritJens Harbott (frickler) proposed openstack-infra/project-config master: Increase timeout for running coverage job on Neutron  https://review.openstack.org/52634211:09
fricklerconfig-core: infra-root: ^^ one possible solution, but I'm still open for other options11:09
*** hughhalf has quit IRC11:11
*** hughhalf has joined #openstack-infra11:11
*** jkilpatr has quit IRC11:12
openstackgerritJens Harbott (frickler) proposed openstack-infra/project-config master: Temporarily disable check-reqs for designate  https://review.openstack.org/52631311:12
*** bobh has joined #openstack-infra11:14
*** armaan has quit IRC11:15
*** kiennt26 has quit IRC11:15
*** armaan has joined #openstack-infra11:15
*** bobh has quit IRC11:18
*** kjackal has quit IRC11:21
*** bobh has joined #openstack-infra11:21
*** kjackal has joined #openstack-infra11:21
*** sdague has joined #openstack-infra11:22
*** armaan has quit IRC11:22
*** armaan has joined #openstack-infra11:23
*** jesusaur has quit IRC11:25
*** bobh has quit IRC11:26
*** hughhalf has quit IRC11:26
*** dsariel has joined #openstack-infra11:27
*** hughhalf has joined #openstack-infra11:28
*** pahuang has quit IRC11:28
*** jesusaur has joined #openstack-infra11:28
*** pahuang has joined #openstack-infra11:28
*** udesale has quit IRC11:28
*** jesusaur has quit IRC11:31
Wei_Liufrickler: hello,  on which OS is recommended for zuul-scheduler/executor to run?11:31
*** yamamoto has joined #openstack-infra11:31
*** jesusaur has joined #openstack-infra11:33
*** wolverineav has joined #openstack-infra11:33
*** JasonCL has joined #openstack-infra11:36
*** cody-somerville has joined #openstack-infra11:38
*** janki has quit IRC11:40
*** janki has joined #openstack-infra11:40
*** alexchadin has quit IRC11:43
*** alexchadin has joined #openstack-infra11:43
*** jkilpatr has joined #openstack-infra11:47
*** kjackal has quit IRC11:49
*** bobh has joined #openstack-infra11:52
fricklerWei_Liu: no idea, really. I guess we (infra) run most stuff on Ubuntu, but I don't think that makes it recommended already. maybe ask in #zuul11:53
Wei_Liufrickler: thanks11:54
*** rossella_s has joined #openstack-infra11:56
*** smatzek has joined #openstack-infra11:57
*** slaweq_ has quit IRC11:57
*** bobh has quit IRC11:57
*** ykarel|lunch is now known as ykarel11:59
*** bobh has joined #openstack-infra12:03
*** salv-orlando has joined #openstack-infra12:03
*** slaweq has joined #openstack-infra12:06
*** salv-orlando has quit IRC12:07
*** bobh has quit IRC12:08
*** andreas_s has quit IRC12:12
*** andreas_s has joined #openstack-infra12:13
*** dhajare has quit IRC12:13
*** bobh has joined #openstack-infra12:16
*** bobh has quit IRC12:20
*** bobh has joined #openstack-infra12:21
*** andreas_s has quit IRC12:22
*** andreas_s has joined #openstack-infra12:22
*** jamesmcarthur has joined #openstack-infra12:25
*** bobh has quit IRC12:25
*** rosmaita has joined #openstack-infra12:29
*** dizquierdo has quit IRC12:29
*** dizquierdo has joined #openstack-infra12:30
*** jamesmcarthur has quit IRC12:30
*** bobh has joined #openstack-infra12:33
*** akscram1 has quit IRC12:34
*** dprince has joined #openstack-infra12:34
*** akscram1 has joined #openstack-infra12:35
*** kjackal has joined #openstack-infra12:38
*** slaweq has quit IRC12:38
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] split playbooks  https://review.openstack.org/52525212:39
*** cody-somerville has quit IRC12:40
*** jpena is now known as jpena|lunch12:40
*** ramishra has quit IRC12:42
*** andreas_s has quit IRC12:42
*** andreas_s has joined #openstack-infra12:43
*** lucasagomes is now known as lucas-hungry12:47
*** rossella_s has quit IRC12:49
*** armaan has quit IRC12:51
*** rhallisey has joined #openstack-infra12:51
*** armaan has joined #openstack-infra12:51
*** slaweq has joined #openstack-infra12:52
*** andreas_s has quit IRC12:52
*** JasonCL has quit IRC12:54
rosmaita"VMware NSX CI check" is showing up as a voting job on glance patches, but I'm pretty sure it's supposed to be non-voting ... anyone know what's up with that?  It's not listed in project-config/zuul.d/projects.yaml, I'm not sure where it's coming from (so I can't find the review that added it)12:55
*** armaan has quit IRC12:55
*** armaan has joined #openstack-infra12:55
rosmaitahere's an example patch where it's voting and causing a failure: https://review.openstack.org/#/c/521457/12:55
*** armaan has quit IRC12:55
*** armaan has joined #openstack-infra12:56
*** psachin has quit IRC12:59
*** kgiusti has joined #openstack-infra12:59
*** armaan has quit IRC13:00
*** armaan has joined #openstack-infra13:00
*** dsariel has quit IRC13:01
*** ykarel has quit IRC13:03
*** salv-orlando has joined #openstack-infra13:04
*** dizquierdo has quit IRC13:04
*** andreas_s has joined #openstack-infra13:07
*** erlon has joined #openstack-infra13:09
*** salv-orl_ has joined #openstack-infra13:09
*** slaweq has quit IRC13:09
*** slaweq has joined #openstack-infra13:11
*** salv-orlando has quit IRC13:11
*** raissa has joined #openstack-infra13:11
*** ykarel has joined #openstack-infra13:12
*** janki has quit IRC13:15
*** andreas_s has quit IRC13:16
*** andreas_s has joined #openstack-infra13:18
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Added summit/read/all scope  https://review.openstack.org/52636513:18
*** dave-mccowan has joined #openstack-infra13:18
openstackgerritRico Lin proposed openstack-infra/nodepool master: Fix getProviderManager in alien_list  https://review.openstack.org/52409213:19
*** alexchadin has quit IRC13:21
openstackgerritMerged openstack-infra/openstackid-resources master: Added summit/read/all scope  https://review.openstack.org/52636513:22
*** mat128 has joined #openstack-infra13:23
*** ykarel has quit IRC13:24
*** tobiash_ has joined #openstack-infra13:26
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] split playbooks  https://review.openstack.org/52525213:27
*** andreas_s has quit IRC13:27
fricklerrosmaita: that seems to be a third party ci, if you don't like what it does, best try to talk to the owners. if there is continued misbehaviour, we could only disable the corresponding ci account13:29
*** lucas-hungry is now known as lucasagomes13:29
fricklerrosmaita: you should be able to merge patches even if that CI reports a failure13:30
rosmaitafrickler: thanks ... how does one determine the owners?  (I mean, i know it's vmware, but there must be a place where they registered to get this job connected to glance patches)13:30
*** tobiash_ has quit IRC13:31
*** andreas_s has joined #openstack-infra13:32
*** salv-orl_ has quit IRC13:33
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Fix on all summit scope  https://review.openstack.org/52638413:33
*** salv-orlando has joined #openstack-infra13:33
*** rlandy has joined #openstack-infra13:33
openstackgerritMerged openstack-infra/openstackid-resources master: Fix on all summit scope  https://review.openstack.org/52638413:34
*** janki has joined #openstack-infra13:34
*** slaweq has quit IRC13:35
*** tobiash has joined #openstack-infra13:36
*** slaweq has joined #openstack-infra13:37
*** jkilpatr has quit IRC13:38
fricklerrosmaita: I think in theory any account can vote like normal reviewers do. there is a link on https://wiki.openstack.org/wiki/ThirdPartySystems that would make me assume salv-orlando and wsihan are the owners13:38
*** coolsvap has quit IRC13:38
*** salv-orlando has quit IRC13:39
rosmaitafrickler thanks for your help, i appreciate it13:39
AJaeger_rosmaita: CI systems should not vote unless being approved by the team, so if there was no such communication, ask an infra-root to disable it.13:40
fricklerrosmaita: yw. their page also doesn't list glance as project, so might be misconfiguration13:40
AJaeger_rosmaita: https://docs.openstack.org/infra/system-config/third_party.html13:40
*** andreas_s has quit IRC13:41
rosmaitaAJaeger_ : thanks, i think i may have been misreading the gerrit review13:42
*** adarazs is now known as adarazs_brb13:43
*** andreas_s has joined #openstack-infra13:43
*** jpena|lunch is now known as jpena13:46
*** tobiash has quit IRC13:46
*** andreas_s has quit IRC13:47
*** andreas_s has joined #openstack-infra13:48
*** raissa has quit IRC13:48
*** tobiash has joined #openstack-infra13:49
*** jkilpatr has joined #openstack-infra13:51
*** dsariel has joined #openstack-infra13:51
*** alexchadin has joined #openstack-infra13:52
*** tobiash has quit IRC13:53
*** adarazs_brb is now known as adarazs13:53
*** kiennt26 has joined #openstack-infra13:54
dmsimardfrickler: still around ?13:54
dmsimardAJaeger_, frickler: for https://review.openstack.org/#/c/526313/2/zuul.d/projects.yaml -- can't designate put up a patch to make the requirements job(s) non-voting in-tree ?13:55
dmsimardI've had to do it before for an chicken-and-egg thing13:55
fricklerdmsimard: I already did13:55
fricklerdmsimard: ah, you said in-tree13:56
*** alexchadin has quit IRC13:56
*** annp has joined #openstack-infra13:56
fricklerdmsimard: I could try that if you prefer, yes13:56
dmsimardI'm not sure if it's against the PTI or something, I would at least put the jobs non-voting instead of removing them outright13:56
fricklerdmsimard: well the idea is to just remove it in order to get the designate patch merged and revert immediately afterwards13:57
*** mriedem has joined #openstack-infra13:57
dmsimardfrickler: they're able to do that themselves13:58
dmsimardhttp://git.openstack.org/cgit/openstack/ara/tree/zuul.d/layout.yaml?h=feature/1.0#n15 (note, this is a feature branch, not master)13:58
dmsimardalthough there's the fact that the template has both check and gate jobs, so we have to put gate jobs non-voting :/13:59
fricklerdmsimard: I'll try if it works. last time I had something similar, the job still was run on the patch removing it13:59
*** tobiash has joined #openstack-infra13:59
dmsimardfrickler: you need to define it in the layout like I linked13:59
*** armaan has quit IRC13:59
*** ihrachys has joined #openstack-infra13:59
dmsimardand then add voting: false13:59
*** salv-orlando has joined #openstack-infra13:59
fricklerdmsimard: is there a difference between adding zuul.d/layout.yaml and modifying the existing .zuul.yaml?14:00
*** links has quit IRC14:01
toskyjeblair (and other zuul core): does this change (zuul.projects as dict) require a zuul restart? https://review.openstack.org/#/c/514119/14:02
dmsimardfrickler: if there's only a .zuul.yaml, you can modify that14:02
dmsimardfrickler: zuul.d is just an easy way to split things up logically14:02
dmsimardtosky: that change is probably live at this point, but I'm not sure. Why ?14:03
*** tobiash has quit IRC14:03
toskydmsimard: because the run of https://review.openstack.org/#/c/514836/ from 2 hours ago says it was not applied (zuul.projects is still a list)14:04
openstackgerritCarlos Camacho proposed openstack-infra/tripleo-ci master: Need to add openstack-selinux with OVS  https://review.openstack.org/52560814:04
dmsimardtosky: I know that we used ._projects14:05
dmsimardas a transition path, you can see it here: http://logs.openstack.org/36/514836/2/check/sahara-image-elements-buildimages-ambari/4435c37/zuul-info/inventory.yaml14:05
toskydmsimard: and that change merged 19 hours ago should reenable zuul.projects as dict14:05
toskyif I understand it correctly14:05
*** armaan has joined #openstack-infra14:05
dmsimardyeah, I'm not sure if executors are automatically restarted14:06
dmsimardI've seen cores restart them to make things apply (quickly) but I'm not sure if it happens automatically eventually14:06
toskywho is the person to bug for this, apart from the poor zuul maintainer I already pinged? :)14:07
dmsimardtosky: jeblair but he's not around yet, should be soon14:07
toskyack, thanks14:07
dmsimardtosky: in the meantime, using _projects will get you what you want14:07
dmsimardtosky: _projects is being kept around until we switch things back to .projects14:07
toskydmsimard: I know, and the patch was working with that, but given that the change to dict was "close" I was waiting for it14:08
dmsimardtosky: you're bleeding edge, woah :D14:09
dmsimardtosky++14:09
toskydmsimard: I'm trying to create native zuulv3 jobs directly without importing the legacy ones14:09
toskyand with that change and the devstack multinode support by Andrea, I can finally start merging patches14:10
*** dbecker has joined #openstack-infra14:13
*** tobiash has joined #openstack-infra14:15
pabelangerdmsimard: tosky: zuul-executors are not automatically restarted. We schedule that because if requires us to stop or abort running jobs14:15
*** dsariel has quit IRC14:15
pabelangerthere have been some recent zuul changes, so we can likely schedule something today (stop / start)14:16
toskypabelanger: ack, thanks; if it's possible, I would be happy14:16
*** esberglu has joined #openstack-infra14:16
dmsimardpabelanger: we should try to land https://review.openstack.org/#/c/516740/ too14:17
dmsimardpabelanger: while we're updating executors14:17
dmsimardIt's running bugging me because of that firefox bug that's long been fixed14:18
pabelangerlets wait until west coast is online and see what people think14:18
dmsimards/running/really/ (wow, dyslexia this morning)14:18
*** tobiash has quit IRC14:19
*** dbecker has quit IRC14:22
*** tobiash has joined #openstack-infra14:24
fungirosmaita: AJaeger_: frickler: which ci systems vote on a particular project is also completely under the control of that project. in this case glance doesn't have an acl to allow any third-party ci systems to leave verified votes, and "VMware NSX CI check" is merely adding review comments with no verified +1/-1 vote, and as noted the only thing we can really do in this case besides reach out and ask14:24
fungithem to stop is to disable that account completely until they fix it14:25
*** dprince has quit IRC14:25
rosmaitafungi thanks ... i was misreading the fact that "(non-voting)" wasn't showing up as part of the result info14:25
*** Goneri has joined #openstack-infra14:26
rosmaitai don't have a problem with them leaving normal +1/-1 votes14:26
*** dbecker has joined #openstack-infra14:26
fungilike, in the comment text?14:27
fungiahh, yeah14:27
*** salv-orlando has quit IRC14:27
*** salv-orlando has joined #openstack-infra14:28
*** smatzek has quit IRC14:28
*** armaan has quit IRC14:28
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] split multinode playbook  https://review.openstack.org/52641414:28
fungithat's entirely configurable on there end but basically just arbitrary text in a comment field from gerrit's point of view. there would have been a separate "Patch Set 7: Verified[+-]1" at the start of the comment and a red -1 or green +1 in the reviews table if they were successfully leaving a verified vote14:29
fungis/there/their/14:29
fungiso unless you're expanding the "toggle ci" button at the bottom you'd in theory never notice them even commenting14:30
*** tobiash has quit IRC14:30
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] split ovb playbook  https://review.openstack.org/52525214:30
*** edmondsw has joined #openstack-infra14:30
*** salv-orlando has quit IRC14:33
*** ramishra has joined #openstack-infra14:35
*** tobiash has joined #openstack-infra14:35
*** ttx has quit IRC14:42
*** wolverineav has quit IRC14:48
*** jesusaur has quit IRC14:48
*** pahuang has quit IRC14:48
*** yee37930 has quit IRC14:48
*** electrofelix has quit IRC14:48
*** dhill_ has quit IRC14:48
*** pbourke has quit IRC14:48
*** openstackgerrit has quit IRC14:48
*** joadavis has quit IRC14:48
*** sai has quit IRC14:48
*** sshnaidm|rover has quit IRC14:48
*** tlbr has quit IRC14:48
*** timothyb89 has quit IRC14:48
*** waynr has quit IRC14:48
*** jzerebec1i has quit IRC14:48
*** ericyoung has quit IRC14:48
*** lifeless has quit IRC14:48
*** afazekas has quit IRC14:48
*** antonym has quit IRC14:48
*** akscram1 has quit IRC14:48
*** yamamoto has quit IRC14:48
*** hughhalf has quit IRC14:48
*** jtomasek has quit IRC14:48
*** adisky_ has quit IRC14:48
*** RuiChen has quit IRC14:48
*** lewo` has quit IRC14:48
*** verdurin has quit IRC14:48
*** neith has quit IRC14:48
*** haleyb has quit IRC14:48
*** clayton has quit IRC14:48
*** mmethot has quit IRC14:48
*** chandankumar has quit IRC14:48
*** tinwood has quit IRC14:48
*** amotoki has quit IRC14:48
*** betherly has quit IRC14:48
*** dangers_away has quit IRC14:48
*** mwhahaha has quit IRC14:48
*** fungi has quit IRC14:48
*** johnsom has quit IRC14:48
*** thingee has quit IRC14:48
*** ameade has quit IRC14:48
*** pas-ha has quit IRC14:48
*** TheJulia has quit IRC14:48
*** simonmcc has quit IRC14:48
*** geekinutah has quit IRC14:48
*** charz has quit IRC14:48
*** bradm has quit IRC14:48
*** prometheanfire has quit IRC14:48
*** robcresswell has quit IRC14:48
*** fdegir has quit IRC14:48
*** logan- has quit IRC14:48
*** jogo has quit IRC14:48
*** Adri2000 has quit IRC14:48
*** DuncanT has quit IRC14:48
*** Krinkle has quit IRC14:48
*** johnthetubaguy has quit IRC14:48
*** petems has quit IRC14:48
*** edwarnicke has quit IRC14:48
*** dirk has quit IRC14:48
*** patricku has quit IRC14:48
*** portdirect has quit IRC14:48
*** abelur has quit IRC14:48
*** v1k0d3n has quit IRC14:48
*** yuval has quit IRC14:48
*** filler has quit IRC14:48
*** ChanServ has quit IRC14:48
*** annp has quit IRC14:48
*** hashar has quit IRC14:48
*** florianf has quit IRC14:48
*** ilpianista_ has quit IRC14:48
*** niedbalski has quit IRC14:48
*** AJaeger_ has quit IRC14:48
*** StevenK has quit IRC14:48
*** snuffkin has quit IRC14:48
*** dalvarez has quit IRC14:48
*** ltomasbo has quit IRC14:48
*** eglute has quit IRC14:48
*** cargonza has quit IRC14:48
*** madhuvishy has quit IRC14:48
*** petrovich has quit IRC14:48
*** d34dh0r53 has quit IRC14:48
*** briancurtin has quit IRC14:48
*** therve has quit IRC14:48
*** jonher has quit IRC14:48
*** thorre has quit IRC14:48
*** toabctl has quit IRC14:48
*** inc0 has quit IRC14:48
*** bmjen has quit IRC14:48
*** stevebaker has quit IRC14:48
*** jlk has quit IRC14:48
*** szaher has quit IRC14:48
*** larsks has quit IRC14:48
*** panda has quit IRC14:48
*** kota_ has quit IRC14:48
*** naichuans_ has quit IRC14:48
*** sdoran has quit IRC14:48
*** spotz has quit IRC14:48
*** jpena has quit IRC14:48
*** kazsh has quit IRC14:48
*** rhallisey has quit IRC14:48
*** sdague has quit IRC14:48
*** ldnunes has quit IRC14:48
*** lihi has quit IRC14:48
*** jrist has quit IRC14:48
*** masber has quit IRC14:48
*** hrubi has quit IRC14:48
*** nhicher has quit IRC14:48
*** anthonyper has quit IRC14:48
*** yuanying has quit IRC14:48
*** ianw has quit IRC14:48
*** zigo has quit IRC14:48
*** anupn has quit IRC14:48
*** mhayden has quit IRC14:48
*** fabo has quit IRC14:48
*** witek has quit IRC14:48
*** Nil_ has quit IRC14:48
*** adarazs has quit IRC14:48
*** jhesketh has quit IRC14:48
*** andreas_s has quit IRC14:48
*** erlon has quit IRC14:48
*** daidv has quit IRC14:48
*** Hal has quit IRC14:48
*** mandre has quit IRC14:48
*** tushar has quit IRC14:48
*** rfolco has quit IRC14:48
*** mikal has quit IRC14:48
*** adriant has quit IRC14:48
*** hamzy has quit IRC14:48
*** lin_yang has quit IRC14:48
*** gmann_afk has quit IRC14:48
*** zhongjun has quit IRC14:48
*** electrical has quit IRC14:48
*** andrewbogott has quit IRC14:48
*** knikolla has quit IRC14:48
*** mgkwill_ has quit IRC14:48
*** tbarron has quit IRC14:48
*** oanson has quit IRC14:48
*** auggy has quit IRC14:48
*** tristanC has quit IRC14:48
*** dimak has quit IRC14:48
*** flwang has quit IRC14:48
*** efried has quit IRC14:48
*** vipul has quit IRC14:48
*** spiffxp has quit IRC14:48
*** matbu has quit IRC14:48
*** rwellum has quit IRC14:48
*** jaypipes has quit IRC14:48
*** trown has quit IRC14:48
*** ethfci has quit IRC14:48
*** jbadiapa has quit IRC14:48
*** calebb has quit IRC14:48
*** dave-mccowan has quit IRC14:48
*** liusheng has quit IRC14:48
*** uberjay has quit IRC14:48
*** tdasilva has quit IRC14:48
*** xinni9e has quit IRC14:48
*** fbouliane has quit IRC14:48
*** serverascode has quit IRC14:48
*** Ng has quit IRC14:48
*** mugsie has quit IRC14:48
*** vkmc has quit IRC14:48
*** bgmccollum has quit IRC14:48
*** persia has quit IRC14:48
*** seongsoocho has quit IRC14:48
*** gothicmindfood has quit IRC14:48
*** amito has quit IRC14:48
*** cmurphy has quit IRC14:48
*** melwitt has quit IRC14:48
*** dham1 has quit IRC14:48
*** kencjohnston has quit IRC14:48
*** oomichi_afk has quit IRC14:48
*** dmanchad has quit IRC14:48
*** wfoster has quit IRC14:48
*** russellb has quit IRC14:48
*** pots has quit IRC14:48
*** odyssey4me has quit IRC14:48
*** tobiash has quit IRC14:48
*** mriedem has quit IRC14:48
*** jkilpatr has quit IRC14:48
*** mat128 has quit IRC14:48
*** bobh has quit IRC14:48
*** alex_xu has quit IRC14:48
*** jascott1 has quit IRC14:48
*** jaosorior has quit IRC14:48
*** Qiming has quit IRC14:48
*** _ari_ has quit IRC14:48
*** myoung|ruck has quit IRC14:48
*** askb has quit IRC14:48
*** mdrabe has quit IRC14:48
*** benj_ has quit IRC14:48
*** sambetts has quit IRC14:48
*** lastmikoi has quit IRC14:48
*** sileht has quit IRC14:48
*** aspiers has quit IRC14:48
*** rcarrillocruz has quit IRC14:48
*** dteselkin has quit IRC14:48
*** mika has quit IRC14:48
*** masayukig has quit IRC14:48
*** notmyname has quit IRC14:48
*** jbernard has quit IRC14:48
*** hemna has quit IRC14:48
*** NikitaKonovalov has quit IRC14:48
*** bswartz has quit IRC14:48
*** lennyb has quit IRC14:48
*** janki has quit IRC14:48
*** numans has quit IRC14:48
*** csomerville has quit IRC14:48
*** caphrim007 has quit IRC14:48
*** mnencia has quit IRC14:48
*** EmilienM has quit IRC14:48
*** migi has quit IRC14:48
*** gaelL_ has quit IRC14:48
*** lathiat_ has quit IRC14:48
*** tonyb has quit IRC14:48
*** strigazi has quit IRC14:48
*** GregHouse has quit IRC14:48
*** njohnston has quit IRC14:48
*** ari[m] has quit IRC14:48
*** mdbooth has quit IRC14:48
*** SpamapS has quit IRC14:48
*** otherwiseguy has quit IRC14:48
*** kuromagi has quit IRC14:48
*** Hunner has quit IRC14:48
*** gtmanfred has quit IRC14:48
*** ssbarnea has quit IRC14:48
*** Marx314 has quit IRC14:48
*** eventingmonkey has quit IRC14:48
*** stephenfin has quit IRC14:48
*** pabelanger has quit IRC14:48
*** Krenair has quit IRC14:48
*** timburke has quit IRC14:48
*** freerunner has quit IRC14:48
*** andreykurilin has quit IRC14:48
*** ddurst has quit IRC14:48
*** SergeyLukjanov has quit IRC14:48
*** DinaBelova has quit IRC14:48
*** ruhe has quit IRC14:48
*** kfox1111 has quit IRC14:48
*** xarses has quit IRC14:48
*** Swami has joined #openstack-infra14:49
*** mmethot has joined #openstack-infra14:49
*** aspiers[m] has quit IRC14:51
*** masayukig[m] has quit IRC14:51
*** sdoran has joined #openstack-infra14:53
*** akscram1 has joined #openstack-infra14:54
*** yamamoto has joined #openstack-infra14:54
*** hughhalf has joined #openstack-infra14:54
*** adisky_ has joined #openstack-infra14:54
*** RuiChen has joined #openstack-infra14:54
*** lewo` has joined #openstack-infra14:54
*** verdurin has joined #openstack-infra14:54
*** neith has joined #openstack-infra14:54
*** haleyb has joined #openstack-infra14:54
*** clayton has joined #openstack-infra14:54
*** chandankumar has joined #openstack-infra14:54
*** tinwood has joined #openstack-infra14:54
*** amotoki has joined #openstack-infra14:54
*** betherly has joined #openstack-infra14:54
*** dangers_away has joined #openstack-infra14:54
*** Krinkle has joined #openstack-infra14:54
*** mwhahaha has joined #openstack-infra14:54
*** fungi has joined #openstack-infra14:54
*** johnsom has joined #openstack-infra14:54
*** pas-ha has joined #openstack-infra14:54
*** thingee has joined #openstack-infra14:54
*** ameade has joined #openstack-infra14:54
*** TheJulia has joined #openstack-infra14:54
*** simonmcc has joined #openstack-infra14:54
*** geekinutah has joined #openstack-infra14:54
*** charz has joined #openstack-infra14:54
*** bradm has joined #openstack-infra14:54
*** prometheanfire has joined #openstack-infra14:54
*** robcresswell has joined #openstack-infra14:54
*** fdegir has joined #openstack-infra14:54
*** logan- has joined #openstack-infra14:54
*** jogo has joined #openstack-infra14:54
*** Adri2000 has joined #openstack-infra14:54
*** DuncanT has joined #openstack-infra14:54
*** johnthetubaguy has joined #openstack-infra14:54
*** petems has joined #openstack-infra14:54
*** edwarnicke has joined #openstack-infra14:54
*** dirk has joined #openstack-infra14:54
*** patricku has joined #openstack-infra14:54
*** portdirect has joined #openstack-infra14:54
*** abelur has joined #openstack-infra14:54
*** v1k0d3n has joined #openstack-infra14:54
*** yuval has joined #openstack-infra14:54
*** filler has joined #openstack-infra14:54
*** hashar has joined #openstack-infra14:54
*** florianf has joined #openstack-infra14:54
*** niedbalski has joined #openstack-infra14:54
*** AJaeger_ has joined #openstack-infra14:54
*** dalvarez has joined #openstack-infra14:54
*** snuffkin has joined #openstack-infra14:54
*** StevenK has joined #openstack-infra14:54
*** ltomasbo has joined #openstack-infra14:54
*** eglute has joined #openstack-infra14:54
*** cargonza has joined #openstack-infra14:54
*** madhuvishy has joined #openstack-infra14:54
*** petrovich has joined #openstack-infra14:54
*** d34dh0r53 has joined #openstack-infra14:54
*** briancurtin has joined #openstack-infra14:54
*** therve has joined #openstack-infra14:54
*** ttx has joined #openstack-infra14:54
*** d0ugal has quit IRC14:54
*** antonym has joined #openstack-infra14:54
*** afazekas has joined #openstack-infra14:54
*** lifeless has joined #openstack-infra14:54
*** ericyoung has joined #openstack-infra14:54
*** jzerebec1i has joined #openstack-infra14:54
*** timothyb89 has joined #openstack-infra14:54
*** tlbr has joined #openstack-infra14:54
*** sshnaidm|rover has joined #openstack-infra14:54
*** sai has joined #openstack-infra14:54
*** waynr has joined #openstack-infra14:54
*** joadavis has joined #openstack-infra14:54
*** openstackgerrit has joined #openstack-infra14:54
*** pbourke has joined #openstack-infra14:54
*** dhill_ has joined #openstack-infra14:54
*** electrofelix has joined #openstack-infra14:54
*** yee37930 has joined #openstack-infra14:54
*** pahuang has joined #openstack-infra14:54
*** jesusaur has joined #openstack-infra14:54
*** wolverineav has joined #openstack-infra14:54
*** annp has joined #openstack-infra14:54
*** tobiash has joined #openstack-infra14:54
*** gouthamr has joined #openstack-infra14:54
*** jamesmcarthur has joined #openstack-infra14:54
*** mriedem has joined #openstack-infra14:54
*** jkilpatr has joined #openstack-infra14:54
*** andreas_s has joined #openstack-infra14:54
*** janki has joined #openstack-infra14:54
*** mat128 has joined #openstack-infra14:54
*** dave-mccowan has joined #openstack-infra14:54
*** erlon has joined #openstack-infra14:54
*** rhallisey has joined #openstack-infra14:54
*** bobh has joined #openstack-infra14:54
*** sdague has joined #openstack-infra14:54
*** alex_xu has joined #openstack-infra14:54
*** daidv has joined #openstack-infra14:54
*** ldnunes has joined #openstack-infra14:54
*** Hal has joined #openstack-infra14:54
*** jascott1 has joined #openstack-infra14:54
*** mandre has joined #openstack-infra14:54
*** numans has joined #openstack-infra14:54
*** csomerville has joined #openstack-infra14:54
*** tushar has joined #openstack-infra14:54
*** jaosorior has joined #openstack-infra14:54
*** caphrim007 has joined #openstack-infra14:54
*** rfolco has joined #openstack-infra14:54
*** jonher has joined #openstack-infra14:54
*** liusheng has joined #openstack-infra14:54
*** mikal has joined #openstack-infra14:54
*** adriant has joined #openstack-infra14:54
*** uberjay has joined #openstack-infra14:54
*** hamzy has joined #openstack-infra14:54
*** Qiming has joined #openstack-infra14:54
*** lin_yang has joined #openstack-infra14:54
*** gmann_afk has joined #openstack-infra14:54
*** zhongjun has joined #openstack-infra14:54
*** electrical has joined #openstack-infra14:54
*** andrewbogott has joined #openstack-infra14:54
*** knikolla has joined #openstack-infra14:54
*** mgkwill_ has joined #openstack-infra14:54
*** tbarron has joined #openstack-infra14:54
*** _ari_ has joined #openstack-infra14:54
*** oanson has joined #openstack-infra14:54
*** auggy has joined #openstack-infra14:54
*** tristanC has joined #openstack-infra14:54
*** dimak has joined #openstack-infra14:54
*** kazsh has joined #openstack-infra14:54
*** lihi has joined #openstack-infra14:54
*** flwang has joined #openstack-infra14:54
*** jrist has joined #openstack-infra14:54
*** efried has joined #openstack-infra14:54
*** witek has joined #openstack-infra14:54
*** vipul has joined #openstack-infra14:54
*** spiffxp has joined #openstack-infra14:54
*** masber has joined #openstack-infra14:54
*** matbu has joined #openstack-infra14:54
*** hrubi has joined #openstack-infra14:54
*** rwellum has joined #openstack-infra14:54
*** nhicher has joined #openstack-infra14:54
*** myoung|ruck has joined #openstack-infra14:54
*** jaypipes has joined #openstack-infra14:54
*** trown has joined #openstack-infra14:54
*** ethfci has joined #openstack-infra14:54
*** jbadiapa has joined #openstack-infra14:54
*** anthonyper has joined #openstack-infra14:54
*** askb has joined #openstack-infra14:54
*** yuanying has joined #openstack-infra14:54
*** calebb has joined #openstack-infra14:54
*** thorre has joined #openstack-infra14:54
*** mdrabe has joined #openstack-infra14:54
*** ianw has joined #openstack-infra14:54
*** mnencia has joined #openstack-infra14:54
*** toabctl has joined #openstack-infra14:54
*** Nil_ has joined #openstack-infra14:54
*** adarazs has joined #openstack-infra14:54
*** fabo has joined #openstack-infra14:54
*** mhayden has joined #openstack-infra14:54
*** anupn has joined #openstack-infra14:54
*** zigo has joined #openstack-infra14:54
*** EmilienM has joined #openstack-infra14:54
*** inc0 has joined #openstack-infra14:54
*** bmjen has joined #openstack-infra14:54
*** stevebaker has joined #openstack-infra14:54
*** jlk has joined #openstack-infra14:54
*** migi has joined #openstack-infra14:54
*** gaelL_ has joined #openstack-infra14:54
*** lathiat_ has joined #openstack-infra14:54
*** benj_ has joined #openstack-infra14:54
*** szaher has joined #openstack-infra14:54
*** sambetts has joined #openstack-infra14:54
*** lastmikoi has joined #openstack-infra14:54
*** larsks has joined #openstack-infra14:54
*** sileht has joined #openstack-infra14:54
*** aspiers has joined #openstack-infra14:54
*** tonyb has joined #openstack-infra14:54
*** panda has joined #openstack-infra14:54
*** rcarrillocruz has joined #openstack-infra14:54
*** amito has joined #openstack-infra14:54
*** dteselkin has joined #openstack-infra14:54
*** mika has joined #openstack-infra14:54
*** jhesketh has joined #openstack-infra14:54
*** tdasilva has joined #openstack-infra14:54
*** masayukig has joined #openstack-infra14:54
*** kota_ has joined #openstack-infra14:54
*** notmyname has joined #openstack-infra14:54
*** naichuans_ has joined #openstack-infra14:54
*** spotz has joined #openstack-infra14:54
*** jpena has joined #openstack-infra14:54
*** jbernard has joined #openstack-infra14:54
*** strigazi has joined #openstack-infra14:54
*** hemna has joined #openstack-infra14:54
*** NikitaKonovalov has joined #openstack-infra14:54
*** bswartz has joined #openstack-infra14:54
*** lennyb has joined #openstack-infra14:54
*** xinni9e has joined #openstack-infra14:54
*** fbouliane has joined #openstack-infra14:54
*** serverascode has joined #openstack-infra14:54
*** GregHouse has joined #openstack-infra14:54
*** Ng has joined #openstack-infra14:54
*** njohnston has joined #openstack-infra14:54
*** mugsie has joined #openstack-infra14:54
*** vkmc has joined #openstack-infra14:54
*** bgmccollum has joined #openstack-infra14:54
*** mdbooth has joined #openstack-infra14:54
*** ari[m] has joined #openstack-infra14:54
*** SpamapS has joined #openstack-infra14:54
*** otherwiseguy has joined #openstack-infra14:54
*** kuromagi has joined #openstack-infra14:54
*** Hunner has joined #openstack-infra14:54
*** persia has joined #openstack-infra14:54
*** gtmanfred has joined #openstack-infra14:54
*** ssbarnea has joined #openstack-infra14:54
*** Marx314 has joined #openstack-infra14:54
*** eventingmonkey has joined #openstack-infra14:54
*** stephenfin has joined #openstack-infra14:54
*** seongsoocho has joined #openstack-infra14:54
*** pabelanger has joined #openstack-infra14:54
*** gothicmindfood has joined #openstack-infra14:54
*** kfox1111 has joined #openstack-infra14:54
*** Krenair has joined #openstack-infra14:54
*** cmurphy has joined #openstack-infra14:54
*** melwitt has joined #openstack-infra14:54
*** timburke has joined #openstack-infra14:54
*** dham1 has joined #openstack-infra14:54
*** kencjohnston has joined #openstack-infra14:54
*** freerunner has joined #openstack-infra14:54
*** andreykurilin has joined #openstack-infra14:54
*** oomichi_afk has joined #openstack-infra14:54
*** ddurst has joined #openstack-infra14:54
*** SergeyLukjanov has joined #openstack-infra14:54
*** DinaBelova has joined #openstack-infra14:54
*** ruhe has joined #openstack-infra14:54
*** dmanchad has joined #openstack-infra14:54
*** wfoster has joined #openstack-infra14:54
*** russellb has joined #openstack-infra14:54
*** pots has joined #openstack-infra14:54
*** odyssey4me has joined #openstack-infra14:54
*** sdoran has quit IRC14:55
*** sdoran has joined #openstack-infra14:55
*** d0ugal has joined #openstack-infra14:55
*** mtreinish has quit IRC14:55
*** jkilpatr has quit IRC14:55
*** tobiash has quit IRC14:56
*** florianf is now known as florianf|afk14:56
*** jkilpatr has joined #openstack-infra14:57
*** salv-orlando has joined #openstack-infra14:57
*** tobiash has joined #openstack-infra14:58
*** yamahata has joined #openstack-infra14:58
*** mtreinish has joined #openstack-infra14:59
sshnaidm|roverdmsimard, pabelanger we have some problem with mirror on rh1 cloud, could you please check this in your time? http://logs.openstack.org/95/522295/2/check-tripleo/tripleo-ci-centos-7-ovb-ha-oooq/94f2eaa/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz#_2017-12-07_13_46_5615:00
sshnaidm|roverdmsimard, pabelanger it's not for every job, but frequently today15:00
dmsimardsshnaidm|rover: can you check if the expected package is there in the upstream (delorean-deps, buildlogs) ?15:01
sshnaidm|roverdmsimard, it seems like problem with connection15:02
dmsimardThe forbidden from the other day was a missing package upstream, just want to make sure it's not something similar (I'd check but on phone right now)15:02
sshnaidm|roverdmsimard, ok, it's completely different :)15:03
fricklerdmsimard: the designate update seems to be working in-repo now, though with the drawback that the patches now run all jobs instead of docs-only and will take some time to complete15:04
dmsimardfrickler: ok, I *think* this is something worth encouraging in the future -- not only to reduce our review load but to make the projects realize they're independant for that kind of thing15:06
*** dsariel has joined #openstack-infra15:06
pabelanger[Thu Dec  7 13:59:27 2017] apache2[15141]: segfault at 70 ip 00007fc9ff2c9968 sp 00007fc9e97e54e0 error 4 in libapr-1.so.0.5.1[7fc9ff2af000+30000]15:06
*** tobiash has quit IRC15:06
pabelangerapache2 crashed15:06
dmsimardfrickler: What bothers me a slight bit is that it's still a PTI template and I guess we don't have a way to prevent projects from turning these jobs non-voting in-tree15:07
dmsimardfrickler: I wonder if job: final prevents that /me looks it up15:07
pabelangerwhich is around the timestamp15:07
*** tobiash has joined #openstack-infra15:07
dmsimardfrickler: "Some attributes of jobs marked job.final may not be overidden" :/15:07
pabelanger[Thu Dec 07 13:59:28.682833 2017] [log_config:warn] [pid 15116:tid 140505342007040] (28)No space left on device: [client 66.187.229.69:43024] AH00646: Error writing to /var/log/apache2/mirror.regionone.tripleo-test-cloud-rh1.openstack.org_8081_access.log, referer:15:08
pabelangerhttp://mirror.regionone.tripleo-test-cloud-rh1.openstack.org:8081/registry-1.docker/v2/tripleomaster/centos-binary-swift-proxy-server/blobs/root@mirror:/home/pabelanger# date15:08
pabelangerso, HDD issue likey again15:08
dmsimardpabelanger: again ?15:08
pabelangerlikely caching faster then we can expire15:08
dmsimardcan we resize the VM with a larger disk ?15:08
jeblairdmsimard: letting projects flip jobs to non-voting on their own was a design goal.  if they abuse it, the tc can take it up with them.  this is us getting out of being the policy police.  :)15:08
openstackgerritMerged openstack-infra/tripleo-ci master: Send graphite stats from ARA  https://review.openstack.org/48012115:08
jeblairdmsimard: voting can be overridden on a final job.15:08
*** annp has quit IRC15:08
dmsimardjeblair: yeah I think that's fair, but we're still in the business of "enforcing" PTI jobs through project-config15:09
dmsimardjeblair: whether projects ultimately end respecting that or not is TC business, you're right :D15:09
jeblairdmsimard: not really -- we're in business of facilitating compliance.  subtle but important distinction.15:09
dmsimardjeblair: indeed, wrong vocabulary15:09
jeblairdmsimard: we've already said if projects need to drop the pti templates and go in-tree, they can.15:09
fricklerjeblair: there still seems to be an issue with the new mistral repo, in case you haven't read backlog yet15:10
jeblairor, at least we said they can drop the templates15:10
pabelangerwe'll need to get more HDD on tripleo-test-cloud-r1 or decrease caching there15:10
dmsimardpabelanger: we already decreased caching earlier this week, can we consider getting more disk allocation ?15:11
jeblairdmsimard: only attributes that affect what happens inside the job can not be overridden on final jobs.  so voting, success-message, tags, etc are all fair game.15:11
*** tobiash has quit IRC15:11
*** alexchadin has joined #openstack-infra15:11
jeblairfrickler: zuul needs another reconfiguration.  either another change to main.yaml, or a root to run the command manually.15:11
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Fix on Published Events Query  https://review.openstack.org/52642815:11
dmsimardjeblair: I'll try and improve the docs to mention that, thanks15:11
pabelangerdmsimard: yes, but the cloud doesn't have much to spare, last I looked15:11
openstackgerritMerged openstack-infra/openstackid-resources master: Fix on Published Events Query  https://review.openstack.org/52642815:12
jeblairdmsimard: you can get the list from https://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/model.py?id=feature/zuulv3#n81415:12
*** tobiash has joined #openstack-infra15:12
dmsimardpabelanger: that's ephemeral, right ? sshnaidm|rover who is the rh1 contact now that bnemec is moving on to other things ?15:12
dmsimardjeblair: thanks!15:13
*** alexchadin has quit IRC15:13
pabelangerdmsimard: right, today we have 100GB partition, we want to bump that to 200GB15:14
pabelangeror add volume15:14
dmsimardI think fungi mentioned rh1 didn't have cinder, does it ?15:14
pabelangerno, don't think they do15:14
*** dbecker has quit IRC15:15
*** ramishra has quit IRC15:15
pabelangerokay, I have to run an errand this morning, I did purge some puppet reports on the mirror. That is something we could do in puppet now, is disable report collections, as we no longer publish them any place15:15
jeblairfrickler: i'm kicking off a manual full reconfiguration.  it will pause for about 6 minutes, then should know about the new repos.15:15
pabelangerdmsimard: if you want to look into that, that would be great, otherwise, I'll push up a patch when I return15:16
dmsimardpabelanger: I'm not root yet :(15:16
*** ChanServ has joined #openstack-infra15:16
*** barjavel.freenode.net sets mode: +o ChanServ15:16
pabelangerdmsimard: don't need root for that, it will be in our puppet modules15:16
dmsimardpabelanger: oh, disabling report collections you mean -- I'll look15:16
pabelangeryah15:16
dmsimardpabelanger: is the mirror okay now though ?15:16
jeblairmaking changes never requires root15:16
*** myoung|ruck is now known as myoung|rover15:16
pabelangerI think it will be in ansible-puppet (puppet.conf)15:16
fricklerjeblair: or we could merge https://review.openstack.org/52505215:16
dmsimardjeblair: fixing the broken rh1 mirror does, I misread15:16
pabelangerdmsimard: there is 3.3GB avail now15:17
dmsimardpabelanger: ok, did you flush afs ? just making sure we don't need another root looking at it immediately before you run off :D15:17
jeblairfrickler: feel free, but the reconfiguration is already in progress, so don't rush it on account of this.15:17
*** ramishra has joined #openstack-infra15:17
*** dprince has joined #openstack-infra15:17
*** jbadiapa has quit IRC15:17
pabelangerdmsimard: no, flush. Only thing I see was apache2 crash, I think because it ran out of disk space15:18
pabelangerI don't see AFS errors15:18
jeblairdmsimard, pabelanger: why don't you reduce the afs cache size so it doesn't fill up the disk?15:18
pabelangerjeblair: yah, we can also do that too15:18
dmsimardjeblair: I think fungi already reduced that earlier this week but I don't have details15:18
pabelangerIIRC, we set to 50GB15:18
dmsimardthere was also an htcacheclean running for longer than usual15:18
jeblaircool15:18
dmsimardpabelanger: ah ok so it was just apache segfaulting then, ok15:18
pabelangerokay, need to run, back shortly15:18
*** dhill_ has quit IRC15:19
dmsimardsshnaidm|rover: should be addressed for the time being, please let us know if you see new errors -- we'll need to plan a maintenance to increase the disk size of that VM15:19
*** d0ugal is now known as Guest2746015:20
*** xarses has joined #openstack-infra15:21
dmsimardoh, now that I'm thinking about it, no need to plan a maintenance -- let's just get a bigger VM when we upgrade to xenial /me adds note to sprint pad15:21
*** janki has quit IRC15:21
*** smatzek has joined #openstack-infra15:21
AJaeger_config-core, could you go over gerrit and review some time this week, please? We have a small backlog including 10+ changes that have already one +2. Would be nice to merge what is ready...15:22
*** Guest27460 has quit IRC15:23
*** d0ugal has joined #openstack-infra15:23
*** d0ugal has quit IRC15:23
*** d0ugal has joined #openstack-infra15:23
*** hongbin has joined #openstack-infra15:23
jeblairfrickler: zuul is operating again15:23
fungidmsimard: pabelanger: i didn't reduce any settings, just removed a portion of the apache cache. it looks like tripleo wants to cache more than 40gb through its apache proxy recently and htcacheclean just doesn't keep up. we could reduce the cache size manually on that mirror and stop puppeting it until it goes away, or we could make it possible to fine-tune different cache sizes on different mirrors15:24
dmsimardfungi: I added a note on the sprint pad, we need to upgrade that mirror to xenial anyway so let's take that opportunity to pick a flavor with sufficient disk allocation15:24
dmsimardjust need to make sure it can survive until next week15:25
fungidmsimard: pabelanger: to clarify, we could reduce the afs cache size on it. reducing the apache proxy cache size won't help as it already seems to accumulate many gigabytes of additional data in the time it takes htcacheclean to run15:25
*** dhill_ has joined #openstack-infra15:25
*** links has joined #openstack-infra15:25
openstackgerritMerged openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/52628915:26
*** hemna_ has joined #openstack-infra15:27
*** alexchadin has joined #openstack-infra15:28
jeblairoh, reducing the afs cache (at least to anything > 0) probably won't help then -- apache will fill the disk no matter what.15:30
jeblairputting the afs cache on a different partition (and, then, likely reducing its size to fit) probably would help avoid afs data getting corrupted.15:31
*** jbadiapa has joined #openstack-infra15:31
*** ramishra has quit IRC15:31
*** masayukig[m] has joined #openstack-infra15:32
*** yamamoto has quit IRC15:33
*** ramishra has joined #openstack-infra15:34
fricklerdmsimard: AJaeger_: for https://review.openstack.org/525052 Anusha is listed like all congress cores at https://wiki.openstack.org/wiki/CrossProjectLiaisons#Infra, so should be o.k. to proceed?15:35
*** nikhil has joined #openstack-infra15:35
*** yamahata has quit IRC15:35
openstackgerritMerged openstack-infra/project-config master: Run monascaclient docs jobs on master only  https://review.openstack.org/52604315:40
*** rbrndt has joined #openstack-infra15:40
*** kiennt26 has quit IRC15:40
openstackgerritMerged openstack-infra/project-config master: Add log-classify project template  https://review.openstack.org/52563915:43
fungijeblair: yep, but given the limitations of the tripleo test cloud, it sounds like additional partitions may mean some design changes in their deployment15:44
dmsimardfrickler: I pinged ecks15:44
sshnaidm|roverdmsimard, I don't think we have somebody instead of bnemec, I'm not aware of that at least. But he didn't manage the proxy itself. I have admin access, so I can help there is need.15:44
dmsimardfrickler: usually we wait for an ack from the ptl15:44
AJaeger_frickler: we explictely ask for PTL for new projects15:44
jeblairfungi: yes, though i didn't necessarily mean additional devices.15:44
fungielsewhere we either have a very large rootfs or we have cinder15:44
AJaeger_dmsimard, frickler : Thanks for reviewing!15:44
dmsimardAJaeger_: thanks for pinging15:45
fungijeblair: i suppose we could find some way to repartition the root device?15:45
dmsimardAJaeger_: I always open up the review dashboard, do a few reviews but then end up getting sidetracked15:45
jeblairfungi: yes, or bind-mount a file15:45
fungioh, i didn't consider a loopback device, yeah that could work15:45
dmsimardjeblair, fungi: Do we really need to consider that if we're going to reinstall the mirror on xenial with a bigger disk next week ?15:46
jeblairsorry, yes, i used wrong words before breakfast15:46
*** sshnaidm|rover is now known as sshnaidm|ruck15:46
jeblairdmsimard: we should *consider* it, but if you think you can duct tape it until then, we probably shouldn't do it.15:46
jeblair(ie, if we only run out of space every 10 days, we're probably good)15:46
dmsimardwe ran out of space sometime earlier this week, causing issues on the pypi mirror15:47
dmsimardI don't have a lot of visibility on what we can possibly duct tape to make us survive until, like, monday15:47
dmsimardI mean, other than what has been proposed so far15:47
*** yamamoto has joined #openstack-infra15:48
*** yamamoto has quit IRC15:48
*** jamesmcarthur has quit IRC15:49
*** ramishra has quit IRC15:50
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Correctly stream the remaining buffer  https://review.openstack.org/51547015:51
*** caphrim007 has quit IRC15:53
*** caphrim007 has joined #openstack-infra15:54
*** dizquierdo has joined #openstack-infra15:55
*** caphrim007 has quit IRC15:58
*** links has quit IRC15:59
TheJuliaIs it just me, or did the bot not return from the netsplit?16:02
jeblairTheJulia: which bot?16:03
TheJuliajeblair: nvmd, i see openstackbot, I was looking for unvoiced16:03
*** tosky has quit IRC16:03
*** rhallisey has quit IRC16:06
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test ci  https://review.openstack.org/52644316:08
*** alexchadin has quit IRC16:09
openstackgerritCarlos Camacho proposed openstack-infra/tripleo-ci master: Need to add openstack-selinux with OVS  https://review.openstack.org/52560816:09
*** ilpianista_ has joined #openstack-infra16:10
*** aspiers[m] has joined #openstack-infra16:10
*** tosky has joined #openstack-infra16:10
dmsimardjeblair: I thought I'd check the memory usage as I was approving a batch of reviews and it's pretty high.. http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=63979&rra_id=all16:11
jeblairdmsimard: it's been restarted without the manually applied repl patch, so i can't really investigate why.  we could restart it again with the patch and poke at it next week...16:13
dmsimardyeah, at this rate we're going to be in swapping territory soon16:13
dmsimardif not already16:13
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] split multinode playbook  https://review.openstack.org/52641416:16
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add finger gateway  https://review.openstack.org/52527616:17
*** smatzek has quit IRC16:18
*** slaweq has quit IRC16:18
*** dhajare has joined #openstack-infra16:21
*** thiagolib has joined #openstack-infra16:22
*** slaweq has joined #openstack-infra16:23
thiagolibHello I would like to know where I can find a step by step to install os_loganalyze on my log server16:23
*** Apoorva has joined #openstack-infra16:24
dmsimardthiagolib: https://github.com/openstack-infra/puppet-openstackci/blob/master/manifests/logserver.pp would probably be a good place to look at16:26
dmsimardthiagolib: you might also find https://docs.openstack.org/infra/openstackci/ helpful16:26
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Collect unbound ttl config file  https://review.openstack.org/52644716:27
*** slaweq has quit IRC16:28
pabelangerfungi: agree, I think we are caching faster then htcacheclean can run. We should see about adding more HDD space, as you mentioned the otherday16:29
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Use playbook from tripleo-quickstart-extras for OVB  https://review.openstack.org/51350816:31
openstackgerritMerged openstack-infra/zuul-jobs master: revoke-sudo: only revoke when zuul is sudoer  https://review.openstack.org/52226116:33
openstackgerritMerged openstack-infra/zuul-jobs master: bindep: fix install bindep_command fact name  https://review.openstack.org/52550316:33
openstackgerritMerged openstack-infra/zuul-jobs master: bindep: skip package install when not needed  https://review.openstack.org/52550616:33
openstackgerritMerged openstack-infra/zuul-jobs master: Fix npm-run-test  https://review.openstack.org/51887916:33
*** dmsimard is now known as dmsimard|afk16:34
dmsimard|afkpabelanger: I added a note on the sprint pad about rh1 mirror server16:34
pabelangerjeblair: there was some discussion this morning in backscroll about restart zuul-executors to pickup some recent changes in zuul code base. I know we landed a few patches this week, is that something we want to schedule?16:34
* dmsimard|afk afk16:34
pabelangerdmsimard|afk: ok16:34
*** hashar is now known as hasharAway16:35
jeblairpabelanger: yeah, based on the memory graphs, we should probably plan on restarting scheduler and executors together16:37
*** dsariel has quit IRC16:38
*** ilpianista_ has quit IRC16:39
*** aspiers[m] has quit IRC16:39
*** masayukig[m] has quit IRC16:39
*** ccamacho has quit IRC16:39
*** cshastri has joined #openstack-infra16:39
pabelangerah, good point. I didn't look at cacti.o.o16:40
*** dizquierdo has quit IRC16:41
fricklerinfra-root: neutron is still having a bit of an issue with job timeouts, see scrollback and my proposal at https://review.openstack.org/52634216:41
openstackgerritMerged openstack-infra/tripleo-ci master: Creating a function that calls /usr/bin/timeout  https://review.openstack.org/52559416:43
*** isq_ has joined #openstack-infra16:45
*** slaweq has joined #openstack-infra16:45
*** caphrim007 has joined #openstack-infra16:45
jeblair#status log This message is to inform you that the host your cloud server 'ze04.openstack.org' resides on alerted our monitoring systems at 16:35 UTC. We are currently investigating the issue and will update you as soon as we have additional information regarding what is causing the alert.  Please do not access or modify 'ze04.openstack.org' during this process.16:45
openstackstatusjeblair: finished logging16:45
jeblairemail from rax ^16:46
pabelangerfrickler: what is the timeout on normal tox job runs for neutron?16:46
*** luzC has quit IRC16:46
*** yamahata__ has quit IRC16:46
*** anupn has quit IRC16:46
*** katkapilatova has left #openstack-infra16:47
*** katkapilatova has quit IRC16:47
*** caphrim007 has quit IRC16:47
*** caphrim007 has joined #openstack-infra16:47
*** yamamoto has joined #openstack-infra16:49
openstackgerritMerged openstack-infra/project-config master: Move governance tag validation to experimental  https://review.openstack.org/52456316:49
openstackgerritMerged openstack-infra/project-config master: Publish governance-sigs content to static.o.o  https://review.openstack.org/52455716:49
openstackgerritMerged openstack-infra/project-config master: Remove legacy jobs for freezer  https://review.openstack.org/52425616:49
openstackgerritMerged openstack-infra/project-config master: Make stable jobs voting on Tempest  https://review.openstack.org/52474116:49
openstackgerritMerged openstack-infra/project-config master: Set up translation for Heat Dashboard  https://review.openstack.org/52502816:49
openstackgerritMerged openstack-infra/project-config master: Add an openstack/self-healing-sig repository  https://review.openstack.org/52527016:49
openstackgerritMerged openstack-infra/project-config master: Use  publish-openstack-sphinx-docs-neutron  https://review.openstack.org/52457416:49
*** luzC has joined #openstack-infra16:49
openstackgerritMerged openstack-infra/project-config master: Don't attempt to run py27 on nova-specs  https://review.openstack.org/52068416:49
openstackgerritMerged openstack-infra/project-config master: Remove freezer-web-ui legacy jobs  https://review.openstack.org/52553016:49
openstackgerritMerged openstack-infra/project-config master: Publish deploy guide updates for all branches  https://review.openstack.org/52515416:49
openstackgerritMerged openstack-infra/project-config master: Remove freezer-api legacy jobs  https://review.openstack.org/52517516:49
openstackgerritMerged openstack-infra/project-config master: Add release notes job to python-octaviaclient  https://review.openstack.org/52534716:49
*** slaweq has quit IRC16:49
*** anupn has joined #openstack-infra16:51
openstackgerritTobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Add connection-port to provider diskimage  https://review.openstack.org/50411216:52
*** salv-orlando has quit IRC16:53
fricklerpabelanger: the default timeout is 30 mins, inherited from base16:54
*** yamamoto has quit IRC16:55
fricklerpabelanger: but I'm seeing successful runs take around 25 mins, so I'd want to start rather high. if they all finish within 32 minutes, we could also reduce to 40 minutes. but my priority in the current situation would be trying to reduce the number of gate resets16:56
*** yamahata has joined #openstack-infra16:58
fricklerjeblair: mistral-tempest-plugin addition seems to have been successful now, config-core please have another look at https://review.openstack.org/52495216:58
pabelangerfrickler: we should check what the timeout was in zuulv2.5 too, but sounds like we are on the edge. 40mins seems a little better to me16:58
*** jascott1 has quit IRC17:02
*** Apoorva has quit IRC17:02
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Drop legacy-horizon-dsvm-tempest-plugin  https://review.openstack.org/52412417:02
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Add heat-tempest-plugin to grenade/sahara job  https://review.openstack.org/52525117:02
*** jascott1 has joined #openstack-infra17:02
fungipabelanger: not sure about neutron but i vaguely recall nova's unit test timeout was 40 minutes (i may also have imagined this)17:03
*** Zara has quit IRC17:04
fricklerpabelanger: fungi: indeed "{name}-coverage-{node}{suffix}" had 40 mins17:04
openstackgerritTobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Add connection-port to provider diskimage  https://review.openstack.org/50411217:05
fricklerhttps://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/python-jobs.yaml?h=before-jenkins-config-removed#n617:05
fricklerso should we do this globally instead of only for neutron?17:05
AJaeger_config-core, could I get another +2 on a job removal, please? https://review.openstack.org/52571717:05
*** shardy is now known as shardy_afk17:06
AJaeger_frickler: I'm fine with doing it globally17:06
*** felipemonteiro_ has joined #openstack-infra17:06
*** yamahata has quit IRC17:06
fricklerAJaeger_: +3 to the i18n removal17:07
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: WIP: Git driver  https://review.openstack.org/52561417:07
*** harlowja has joined #openstack-infra17:08
* frickler will step out for a bit, will do the job timeout later if noone beats me to it ... hint, hint ;)17:08
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add command socket support to zuul-merger  https://review.openstack.org/52319717:08
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add command_socket setting to executor section  https://review.openstack.org/52346517:08
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Move send_command() into ZuulApp  https://review.openstack.org/52321117:08
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add command socket support to zuul-scheduler  https://review.openstack.org/52346617:08
jeblaircool ^ we should keep a close eye on that when we restart17:09
openstackgerritRico Lin proposed openstack-infra/project-config master: Add release-notes-jobs for heat-agents and python-heatclient  https://review.openstack.org/52646117:09
pabelangerfrickler: fungi 40min++17:09
jeblair(hopefully it hasn't broken anything, but some of that is hard to test in unit tests)17:09
openstackgerritTobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Don't gather host keys for non ssh connections  https://review.openstack.org/50316617:09
pabelangeragree, i tried to test it locally17:09
openstackgerritTobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Add connection-port to provider diskimage  https://review.openstack.org/50411217:09
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Rename ssh_port to connection_port  https://review.openstack.org/50080017:10
pabelangerwe'll also need to update init scripts too17:10
pabelangerI can propose that now17:10
AJaeger_thanks, frickler17:10
AJaeger_pabelanger, frickler , dmsimard|afk (once you're back), could either of you look at http://lists.openstack.org/pipermail/openstack-infra/2017-December/005709.html and either do it - or tell me what to do, please?17:11
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Support username also for unmanaged cloud images  https://review.openstack.org/50080817:12
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Add connection-type to provider diskimage  https://review.openstack.org/50314817:12
*** Swami has quit IRC17:15
*** salv-orlando has joined #openstack-infra17:15
openstackgerritPaul Belanger proposed openstack-infra/puppet-zuul master: Update merger / scheduler init script to use command socket  https://review.openstack.org/52646317:15
*** dtantsur is now known as dtantsur|afk17:15
pabelangerjeblair: puppet change^ but will break v2.5 installs. Should we consider creating a v2.5 tag before landing it?17:16
*** rkukura has quit IRC17:18
*** csomerville has quit IRC17:18
*** andreas_s has quit IRC17:19
*** csomerville has joined #openstack-infra17:19
*** andreas_s has joined #openstack-infra17:19
*** rhallisey has joined #openstack-infra17:21
openstackgerritMerged openstack-infra/project-config master: Added initial jobs for mistral-tempest-plugin  https://review.openstack.org/52495217:21
openstackgerritMerged openstack-infra/project-config master: Remove build-openstack-sphinx-docs from i18n  https://review.openstack.org/52571717:21
*** cody-somerville has joined #openstack-infra17:22
*** cshastri has quit IRC17:23
*** salv-orlando has quit IRC17:24
*** rfolco is now known as rfolco|brb17:25
*** csomerville has quit IRC17:26
*** andreas_s has quit IRC17:28
*** jpich has quit IRC17:29
*** andreas_s has joined #openstack-infra17:29
*** andreas_s has quit IRC17:29
*** andreas_s has joined #openstack-infra17:29
*** e0ne has quit IRC17:29
*** kjackal has quit IRC17:30
*** jascott1 has quit IRC17:31
*** jascott1 has joined #openstack-infra17:31
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Rename ssh_port to connection_port  https://review.openstack.org/50079917:32
jeblairpabelanger: we have /etc/defaults files for v3.  let's use that to switch behavior in the init script to support both17:33
*** andreas_s has quit IRC17:34
*** ccamacho has joined #openstack-infra17:36
*** gyee has joined #openstack-infra17:37
*** ccamacho has quit IRC17:39
*** Apoorva has joined #openstack-infra17:39
pabelangerjeblair: yah, good idea17:39
jeblair#status log This message is to inform you that the host your cloud server 'ze04.openstack.org' resides on became unresponsive. We have rebooted the server and will continue to monitor it for any further alerts.17:40
openstackstatusjeblair: finished logging17:40
fungiinfra-root: i'm going to grab lunch in a bit, but when i get back looks like maybe i can work on tearing out the docs-draft.o.o site and reclaiming its volume for use by the logs.o.o site? http://cacti.openstack.org/cacti/graph.php?local_graph_id=718&rra_id=all17:41
jeblairfungi: ya, i think that time has elapsed, thanks!17:42
fungiit's been empty for about a month now17:42
*** salv-orlando has joined #openstack-infra17:42
* fungi makes a note to also simplify the expiration/pruning cronjob for it in puppet-openstackci too17:43
*** cshastri has joined #openstack-infra17:44
fungihttp://paste.openstack.org/show/628379/17:45
fungithat'll be a nice bump in available space _and_ inode capacity17:45
*** inc0 has quit IRC17:46
fungiwhich we need since it now takes nearly 2 weeks for our expiration pass to completely crawl the tree17:46
*** inc0 has joined #openstack-infra17:46
*** dhajare has quit IRC17:47
*** jpena is now known as jpena|off17:48
*** kjackal has joined #openstack-infra17:49
*** yamamoto has joined #openstack-infra17:51
*** masayukig[m] has joined #openstack-infra17:51
*** lucasagomes is now known as lucas-afk17:52
*** rkukura has joined #openstack-infra17:52
*** rfolco|brb is now known as rfolco17:53
*** salv-orlando has quit IRC17:53
*** salv-orlando has joined #openstack-infra17:53
*** derekh has quit IRC17:54
*** Wei_Liu has quit IRC17:55
*** yamamoto has quit IRC17:56
*** salv-orlando has quit IRC17:57
*** jamesmcarthur has joined #openstack-infra17:58
fungithus, more headroom needed to allow for the flux17:59
*** slaweq has joined #openstack-infra17:59
*** salv-orlando has joined #openstack-infra18:00
pabelanger++18:00
*** salv-orlando has quit IRC18:04
*** slaweq has quit IRC18:04
*** salv-orlando has joined #openstack-infra18:04
*** masayukig[m] has quit IRC18:05
*** andreas_s has joined #openstack-infra18:05
*** bandini has quit IRC18:07
*** bandini has joined #openstack-infra18:07
*** [HeOS] has quit IRC18:08
*** Hal has quit IRC18:08
*** salv-orlando has quit IRC18:09
*** sambetts is now known as sambetts|afk18:09
*** andreas_s has quit IRC18:09
*** salv-orlando has joined #openstack-infra18:10
*** tesseract has quit IRC18:13
*** jascott1 has quit IRC18:15
*** salv-orlando has quit IRC18:16
*** salv-orlando has joined #openstack-infra18:16
*** trown is now known as trown|lunch18:18
*** salv-orlando has quit IRC18:19
*** salv-orlando has joined #openstack-infra18:20
openstackgerritJens Harbott (frickler) proposed openstack-infra/openstack-zuul-jobs master: Increase timeout for openstack-tox-cover job  https://review.openstack.org/52647418:20
*** ldesimone has quit IRC18:21
*** ldesimone has joined #openstack-infra18:21
*** felipemonteiro_ has quit IRC18:22
*** e0ne has joined #openstack-infra18:23
*** vivsoni__ has joined #openstack-infra18:24
*** vivsoni_ has quit IRC18:24
*** salv-orlando has quit IRC18:24
*** esberglu has quit IRC18:24
*** esberglu has joined #openstack-infra18:25
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Add support for protected jobs  https://review.openstack.org/52298518:25
*** ldesimone has quit IRC18:25
*** esberglu_ has joined #openstack-infra18:29
fricklerclarkb: pabelanger: added our sprint to https://wiki.openstack.org/wiki/VirtualSprints mentioning you two as organizers, please review if needed18:29
*** numans has quit IRC18:30
*** esberglu has quit IRC18:30
pabelangerfrickler: Ah, thanks. I was meaning to do that18:30
*** thorre has quit IRC18:31
*** thorre has joined #openstack-infra18:31
*** numans has joined #openstack-infra18:32
*** SumitNaiksatam has joined #openstack-infra18:34
*** cody-somerville has quit IRC18:34
*** nikhil has quit IRC18:34
*** cody-somerville has joined #openstack-infra18:34
*** cody-somerville has quit IRC18:34
*** cody-somerville has joined #openstack-infra18:34
*** masayukig[m] has joined #openstack-infra18:35
*** slaweq has joined #openstack-infra18:35
inc0hey, how do I make pipeline of jobs in zuulv3? as in run job 2 when job 1 succeeds18:35
*** martinkopec has quit IRC18:35
*** dprince has quit IRC18:36
jeblairinc0: https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html#attr-job.dependencies18:36
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove implied playbook extensions  https://review.openstack.org/52523918:36
inc0thanks jeblair18:37
jeblairinc0: note that lets you create a full DAG18:37
inc0DAG?18:38
jeblairinc0: directed acyclic graph18:38
inc0ah, right, thanks, what's what I'm after:)18:38
jeblairso one job can spawn two more, and then a fourth job can run only after both of those have completed18:39
*** andreas_s has joined #openstack-infra18:39
jeblairthat's the main reason for the syntax change from v2, which could only support a tree18:39
inc0hmm...job spawning a job? how that works?18:40
jeblairinc0: i just mean that the two in the middle run after the first completes18:40
*** slaweq has quit IRC18:40
inc0ah ok18:40
jeblairi guess 'release' would be a better word18:40
*** rbrndt has quit IRC18:40
*** ldnunes has quit IRC18:42
*** sshnaidm|ruck is now known as sshnaidm|off18:43
*** andreas_s has quit IRC18:44
*** csomerville has joined #openstack-infra18:45
*** tosky has quit IRC18:47
*** cody-somerville has quit IRC18:49
*** cody-somerville has joined #openstack-infra18:50
*** csomerville has quit IRC18:50
fricklerconfig-core: openstack-tox-cover timeout patch: https://review.openstack.org/52647418:52
*** jkilpatr has quit IRC18:52
*** yamamoto has joined #openstack-infra18:52
*** ldnunes has joined #openstack-infra18:54
*** shardy_afk has quit IRC18:54
*** dhinesh has joined #openstack-infra18:54
*** yamamoto has quit IRC18:57
*** dhinesh_ has joined #openstack-infra18:57
*** bobh has quit IRC18:59
*** dhinesh has quit IRC18:59
*** edmondsw has quit IRC19:00
*** ldnunes has quit IRC19:01
*** dprince has joined #openstack-infra19:02
*** chandankumar has quit IRC19:04
*** csomerville has joined #openstack-infra19:04
*** rhallisey has quit IRC19:05
dirkDoes anyone know what happened to the periodic requirement generate upper constraints job? It doesn't seem to be running for the last two days19:05
dirkAnd I can't find the logs for it anymore19:05
*** aspiers[m] has joined #openstack-infra19:06
*** ilpianista_ has joined #openstack-infra19:06
openstackgerritJames E. Blair proposed openstack-infra/puppet-mailman master: WIP Add multihost support  https://review.openstack.org/52648319:06
*** baoli has joined #openstack-infra19:07
*** cody-somerville has quit IRC19:08
*** electrofelix has quit IRC19:08
*** dhinesh_ has quit IRC19:08
jeblairfungi, clarkb: ^ that's very much a WIP.  it shows the approach i'm taking to support multiple installs on the same host, while actually using the underlying debuntu-provided mailman package.19:09
*** felipemonteiro_ has joined #openstack-infra19:10
*** adisky_ has quit IRC19:10
jeblairi've verified that works for list creation, web hosting, and the mailman daemons themselves.19:10
jeblairi'm pushing it up now so we can take a look at it and see if we think it's tenable, or if it's insane and we should just run multiple servers or wait for containers to solve our problems19:11
*** felipemonteiro__ has joined #openstack-infra19:11
*** trown|lunch is now known as trown19:11
*** tosky has joined #openstack-infra19:12
*** [HeOS] has joined #openstack-infra19:12
*** rhallisey has joined #openstack-infra19:12
*** slaweq has joined #openstack-infra19:12
*** jkilpatr has joined #openstack-infra19:13
*** Goneri has quit IRC19:14
*** felipemonteiro_ has quit IRC19:15
*** slaweq has quit IRC19:17
*** bobh has joined #openstack-infra19:17
*** jkilpatr has quit IRC19:18
*** jkilpatr has joined #openstack-infra19:19
*** andreas_s has joined #openstack-infra19:20
*** harlowja has quit IRC19:21
*** armaan has joined #openstack-infra19:23
*** andreas_s has quit IRC19:25
*** jbadiapa has quit IRC19:28
*** armaan has quit IRC19:28
*** kjackal has quit IRC19:30
*** kjackal has joined #openstack-infra19:30
*** jamesmcarthur has quit IRC19:30
*** esberglu_ is now known as esberglu19:33
*** kjackal has quit IRC19:33
*** ilpianista_ has quit IRC19:33
*** aspiers[m] has quit IRC19:33
*** masayukig[m] has quit IRC19:34
*** edmondsw has joined #openstack-infra19:35
*** kjackal has joined #openstack-infra19:35
*** dhinesh has joined #openstack-infra19:35
*** e0ne has quit IRC19:36
*** bandini has quit IRC19:36
*** bandini has joined #openstack-infra19:38
*** [HeOS] has quit IRC19:39
*** edmondsw has quit IRC19:39
*** e0ne has joined #openstack-infra19:40
*** edmondsw has joined #openstack-infra19:41
*** Wei_Liu has joined #openstack-infra19:41
*** andreas_s has joined #openstack-infra19:43
*** rbrndt has joined #openstack-infra19:43
*** [HeOS] has joined #openstack-infra19:43
*** slaweq has joined #openstack-infra19:43
*** edmondsw has quit IRC19:45
*** andreas_s has quit IRC19:47
*** slaweq has quit IRC19:47
fungicontainers solve everything19:50
fungior that's what the people who are at kubecon keep saying19:50
fungii didn't go, so i missed the mandatory kool-aid19:50
clarkbnot really here because sick but ya I think it likely much easier to sort out hosting things like etherpad and ethercalc via containers before trying to sort out mailman19:52
*** edmondsw has joined #openstack-infra19:52
fungietherpad and ethercalc don't seem like they should need it (i say we just rebrand those to something neutral)19:53
*** yamamoto has joined #openstack-infra19:53
fungimailing lists as a point of contact for projects are much more prominent and tend to want some tailored brand identity, so the multi-hosting approach makes a bit more sense to me19:54
clarkbya they don't need it in the context of friendly hosting, more they should be easy to get running on such a system so would be good canaries19:54
*** mikal_ has joined #openstack-infra19:55
*** e0ne has quit IRC19:55
*** edmondsw has quit IRC19:56
*** mikal has quit IRC19:57
*** yamamoto has quit IRC19:58
*** edmondsw has joined #openstack-infra19:58
*** edmondsw has quit IRC19:58
*** edmondsw has joined #openstack-infra19:58
fungii'm still unconvinced there's any containerization which makes any of what we're doing now particularly easier20:00
*** mikal has joined #openstack-infra20:00
*** andreas_s has joined #openstack-infra20:01
*** andreas_s has quit IRC20:01
*** andreas_s has joined #openstack-infra20:01
fungie.g., how would you run multiple receiving mtas in different containers on a vm which gets only one globally routable ipv4 address? they can't all listen on 25/tcp on the same address20:01
*** mikal_ has quit IRC20:01
fungii'm guessing you get to run a single mta receiving e-mail and routing it for multiple domains each on private container-to-container networks. that seems a lot _less_ simple20:02
*** edmondsw has quit IRC20:02
fungisimilarly, you can't have multiple containerized apache daemons serving their individual webuis all listening on the same address+port, so you end up with another routing proxy which works out the sitename to c2c network addressing and forwards the requests and responses on their behalf?20:04
*** andreas_s has quit IRC20:05
*** armaan has joined #openstack-infra20:05
clarkbfungi: yup, this is one of the big benefits of using k8s aiui, the load balancing is built in20:05
clarkb(so you don't have to figure that out on your own if using eg docker)20:05
clarkbgranted I'm not sure smtp load balancing is a thing20:05
clarkbbut http should mostly work20:05
*** edmondsw has joined #openstack-infra20:07
fungiseems like even for http(s) it would need to be layer-7 "load balancing" (it's not really load balancing, it's a request routing proxy anyway)20:08
*** jkilpatr has quit IRC20:08
fungito call it "load balancing" implies you're using it to balance out load20:08
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove legacy jobs for freezer  https://review.openstack.org/52454920:09
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Use playbook from tripleo-quickstart-extras for OVB  https://review.openstack.org/51350820:10
ianwfrickler: what ended up happening with https://review.openstack.org/524952 and mistral-tempest-plugin?  did we just wait and it appeared?20:10
*** caphrim007 has quit IRC20:10
openstackgerritMerged openstack-infra/project-config master: Add new repo for congress tempest plugin  https://review.openstack.org/52505220:11
clarkbfungi: well they do both I think, load balance to multiple backends for a specific "vhost" but also direct to the proper backends for various vhosts.20:11
clarkbfungi: I want to say the default open source implentation uses nginx20:11
jeblairfungi, clarkb: it sounds like for the moment you're both more inclined to either (a) continue with my crazy multi-site approach, or (b) use multiple vms?20:12
clarkbjeblair: yes I think so20:12
*** edmondsw has quit IRC20:12
*** caphrim007 has joined #openstack-infra20:12
*** slaweq has joined #openstack-infra20:14
jeblairmake sure you've read mm_cfg_multihost.py.erb before you decide between the two :)20:14
fungiyup20:16
fungiheh20:16
fungithat's rather a bit of ruby in the implementation20:18
jeblairisn't it fun?  i think i can drop the mailman_install type and make it a class, for which it is better suited.  the mailman_list type and provider will need to remain, but that's basically yet another copy of the provider we already have with just a couple extra lines.20:20
*** slaweq_ has joined #openstack-infra20:20
jeblair(extra lines to do the env setting)20:20
*** caphrim007 has quit IRC20:20
jeblairto clarify: mailman_install/default.rb becomes a class20:21
*** slaweq has quit IRC20:21
fungigot it, saw it mentioned in the commit message but the explanation helps20:21
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Added relations parameter to get summits endpoint  https://review.openstack.org/52650320:21
fungii'm guessing the exim portion of this, when written, will mostly be config templating since it already has name-based virtual hosting support in the mta20:22
openstackgerritMerged openstack-infra/openstackid-resources master: Added relations parameter to get summits endpoint  https://review.openstack.org/52650320:23
fungiand probably tweaking the mailman-specific router to operate on full addresses rather than only considering the localpart20:23
jeblairfungi: yeah, i'm assuming i'll be able to look up the install from the email address.  i may need to have the mailman_install class maintain a mapping file of domain -> install.20:24
*** jkilpatr has joined #openstack-infra20:24
jeblairbut it should end up being a single router/transport that's highly parameterized.20:25
fungiso this is mostly "chrooting" (but not with an actual chroot) the mailman data and web theming20:26
*** thorre has quit IRC20:27
*** thorre has joined #openstack-infra20:27
jeblairyep.  i haven't actually addressed multiple themes yet, but i think it can be done.20:29
jeblairthe web stuff that's there gets us listing the correct lists on the correct hostnames, but they all still have the openstack logo/theme20:30
openstackgerritPaul Belanger proposed openstack-infra/puppet-zuul master: Update merger / scheduler init script for zuulv3  https://review.openstack.org/52646320:30
openstackgerritPaul Belanger proposed openstack-infra/puppet-zuul master: Update merger / scheduler init script for zuulv3  https://review.openstack.org/52646320:31
*** hasharAway is now known as hashar20:31
*** salv-orlando has joined #openstack-infra20:31
*** baoli has quit IRC20:32
pabelangerjeblair: i figure it might just be easier to do a conditional install of v3 init scripts. That way, once we release 3.0, we can go back into puppet-zuul and start to remove all the 2.0 legacy stuff20:32
*** jkilpatr has quit IRC20:32
*** jascott1 has joined #openstack-infra20:32
*** sean-k-m1 has joined #openstack-infra20:36
*** dmellado has quit IRC20:37
*** sean-k-mooney has quit IRC20:37
*** salv-orlando has quit IRC20:39
*** salv-orlando has joined #openstack-infra20:39
*** stevebaker has quit IRC20:41
*** caphrim007 has joined #openstack-infra20:41
*** [HeOS] has quit IRC20:42
*** Goneri has joined #openstack-infra20:43
*** stevebaker has joined #openstack-infra20:44
*** salv-orlando has quit IRC20:44
*** slaweq has joined #openstack-infra20:45
*** caphrim007 has quit IRC20:46
AJaeger_mordred: do you want to merge https://review.openstack.org/#/c/525688 ?20:46
*** salv-orlando has joined #openstack-infra20:48
rm_workanyone know what channel for the openstack-client?20:49
fungirm_work: maybe #openstack-sdks since it's getting closely tied to the converged sdk now, but dtroyer would know if he's around20:49
*** sboyron has joined #openstack-infra20:50
*** dmellado has joined #openstack-infra20:50
rm_workactually, anyone here know if you can force identity v3 in the clouds.yaml or if you HAVE to do the bash var20:50
*** slaweq has quit IRC20:50
fungirm_work: ahh, yep description for that channel even says "and the unified OpenStack command line tool" https://wiki.openstack.org/wiki/IRC20:52
rm_workcool, kk20:52
mordredrm_work: you can set versions using clouds.yaml20:52
fungi^ mordred might know about forcing ks v3 in config though if he's around20:52
fungiand indeed he is20:52
rm_worklolol20:52
rm_workthe timing20:52
rm_workmordred: what's the line?20:53
mordredrm_work: however, it's important to note that identity api version and auth api version are different concepts20:53
*** cshastri has quit IRC20:53
rm_workhmmmmmmm20:53
rm_workexport OS_IDENTITY_API_VERSION=320:53
rm_workis what i'm trying to do in config20:53
mordredyah- for that, it's "identity_api_version: 3"20:53
*** salv-orlando has quit IRC20:53
rm_workah, thanks20:53
rm_worki thought i tried that...20:53
jeblairpabelanger: wfm.20:53
rm_workah well20:53
mordredrm_work: oh - wait ...20:53
rm_workyeah i did try that20:54
mordredyou're talking about openstackclient - at the moment I believe you need to do it by env vars - until we get some of the new sdk stuff merged20:54
rm_workok20:54
mordredAJaeger_: yah20:55
*** armaan has quit IRC20:56
* mordred waves to people, having been otherwise useless for the last two days...20:56
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Increase timeout for openstack-tox-cover job  https://review.openstack.org/52647420:57
*** xarses_ has joined #openstack-infra20:57
* AJaeger_ waves to mordred from kubecon21:00
*** rfolco is now known as rfolco|off21:01
mordredAJaeger_: you're at kubecon? darn, I was just at kubecon but am no longer there21:01
openstackgerritJeremy Stanley proposed openstack-infra/system-config master: Remove docs-draft vhost from static.o.o  https://review.openstack.org/52651021:02
AJaeger_mordred: heard that you were here yesterday but seems you did not see my message earlier21:02
mordredAJaeger_: I did not - I wound up failing at opening my laptop21:03
AJaeger_mordred: no worries21:03
*** slaweq_ has quit IRC21:04
openstackgerritJeremy Stanley proposed openstack-infra/puppet-openstackci master: Stop expiring contents for docs-draft.o.o  https://review.openstack.org/52651121:04
AJaeger_mordred: regarding 525688 - it needs second +2A21:05
AJaeger_mordred: argh, your change, uploaded by dmsimard|afk - so, we need to ping for other config-core reviewers of 525688.21:07
*** jkilpatr has joined #openstack-infra21:07
*** ganso has quit IRC21:07
fungii'll take a quick look21:08
AJaeger_thanks, fungi21:11
fungilooks potentially disruptive, but i guess no projects are actually running build-sphinx-docs yet?21:11
*** chandankumar has joined #openstack-infra21:11
mordrednot yet, no - this is in service of being able to roll out that change21:12
fungithat's what i thought, so should be safe for now21:12
fungiand it's a little deeper ansible than i feel confident in, but i trust mordred and dmsimard|afk to ansible well21:12
mordredfungi: it has a test depends-on ... https://review.openstack.org/#/c/525690/21:12
fungiahh, nice!21:13
fungii missed that21:13
mordredfungi: well, we didn't needed-by because we're bad people21:13
fungii was more just worried about projects needing to suddenly get specific about setting python versions if they were already using it21:13
mordred++21:14
*** hemna_ has quit IRC21:15
*** thorre has quit IRC21:16
*** thorre has joined #openstack-infra21:17
*** linkmark has quit IRC21:18
*** gyee has quit IRC21:20
*** [HeOS] has joined #openstack-infra21:20
*** camunoz has joined #openstack-infra21:20
*** thorre has quit IRC21:22
*** thorre has joined #openstack-infra21:22
openstackgerritMerged openstack/diskimage-builder master: ironic-agent: don't remove make  https://review.openstack.org/52625721:23
*** slaweq has joined #openstack-infra21:25
pabelangerinfra-root: could use a +3 on https://review.openstack.org/526194/ to remove the last of long lived slaves from puppet21:25
*** kgiusti has left #openstack-infra21:26
pabelangermordred: also fix https://review.openstack.org/526157/ and depends on from api-timeout discussions yesterday for nodepool / clouds.yaml21:26
*** thorre has quit IRC21:27
*** thorre has joined #openstack-infra21:27
clarkbinfra-root, I'm not really in a great place to debug much but I think the reason the elasticsearch indexing is behind is because all of the logstash java daemons got wedged (we've seen this happen when elasticsaerch servers flap)21:28
clarkbI think `sudo service logstash stop` confirming it actually stopped (use manual kill if necssary) then `sudo service logstash start` should make it happy again21:28
*** sambetts|afk has quit IRC21:29
fungiclarkb: i'll give that a shot in a sec21:29
fungipabelanger: looking now21:29
clarkbtelltale behavior that this is a problem is that the service stop won't actually stop it21:29
*** slaweq has quit IRC21:29
fungiyeah, i've handled that at least once in the past too21:29
*** xarses_ has quit IRC21:29
fungiyou're talking about on the logstash-workerNN servers, right?21:30
*** dmsimard|afk is now known as dmsimard21:30
clarkbfungi: no logstash itself21:30
clarkbthe java process21:30
*** florianf|afk has quit IRC21:30
fungioh, just on on logstash.o.o?21:30
ianwyeah, i saw that too and added it to the notes i made in -> https://review.openstack.org/#/c/512467/1/doc/source/logstash.rst21:31
clarkbno we run logstash daemons on each of the worker nodes21:31
*** slaweq has joined #openstack-infra21:31
clarkbfungi: the flow is gearman workers grab files and run crm114 on things then inject the data into logstash which does format processing and indexing into elasticsearch21:31
clarkbin this case I think the gearman workers are all fine, its the java logstash daemon on each of the worker nodes that is unhappy21:31
fungiwait, which worker nodes if not the logstash-workerNN servers?21:32
*** thorre has quit IRC21:32
clarkboh sorry I misread your original question21:32
clarkbyes on the logstash-workerNN servers but not the log-worker processes21:32
*** sambetts_ has joined #openstack-infra21:32
fungigot it, okay i can churn through those21:32
*** thorre has joined #openstack-infra21:32
clarkblogstash service on logstash-workerNN21:32
clarkb(also thats probably an indication I should stop computering, but my head feels so much better sitting up)21:33
fungipabelanger: turnabout, i have similar zuulv3-enabled cleanup proposed in 526510 and 52651121:33
*** thorre has quit IRC21:33
fungiripping out the old docs-draft bits21:34
*** thorre has joined #openstack-infra21:34
fungilooking at http://codesearch.openstack.org/?q=docs-draft it's funny that things like neutron specs and candidate platforms embedded links to ephemeral docs-draft.o.o urls which would only have existed for a few weeks21:35
*** slaweq has quit IRC21:36
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Add query for nova functional test bug 1736976  https://review.openstack.org/52651621:36
openstackbug 1736976 in OpenStack Compute (nova) "test_live_migration_actions functional test randomly fails with "AssertionError: The migration table left empty."" [Undecided,New] https://launchpad.net/bugs/173697621:36
*** rlandy is now known as rlandy|mtg21:38
*** salv-orlando has joined #openstack-infra21:40
jeblairi would not have expected that21:42
*** salv-orlando has quit IRC21:43
*** sambetts_ has quit IRC21:43
*** chandankumar has quit IRC21:43
*** esberglu has quit IRC21:43
*** inc0 has quit IRC21:43
*** dhill_ has quit IRC21:43
*** hashar has quit IRC21:43
*** niedbalski has quit IRC21:43
*** AJaeger_ has quit IRC21:43
*** StevenK has quit IRC21:43
*** snuffkin has quit IRC21:43
*** dalvarez has quit IRC21:43
*** ltomasbo has quit IRC21:43
*** eglute has quit IRC21:43
*** cargonza has quit IRC21:43
*** madhuvishy has quit IRC21:43
*** petrovich has quit IRC21:43
*** d34dh0r53 has quit IRC21:43
*** briancurtin has quit IRC21:43
*** therve has quit IRC21:43
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Add query for cirros 0.3.5 kernel panic bug 1737039  https://review.openstack.org/52651821:43
openstackbug 1737039 in OpenStack-Gate "Random "Kernel panic - not syncing" with cirros 0.3.5 image" [Undecided,New] https://launchpad.net/bugs/173703921:43
fungijeblair: i _didn't_ expect that but, in retrospect, i'm surprised there weren't more21:43
*** bandini has quit IRC21:45
*** Jeffrey4l has quit IRC21:45
*** slaweq_ has joined #openstack-infra21:47
*** bandini has joined #openstack-infra21:48
*** salv-orlando has joined #openstack-infra21:48
*** sambetts_ has joined #openstack-infra21:48
*** chandankumar has joined #openstack-infra21:48
*** esberglu has joined #openstack-infra21:48
*** inc0 has joined #openstack-infra21:48
*** dhill_ has joined #openstack-infra21:48
*** hashar has joined #openstack-infra21:48
*** niedbalski has joined #openstack-infra21:48
*** AJaeger_ has joined #openstack-infra21:48
*** dalvarez has joined #openstack-infra21:48
*** snuffkin has joined #openstack-infra21:48
*** StevenK has joined #openstack-infra21:48
*** ltomasbo has joined #openstack-infra21:48
*** eglute has joined #openstack-infra21:48
*** cargonza has joined #openstack-infra21:48
*** madhuvishy has joined #openstack-infra21:48
*** petrovich has joined #openstack-infra21:48
*** d34dh0r53 has joined #openstack-infra21:48
*** briancurtin has joined #openstack-infra21:48
*** therve has joined #openstack-infra21:48
*** Jeffrey4l has joined #openstack-infra21:48
*** openstackgerrit has quit IRC21:48
*** salv-orlando has quit IRC21:49
*** salv-orlando has joined #openstack-infra21:49
*** jkilpatr has quit IRC21:53
*** armax has joined #openstack-infra21:53
*** salv-orlando has quit IRC21:54
dmsimardI'm back o/21:54
*** thorre has quit IRC21:56
*** slaweq has joined #openstack-infra21:57
*** thorre has joined #openstack-infra21:57
pabelangerfungi: question on 52651021:57
*** felipemonteiro__ has quit IRC21:57
*** baoli has joined #openstack-infra21:57
*** tobiash has quit IRC22:01
*** xarses_ has joined #openstack-infra22:01
*** slaweq has quit IRC22:01
*** openstackgerrit has joined #openstack-infra22:01
openstackgerritMerged openstack-infra/tripleo-ci master: cistatus.html: update OVB jobs  https://review.openstack.org/52575122:01
*** tobiash has joined #openstack-infra22:01
*** wolverineav has quit IRC22:02
*** baoli has quit IRC22:02
*** rlandy|mtg is now known as rlandy22:02
*** wolverineav has joined #openstack-infra22:02
*** andreww has joined #openstack-infra22:04
*** threestrands has joined #openstack-infra22:04
*** threestrands has quit IRC22:04
*** threestrands has joined #openstack-infra22:04
*** baoli has joined #openstack-infra22:04
*** andreww has quit IRC22:04
dmsimardAre we planning on restarting executors/scheduler after all ?22:06
*** xarses_ has quit IRC22:06
*** trown is now known as trown|outtypewww22:06
*** hemna_ has joined #openstack-infra22:06
*** andreww has joined #openstack-infra22:06
*** mat128 has quit IRC22:06
*** wolverineav has quit IRC22:06
pabelangersome discussions, yah22:07
openstackgerritMerged openstack-infra/elastic-recheck master: Add query for nova functional test bug 1736976  https://review.openstack.org/52651622:08
openstackgerritMerged openstack-infra/elastic-recheck master: Add query for cirros 0.3.5 kernel panic bug 1737039  https://review.openstack.org/52651822:08
openstackbug 1736976 in OpenStack Compute (nova) "test_live_migration_actions functional test randomly fails with "AssertionError: The migration table left empty."" [Undecided,New] https://launchpad.net/bugs/173697622:08
openstackbug 1737039 in OpenStack-Gate "Random "Kernel panic - not syncing" with cirros 0.3.5 image" [Undecided,New] https://launchpad.net/bugs/173703922:08
*** rcernin has joined #openstack-infra22:09
*** rcernin has quit IRC22:09
*** rcernin has joined #openstack-infra22:09
*** armaan has joined #openstack-infra22:09
*** ihrachys has quit IRC22:14
*** hashar has quit IRC22:14
fungipabelanger: replied--great question--thanks!22:15
*** armaan has quit IRC22:17
*** dprince has quit IRC22:19
fungiclarkb: bingo. logstash service wasn't gracefully stopping on the workers and ultimately needed sigkill to cleanup (not even sigsegv was getting them to terminate much less sigterm even when followed by sighup)22:21
fungiall started cleanly again now22:21
*** thorre has quit IRC22:21
fungi#status log logstash service stopped, killed and started again on all logstash-worker servers22:22
openstackstatusfungi: finished logging22:22
*** thorre has joined #openstack-infra22:22
jeblairpabelanger, dmsimard: i won't be able to restart it tonight, but if someone else can, that's great (scheduler+executors)22:22
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add finger gateway  https://review.openstack.org/52527622:23
*** thorre has quit IRC22:23
*** weshay is now known as weshay|ruck22:23
*** thorre has joined #openstack-infra22:23
*** cody-somerville has joined #openstack-infra22:24
*** cody-somerville has quit IRC22:24
*** cody-somerville has joined #openstack-infra22:24
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add finger gateway  https://review.openstack.org/52527622:27
*** slaweq has joined #openstack-infra22:29
*** ianychoi has quit IRC22:30
*** ianychoi has joined #openstack-infra22:33
*** slaweq has quit IRC22:34
openstackgerritDavid Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Rename base and multinode integration jobs  https://review.openstack.org/52403922:35
*** andreww has quit IRC22:36
*** thorre has quit IRC22:36
*** rcernin has quit IRC22:36
*** hemna_ has quit IRC22:36
*** sambetts_ has quit IRC22:36
*** chandankumar has quit IRC22:36
*** esberglu has quit IRC22:36
*** inc0 has quit IRC22:36
*** dhill_ has quit IRC22:36
*** niedbalski has quit IRC22:36
*** AJaeger_ has quit IRC22:36
*** StevenK has quit IRC22:36
*** snuffkin has quit IRC22:36
*** dalvarez has quit IRC22:36
*** ltomasbo has quit IRC22:36
*** eglute has quit IRC22:36
*** cargonza has quit IRC22:36
*** madhuvishy has quit IRC22:36
*** petrovich has quit IRC22:36
*** d34dh0r53 has quit IRC22:36
*** briancurtin has quit IRC22:36
*** therve has quit IRC22:36
*** thorre has joined #openstack-infra22:37
openstackgerritIan Wienand proposed openstack-infra/openstack-zuul-jobs master: [WIP] Move prepare-zanata-client to o-z-j  https://review.openstack.org/52576022:38
openstackgerritDavid Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: base/multinode rename #1: Add renamed jobs  https://review.openstack.org/52653222:41
*** rcernin has joined #openstack-infra22:42
*** hemna_ has joined #openstack-infra22:42
*** sambetts_ has joined #openstack-infra22:42
*** chandankumar has joined #openstack-infra22:42
*** esberglu has joined #openstack-infra22:42
*** inc0 has joined #openstack-infra22:42
*** dhill_ has joined #openstack-infra22:42
*** niedbalski has joined #openstack-infra22:42
*** AJaeger_ has joined #openstack-infra22:42
*** dalvarez has joined #openstack-infra22:42
*** snuffkin has joined #openstack-infra22:42
*** StevenK has joined #openstack-infra22:42
*** ltomasbo has joined #openstack-infra22:42
*** eglute has joined #openstack-infra22:42
*** cargonza has joined #openstack-infra22:42
*** madhuvishy has joined #openstack-infra22:42
*** petrovich has joined #openstack-infra22:42
*** d34dh0r53 has joined #openstack-infra22:42
*** briancurtin has joined #openstack-infra22:42
*** therve has joined #openstack-infra22:42
*** yamahata has joined #openstack-infra22:44
*** dhill_ has quit IRC22:46
ianwdmsimard: hmm, i don't think that caching of the client is working as i expected ... looking into the dib side22:46
openstackgerritDavid Moreau Simard proposed openstack-infra/project-config master: base/multinode rename #2: Rename project-config jobs  https://review.openstack.org/52653422:46
*** sboyron has quit IRC22:46
dmsimardianw: caching of the client ?22:46
ianwthe zanata stuff22:47
dmsimardmissing context22:47
*** jaypipes has quit IRC22:47
*** dhill_ has joined #openstack-infra22:47
dmsimardoh, you mean the file isn't there or something ?22:47
*** esberglu has quit IRC22:50
*** salv-orlando has joined #openstack-infra22:50
*** gyee has joined #openstack-infra22:50
*** slaweq_ has quit IRC22:50
ianwyeah, dib seems confused.  maybe the url, we'll see22:50
openstackgerritMichael Turek proposed openstack/diskimage-builder master: Add iscsi-boot element  https://review.openstack.org/51149422:51
*** thorre has quit IRC22:51
*** thorre has joined #openstack-infra22:52
dmsimardianw: so ansible ends up downloading it all the time or something ?22:52
openstackgerritDavid Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: base/multinode rename #1: Add renamed jobs  https://review.openstack.org/52653222:54
openstackgerritDavid Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: base/multinode rename #3: Remove old jobs  https://review.openstack.org/52653622:56
*** salv-orlando has quit IRC22:57
*** thorre has quit IRC22:57
*** felipemonteiro_ has joined #openstack-infra22:57
*** nhandler has quit IRC22:57
*** thorre has joined #openstack-infra22:57
pabelangerjeblair: dmsimard: okay, I too won't be able to this evening. Lets aim for tomorrow morningish22:58
*** gouthamr has quit IRC22:59
dmsimardpabelanger: is swap enabled on zuulv3.o.o %23:00
dmsimard?23:00
dmsimardI remember seeing that it wasn't enabled at one point23:00
*** nhandler has joined #openstack-infra23:00
pabelangerYup: http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=6398123:01
dmsimardok23:02
*** markvoelker has quit IRC23:03
*** onovy has quit IRC23:04
*** markvoelker has joined #openstack-infra23:04
dmsimardjeblair: just fyi, finally got around to do the integration renames https://review.openstack.org/#/q/topic:base-rename23:04
*** onovy has joined #openstack-infra23:05
openstackgerritDavid Moreau Simard proposed openstack-infra/project-config master: base/multinode rename #2: Rename project-config jobs  https://review.openstack.org/52653423:06
openstackgerritDavid Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: base/multinode rename #3: Remove old jobs  https://review.openstack.org/52653623:06
*** harlowja has joined #openstack-infra23:07
*** slaweq has joined #openstack-infra23:07
*** slaweq has quit IRC23:11
*** salv-orlando has joined #openstack-infra23:16
*** thorre has quit IRC23:17
*** thorre has joined #openstack-infra23:17
*** thorre has quit IRC23:21
*** thorre has joined #openstack-infra23:22
mwhahahaquestion, when zuul preps the git repo with the changes where does it put them on the node? is it /var/lib/zuul/builds/<hash>/work/... or is there an easier location to consume from23:22
*** felipemonteiro_ has quit IRC23:22
pabelangermwhahaha: they will be in /home/zuul/src/git.o.o/openstack/foo23:25
pabelangeron the remote node23:25
mwhahahapabelanger: ok thanks23:25
fungigit.openstack.org is spelled out, in case that was non-obvious23:25
*** salv-orlando has quit IRC23:26
mwhahahawell it's better than /var/lib/zuul/blah/blah :D23:26
pabelangermwhahaha: fungi: yah, there is also inventory varaibles in anasible. eg: zuul.project.src_dir23:26
pabelangerhttp://logs.openstack.org/38/526238/1/gate/openstack-tox-pep8/9513211/zuul-info/inventory.yaml for example23:26
*** salv-orlando has joined #openstack-infra23:26
mwhahahayea i founds those23:26
pabelangerbut in general we use the go style of git repos now (I think it is git) in the ~/src23:27
pabelangers/git/go*23:27
*** erlon has quit IRC23:28
*** yamahata has quit IRC23:29
*** salv-orlando has quit IRC23:30
ianw... ohhh, the file is a single line with no trailing newline, so "read" misses it silently :/  now that's a gotcha23:31
fungiianw: which file? i may be a pedant, but i consider text files with no final newline to be a bug worth fixing if it's something under our control or we can at least report about23:33
ianwfungi: https://review.openstack.org/#/c/525050/1/nodepool/elements/cache-devstack/source-repository-zanatacli ... i can definitely fix it; i can either figure out a dib-lint to ensure it, or make the caching script more tolerant23:35
*** jkilpatr has joined #openstack-infra23:41
*** gouthamr has joined #openstack-infra23:41
*** thorre has quit IRC23:41
*** thorre has joined #openstack-infra23:42
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: update zuul cloned repo directory for zuulv3  https://review.openstack.org/52654623:43
mwhahahaweshay|ruck: /home/zuul/src/git.openstack.org/openstack23:43
* mwhahaha applies appropriate -123:43
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: update zuul cloned repo directory for zuulv3  https://review.openstack.org/52654623:44
weshay|ruckthanks23:44
mwhahahanow lets see if it actually works23:44
mwhahaha:D23:44
*** slaweq has joined #openstack-infra23:45
mwhahahawe should also remove the regex_replace https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/build-test-packages/tasks/dlrn-build.yml#L5223:46
mwhahahabut that'll require some fun patch landing23:46
*** slaweq has quit IRC23:49
*** mat128 has joined #openstack-infra23:51
*** stakeda has joined #openstack-infra23:53
*** threestrands has quit IRC23:55
*** dingyichen has joined #openstack-infra23:55
smcginnisfungi: I've been offline too much, and just passing through now, but wondering if you knew if anything had happened with those npm job failures?23:56
mgagneFYI, we fixed some instability issues with inap-mtl01 region where some instances could have failed to schedule successfully23:58
*** yamamoto has joined #openstack-infra23:58
*** mat128 has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!